Beyond  Shi+  Le+  5  Steps  to  Detec+ng  Issues  Earlier  in  Your  Release  Cycle  

#beyondshi+le+  @perfectomobile  

What  We’ll  Cover  

•  Our  Climate  Is  Changing  •  Why  Are  Top  Performers  Winning?  •  Less  Time  Fixing,  More  Time  CreaEng  •  Fast  Feedback  in  5  Steps  

•  Speak  the  Same  Language  •  Write  Stable  Tests  •  Break  Up  the  Monoliths  •  Test  More  in  Every  Build  •  Provide  Context  with  Results  

•  Q  &  A  

#beyondshi+le+  @perfectomobile  

Our  Presenters  

Paul  Bruce  Developer  Evangelist  

@paulsbruce  

Nick  Sanjines  Systems  Engineer  @NickSanjines  

work  fast  learn  o+en  

#beyondshi+le+  @perfectomobile  

The Effect of “Climate Change” on App Development

Infrastructure  is  cheap  

Releasing  is  easy  

Faster  to  create  apps  

Automate  everything  

Release  more  frequently  

People  expect  great  apps  

Maintenance  is  costly  

At  risk  more  o+en  

Higher  stakes  for  business  

#beyondshi+le+  @perfectomobile  

#defectshappen  :  

#beyondshi+le+  @perfectomobile  

Top  Performers  

● Releasing  is  run-­‐rate      more  o+en  

 ● Manage  change    

 early  recovery    

● Small  batch  sizes      less  lead  Eme  

 ● Less  unplanned  work    

 more  new  work  

State  of  DevOps  2016,  Puppet  Labs  

quality  impacts  velocity  

#beyondshi+le+  @perfectomobile  

Less  Cme  fixing,  more  Cme  creaCng,  

From  this:    ● Unclear  goals  ● Technical  debt  ● Fire  fighEng  ● Release  risk  

To  this:    ● User  saEsfacEon  ● Be^er  decisions  ● Higher  confidence  

Defects  

Backlog  

Features  

Time  

#beyondshi+le+  @perfectomobile  

How  do  we  spend  less  Cme  on  re-­‐work?  

•  Catch  defects  earlier  •  Simplify  diagnosis  and  fixing  •  Automate  repeEEve  tesEng  •  Foster  fast  feedback  

 

“As  an  engineer,  you  should  constantly  work  to  make  your  feedback  loops  shorter  in  +me  and/or  wider  in  scope.   ”    —   @KentBeck  

Source:  IBM  System  Science  InsEtute  

#beyondshi+le+  @perfectomobile  

5  Steps  to  DetecCng  Issues  Earlier  

1.  Speak  the  Same  Language  

2.  Write  Stable  Tests  

4.  Test  More  in  

Every  Build  

3.  Break  Monoliths  Apart  

5.  Provide  Context  with  

Results  

1  speak the same language

#beyondshi+le+  @perfectomobile  

Why  speak  the  same  language?  

•  Simplifies  collaboraEon  on  tesEng  •  Enables  tests  to  reuse  code  arEfacts  •  Makes  code  and  tests  equal  ciEzens  

 

Selenium  

App  Code   Java,  Javascript,  C#,  etc.  

2  write  stable  tests  

#beyondshi+le+  @perfectomobile  

How  do  we  write  stable  tests?  

•  Reduce  technical  gaps  in  object  idenEficaEon  •  Clean  setup  /  teardown  procedures  •  Be  as  hermeEc  as  possible  •  Soak  new  tests  before  checking  them  in  

 

3  break  monoliths  

apart  

#beyondshi+le+  @perfectomobile  

Job   Total  CI  Eme  budget  

Test  Time  (avg.)  

Scope  

Commit  (VCS)   15-­‐30  mins   <100ms   Unit,  API,  IntegraEon  

Hourly   20-­‐40  mins   Varies   New  Tests,  Key  Smoke  (BVT)  

MulEple/day   30-­‐60  mins   <2s   All  Smoke,  Key  FuncEonal,  Some  Data  

Nightly   2-­‐7  hrs   <120s   All  FuncEonal,  All  Data  

Weekend   10-­‐48  hrs   all   All  Tests  incl.  Performance  

Don’t  leave  all  the  regression  tesCng  to  the  end!  

An  anecdotal  example,  your  own  mileage  will  vary  

4  test  more  in  every  build  

#beyondshi+le+  @perfectomobile  

What  does  “more”  mean  in  terms  of  your  audience?  

Digital  Test  Coverage  Index  

Q4  2016,  5th  EdiEon    

bit.ly/DTC-­‐index-­‐5  

5  provide  context  

with  results  

#beyondshi+le+  @perfectomobile  

5  Steps  to  DetecCng  Issues  Earlier  

1.Speak  the  Same  

Language  

2.  Write  Stable  Tests  

4.  Test  More  in  

Every  Build  

3.  Break  Monoliths  Apart  

5.  Provide  Context  with  Results  

#beyondshi+le+  @perfectomobile  

Q  &  A  

Paul  Bruce  Developer  Evangelist  

@paulsbruce  

Nick  Sanjines  Systems  Engineer  @NickSanjines