Post on 15-Jan-2015
description
transcript
�
AW12 Concurrent�Session�11/13/2013�3:45�PM�
�����
"Test (and More) Patterns for Continuous Software Delivery"
���
Presented by:
Andy Singleton Assembla
���������
Brought�to�you�by:��
��
340�Corporate�Way,�Suite�300,�Orange�Park,�FL�32073�888Ͳ268Ͳ8770�ͼ�904Ͳ278Ͳ0524�ͼ�sqeinfo@sqe.com�ͼ�www.sqe.com
Andy Singleton Assembla
Andy Singleton is the founder of Assembla, a SaaS company providing tools for distributed agile development teams. Andy moved Assembla from bi-weekly releases to multiple daily releases. His forthcoming book Unblock! A Guide to the New Continuous Agile describes workflows for product management, task management, coding, and testing. Previously, Andy founded PowerSteering Software, a provider of enterprise project management software, and Cambridge Interactive, an eBusiness consultancy. He developed financial information systems for Reuters, Bloomberg, SNL Kagan, and Thomson. During his career, Andy has produced more than twenty major software and financial information products.
Be#er%So(ware%East%From%Andy%Singleton,%h#p://andysingleton.com%
www.assembla.com%%
In%this%session%! IntroducAon%to%CD,%benefits%of%CD,%and%ways%to%pitch%it%! MulAple%code%and%test%pa#erns%to%choose%from%! How%to%get%the%automated%tests%you%need%! Changes%in%the%role%of%QA%%
Program% Launch%
Releases%
Plan% Program% Test% Doc% Deploy%
Skip%Automate%&%Blend% Lag% Automate%
Pull%
CI%&%CD%
End%up%with%
Waterfall%to%ConAnuous%
Measure%
Assembla%in%2011%! “Scrumban”%with%iteraAve%releases,%but%conAnuous%planning%to%accommodate%a%distributed%team.%
! Releases%took%longer%as%system%got%bigger%and%there%was%more%to%test.%2%weeks%V>%3%weeks%
! Bugs%in%producAon.%%2%days%for%fixes.%%Stressful)! CompeAtors%achieved%faster%velocity%with%conAnuous%delivery%
! Made%a%study%of%conAnuous%methods%with%our%own%team,%customers,%and%tools.%
! Now%–%releasing%about%250%Ames%per%month.%%Fewer%bugs.%%Much%less%stress.%
You%can%do%it%incrementally%1. Release%more%frequently%2. Improve%%%%%%%
When%to%use%ConAnuous%Delivery?%! You%provide%an%online%service.%%CompeAAve%pressure%will%force%you%to%conAnuous:%Office%365%vs%Google%Docs.%
! You%provide%any%service%with%so(ware%inside%! Your%release%Ames%are%gedng%longer,%or%the%release%process%is%stressful%
! You%are%developing%a%new%product%with%lean%startup%and%MVP%techniques%
! You%have%a%big%project%with%a%lot%of%contributors%! This%applies%to%hardware,%also%
Benefits%for%the%test%process%! Eliminate%stress%from%the%release%process.%%There%is%never%a%big%batch%of%stuff%to%test%under%pressure.%%
! Add%or%remove%test%layers%to%increase%quality%or%increase%speed%
! ConAnuous%integraAon%removes%the%stress%of%acquiring%and%fidng%components%from%many%teams%and%suppliers%
! Your%CEO%wonders%why%the%compeAAon%is%releasing%faster%and%moving%faster%
%
ConAnuous%Delivery%Dial%
Dev
Business
Ops
DEV TEST UAT PROD
CONTINUOUS DELIVERY
RELEASE TRAIN
Self-Service
Controlled
INTEG STAGING
From%Steve%Brodie%and%Rohit%Jainendran%
The%big%quesAon:%How%to%test?%! We%release%so(ware%in%batches%so%that%we%can%test%it.%%That%is%the%whole%reason%for%doing%it.%%We%test%so(ware%%“release%candidates”%to%make%sure%everything%works%together.%
! In%conAnuous%delivery,%we%might%get%as%li#le%at%10%minutes%to%test%a%release%candidate%
! If%we%can’t%test%the%complete%version,%how%do%we%guarantee%quality?%%Is%it%magic?%
Test%Layering%–%A%Golden%Ticket%Monitor%your%released%so(ware:%Errors,%Usage%volume,%usage%pa#erns,%user%feedback%
QA%System%with%Human%test%consultants%
Code%review:%Both%a%manual%test,%and%a%place%to%ask%for%test%scripts.%
ConAnuous%integraAon:%Run%automated%tests%before%using%human%review%Ame%
Unit%tests%in%the%development%environment%
Switch%new%features%and%architecture%
Go%Both%Ways%
Veloc i ty )
Quality)
Increase)Quality)
Increase)Velocity)
Switch%and%Unveil%
Hidden%Programmer%sees%a%change%locally.%Change%is%tested%in%the%%main%version%but%not%seen.%
Test% Story%Owner%and%testers%see%the%change%on%test%systems.%
Beta%Insiders%see%it%and%use%it.%%Story%Owner%can%show%it%to%users%for%feedback%or%A/B%tesAng.%
UNVEIL!% The%big%event.%%Communicate%with%all%users.%%Measure%reacAon.%
One%code%version%
No%special%test%builds%
No%longVrunning%branches%
Code%and%Test%Pa#erns%! Centralized%
! TradiAonal%process%with%no%branches%finds%and%fixes%problems%as%early%as%possible%
! Distributed%! Release%every%change%with%its%own%branch%and%test%
! Temporary%branches%! Efficient%review,%combines%benefits%of%centralized%and%distributed%
! MAXOS%! Use%centralized%conAnuous%integraAon%to%manage%a%massively%scalable%IT%system%
Centralized%CI/CD%Contributor%Commits%–%“as%early%as%possible”%to%find%problems%
ConAnuous%IntegraAon%tests%
Fail%V%alarm%
Release%Candidate%Test%System% Release%
QA%TesAng%
Pass%
Distributed%ConAnuous%Delivery%Contributor%Commits%
Branch%or%Fork%
Deployed%version%
Peer%review%merge%requests%
Other%contribuAons%merged%and%released%“as%late%as%possible”%
QA%Consults%Pass%Final%Auto%Test?%
Merge%back%Current%
Deploy%
Distributed:%MulAple%Test%Systems%Contributor%1%
Contributor%2%
ProducAon%Revision%
Release%AnyAme%
CI%System%QA%Team%
Test%System%1%
Test%System%2%
Assembla%spins%up%test%servers%
Test%and%Review%on%Temp%Branches%
Contributor%Commits%
Auto%tests%
Mainline%
Review% Auto%tests% Review%
• Github%style%–%Pull%requests%or%merge%requests%going%to%CI%system%• Gerrit%style%–%Temporary%branches%for%each%contribuAon%• Assembla%git%supports%both%–%merge%requests,%and%temp%branches%%created%%
by%a%push%to%a%protected%branch.%
Test%before%spending%human%Ame%on%review%
The%Services%Megatrend%Desktop Web App Cloud Services
App
DB
Service
Service
Service
Matrix%of%Services%V%MAXOS%
PrioriAzed%Backlog%
Current%Work%
Each%team%releases%when%ready%
Service%team%IntegraAon%test%env%
ProducAon%service%
Service%team%IntegraAon%test%env%
ProducAon%service%
Service%team%IntegraAon%test%env%
ProducAon%service%
Feedback%on%speed,%errors,%usage,%and%requests%
Organize%by%tesAng%as%one%system%
Coordinate%without%big%meeAngs%
PrioriAzed%Backlog%
Current%Work%
Service%team%IntegraAon%test%env%
Service%team%IntegraAon%test%env%
Service%team%IntegraAon%test%env%
ConAnuous%IntegraAon%between%%latest%dev%version%of%each%service%
• ConAnuous%integraAon%helps%teams%coordinate.%
• See%dependencies%between%“producers”%and%“consumers”%
• Errors%and%conflicts%show%related%team%contact%info%
• MeeAngs%and%changes%negoAated%between%two%teams,%not%many%
Role:%Developer%! Developers%have%more%power%and%responsibility.%! Developers%have%more%responsibility%for%tesAng.%! Developers%(not%QA%or%PM)%decide%when%to%release.%This)is)a)strong)finding.%! IncenAves%are%correct.%%Developer%might%have%to%come%back%from%Friday%night%beers%to%fix%a%problem.%%This%provides%a%moAvaAon%to%make%good%decisions%and%automate%tesAng.%
! Features%can%be%released%but%hidden.%%Product%Managers%and%Marketers%will%unveil%when%they%are%ready.%Unblock!%
Role:%QA%! QA%is%a%consultant.%! QA%gets%more%respect.%%Developers%have%to%ASK%for%service.%
! Developers%do%more%of%the%tesAng%work.%%They%should%organize%reviews%and%automated%tests%so%that%bugs%don’t%go%through%into%the%manual%test%process.%
! QA%has%more%Ame%to%invesAgate%usability%! QA%monitors%producAvity%and%quality%metrics%
IncenAves%for%ConAnuous%Flow%You%don’t%need%culture%change.%%You%only%need%to%release%more%frequently%1. If%you%do%code)review,%you%can%get%automated.%No%
browbeaAng%and%cajoling%is%required.%%Developers%will%ask%for%tests%when%they%review.%
2. If%developers%decide%to%release,%they%will%take%more%responsibility%for%tesAng%and%automated%tests.%
3. If%you%release%more%frequently,%developers%will%quickly%learn%not%to%break%the%build%
4. If%PM’s%take%unveil)and)measurement%responsibility,%they%will%make%be#er%features%
Our%Master%Plan%1. Release%more%frequently%2. Improve%%%%%%%
www.conAnuousagile.com/unblock%