Post on 22-Jan-2018
transcript
Managing So)ware Debt in Prac2ce
Quality Debt Focus
2
First Things First...
@csterwa #swdebt
Get Rally! www.rallydev.com
3
Chris Sterling
• Director of Engineering at Rally So)ware in Kirkland, WA office
• Author of Book “Managing So)ware Debt: Building for Inevitable Change”
• Consults on so)ware technology, Agile technical prac2ces, Scrum, and effec2ve management techniques
• Innova2on Games® Trained Facilitator • Cer2fied Scrum Trainer • Open Source Developer
Email: csterling@rallydev.com Web: hWp://www.rallydev.com Follow me on TwiWer: @csterwa Blog: hWp://www.geYngagile.com Hashtag for presenta2on: #swdebt
4
Agenda
• Managing So)ware Debt Overview
• So)ware Debt Types • Technical • Quality • Configura2on Management • Design • Pla]orm Experience
• Asser2ng Quality • Defini2on of Done • Test Automa2on
• The “3 Amigos” PaWern • Test-‐Driven Design
• Monitoring Quality • The Power of 2 Scripts • Con2nuous Integra2on • Automated Promo2on • Advanced Quality Metrics Trending and Analysis
• Wrap Up • So)ware Debt Management Strategy • The “No Defect” Mindset
Managing So)ware Debt Overview Let me tell you a poten2ally familiar story…
Lack of emphasis on so)ware quality aWributes contributes to decay
Principle: No maWer what, the cost of addressing so)ware debt increases with 2me.
Types of So)ware Debt Technical, Quality, Configura2on Management, Design, and Pla]orm Experience
10
• Technical debt tended to focus more on programming aspects of so)ware delivery and le) out full so)ware development life cycle
• Each type of so)ware debt can be managed and monitored using different tools and approaches
• Focusing on managing each type of so)ware debt simplifies crea2on of an overall strategy that promotes holis2c perspec2ve
Why not just call it all “Technical Debt”
Types of So)ware Debt • Technical Debt: These are the ac2vi2es that a team or team members take shortcuts on now that will impede future development if le) as is.
• Quality Debt: There is a diminishing ability to verify the func2onal and technical quality of so)ware: the “Break/Fix” mentality.
• Configura2on Management Debt: Integra2on and release management become more risky, complex, and error-‐prone.
• Design Debt: The cost of adding features is increasing toward the point where it is more than the cost of wri2ng from scratch.
• Pla]orm Experience Debt: The availability and alignment of people to business objec2ves that involve so)ware changes is becoming more limited or cost-‐prohibi2ve.
8
Asser2ng Quality Teams must focus on asser2ng sustainable quality to support future customer needs in a 2mely manner
Defini2on of Done So)ware developments assert what finished, working so)ware entails to support predictable delivery into the future
14
Defini2on of Done -‐ Assert Quality " Acceptance defined criteria for each
user story �
" Unit tests written and passed �
" Code compiles with no errors and no warnings �
" New code doesn’t break existing code �" Test case review (Dev to review test
case written) �
" Architectural impact assessed and artifacts updated if necessary�
" Comments in code �
" Error codes added�
" Code reviewed by peer �
" Code checked in with reference to US#/Task# �
" Tested on FE�
" Integration test written & passes �
" Test code reviewed�
" Environment requirements documented �
" Interface document updated/added and checked in to SVN �
" Acceptance criteria verified complete �
" All P1-P3 bugs for the story are closed �
" Test approves user story �
" Story demonstrated to product owner and accepted on Target Platform�
Release Defini2on of Done
• Every release should have clear quality criteria • With a “Release Defini2on of Done” you can understand targets beWer
• Measure the gap between the teams’ Defini2on of Done and a Release Defini2on of Done. • This gap is a source of quality issues and represents significant risk to schedule
Case Study: Test Automa2on Reduces Cost of Change
17
Manual Regression Tes2ng
• Tes2ng was taking 75 person hours during 2 full test runs consis2ng of: • Comprehensive manual regression tes2ng • Data conversion and valida2on
• Cost for tes2ng was $17,000 each itera2on
18
Introducing Fit into Tes2ng Process
• A)er 8 itera2ons team had introduced healthy amount of Fit fixtures and automated tests
• Reduced 70+ hour test run2me down to 6 hours which now included: • Fit automated regression tes2ng • Data conversion and valida2on automated with Fit fixtures
• Reduced cost of tes2ng each itera2on from $17,000 to $7,000
19
The Agile Regression Tes2ng Triangle*
* The Agile Triangle has been modified from Mike Cohn’s original version
Automated Unit Tests Make up largest por2on of regression tests and are developed by programmers
Integra2on Tests Automated & Exploratory
Smoke++ Tests Risk-‐based UI & API Automated Tests
The “3 Amigos” PaWern* Quickly get testers, coders, and business on the same page before building based on a requirement
* The Three Amigos paWern originally coined by George Dinwiddie hWp://www.s2ckyminds.com/s.asp?F=S17232_COL_2
21
The Three Amigos PaWern
As a Shopper I want to receive updates on incredible deals that are located near my home so that I can save money on my purchases
Acceptance Criteria: • Save Shopper’s location • Ask Shopper if they want to receive localized deals daily • Send notification of incredible deals to Shoppers located within 10 miles each morning • Allow Shopper to stop receiving localized deals
22
The Three Amigos PaWern
As a Shopper I want to receive updates on incredible deals that are located near my home so that I can save money on my purchases
• What areas of the applica2on will this affect? • What is the overall design? (UI, API, UX, etc…) • What are the details test cases for this user story and it’s acceptance criteria? • What about nega2ve test condi2ons? • What about boundary condi2ons? • How might we put exis2ng func2onality at risk?
23
The Three Amigos PaWern
• At minimum include tester, coder & business rep in discussion
• Should only take 30 minutes to 1 hour for user stories
• Focus on clarifica2on and design through testable inputs/outputs
24
Acceptance Test-‐Driven Development
Release Management “If releases are like giving birth, then you must be doing something wrong.” -‐ Robert Benefield
26
Case Study: Enterprise Agile Adop2on • 180+ person “Web 2.0” product organiza2on • Waterfall SDLC that development uses to deliver in 6 month release cycles
• Want to use Agile methods to be more responsive to users and keep up with other “Web 2.0” companies
• Transi2oned to Agile methods on 15 teams in 3 months • Changed release management strategy, added XP technical prac2ces, and implemented Scrum product development framework for scaled coordina2on
• Able to release every week to users within 4 months • Used streamlined deployment environment process to validate product changes daily using Con2nuous Integra2on and automated promo2ons
27
The Power of 2 Scripts: Deploy & Rollback
28
Tradi2onal Source Control Management
Main Branch Debt
Death March { Debt accrues quickly within stabiliza2on periods
Version 1 Branch
Integrate for Version 2
Code Complete
29
Flexible Source Control Management
Main Branch
Version 1 Version 2 {Not Easy! Must have proper infrastructure to do this.
30
Scaling Con2nuous Integra2on
Component Valida2on
Integrated Component Valida2on
End-‐to-‐End & Load/Stress
31
Automated Promo2on to Environments
Advanced Quality Asser2ons Using Automated Tools and Dashboards
33
Con2nuous Integra2on
34
Quality Dashboard -‐ Sonar
35
Quality Dashboard -‐ Sonar
36
Quality Dashboard -‐ Sonar
37
Quality Dashboard -‐ Sonar
38
Early Warning Signs
Early Warnings: • Broken Builds • Broken Automated Tests • Broken Custom Thresholds
39
Early Warnings: • Design Debt in Duplica2on (DRY) • Technical Debt in Code Complexity • Quality Debt in Bug DB (Break/Fix) • Other Custom Thresholds
Early Warning on Quality Dashboard
The “No Defect” Mindset “What he needs is some way to pay back. Not some way to borrow more.” -‐-‐ Will Rogers
39
Ken Schwaber “For every [dollar] of compe22ve advantage gained by cuYng quality, it costs $4 to restore it; and so)ware is an organiza2onal asset and decisions to cut quality must be made by execu2ve management and reflected in the financial statements.” hWp://www.infoq.com/presenta2ons/agile-‐quality-‐canary-‐coalmine
Case Study: Field Support Applica2on
• 2000+ users access applica2on each day • Applica2on supports mul2ple perspec2ves and workflows from Field Support Opera2ons to Customer Service
• Team of 5 people delivering features on exis2ng Cold Fusion pla]orm implementa2on
• Migra2ng Architecture to Spring/Hibernate in slices while s2ll delivering valuable features
• 36 2-‐week Sprints, 33 produc2on releases, and only 1 defect found in produc2on
• So, what was the defect you say? Let me tell you…
40
Can We Afford a “No Defect” Policy? • This team worked on legacy codebase inherited from another vendor
• Other vendor had been slowing down month a)er month and cost of development was increasing
• In first itera2on this team was able to deliver more than other vendor was able to in previous 2 months
• A)er 24 itera2ons this team was 10 2mes faster delivery than1st itera2on
• Acceptance Test-‐Driven Development and Con2nuous Integra2on were greatest technical factors to support team in these results
• Can you afford not to have a “No Defect” policy?
41
Thank you! Ques2ons and Answers [Time permiYng]
45
Chris Sterling Director of Engineering at Rally So)ware in Kirkland, WA office Author of Book “Managing So)ware Debt: Building for Inevitable Change” Consults on so)ware technology, Agile technical prac2ces, Scrum, and effec2ve management techniques Innova2on Games® Trained Facilitator Cer2fied Scrum Trainer Open Source Developer
Email: csterling@rallydev.com Web: hWp://www.rallydev.com Follow me on TwiWer: @csterwa Blog: hWp://www.geYngagile.com Hashtag for presenta2on: #swdebt