1. W6 Special Topics 5/1/2013 11:30:00 AMYin and Yang: Metrics within Agile and Traditional Lifecycles Presented by: Shaun Bradshaw, Zenergy Technologies, Inc. Bob Galen, Rgalen ConsultingBrought to you by:340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 904-278-0524 sqeinfo@sqe.com www.sqe.com

2. Shaun Bradshaw Cofounder of Zenergy Technologies, Inc., a QA and agile solutions consulting firm (zenergytechnologies.com), Shaun Bradshaw has spent the past fifteen years advising, teaching, and mentoring clients to improve their QA and test processes with effective testing and test management techniques. He is the coauthor, editor, and trainer of a suite of methodologies covering testing, test management, and test metrics. A popular speaker at many of the major QA industry conferences, Shaun is known for his presentations on test metrics, the S-Curve, and the Zero Bug Bounce.Bob Galen Bob Galen is an agile coach at RGalen Consulting and director of agile solutions at Zenergy Technologies, a North Carolina-based firm specializing in agile testing and leading agile adoption initiatives. Bob regularly speaks at international conferences and professional groups on topics related to software development, project management, software testing, and team leadership. He is a Certified Scrum Master Practicing (CSC), Certified Scrum Product Owner (CSPO), and an active member of the Agile Alliance and Scrum Alliance. Bob published Scrum Product OwnershipBalancing Value from the Inside Out, which addresses the gap in guidance toward effective agile product management. Contact Bob at bob@rgalen.com or bob.galen@zenergytechnologies.com. 3. 4/16/2013The Yin & Yang Title of Software MetricsSTAREAST 2013 Presented By: Bob Galen and Shaun BradshawSlide 1Software Peace of Mind in Action??? Introductions Bob Galen & Shaun BradshawDirector, Agile Solutions Zenergy Technologies Determined agile adoption coach, trainer, and consultant 10+ years of agile experience across Software, QA/Test, and Project Management Specialty in Agile at-Scale challenges bob.galen@zenergytechnologies.comSlide 2VP of Consulting Solutions Zenergy Technologies Experienced test manager, consultant, trainer IT Testing strategist 15 years of multi-domain experience Agile experience; Scrum Master shaun.bradshaw@zenergytechnologies.comSoftware Peace of Mind in Action1 4. 4/16/2013??? Format For This Session Focus for this session: QA & Testing Metrics Panel format 15 minutes for Shaun 15 minutes for BobWrite down your questions, ideas, metrics, etc. on 3x5 cards 30 minutes of Q&A interaction and debate Slide 3Software Peace of Mind in ActionWhat is Measurement? Measurement is the process by which numbers or symbols are assigned to attributes of entities in the real world in such a way as to characterize them according to clearly defined rules. Measurement is the empirical, objective assignment of numbers, according to a rule derived from a model or theory, to attributes of objects or events with the intent of describing them.2 5. 4/16/2013??? Software Metrics Traps Karl Wiegers 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.http://www.processimpact.com/articles/mtraps.htmlLack of Management Commitment Measuring Too Much, Too Soon Measuring Too Little, Too Late Measuring the Wrong Things Imprecise Metrics Definitions Using Metrics Data to Evaluate Individuals Using Metrics to Motivate, Rather than to Understand Collecting Data That is Not Used Lack of Communication & Training Misinterpreting Metrics DataSlide 5Software Peace of Mind in Action??? Case Study Large, traditional financial / insurance organization 100 year company, traditionally staffed IT organization Regulatory support Large-scale project; joint with other consulting firmsSlide 6Software Peace of Mind in Action3 6. 4/16/2013??? Waterfall is Real Not every project can be broken down into 2-week chunks Not every company wants to be agile Scale of the project impacts everything Well know were done when were done is NOT an acceptable answer in many contexts Waterfall and AgileSlide 7Software Peace of Mind in Action??? Monitoring Phases & Transitions The overriding goal was to plan a series of maturation phases Testing would serve as the guide between phases Qualifying progress Verifying functionality Measuring variation / defects Guiding next stepsSlide 8Software Peace of Mind in Action4 7. 4/16/2013??? Risk-Based TestingAll testing was focused towards what Was delivered Worked Was planned Was considered Higher Priority features Was considered Higher Risk itemsSlide 9Software Peace of Mind in Action??? Key Metrics Exit Criteria Test Execution by priority Have we tested enough? Did we cover our plans?Overall pass rates Is the software working? Where are we Blocked? Are there any Patterns to the failuresPercent of defects corrected by severity Are we catching up on repairs? Is the software maturing at a sufficient rate to support the plan? Slide 10Software Peace of Mind in Action5 8. 4/16/2013??? Customer Override Basically, the customer has the final authority to adjust Metrics Analysis (Re)Actions Goals So, agility from a leadership & governance perspectiveSlide 11Slide 12Software Peace of Mind in ActionSoftware Peace of Mind in Action6 9. 4/16/2013TransitionSlide 13Software Peace of Mind in Action??? Agile Metrics Agile methods are generally not metrics centered, Theyre much more team-centered and progress or outcome centered Rarely do you want to examine the individual; leave that to the team Exceptions experienced coachesAvoid Metrics Dysfunction metrics Lines of code produced Bugs found Estimate vs. Actual Slide 14Software Peace of Mind in Action7 10. 4/16/2013??? Big 4 Agile Metrics 1. 2. 3. 4.Value Delivered Velocity & Throughput Quality Team Morale & SatisfactionSlide 15Software Peace of Mind in Action??? Value-Delivered Prioritization Always deliver the highest priority work first Earned value its DoneCustomer Acceptance User Story sign-off Active customer demonstration; embrace changesROI Usage Sales & usage targets; confirm Did we make the right decisions on value? Adjustments? Slide 16Software Peace of Mind in Action8 11. 4/16/2013??? Velocity & Throughput Productivity matters in agile teams First, the software is only accepted if it meets clear Done-Ness criteria Velocity the number of Story Points produced per Sprint Throughput the number of Days that a story was InProgress, relative to its size; also a risk playContinuous improvement Velocity UP; Throughput - DOWN Slide 17Software Peace of Mind in Action??? Quality Test Automation Continuous Integration & Continuous Deployment Every check-inrun automation; Stop-the-LineDefect Escapes Escaping your Done-Ness Escaping your Sprint Escaping into the wildCustomer EscapesPairing, inspections, RCA Continuous Improvement downward trending Slide 18Software Peace of Mind in Action9 12. 4/16/2013??? Team Morale & Satisfaction Self-directed; Team autonomy; Listened to Attrition is analyzed; New hire onboarding Sustainable pace Periodic surveys Team overall health & satisfaction Feedback on adjustments Agile valuesMake the survey and the actions Transparent Make it clearpeople are central to your successSlide 19Software Peace of Mind in Action??? Thats it Aligned with Goals Sprint Goals & Release Goals Organizational Goals & Company GoalsAttributes: Theyre all importantperhaps People First There are no fixed targets Keep them simple; everyone is aware of the goalMeasure as a team Stop the individual metrics dysfunction Slide 20Software Peace of Mind in Action10 13. 4/16/2013TransitionSlide 21Software Peace of Mind in ActionNow Its YOUR Turn Questions? Debate!Slide 22Software Peace of Mind in Action11 14. 4/16/2013??? Questions for your Consideration What healthy metrics have you personally experienced? What about stories of dysfunctional metrics? Challenges youve seen in moving from Waterfall to Agile or vice versa? If you only had 1-3 metrics to measure software in your own company, what would they be?Slide 23Software Peace of Mind in Action??? Questions for your Consideration Example of what we could discuss Estimation vs. Actuals Planning Poker & Relative estimationAnyone use GQM?Slide 24Software Peace of Mind in Action12 15. 4/16/2013Any other questions?Thank you!Slide 25Software Peace of Mind in Action13