Do less testing

Mark Hrynczak

Atlassian

$

Testing ≠ Quality

“Effective software teams are all alike; every dysfunctional team is dysfunctional in its own way”

L. TOLSTOY

ProcessDeveloperRequirements Quality Software

Dysfunctional software team

ProcessDeveloperRequirements Low-qualitySoftware

Dysfunctional software team

ProcessDeveloperRequirements

Testing Phase

Quality Software

Dysfunctional software team

ProcessDeveloperRequirements

Testing Phase

Quality Software

Dysfunctional software team

ProcessDeveloperRequirements

Testing Phase

Quality Software

Dysfunctional software team

ProcessDeveloperRequirements

Testing Phase

Quality Software

Dysfunctional software team

ProcessDeveloperRequirements

Testing Phase

Quality Software

Dysfunctional software team

Conflict in roles

TesterWrites the test code

DeveloperWrites the functional code

Conflict in aims

TesterWants to block release

DeveloperWants to release

Conflict in attitude

TesterCares about quality

DeveloperDoes not care about quality

Conflict in perception

TesterValue is implicit and unmeasurable

DeveloperDelivers explicit value to the business

Process Quality SoftwareDeveloperRequirements

Dysfunctional software team

High-qualityProcess

Quality SoftwareDeveloperRequirements

Dysfunctional software team

High-qualityProcess

Quality SoftwareRequirements High-qualityDeveloper

Dysfunctional software team

High-qualityProcess

Quality SoftwareHigh-qualityDeveloper

High-qualityRequirements

Effective software team

Testing ≠ Quality

Do less testing

What is testing for?What are the ongoing costs?

Automated testingIs it safe to release?

Manual testingCan we find bugs?

Automated testing is not cost free

(Virtual) Hardware Time to run

Framework & implementation Ongoing maintenance

Automated testing is not cost free

(Virtual) Hardware Time to run

Framework & implementation Ongoing maintenance

Acceptable testing:

Several days of manual testing to find bugs

Absolute reliance on fully automated pipeline

Reducing time between releases from:

Months to weeks

Weeks to days

Days to hours

Every commit

Journey to continuous deployment

How to do less testing?

How to spend less time doing testing?

Prevent

Identify potential problems

before coding

Mitigate

Reduce the impact of

problems to end users

Listen

Adjust the quality bar based

on your users’ actual

experience

Tactics to reduce/replace testing

Listen

• Realistic and consistent bug

policy

• Impact-driven priorities

• Don’t sweat the small stuff

Mitigate

• Monitoring

• Staged deployments with

automatic rollbacks

• Feature flags

Prevent

• Kickoffs to identify edge cases

• Paired development

• Train your developers in quality

thinking

Examples

AutomateMachines tell you that deployment is safe

TrustDevelopers are allies with the same goals

MeasureTesting has costs, especially time-related

ReplaceAlternative ways to engineer quality software

Takeaways MindsetGoal is quality software, not testing activity