© 2010 Seapine Software, Inc. All rights reserved.

Monitoring Quality Metrics to Know When to Ship

Seapine ALM Tour

Huntsville, Alabama

Peter VarholSolutions Evangelist, Seapine Software

© 2010 Seapine Software, Inc. All rights reserved.

Agenda

• Why we don’t know when to ship

• How we can know when to ship

• Data, reports, and shipping

• Demonstration

• Really knowing when to ship

© 2010 Seapine Software, Inc. All rights reserved.

When the schedule tells us to

We All Know When to Ship

© 2010 Seapine Software, Inc. All rights reserved.

We All Know When to Ship

• The schedule isn’t a good metric for shipping

• We don’t know if the software is ready

• In fact,

• What does “ready to ship” really mean?

• And if we don’t know what it is, we can’t measure it.

© 2010 Seapine Software, Inc. All rights reserved.

We All Know When to Ship

• Ready to ship may mean:

• All features seem to be working properly

• Failures aren’t replicable

• We think it’s ready

• But there are more objective indicators:

• No P0 or P1 bugs

• New bug count is asymptotic

• All functional tests execute for 10 consecutive days

© 2010 Seapine Software, Inc. All rights reserved.

We All Know When to Ship

• Any definition will do:

• As long as it meets our needs

• And can be measured

• You need:

• A platform for data collection

• The ability to analyze data and create ongoing reports

• The ability to perform ad hoc queries

• The ability to share reports and data

© 2010 Seapine Software, Inc. All rights reserved.

We All Know When to Ship

© 2010 Seapine Software, Inc. All rights reserved.

We All Know When to Ship

• Data tells you when to ship

• You need reports on your data

• TestTrack provides a powerful facility for defining reports

• Built-in reports

• Custom reports

• Easily creatable

• Live charts

© 2010 Seapine Software, Inc. All rights reserved.

Building Reports in TestTrack Pro

• Open versus Closed defects

• Defects by type

• Defects by priority

• Live charts

• Custom reports

© 2010 Seapine Software, Inc. All rights reserved.

Open Versus Closed Defects

© 2010 Seapine Software, Inc. All rights reserved.

Defects by Type

© 2010 Seapine Software, Inc. All rights reserved.

Custom Reports

© 2010 Seapine Software, Inc. All rights reserved.

Custom Reports

© 2010 Seapine Software, Inc. All rights reserved.

Live Charts

• Live charts visually display information in real time:

• Within the TestTrack dashboard

• In a separate window

• Useful for:

• Sharing changing data across the team

• Focusing on the most important data

© 2010 Seapine Software, Inc. All rights reserved.

Live Charts

© 2010 Seapine Software, Inc. All rights reserved.

Summary

• Define your criteria for shipping

• Determine information you need about that criteria

• Generate reports and live charts

• Now you know when to ship

© 2010 Seapine Software, Inc. All rights reserved.

Thank you