Lean testing

transcript

Lean Management for IT Testing by Jean Louis Lequeux is available under the terms of Creative Commons License - No Commercial Use - No Modification 3.0 Unported. Based on a works

at http://weltram.eu. For further inquiries: http://weltram.eu/contactus.aspx

Lean Management for IT Testing

Jean Louis LEQUEUX http://weltram.eu

V 3.0 – December 6th, 2010

December 6th, 2010 QHSE Series, Jean Louis LEQUEUX http://weltram.eu2

Grand Summary

1. What is Lean Management?2. Pledging for a Test repository3. How to apply Lean Management to

Testing

Lean Management for IT

Reduce Waste Optimize Processes Increase Performances Get better Quality, better Customer

Satisfaction

Where does it apply?

In all Service Areas where production is involved: Maintenance Operation Customer Support

But also in any time & energy consuming subset of the Development Cycle

The Development Cycle

1. Specification2. Prototyping3. Conception4. Coding5. Integration6. End-to-end tests7. Pre-production8. Production9. Evolution

1. Specification2. Prototyping tests3. Conception4. Coding tests5. Integration tests6. End-to-end tests7. Pre-production

tests8. Maintenance tests9. Evolution tests

Test issues for Lean Management Issue 1: different teams Issue 2: different periods Issue 3: an everlasting “do it again!”

Solution!

A Test Repository is necessary

Lean Management on the different issues of the Test Repository

A Test Repository for the IS

Jean Louis LEQUEUXlequeux.jean-louis@weltram.eu

Un référentiel de tests pour le SI

Summary of Repository

What does test means? The IS Tests

methodology 8 steps A Tests Repository for

the Information System ISO Approach

Tests are the basement of a good IT System

Que signifie « tests »? Méthodologie de tests

8 étapes Un référentiels de

tests pour le système d’information

L’approche ISO Les tests sont

fondamentaux!

Testing

Functional tests Unit testing Assembly testing Non regression

tests Performance tests

Load tests Technical tests

Security tests

Tests fonctionnels Unitaires D’intégration De non-régression

Tests de performance Montée en charge Tests techniques

Tests de sécurité

Objectivity, Conformity…

Manage a trustful tests program

Rely on repeatable results

Tests must be Conform to reality Usable for

predictive behavior of SI

Gérer un programme de tests fiables

Se baser sur des résultats répétables

Les tests doivent être Conformes à la réalité Utilisables pour le

comportement prédictif du SI

Today’s Situation

For each application, different scenarios, different tools, even different sets of data are used…

No leading concepts for testing!

Pour chaque application, de différents scénarios, différentes données, différents outils…

Pas d’idées directrices pour les tests!

How Did We Get Here?

Tests are often “forgotten” until the last stages of integration

Only functional tests draw the attention: Because the end-

users are particularly interested

Les tests sont souvent « oubliés » jusqu’aux dernières étapes de l’intégration…

Seuls les tests fonctionnels attire l’attention: Parce que les

utilisateurs finaux s’y intéressent plus particulièrement

Types of tests

Tests apply toManagement

Information Systems

Technical Information Systems

Les tests s’appliques aux:

Systèmes de gestion

Systèmes d’information techniques

Types of tests

Typical transactions

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Tests Procedures

Define tests procedures at early stages of project

Requested Performances must be listed

Define clearly current operational conditions

Propose tests records and tests validation chains

Définir les procédures de tests dès les phases initiales du projet

Liste obligatoires des performances requises

Définir clairement les conditions d’exploitation

Définir les rapports de tests et la chaîne de validation des tests

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Typical Transactions

Define and describe typical transactions, with chaining of elementary transactions

Define end-to-end transactions

Définir et décrire les transactions types avec l’enchaînement des transactions élémentaires

Définir les chaînes de transactions de bout en bout

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Typical Data

Build test data (the tests data set must be reusable)

Tests data must be realistic

Ideally, tests data structure derived from the data models

Construire des données de tests permanentes

Les données de tests doivent refléter la réalité

Idéalement, la structure des données de tests dérive des modèles de données.

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Re-usable Tools

Choose pertinent tools, according to environment, tests objectives, etc.

Develop necessary complementary tools, such as injectors

Choisir les outils de tests en fonctions de l’environnement, des objectifs, etc.

Développer les outils complémentaires nécessaires, tels que les injecteurs

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Re-usable scenarios

Write test scenarios as close as possible to real applications

Scenarios should be reusable

Scripts must be adaptable according to the evolutions of the IT System

Écrire des scénarios de tests aussi proches que possible de la réalité

Les scénarios doivent être réutilisables

Les scripts doivent pouvoir évoluer avec le SI

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Tests Platforms

Tests platforms must be as much conformant as possible with the operational systems

Use simulators to build a model of target systems

Emulate physical and technical environment (bandwidth, traffic, etc.)

Les plateformes de tests doivent être aussi conformes que possibles aux systèmes

Utiliser des simulateurs pour modéliser le système cible

Émuler l’environnement technique et physique (bande passante, etc.)

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Tests Results

Create templates for tests reports

Manage Tests Results

Use Document Management System

Results of past tests must be accessible

Créer des modèles de rapports de tests

Gérer les résultats des tests

Utiliser un Système de Gestion Documentaire

Résultats accessibles!

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Recommendation

Urbanize First! Define tests on an

Urbanized Model

Define Domain for tests on Urbanized System

Urbaniser d’abord le SI!

Les tests sont défini sur le modèle urbanisé

Définir les domaines de tests!

A Tests Repository

Build-up the Test Repository with :

Best Practices recommendations

• Typical tests requested• Tests Procedures

Tests objects• Re-usable Tools• Test Data extracted

from real Data Models• Typical transactions for

each urbanized domain• Re-usable scenarios• Adaptable Scripts

Tests Results

Construire le référentiel de test comprenant:

Les Meilleures Pratiques:• Procédures• Tests requis

Items de tests …/• Outils réutilisables• Données de tests

extraites de modèles réels

• Description des transactions typiques dans chaque domaine urbanisé

• Scénarios réutilisables• Scripts adaptables

Résultats des tests

Pledging for an ISO Approach Beyond ISO 9001, a

normative approach is necessary

IT tests are definitely “measures”

Is ISO/IEC 17025 adaptable to IT Tests Labs?

Au-delà de l’ISO 9001, une approche normative est nécessaire

Les tests informatiques font partie du domaine de la mesure

L’ISO/IEC/17025 est-elle adaptable à nos professions?

Lean Management Applied to IT Testing

Jean Louis Lequeuxhttp://weltram.eu

Lean Testing Summary

Starting point Where to optimize the processes How to manage coordination

Lean Management of the Test Repository

1. Internal processes2. Inter processes3. Specificity of each

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Internal processes

Lean Management for each team: Optimize Resources Optimize tools Optimize exchanges

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Inter processes

Make sure each team pass all necessary information

Allow fluidity between teams Cases and discussions: focus

on solutions!

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Types of tests

Apply Agile Method for Unit Testing

Use tools for detecting exhaustively cases

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Capitalize procedures Make them reusable Discuss with leaders

of other testers’ teams

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Customers First! Only real customers’

transactions should be considered

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Typical data

Avoid costy tools Extracts of real data are

better, however They must be anonymized …tools could be necessary

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Reusable tools

Tools are an important issue in terms of costs

Optimize tool set Optimize training of

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Reusable Scenarios

Users’ experience is absolutely necessary

Listen customers Reusability submitted to

identical environment

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Tests platforms

Evaluate costs TDaaS would cut down

budget! However TDaaS means

also Management!

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Tests results

The test results must be capitalized and centralized on a site

Use Wiki in order to exchange and enrich testing knowledge!

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

There is no dichotomy!

While building up the Test Repository, think in terms of Lean Management

The Repository is the first step of Lean Management for testing

Lean Management applied to testing is a good start for

Apply Lean Management to testing

Lean Management for testing is applicable on the whole cycle (horizontal bar….)!

Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Why Lean?

Avoid over-testing Insure complete transfer of testing

information: Thru the complete life cycle Thru the different teams of the project

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

5S? OK

Seiri: Sort out

Seiton: Systemize

Seso: Cleaning/Shining Tools! Docs!

Seiketsu: Standardize

Shitsuke - Sustain/Discipline

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

DMAIC Testing

Define Measure Analyze Improve Control

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Six Sigma: Metrics ?

In terms of Test Cases Type of errors Type of solution

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Key indicators

Test program Daily follow-up Tests achievement

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Beyond technical tools: Communication tools forever!

Testing should be perennial! Capitalize Use forums, Wiki, etc. Update documents on Intranet Create or join Communities

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Don’t Forget CMMI!

If CMMI is requested: CMMI applies to the complete cycle Lean Management helps in

maintaining CMMI level and classification

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Don’t forget ITIL!

Important points: Production! ITIL Approach is necessary

Types of tests

Typical data

Re-usable tools

Re-usable scenarios

Tests platforms

Tests results

Tests procedures

Lean testing

Technology