of 23
8/14/2019 Process Improvement With Test Assessment Models
1/23
Proc ess Imp rovement WithTest Assessment Mod elsMa rwa n Zeined d ineTesting Pra c tic e Ma na g erLionb rid ge Tec hnolog ies
March 31 April 2, 2009
8/14/2019 Process Improvement With Test Assessment Models
2/23
8/14/2019 Process Improvement With Test Assessment Models
3/23
Why Should I Consid er TestProc ess Assessments?
Produce Higher Quality SoftwareRaise Awareness of Testing
Discover Strengths and WeaknessesIncrease Test Productivity
Boost Test EfficiencyInstitute Continuous Test ImprovementManagement DirectivesContractual Fulfillment Specific Projects
8/14/2019 Process Improvement With Test Assessment Models
4/23
Why Use a TestImp rovement Mod el?
Model-Based Test Process Improvement:
An improvement effort that uses a model to appraisethe quality of test processes, identify and prioritizeneeded changes, and guide its improvementactivities.
Global AcceptanceStructured Framework: Measurement and ScoringCertifications and AwardsIncreased Improvement Efficiency and PredictabilityGreater Industry Recognition: Customer Realization
8/14/2019 Process Improvement With Test Assessment Models
5/23
Risks of Using TestImp rovement Mod els
All models are wrong. Some are useful. George Box, Statistician
How wrong can a model be before it is useless?There is no silver bulletModel Evolution and Maintenance
8/14/2019 Process Improvement With Test Assessment Models
6/23
Test Imp rovement Mod elSelec tion Criteria
Should improve the test process
Must have a maturity structureShould have sufficient information
Must be adequately maintained
8/14/2019 Process Improvement With Test Assessment Models
7/23
8/14/2019 Process Improvement With Test Assessment Models
8/23
Comp a ring TMM/ TMMi to TPITMM TPI
Type Model Maturity Maturity
Year of development 1996 1997
Approach Conceptual Practical
Maturity Model Foundation Gelperin and Hetzel None
Number of Levels 5 13
Number of KPAs 13 20
Assessment Type Questionnaire ChecklistAssessment Elements Assessment Procedure,
Questionnaire, Training andTeam Selection criteria
Assessment Guideline,Checklist and Test MaturityMatrix
Assessment Foundation CMM, ISO, SPICE Practical Experience
Information about Model Articles, Dissertations andBook
Articles, tools and Book
Sources: Burnstein & Koomen/Pol
8/14/2019 Process Improvement With Test Assessment Models
9/23
Key Proc ess Area sTMM/ TMMi vs. TPITMM TPI
Test Planning Test StrategyTesting/Debugging Policies/Goals Life-cycle ModelBasic Testing Techniques/Methods Moment of InvolvementControlling and Monitoring Tests Estimating and PlanningIntegration of Test Test Specification Techniques
Test Training Static Test TechniquesTest Organization MetricsSoftware Quality Evaluation Test Automation (tools)Test Measurement Test EnvironmentReview Program Office EnvironmentTest Process Optimization Commitment and MotivationQuality Control Test Functions and TrainingDefect Prevention Scope of Methodology
CommunicationReportingDefect Management
Testware ManagementTest Process ManagementEvaluationLow Level Testing Sources: Burnstein & Koomen/Pol
8/14/2019 Process Improvement With Test Assessment Models
10/23
Assessing TMM Ma turityLevels
Source: I. Burnstein, Practical Software Testing
8/14/2019 Process Improvement With Test Assessment Models
11/23
Five Ma turity Levels of TMM
Source: I. Burnstein, Practical Software Testing
8/14/2019 Process Improvement With Test Assessment Models
12/23
Three Pha ses of Test Proc essAssessment
Phase 1o Plan Assessmento Prepare Questionnaire
o Review Documentationo Schedule Interviews
Phase 2o Conduct Interviews
Phase 3o Deliver Assessment Findingso Provide Improvement Recommendation
http://www.movieweb.com/features/pop.php?p=PHseTvwxGGnQww8/14/2019 Process Improvement With Test Assessment Models
13/23
Assessment Planning
Define Assessment Scope/Goals
Assemble Assessment Team
Collect Collaterals to be Reviewed
Notify Stakeholders to beInterviewed
Define Expected Deliverables
Publish Assessment Questionnaire
Distribute Assessment Schedule
Milestones Sample Deliverables
Confirm assessment objectives, collect collaterals and notify stakeholders
Assessment Project PlanPresentation to Stakeholders
Notification Email toStakeholders
Assessment Questionnaire
Interview Schedule
1
8/14/2019 Process Improvement With Test Assessment Models
14/23
AssessmentAna lysis & Interviews
Analyze test processes, practices & tools and conduct interviews
Review Received Collaterals
Review Questionnaire Answers
Solicit Missing or AlternativeCollaterals
Generate Additional Questions
Validate Presence vs. Awareness
Confirm Usage and Consistency
Milestones Sample Deliverables
Updated Assessment ProjectPlan
Updated Interview Schedule
Progress Reports
Interview Matrix
8/14/2019 Process Improvement With Test Assessment Models
15/23
Find ing s a nd Imp rovementRecommendations
Analyze additional collaterals and questionnaire answers and produce reports
Analyze Additional Collaterals
Review Interview QuestionnaireAnswers
Combine All Answers into ScoringSpreadsheet
Identify Strengths andWeaknesses
Provide Recommendations withComplexity and Priority Ratings
Milestones Sample Deliverables
Assessment Findings ReportImprovement RecommendationDetailed Report
Improvement Roadmap
Findings Presentation toStakeholders
Professional Services Proposal
8/14/2019 Process Improvement With Test Assessment Models
16/23
TMM Ma turity Results
8/14/2019 Process Improvement With Test Assessment Models
17/23
TMM Level 2: Pha se DefinitionGoa l 1 : Develop Testing a nd Deb ugg ing Goa ls a nd Polic iesNot Sa tisfied
8/14/2019 Process Improvement With Test Assessment Models
18/23
TMM Level 2: Pha se DefinitionGoa l 2 : Initiate a Test Planning Proc essSa tisfied
8/14/2019 Process Improvement With Test Assessment Models
19/23
TMM Level 2: Pha se DefinitionGoa l 3 : Ba sic Testing Tec hniq ues a nd Method s
Sa tisfied
8/14/2019 Process Improvement With Test Assessment Models
20/23
Imp rovement Roa d ma p
8/14/2019 Process Improvement With Test Assessment Models
21/23
Imp lementa tion Benefits
Organization wide committee orgroup on test planning
Test planning procedures, goals,and policies are developed,documented, distributed, andmaintained.
Technical training and basic tools
are availableStaffing and on-boarding of test andQA staff is managed and governed
Consistent high quality test planningand execution collaterals
8/14/2019 Process Improvement With Test Assessment Models
22/23
Imp rovement Benefits6-12 Months
Justifiable and Defendable StrategiesPrescriptive Guidelines and PoliciesConsistent Test Planning and ExecutionDefined Product Quality AttributesQuality Audit Guidelines
Monitoring and Control Policies.Fully Integrated Testing Activities into SDLCEstablished Training Protocols.
Consistent Quality Evaluation Metrics.Achieve TMM Maturity Level 3
8/14/2019 Process Improvement With Test Assessment Models
23/23
Tha nk you!
For m ore inform a tion:
Ma rwa n Zeined d inePra c tic e Ma na g er, Ma na ged Test Serives
Lionb rid ge Tec hnolog ies(704) 528-3730
Ma rwa n.zeined d ine@lionb rid ge.c omwww.lionbridge.com