Date post: | 05-Apr-2018 |
Category: |
Documents |
Upload: | changsoo-lee |
View: | 215 times |
Download: | 0 times |
of 32
7/31/2019 2012 q1 Reid Mette
1/32
ISO/IEC 29119The New International
Software Testing Standard
Stuart Reid
Testing Solutions Group
117 Houndsditch
London EC3
UKTel: 0207 469 1500Fax: 0207 623 8459
www.testing-solutions.comStuart Reid, 2012
http://www.testing-solutions.com/http://www.testing-solutions.com/http://www.testing-solutions.com/http://www.testing-solutions.com/7/31/2019 2012 q1 Reid Mette
2/32
Scope
The purpose of standards
Overview of ISO 29119
Applicability
Recent developments
Timeline
Future work
7/31/2019 2012 q1 Reid Mette
3/32
What are standards?
Guidelines documents as they are not compulsoryunless mandated by an individual or anorganization
Agreements because they often reflect a certain
level of consensus
Guidelinedocumentation that reflectsagreementsonproducts,practices, oroperationsbynationally orinternationally
recognizedindustrial, professional, tradeassociations or governmentalbodies
- ISO
7/31/2019 2012 q1 Reid Mette
4/32
Pre-standardization?
7/31/2019 2012 q1 Reid Mette
5/32
Why use standards?
Consumers
Confidence in compliant products
Authors provide expertise in standards
Manufacturers
Conformance and Marketing
Safety from liability
Guidelines on production
But not Best Practice
7/31/2019 2012 q1 Reid Mette
6/32
Quality and Standards
7/31/2019 2012 q1 Reid Mette
7/32
What use are standards?
Basis for:
Communication common terminology
Professional qualifications
Certification/compliance schemes Benchmark of good industry practice
Contracts
Interoperability and consistency
Standards describe a current body ofknowledge that provides the basis for aprofessional discipline
7/31/2019 2012 q1 Reid Mette
8/32
A Case in Point
7/31/2019 2012 q1 Reid Mette
9/32
Standardization Bodies
http://www.brandsoftheworld.com/download/brand/61142.htmlhttp://www.sis.se/Default.aspxhttp://www.bsi-global.com/British_Standards/index.xalterhttp://www.omg.org/http://www.iec.ch/7/31/2019 2012 q1 Reid Mette
10/32
ISO/IEC Software Standards
0
20
40
60
80
100
120
140
1
987
1
988
1
989
1
990
1
991
1
992
1
993
1
994
1
995
1
996
1
997
1
998
1
999
2
000
2
001
2
002
2
003
2
004
2
005
2
006
2
007
2
008
2
009
2
010
Published
Maintained
7/31/2019 2012 q1 Reid Mette
11/32
Motivation for ISO 29119
Conflicts in definitions, processes & procedures
plethora of standards to be replaced by one
e.g. IEEE 829, IEEE 1008, BS 7925-1/-2, IEEE 1028
practitioners do not know which standard to follow Lacking in current standards:
Organizational Testing not covered
e.g. Test Policy and Organizational Test Strategy
no Project Test Management BS 7925 only covers unit testing
common functional techniques missing
poor coverage of non-functional testing
7/31/2019 2012 q1 Reid Mette
12/32
ISO/IEC SC7 WG26
ISO
TC176
Quality
JTC1
InformationTechnology
SC7
Software &Systems
Engineering
SWG1 ..................
WG26
SoftwareTesting
Representativesof National
Standards Bodies
SC22
Programming
Languages
SC27
IT Security
SC32
DataManagement &
Interchange
IEC
TC56
Dependability
7/31/2019 2012 q1 Reid Mette
13/32
BS 7925-1
BS 7925-2 IEEE 829
Concepts & Vocabulary
Part 1
ISO/IEC 29119 Structure
Process
Assessment
Part 5
TestingTechniques
Part 4
Documentation
Part 3Part 2
Processes
Actually ISO/IEC 33063
7/31/2019 2012 q1 Reid Mette
14/32
SOFTWARE TESTING CONCEPTS
Scope, Conformance, Normative References
TESTING IN DIFFERENT LIFE CYCLE MODELS
ROLES AND RESPONSIBILITIES IN TESTING
ANNEXES Metrics, Examples, Bibliography
DEFINITIONS
Part 1: Concepts & Vocabulary
7/31/2019 2012 q1 Reid Mette
15/32
Part 2: Testing Processes
TEST MANAGEMENT PROCESSES
ORGANIZATIONAL TEST PROCESS
DYNAMIC TEST PROCESSES
7/31/2019 2012 q1 Reid Mette
16/32
TEST MANAGEMENT PROCESSES
ORGANIZATIONAL TEST PROCESS
DYNAMIC TEST PROCESSES
Instantiating Testing Processes
7/31/2019 2012 q1 Reid Mette
17/32
ORGANIZATIONAL TEST PROCESS
TEST MANAGEMENT PROCESSES
TEST
PLANNING
TEST
MONITORING &
CONTROL
TEST
COMPLETION
ORGANIZATIONALTEST
DOCUMENTATION
FEEDBACK ONORGANIZATIONAL TESTDOCUMENTATION
TEST PLAN UPDATES
TESTPLAN
TESTCOMPLETIONREPORT
DYNAMIC TEST
PROCESSES
TEST
MANAGEMENT
PROCESSES
TEST PLAN,TEST COMPLETIONREPORT,TEST MEASURES
TESTMEASURES
TEST PLAN,CONTROLDIRECTIVES
TEST PLAN,CONTROLDIRECTIVES
Test Management Processes
7/31/2019 2012 q1 Reid Mette
18/32
OrganiseTest Plan
Development
Identify &Estimate Risks
Design TestStrategy
DetermineStaffing and
SchedulingDocumentTest Plan
Schedule,Staffing Profile
TestStrategy
AnalyzedRisks
Scope
Identify RiskTreatmentApproaches
GainConsensuson Test Plan
ApprovedTest Plan
DraftTest Plan
TestPlan
PublishTest Plan
UnderstandContext
TreatmentApproaches
Test Planning Process
7/31/2019 2012 q1 Reid Mette
19/32
Test Monitoring &Control Process
Monitor
Report
measures
ControlSet-Up
[testingcomplete]
[testing incomplete]
controldirectives
testprogress
info
test progressinfo
testplan
testmeasures
test status report
testcontrolinfo
...TEST PROCESSES...
Dynamic/Management
7/31/2019 2012 q1 Reid Mette
20/32
DYNAMIC TEST PROCESSES
TESTSPECIFICATION
[NO ISSUESNOTICED]
TESTENVIRONMENTREADINESS
REPORT
TESTENVT
REQUIREMENTS
INCIDENTREPORT
TEST DESIGN &
IMPLEMENTATION
TEST
ENVIRONMENT
SET-UP
TEST
EXECUTION
TEST
INCIDENT
REPORTING
(LEVEL) TEST MANAGEMENT PROCESS
[ISSUE NOTICED or
RETEST RESULT]
TESTRESULTS
CONTROLDIRECTIVES TEST
MEASURES
(LEVEL)TEST PLAN
Dynamic Test Processes
7/31/2019 2012 q1 Reid Mette
21/32
Test Process Descriptions Example Organizational Test Process
Each testing process is described using a standard template (followingISO 24774), for example:
Purpose
The purpose of the Organizational Test Process is to develop and maintain organizational test specifications,such as the Test Policy and Organizational Test Strategy.
Outcomes
As a result of the successful implementation of the Organizational Test Process:
The organizational test specification is developed based on stakeholder requirements;
The organizational test specification is agreed by stakeholders;
The organizational test specification is published and accessible to stakeholders;
Conformance of stakeholders with the organizational test specification is managed;
The organizational test specification is reviewed on a periodic basis;
Minor updates to the organizational test specification are made as necessary.
Activities and tasks
The tester shall implement the following activities and tasks in accordance with applicable organization policiesand procedures with respect to the Organizational Test Process.
Develop Test Specification
This activity consists of the following tasks:
Analyze any relevant source documents and the current testing practices within the organization toidentify requirements for the organizational test specification.
etc.
Gain Consensus on Test Specification
etc.
Information items
The output of the Organizational Test Process is the organizational test specification. The Organizational TestPolicy and Organizational Test Strategy are typical examples of organizational test specifications.
Note that ISO 24774 (and this format) has changed since we started, sowe have had to rewrite all the process descriptions
7/31/2019 2012 q1 Reid Mette
22/32
Part 3 Test Documentation
TEST DOCUMENTATION
ANNEXES - EXAMPLES
Scope, Conformance,Normative References
7/31/2019 2012 q1 Reid Mette
23/32
Part 3: Test Documentation
Organizational test documentation Test policy
Test strategy
Project test documentation
Project test plan
Test project completion report Test Level documentation
Test plan
Test specification
Test results
Anomaly reports Level test status report
Test environment report
Test level completion report
Appendices
examples of documents at each level of testing
7/31/2019 2012 q1 Reid Mette
24/32
Part 4 Test Techniques
TEST COVERAGE MEASUREMENT
Scope, Conformance, Normative References
ANNEXE
TESTING OF QUALITY CHARACTERISTICS
ANNEXE SELECTION OF TECHNIQUES
ANNEXE TEST TECHNIQUE EFFECTIVENESS
TEST DESIGN TECHNIQUES
BLACK BOX WHITE BOX
7/31/2019 2012 q1 Reid Mette
25/32
Applicability
Intended to be a generic standard support all domains
Some domains are more standards-oriented, e.g.
safety-related
telecoms
Some domains will become more regulated, e.g.
financial banks, stock markets, etc.
Needs trialling in all domains and project types
Applicability to sequential, iterative and agile Applicability to new and maintenance projects
Applicability to IS and embedded
E.g. when wouldnt you be selecting and usingtechniques? When wouldnt you plan?
h
7/31/2019 2012 q1 Reid Mette
26/32
When can you start using itand when can you get involved?
Typical ISO standards take over 7 years
ISO 12207 was conceived in 1988 andpublished in 1995 and represents 17,000
person hours (8.5 person years)
BS 7925-1 & -2 took 8 years to develop
IEEE estimates 2-4 years to develop astandard, at a cost of between $2,000 and$10,000 per page [1998]
7/31/2019 2012 q1 Reid Mette
27/32
May
10
May
11
May
12
May
13
Timelines all going to plan!
Working Draft (WD)Committee Draft (CD)Draft International Standard (DIS)Final Draft International Standard (FDIS)Final International Standard (FIS)
Parts 2 & 3
Parts 1 & 4
WDCD1
DIS FDIS
FIS
WD
CD1
DISFDIS
FIS
CD2CD3
CD2CD3
7/31/2019 2012 q1 Reid Mette
28/32
Do you want to be involved?
Join the working group
representing your national standards body
6 day meetings, every 6 months
contribute between meetings
Join a mirror group
for your national standards body
Contribute materials
Review drafts
7/31/2019 2012 q1 Reid Mette
29/32
Current status
Drafts are out for review Unsurprisingly (for a testing standard) we get
plenty of comments (1000s)
Countries attending meetings include:
Denmark, India, China, UK, South Africa, HongKong, Spain, Germany, USA, Finland,Australia, Korea, Canada, France, Poland,Japan, Russian Federation, Columbia, Sweden,
Brazil, Malaysia, New Zealand, Argentina,India, Peru, Colombia, Turkey
Last meeting Mumbai, Nov 2011
Next meeting Korea, May 2012
7/31/2019 2012 q1 Reid Mette
30/32
Conclusions
International standard will provide practitionerswith guidelines for testing that cover all aspectsof the life cycle
Provides a consistent set of definitions,
processes, procedures & techniques forsoftware testing
Will be adopted by IEEE, BSI, ISO and othernational standards bodies
Currently has representation from 18 nations &is being reviewed by software testingprofessionals world-wide
We still need to ensure widespread applicability
7/31/2019 2012 q1 Reid Mette
31/32
Any Questions?
Any Volunteers?
7/31/2019 2012 q1 Reid Mette
32/32
For more information
if interested in trialling the standard on aproject, reviewing drafts or writing examples
http://softwaretestingstandard.org/
WG26 website
http://www.jtc1-sc7.org/
access to official documents released by WG 26
mailto:[email protected]://www.jtc1-sc7.org/http://www.jtc1-sc7.org/http://www.jtc1-sc7.org/http://www.jtc1-sc7.org/http://www.jtc1-sc7.org/http://www.jtc1-sc7.org/http://www.jtc1-sc7.org/http://www.jtc1-sc7.org/http://www.jtc1-sc7.org/mailto:[email protected]:[email protected]:[email protected]