Ch6 performinng to asessment

transcript

SE423 SPICH6-Performing Assessment & ISO/IEC15504

Kittitouch Suteeca

Outline The Process Assessment Process Process Assessment Model Process Assessment Model Activities in Process Assessment ISO 15504 The Measurement Framework Roles and responsibilities

Performing an Assessment The requirements for performing an assessment defined in

ISO/IEC 15504 aim at achieving a greater degree of uniformity in the approach to process assessment, so as to maximize the reliability of different approaches and provide a degree of comparability between the results of different assessments.

It may make sense to verify the requirements prior to and during the course of the assessment so that corrective actions can occur.

The Process Assessment Process

ROLES AND RESPONSIBILITIES

SponsorCompetent Assessor

Assessors

ASSESSMENT PROCESS

PlanningData CollectionData Validation

Process Attribute RatingReporting

INPUTSponsor identity

PurposeScope

ConstraintsAssessment

OUTPUTIdentification of Evidence

Process UsedProcess Profiles

PROCESS REFERENCE MODEL

Domain and ScopeProcess Purpose

Process Outcomes

PROCESS ASSESSMENT

MODELScope

IndicatorsMapping

Translation

MEASUREMENT FRAMEWORK

Capability LevelsProcess Attributes

Rating Scale

PRM:Process Reference Model The specific requirements for a candidate

process reference model A Process Reference Model shall contain:

A declaration of the domain of the PRM. Descriptions of the process within the scope of the

PRM., in term of process outcome and purpose. a description of the relationship between the

Process Reference Model and its intended context of use.

Example of PRM. ISO 9000, 12207, 29110, CMMi

Process Assessment Model

PAM:Process Assessment Model A Process Assessment Model forms the basis for the

collection of evidence and rating of process capability.

Any Process Assessment Model is related to one or more Process Reference Models.

A Process Assessment Model shall contain: a definition of its purpose, scope, elements and

indicators; its mapping to the Measurement Framework and the

specified Process Reference Model(s); a mechanism for consistent expression of results.

Assessors

ASSESSMENT PROCESS

PurposeScope

Process Outcomes

PROCESS ASSESSMENT

MODELScope

IndicatorsMapping

Translation

Rating Scale

Activities in Process Assessment

PlanningData collection

DataValidation

Rating Reporting

1.Planning

A plan for the assessment shall be developed and documented, including at minimum:

1) the required inputs defined in this part of ISO/IEC

15504(PAM.); 2) the activities to be performed in conducting the

assessment; 3) the resources and schedule assigned to these activities; 4) the identity and defined responsibilities of the

participants in the assessment; 5) the criteria to verify that the requirements of this

International Standard have been met; 6) a description of the planned assessment outputs.

Assessment input

Identify of the sponsor of the assessment;

Assessment purpose. Assessment scope. Assessment approach. Assessment constraints. The model(s) used within the

assessment.

Assessment input (cont’)

The role and identify of the assessment team, including the competent assessor.

The criteria of competent of the assessor who is responsible for the assessment.

The identify of assesse and support staff.

Any additional information to be collected during the assessment.

DataValidation

Rating Reporting

2.Data collection

Data required for evaluating the processes within the scope of the assessment and additional information shall be collected in a systematic manner, applying at minimum the following:

1) the strategy and techniques for the selection, collection, analysis of data and justification of the ratings shall be explicitly identified and shall be demonstrable

2.Data collection

2) correspondence shall be established between the organizational unit’s processes, specified in the assessment scope, and the elements in the Process Assessment Model;

3) each process identified in the assessment scope shall be assessed on the basis of objective evidence;

2.Data collection4) the objective evidence gathered for

each attribute for each process assessed shall be sufficient to meet the assessment purpose and scope;

5) the identification of the objective evidence gathered shall be recorded and maintained to provide the basis for verification of the ratings.

DataValidation

Rating Reporting

3.Data validation

The data collected shall be validated to:

1) confirm that the evidence collected is objective;

2) ensure that the objective evidence is sufficient and representative to cover the scope and purpose of the assessment;

3) ensure that the data as a whole is consistent.

DataValidation

Rating Reporting

4.Process attribute rating A rating shall be assigned based on validated

data for each process attribute.1) the set of process attribute ratings shall be

recorded as the process profile for the defined organizational unit;

2) during the assessment, the defined set of assessment indicators in the Process Assessment Model shall be used to support the assessor's judgment in rating process attributes in order to provide the basis for repeatability across assessments;

4.Process attribute rating 3) the decision-making process that is used to

derive rating judgments shall be recorded; 4) traceability shall be maintained between an

attribute rating and the objective evidence used in determining that rating;

5) for each process attribute rated, the relationship between the indicators and the objective evidence shall be recorded.

ISO 15504 The Measurement Framework

Level 1 PerformedPA.1.1 Process Performance

Level 2 ManagedPA.2.1 Performance ManagementPA.2.2 Work Product Management

Level 3 EstablishedPA.3.1 Process DefinitionPA.3.2 Process Deployment

Level 4 PredictablePA.4.1 Process MeasurementPA.4.2 Process Control

Level 5 OptimizingPA.5.1 Process InnovationPA.5.2 Process Optimisation

Level 0 IncompleteLevel 0 Incomplete IncompleteThe process is not implemented or fails to achieve its purpose

PerformedThe process is implemented and achieves its process purpose

ManagedThe process is managed and work products are established, controlled and maintained.

PredictableThe process is enacted consistently within defined limits

OptimizingThe process is continuously improved to meet relevant current and projected business goals

EstablishedA defined process is used based on a standard process.

Level 0: Incomplete process The process is not implemented, or fails to

achieve its process purpose. At this level there is little or no evidence of any

systematic achievement of the process purpose.

The Incomplete process is one that is either not performed at all, or for which there is little or no evidence of systematic achievement of the process purpose. Systematic achievement is characterized by the routine performance of necessary actions and the presence of appropriate input and output work products which, collectively, ensure that the process purpose is achieved.

Level 1: Performed process The implemented process achieves its

process purpose. The following attribute of the process

demonstrates the achievement of this level:

The Performed process achieves its process purpose through the performance of necessary actions and the presence of appropriate input and output work products which, collectively, ensure that the process purpose is achieved.

Presenting the ResultsProcess Profiles and

Capability Level Rating

Rating process attributes ISO/IEC 15504 provide the basic

elements of a measurement framework for rating process capability. To complete the framework, a scale for rating the extent of achievement of a process attribute is defined.

Rating process attributes Process attribute rating values: The ordinal

rating scale defined below shall be used to express the levels of achievement of the process attributes. N Not achieved: There is little or no evidence

of achievement of the defined attribute in the assessed process.

P Partially achieved: There is some evidence of an approach to, and some achievement of, the defined attribute in the assessed process. Some aspects of achievement of the attribute may be unpredictable.

Rating process attributes L Largely achieved: There is evidence of

a systematic approach to, and significant achievement of, the defined attribute in the assessed process. Some weakness related to this attribute may exist in the assessed process.

F Fully achieved: There is evidence of a complete and systematic approach to, and full achievement of, the defined attribute in the assessed process. No significant weaknesses related to this attribute exist in the assessed process.

Rating process attributes The corresponding values shall be:

N Not achieved 0 to 15% achievement P Partially achieved >15% to 50%

achievement L Largely achieved >50% to 85%

achievement F Fully achieved >85% to 100%

achievement.

DataValidation

Rating Reporting

5.Reporting The assessment results, including at

minimum the outputs specified PAM., shall be documented and reported to the assessment sponsor or to their delegated representative.

Assessment FeedbackShall contain Introduction of organization context. Assessment Scope Rating Results Process Related Risk Strengths Weaknesses Potential Improvement Actions

Objective No.

O.1 O.2 O.3 O.4 O.5 O.6 O.7

PM. N L P P F P P

SI. L L N F F P P

Attribute

N Not Achieved

P Partially Achieved

L Largely Achieved

F Fully Achieved

Rating Result

Assessors

ASSESSMENT PROCESS

PurposeScope

Process Outcomes

PROCESS ASSESSMENT

MODELScope

IndicatorsMapping

Translation

Rating Scale

Roles and responsibilities

Responsibilities of the Sponsora) verify that the individual who is to take

responsibility for conformity of the assessment is a competent assessor;

b) ensure that resources are made available to conduct the assessment;

c) ensure that the assessment team has access to the relevant resources.

Roles and responsibilities Responsibilities of the Competent Assessora) confirm the sponsor's commitment to

proceed with the assessment;

b) ensure that the assessment is conducted in accordance with the requirements of this part of ISO/IEC 15504;

c) ensure that participants in the assessment are briefed on the purpose, scope and approach of the assessment;

Responsibilities of the Competent Assessor

d) ensure that all members of the assessment team have knowledge and skills appropriate to their roles;

e) ensure that all members of the assessment team have access to appropriate documented guidance on how to perform the defined assessment activities;

Roles and responsibilities Responsibilities of the Assessors a) carry out assigned activities associated

with the assessment, e.g. detailed planning, data collection, data validation and reporting;

b) rate the process attributes.

References ISO/IEC 15504-3 manual

Ch6 performinng to asessment

Technology