Date post: | 11-Apr-2015 |
Category: |
Documents |
Upload: | venkateshr |
View: | 735 times |
Download: | 1 times |
GST
Metrics for Managing
CMM Based
Metric Workshop
GST
Metrics for Managing
• Metrics for Management– Metric program should be simple– Metric program should be vertically and
horizontally consistent– Metrics should be factual– Metrics serve as the catalyst for effective
management decisions at all organization levels
GST
CMM Review– Good Business Model– L2/Tracking to plan (compliance to spec)– L3 Tailoring to client needs (efficiency)– L4 Performance and Goals at project level– L5 Performance and Goals at organization level
Metrics for Managing
GST
Metrics for Managing
Measurement improves Visibility-Need for timeliness
-Need for accuracy
-Need for consistency
-Need for relevance
-Need for completeness
Measurement data sources are varied-quality reviews and audits; peer and technical reviews;
phased, management, and project reviews; testing
GST
Metrics for Managing
CMM ReviewStatus; used in each KPA (Why? What is
implication?)
Quality; part of most KPAs (Cost of Quality?)
Management Reviews; part of all KPAs
All KPAs are made up of key practicesExecution of practice requires resources. Resources
need to be controlled and managed.
GST
Metrics for Managing
• Simplicity of Measures:– Measures for managing activity– Measures for managing process quality– Measures for managing product quality– Measures for managing performance
GST
Metrics for Managing
•Vertical and Horizontal Consistency
GST
Metrics for Managing
Activity Activity
Developer Loop
Project Management Loop
Quality Management Loop
Organization Management Loop
Developer Loop
GST
Metrics for Managing
• Factual (accurate, complete, consistent)– Metrics are derived from factual data
• Test data
• Review and Audit data
• Accounting and Timekeeping data
• Project Status Data
• Fact-based metrics facilitate timely and management decisions.
GST
Metrics for Managing
• Metrics Act as a Catalyst– Provides management visibility– Eliminates guessing, removes subjectivity
from decision making process– Meetings are focused on facts– Consensus on status leads to consensus to
the actions to be taken
GST
Metrics for Managing
• Who needs metrics– Decision makers
– Analytical support activities like SEPG, Quality, Tools Group
– All levels of management
• Task Manager
• Project Leader
• Program Manager
• Group (Business Unit) Manager
• Organization Manager
GST
Metrics for Managing
• Basic Premise
“ If the metric does not support a management decision process, then it is not necessary and
should not be collected.”
GST
Metrics for Managing
• What Metrics should be collected?– Metrics that show the status of work activities in
the organization (Time, Effort, Current State, number of occurrances)
– Metrics that detail performance of the activities that are performed (actual vs plan)
– Metrics that detail the quality attributes of the products (error density, error rate, MTTR, MTBF, Reliability)
– Metrics that detail the quality attributes of the process (error density, error rate)
GST
Metrics for Managing
• Metrics that show status allow manage-ment to take effective actions to meet project requirements– Schedule status (effort of work items completed
less the effort of work scheduled to date) shows the effort that will need to be added to complete on schedule
– Effort status (effort of work items completed less the effort expended to date) shows whether there is overage or underage on budget
GST
Metrics for Managing
• Other status metrics– Number of changes (opened, received, closed, etc)
– Elements of system (requirements, design objects, units of software code, tests, etc) work/time
– Comparison of planned activities vs actual activities
• Value of these metrics is that they assist management in allocation of resources to achieve the project objectives
GST
Metrics for Managing
• Quality of product metrics– Number and type of defects found
– Distribution and value of defects in product
– Value of actions to remove and repair defects
• Value of product quality metrics is they:– Identify problems so that they can be fixed
– Provide insight into the product being developed
– Assist in identifying product quality issues
– Enable project to enhance overall product quality
GST
Metrics for Managing
• Quality of process metrics– Number and type of defects found
– Distribution and value of defects in process
– Value of actions to remove and repair defects
• Value of process quality metrics is, they:– Identify problems so that they can be fixed
– Provide insight into the process being utilized
– Assist in identifying process quality issues
– Enable organization to enhance overall process quality
GST
Metrics for Managing
• Executive Organization Metrics
Process Lifecycle Phases
Work Scheduled
Work Completed
Work Accomplished
Errors (value)
Error ratio
Moving Averages (3 Month Window)
New Projects
Active Projects
Project man/mth
GST
Metrics for Managing
• Executive Organization Metrics (continued)
Moving Averages (3 Month Window)
Total Staff
Managers
Project Pers
Support
Turnover
Moving Averages (3 Month Window)
Training Ratio
Systems Ratio
HR Ratio
GST
Metrics for Managing
• Executive Organization Metrics (continued)
Moving Averages (4 Week Window, Weekly)
Productivity
Delivery
Defect Rate
Removal Eff.
Defect Ratio
Moving Averages (3 Month Window)
Performance
Technology
GST
Metrics for Managing
• Project metrics ( all stated in effort)
DEFECT DATA (In effort lost)
Schedule Effort REQ HLD DD C&T INT Other Total
Norm/Plan
Task 1
Task 2
Task 3
Task 4
Task 5
Project Total
GST
Business Unit Head TotalBusiness Unit Head Total
Program Manager TotalProgram Manager Total
Project TotalProject Total
Metrics for Managing
Project Total
Program Manager Total
Business Unit Head Total
Organization Total
GST
Metrics for Managing
• Project level metrics– Status: schedule and effort, planned vs
actual– Quality: product and process, allocation of
defects/nonconformance to elements of the product being built and to process by lifecycle phase
– Performance: relationship between accomplishments, time, and quality
GST
Metrics for Managing
• Clarification of Schedule and Effort Metrics (Schedule only as example)– Schedule: Determine variance to schedule in
precise quantitative terms(Effort of items completed-Effort of scheduled items)= Variance
– Example: All effort allocated to tasks of a project scheduled to be complete after three weeks total 200 man hours (10 tasks of 20 hours each). The actual effort of tasks completed is 160 (only 8 tasks are complete)…value is -40 hours (variance)
GST
Metrics for Managing
• Defect Data Characteristics– Project, task, WBS element, product or process
element
– Activity Type, evaluator, sample size (if audit)
– Defect Data• Location in product (or facility)• Description• Defect Code• Priority• Effort (if manually collected)• Disposition • Status
GST
Metrics for Managing
• At this point, will all of the data and metrics, you are still a level 3 company– You have consistency of process and
consistency of metrics used across and vertically in organization
– Information helps projects plan better and assists in understanding processes and ensuring product quality
GST
Metrics for Managing
• Transition from L1 to L2/3 to L4/5– L1 Metrics are known at the end of a
project…no value to management– L2 Metrics used on the project plan vs
actual– L3 Metrics consistent across organization
and used by others– L4 Metrics used to Manage Projects– L5 Metrics used to Manage Organization
GST
Metrics for Managing
Earned Value Metrics
Budgeted Cost of work Performed (BCWP)
Actual Cost of work Performed (ACWP)
Budgeted Cost of work Scheduled (BCWS)
GST
Metrics for Managing
BCWP:Original or Updated Estimate
Time
% of Work or units of effort
(cost orEffort)
ACWP
Cost Variance
Schedule Variance
BCWS
GST
Metrics for Managing
Types of metrics
• Management– Performance, timeliness, status, efficiency,
profitability, return on investment, staff turnover, training days, growth rate, productivity, % of rework, etc.
• Quality– Error Rates, Cost of Quality, Number and value of
errors by type, seepage rate, distribution of errors by projects, by life-cycle phases, etc.
GST
Metrics for Managing
Management
Quality
Organization Sub-organization Project Activity
Each element has a unique set metrics that are important to
that organization element
GST
Metrics for Managing
Activity
ExecuteProcess
ExecutePlan
DefinePlan
Adjust Plan
MeasurePlan/Actual
DefineProcess
ControlProcess
MeasureProcess
Requirements Product
ImproveProcess
ProcessManagement
PerformanceManagement
GST
Metrics for Managing
Developing Metrics…a 4-Step Process
• Establish goals for measurements
• Define the attributes of the goals
• Describe metric measures
• Identify data required
GST
Metrics for Managing
• Goals Examples:– “We need to measure our productivity”– “We need to measure defect frequency”– “Philips needs to measure project
accomplishments”– “We need to measure the accuracy and
completeness of testing”
GST
Metrics for Managing
• Verification:
Establish how metric will be used to improve or manage performance or quality. Example: Productivity will be used to determine how projects are performing to their plan so that actions can be taken when significant deviations are noted.
GST
Metrics for Managing
• Attributes of the Goals: Productivity– What is existing productivity?– What is the expected productivity?– What is the improvement desired?– What was the productivity at the end of
each of the last 3 quarters?– What is our productivity trend?
GST
Metrics for Managing
• Metrics of Productivity
Effort Expended (Man-hours)
Product Units
Estimated Hours per Phase
Actual Hours per Phase
GST
Metrics for Managing
• Data needed for measures: Productivity– Units of work– Actual effort expended on work– Estimated effort of work– History of units of work produced and effort
consumed in production– Distribution of work by phase– Actual work distribution by phase
GST
Metrics for Managing
• Exercise One:– Groups of 5 persons grouped by function:
– Each group is assigned a level of management for metric definition
– Each group will identify at least 10 goals relative to the level of management of the group
– At least 6 goals should be “Management” goals and at least 4 should be “quality” goals
– Time is 1 hour: Then each group will present its goals to the class, 10 Minutes for each presentation
– Discussion by Class
GST
Metrics for Managing
• Exercise Two:– Each Group will verify each of its goals by
defining how the goal will be helpful in improving performance
– 20 Minutes for verification, then 5 minutes per group to explain verification to class
GST
Metrics for Managing
• Exercise Three:– Each Group will identify the attributes needed for so
that the Measurement Goals can be met. Each group will also define at least one metric method (Calculation) for each Measurement Goal.
– Each Group will take 45 minutes to complete; then they 10 minutes to report.
GST
Metrics for Managing
• Exercise Four– Each Group will identify the data elements
needed for each goal/attributes/measurement and determine if the data is currently available.
– 30 minutes. Then present need and availability.