Date post: | 15-Aug-2015 |
Category: |
Software |
Upload: | spin-chennai |
View: | 50 times |
Download: | 0 times |
1
Contents
Key AD / AM Metrics
Who are the Key personnel
Why Metrics
When and how to use Metrics
How to identify Data Quality issues
Some Enablers for Improvement
Effort Metrics Formula
Effort Variance ((Actual Effort – Estimated Effort) / Estimated Effort)*100
Load Factor (Actual Effort / Effort available))
% Review Effort (Total Effort expended on Reviews across all stages)/ (Actual Overall Project Effort) *100
% Cost of Quality (Effort spent on Prevention + Effort spent on Appraisal + Effort spent on Failure) / (Effort spent on Prevention + Effort spent on Appraisal + Effort spent on Failure + Effort spent on Production)) *100
Schedule Metrics Formula
Schedule Variation ((Actual End date – Planned End date) / (Planned End date - Planned Start date))*100
Duration Variation ((Actual End Date – Actual Start Date) – (Planned End Date – Planned Start Date)) / (Planned End Date – Planned Start Date) * 100
AD - Basic Process Metrics and Formula
Schedule Effort
AD - Basic Process Metrics and Formula – Cont.
Defect
Metrics Formula
Defect Removal Efficiency
(Total number of Pre-shipment Defects)/ (Total number of Pre-shipment Defects + Total number of post-shipment Defects + Total number of Post production Defects) *100
Defect Detection Efficiency
(Number of Pre-shipment defects / Appraisal Effort)
Defect Density by Effort
Total no of Defects Detected/Total overall actual effort spent.
Defect Leakage Sum((Number of defects attributed to a stage but only captured in subsequent stages) / (Total number of defects captured in that stage + Total Number of defects attributed to a stage but only captured in subsequent stages)) *100
Metrics Formula
Size Variation ((Actual Overall Size – Planned Overall Size) / (Planned Overall Size))* 100
Productivity Overall Productivity = (Overall Project Size) / (Total Effort for the Project)
Size
AVM - Basic Process Metrics, Definition and Formula
Metrics Formula
Acknowledging Severity 1….5 Incident
(No. of Sev 1/2/3/4/5 incidents acknowledged within the applicable Acknowledgement Time / Total No. of Sev 1/2/3/4/5 Incidents) * 100
Severity 1…..5 Incidents resolved within the allotted time
(Number of Sev 1/2/3/4/5 incidents resolved within the allotted time / number of Sev 1/2/3/4/5 incidents resolved) * 100.
Avoidable Problems / Unforced Errors for Severity level 1 and 2
(No. of incidents/problems caused by the Supplier's Actions / Total No. of incidents/problems) * 100
Metrics Formula
Function Points per $1K Spent
FP count / Total amount spend in ($1K )
Defect Injection Rate - Release or Project
(Total number of defects injected in the Release or Project / size of product)
% of SLAs met % of fixes without escalation
Other set of Product Quality Metrics
Metric / UOM
Formula Operational Definition Tools Usage
.Net Java
Code Review Coverage
Number of impacted programs reviewed / Total number of programs * 100
A measure of the review coverage on the number of programs This is higher the better metric
VSTS-Code Analysis, FxCop, Sonar
SONAR, JCAP, PMD, Checkstyle, FindBugs
Unit Test Coverage
Based on Unit Test Coverage Tools such as Junits/JCoverage
The Code coverage metric identifies the sections of the source code that were either tested/ not tested as part of white box testing This is higher the better metric
NUnit, VSTS- Unit Testing NCover
Junit, Test NG, Code pro analytix, Cobertura, EMMA
Code Quality - Cyclomatic complexity
Cyclomatic Complexity at class level (Highest method CC)
This metric estimates the complexity of the individual functions, modules, methods or classes within a program so as to measure the program's structural complexity. Lower the better metric
IDE,Sonar, VSTS-Code Analysis, FxCop
SONAR, JCAP, PMD, Checkstyle, FindBugs
Requirements to Test Case coverage
% of requirements linked to Test cases
An indication of how extensively the requirements are covered by Test Cases This is higher the better metric
VSTS- Unit Testing, NCover
Junit, Test NG, Code pro analytix, Cobertura, EMMA
Metrics – Testing
Project
Metrics Intent Definition
Reporting
Frequency
Test
Effectiveness
Indicates ability to unearth and fix defects
before they reach UAT and Production
(Number of accepted defects in SIT / (Number of accepted
defects in SIT + UAT + Post UAT)) * 100 Monthly
Test Design
coverage
How much requirements are covered by
test cases ?
(Total number of baselined testable requirements mapped to
test cases / Total number of baselined testable
requirements)*100
Monthly
Test Case
Preparation
Productivity
Test case creation productivity of the team ((No of Test Cases or Test Case points (TCP) prepared)/ (Effort
spent for Test Case Preparation) Monthly
Test Case
Execution
Productivity
Indicates test execution productivity of
the team
((No of Test Cases or TCP Executed)/ (Effort spent for Test
Execution) Monthly
9
Metrics Work Flow
Developer / TL enters/updates
the data
PL Reviews the data
PM Approves the Metrics
Metrics review by DD / DM
Some proactive approaches for reviewing the Metrics data
Metrics Submission date to be done by end of every month.
Review of Metrics by PM/Delivery Manger to be completed subsequently.
Monthly Metrics Review scheduled with Delivery Director by 1st week of subsequent month for critical projects.
12
Customer’s Expectations (i.e. Why Metrics)
Improved Business Value & New Revenue Generation
On time delivery & Improvement in Time to Market
Zero defects
High Quality Business / Technology Solutions
Reduction in IT Operating Cost
Am I getting more work for lesser $ spent over a period
13
Senior Management Expectations (i.e Why Metrics)
Are we fixing more Tickets over a period
Are we building more LoC over a period
Are we delivering Zero Defects Software
Are we making expected Profitability
Are we adopting Best Practices and Reusable
Are we getting accurate Productivity while submitting RFPs
Are we getting repeat business from this engagement
14
Where are we - Now
Follow-up on Project planning & tracking
Data Quality Issues No Data porting from external tool
Expending our energy in process compliance Follow up
mailers for Metrics data submission
Project Health Scorecards
JUST THINK
Are these parameters helping you to meet
Customer Expectations ?
Senior Management expectations ??
Delivery Managers Expectations ???
Are we using these data in a true sense for the success of the project ????
15
Metrics Based Project Management – Work flow
Process Performance Objective ; Process Performance models
Benchmarking
Guidelines for Metrics and QPM, Statistical Techniques
Data Validation, Analysis and reporting ; Facilitation for usage of statistical tools like control charts
Facilitation
Data Trend analysis and validation of trends using hypothesis testing
Metrics Based Project Management
Scenario - 1
Effort Variation Effort Over Run
Schedule Variation On Schedule
Defects (Internal)
Defects (Customer)
Inference • Is there a problem in Estimation • Is the team over burdened? • Is the team possess right skills to carry out the tasks in a stipulated time period? Or takes longer time to find a solution & fix • Is there any scope scale down?
Bring back to Track • Revisit the Estimation • Additional Trainings to Team
Scenario - 2
Effort Variation Effort Over Run
Schedule Variation Schedule Over Run
Defects (Internal)
Defects (Customer)
Inference • Is there a problem in Estimation • Is the team over burdened? • Is the team possess right skills to carry out the tasks in a stipulated time period? Or takes longer time to find a solution & fix • Is there any scope creep ? • Are requirements changed frequently (stability of requirement is low)
Bring back to Track • Revisit the Estimation • Additional Trainings to Team • Reach out to customer, if Scope creep observed, Requirements changed
Data Quality Issues- What is wrong here
A Project with a –ve Effort variation cannot have a +ve Schedule overrun and a high LF
Data Quality Issues- What is wrong here
Failure Cost is zero in spite of Defects recorded and having a very High Appraisal cost. Production Cost shows zero