Date post: | 22-Nov-2014 |
Category: |
Technology |
Upload: | richa-goel |
View: | 943 times |
Download: | 1 times |
Quality in Software IndustryRicha Goel
1
Contents
•Quality : Meaning•Quality in Software Industry•Implementation of Quality•Measuring Quality•Various Types of Metrics•Case Studies•Software Quality Assurance
2
What is quality?
•A product should meet its specification.•This is problematical for software systems
▫Some quality requirements are difficult to specify in an unambiguous way;
▫Software specifications are usually incomplete and often inconsistent.
3
4
Quality is…..
Invisible when GOOD Impossible to ignore when BAD
•Meeting Our Customer’s Requirements
•Doing Things Right the First Time; Freedom from Failure (Defects)
•Consistency (Reduction in Variation)
•Continuous Improvement
•Quality in Everything We Do
Total Quality Is…
5
Quality of a product is satisfied customer
6
5
Software Quality ???
•Software Quality refers to any measurable characteristics such as correctness, maintainability, portability, testability, usability, reliability, efficiency, integrity, reusability and interoperability.
7
Software quality management
•Concerned with ensuring that the required level of quality is achieved in a software product.
•Involves defining appropriate quality standards and procedures and ensuring that these are followed.
•Should aim to develop a ‘quality culture’ where quality is seen as everyone’s responsibility.
8
9
Absence of Quality…
Definition: Software Quality
•What is software quality?•What are the attributes of quality for
software?
?This is high quality software because...
10
Software Quality AttributesPortability
Reliability
Efficiency
Usability
Testability
Understandability
Modifiability
11
Common problems in software processes•Cost overruns•Schedule delays•Low productivity rate•Poor quality - in software, maintenance or
fixes
12
6
Quality Concepts Quality of Design refers to the characteristics
that designer’s specify for an item.
Quality Control is the series of inspections, reviews and tests used throughout the development cycle to ensure that each work product meets the requirements placed upon it.
Quality of Conformance is the degree to which the design specifications are followed during manufacturing.
13
Quality Concepts
• Quality of Design
Design
14
Software Quality MeasurementsWe best manage what we can measure.
15
Quality Measurement• Measurement enables the Organization to
improve the software process; assist in planning, tracking and controlling the software project and assess the quality of the software thus produced.
• Metrics are analyzed and they provide a dashboard to the management on the overall health of the process, project and product.
• The validation of the metrics is a continuous process spanning multiple projects.
16
What to measure?
Why to measure?
How to measure?
17
What to check?
Should we track:•Number of tests?•Pass rate?•Plan versus actual?•Number of defects?•Code coverage?•Functional coverage?•Performance figures?
18
What to measure?
Resource
Defects
Code Process
19
Metric Classification•Products
▫Explicit results of software development activities
▫Deliverables, documentation, by products
•Processes▫Activities related to production of software
•Resources▫Inputs into the software development activities▫hardware, knowledge, people
20
Types of Software Metrics
•Product metrics – e.g., size, complexity, design features, performance, quality level
•Process metrics – e.g., effectiveness of defect removal, response time of the fix process
•Project metrics – e.g., number of software developers, cost, schedule, productivity
21
Why measure? Are we meeting our business objectives? Are our customers satisfied with our products and
services? Are we earning a fair return on our investments? Can we reduce the cost of producing the product or
service? How can we improve the response to our customers’
needs or increase the functionality of our products? How can we improve our competitive position? Are we achieving the growth required for survival?
Without the right information, you are just another person with an opinion
22
Why Measure Software?
•Determine the quality of the current product or process
•Predict qualities of a product/process
•Improve quality of a product/process
23
Why measure?
•By analyzing the metrics the organization can take corrective action to fix those areas in the process, project or product which are the cause of the software defects.
24
How to measure?
25
Purpose of Measuring Cost saving Employee satisfaction Customer satisfaction Quality
26
Purpose of Measuring• Cost saving
▫Quality of Product: Defects per KLOC or FP▫Project Status: Tracking against estimated
schedule, budget, size• Employee satisfaction
▫Work Effort: Each team member utilized• Customer satisfaction
▫Satisfaction: Survey (Six Months)• Quality
▫Quality of Process: COQ, ROI on QA, amount of rework, quality of team time and teamwork
27
Goal of metrics
•to improve product quality and development-team productivity
•concerned with productivity and quality measures· measures of SW development output as
function of effort and time· measures of usability
28
Help in…
•Allows manager to (1) assess status of ongoing project(2) track project risks(3) uncover problem areas(4) adjust tasks or workflow(5) evaluate team’s ability to control quality
29
Define Metrics toBe collected –
Project & Support Group Metrics
Store data forFuture useIn MetricsDatabase
Establish DataCollection Mechanism
And send the metrics data to QA Group
Arrive at Organizational Process Capability Baseline
Analyze Metrics
30
Metrics Data Capture – Frequency & Sources What should be the frequency of collecting the data?
Sources for the data capture of the Metrics
Timesheets Project plan, Schedule (MPP, XLS) Defect tracking system Ticket tracking sheet
31
Goal 1: Improve software project planning How good is the software
effort planning?
How good is the software scheduling?
32
Effort Variance MetricEffort Variance (in %)=[Actual Effort – Planned(Estimated) Effort]*100 [Planned ( Estimated ) Effort ]
Actual Effort (in hrs) = Planning, Tracking, Configuration, Defect Prevention, Requirements, Design, Coding, Review, Rework & Testing
Planned Effort (in hrs) = Planning, Tracking, Configuration, Defect Prevention, Requirements, Design, Coding, Review, Rework & Testing
This is to measure the variance of effort compared with the estimated effort in terms of man days spend with respect to daily hrs
33
Schedule Variance MetricSchedule Variance =[Actual Duration - Planned Duration] *
100 (in %) [ Planned Duration ]
Actual Duration (in Days) : Actual end date - Planned start date
Planned Duration (in Days) : Planned end date - Planned start date
This is to measure the variance of schedule compared with the estimated
schedule in terms of duration in calendar days
34
PhaseRequirementActual
Schedule Planned Schedule
Design Actual Schedule
Planned Schedule
Code/Unit Test Planned Schedule
Actual Schedule
Requirement
Design
Coding
Testing
Delivery / Release
Schedule Metrics - SampleProject Code and Name:
Prepared By : Reviewed By (MR) : Sign Date : Data Given by : Sign/Date :
35
Goal 2 : Improve ProductivityWhat is the productivity in different types of projects?
36
Productivity MetricsDevelopment = ____Total project effort ____
Project size in LOC or FP
Testing = Total effort spent in execution of test cases Total number of test cases executed
Maintenance = Cumulative Actual Effort for a category of CRs completed
Number of CRs completed in the category
37
Goal 3 : Reduce number of defects How many defects are we getting
with respect to the effort spent?
How efficient and Effective are our reviews?
38
Defect Density MetricA "Defect" is a:
Deviation from a standard Deviation from requirement Anything that causes customer dissatisfaction
Defect Density (DD) = Total no. of defects
Total effort (In hrs)
Total no. of defects = Pre Delivery Defects ( reviews & testing ) + Post Delivery Defects ( reported after delivery to customer )
Total Effort ( in hrs ) = Planning, Tracking, Configuration Management, Defect Prevention, Requirements, Design, Coding, Review, Rework & Testing
39
Review Efficiency MetricReview efficiency = Review defects (including unit testing
defects)*100 Review effort ( including unit testing
effort )
Review Defects = No. of defects in ( Requirements, Design, CUT )
Review Effort (in hrs) = ( Requirements, Design, CUT ) Review Efforts
SRS review defects = 1 Design Review defects = 1 Code review defects = 1 Unit Testing defects = 0 Review Effort = 15
Review Efficiency will be 0.2
40
Review Effectiveness Metric Review effectiveness = Total no. of Review defects *
100 Total no. of defects
( Review + Testing )
DEFECT DATA
SRS review defects = 2 Design Review defects = 2 Code review defects = 4 Testing defects = 2 Review effectiveness will be
80%
41
Goal 4 : Reduce Cost of Quality What is the cost of detection (Appraisal Costs)?
What is the cost of Correction (Failure Costs)?
What is the cost of Prevention (Prevention Costs)?
42
Cost Of Quality (COQ) Metric
COQ ( in % ) = ( AC +PRC +FC ) * 100 ( AC +PRC +FC + PDC )
Appraisal Cost ( AC ) = Review + Audit + Testing
Prevention Cost ( PRC ) = Project Management + Training +
CM + Defect Prevention + RFC / change request
Failure Cost ( FC ) = Rework + Idle Time + Complaints +
post sales defects
Production Cost ( PDC ) = Requirements, Design and Coding +
Tools/Scripts Development + manuals
43
Goal 5 : Improve customer satisfaction Are we meeting our commitments to the customer?
How satisfied are customers with our services and products?
44
Customer Satisfaction Metrics Customer Satisfaction Index / Rating
Rating on a scale of 1 – 5 through Customer Satisfaction Surveys
SLA Compliance [ ( Actual Incidents where SLA was met ) * 100 ] / [ Total
No. of incidents ] resolved
SLA Variance [ ( Actual Resolution Time – SLA ) * 100 ] / [ SLA ]
45
Tell me now… Find Cost Of Quality (COQ) ?
Appraisal Cost = 3 % Prevention Cost = 7 % Production Cost = 78 % Failure Cost = 12 %
Review efficiency in a project is 0.1, what does it mean?
Defect Density helps in……Reducing rework and controlling effort variance and schedule variance
One Review defect has been detected after spending 10 hours of review efforts
COQ = 22 %
46
Case Studies
47
Management review report
•Progress reports •Periodic performance reports•Customer satisfaction feedback•Follow up reports•Review of significant findings
48
Software Development Projects
49
Development metrics
•In software development projects, we capture the efficiency of correctness and robustness of the code.
•We track them via:▫LOC (Line of Code)▫Function Points▫Cyclomatic Complexity
50
Line of Code analysis
•Derived by normalizing (dividing) any direct measure (e.g. defects or human effort) associated with the product or project by LOC.
•Size oriented metrics are widely used but their validity and applicability is widely debated.
51
Function oriented Metrics• Function points are computed from direct
measures of the information domain of a business software application and assessment of its complexity.
• Once computed function points are used like LOC to normalize measures for software productivity, quality, and other attributes.
• The relationship of LOC and function points depends on the language used to implement the software.
52
Software Testing Projects
53
Check Points
• Main objectives of a project: High Quality & High
Productivity (Q&P)
• Quality has many dimensions
▫ reliability, maintainability, interoperability etc.
• More defects => more chances of failure => lesser
reliability
• Hence quality goal: Have as few defects as possible
in the delivered software!
54
Metrics for Software Testing
• Defect Removal Effectiveness
DRE= Defects removed during development phase x100% Defects latent in the product
Defects latent in the product = Defects removed during development
phase+ defects found later by user
• Efficiency of Testing Process (define size in KLoC or FP, Req.)
Testing Efficiency= Size of Software TestedResources used
55
Support Groups Metrics
56
SEPG & SQA Metrics Person / Process Trainings PI Index - Process Improvement suggestions per quarter SEPG Efforts
Total effort spent in SEPG Activities Total effort spent in SQA Activities
No. of NCs per project
57
ISS Metrics
% Calls resolved same day =
Number of calls ( 24hrs category ) resolved within one day
Number of user calls ( 24 Hrs category )
% Calls resolved in two days =
Number of calls ( 48hrs category ) resolved within two days
Number of user calls ( 48 Hrs category )
% Network up time = Network up time ( for the month )
Total Available time
58
RMG Metrics
Average hiring cost per hire = Total hiring cost for the month
Total no. of joinees
Selection ratio = Total No. of offers Total no. of candidates interviewed
Offer to joinee ratio = Total no. of joinees Total no. of Offers
Offer to Decline ratio = Total no. of Declines Total no. of Offers
59
Some important metricsIndicator Metric (Unit /
s) Formula Base Measure / s Source / s
Effort Effort Variance(%)
(Actual Effort – Planned Effort) * 100 / (Planned
Effort)
Actual EffortPlanned Effort
TimesheetProject
schedule
ScheduleSchedule Variance
(%)
(Actual End Date – Planned End Date) * 100 /
(Planned Project Duration)
Planned project duration
Planned End DateActual End Date
Project schedule
Size Size Variance (%)
(Actual Software Size – Estimated Software Size) * 100 / (Estimated
Software Size
Actual sizeEstimated size
Estimation sheet
ProductivityProductivity
(KLOC / mandayOr
FP / manday)
(Size) / (Effort) EffortSize
Estimation Sheet
60
Some important metricsIndicat
orMetric
(Unit / s) Formula Base Measure / s Source / s
Quality
Defect Density
(Defects / KLOC
Or
Defects / FP)
(Total No. of Defects) / (Size)
No. of total Defects (Pre
delivery + Post delivery)
Size
Review & testing Defect log
Estimation Sheet
Quality
Review Efficiency
(Defects / Manhour)
(Total no. of review defects) / (review
hours)
Total no. of review defects
Review effort in hours
Review & testing Defect log
QualityCost of Quality
(%)
(Appraisal efforts + Failure efforts +
Prevention efforts) * 100 / (Total Project
Effort)
Review efforts
Testing efforts
Rework efforts
Timesheet
Project schedule
61
Sample Tracking Sheet
62
Software Quality Assurance
63
64
What is Software Quality Assurance?
•Used to Monitor and Improve the Software Development Process
•Making Sure That Standards and Procedures are Followed
•Ensures that Problems are Found and Dealt with
•Orientated to ‘Prevention’
65
Standards and Procedures
•Framework for which Software Evolves•Standards
▫Established Criteria to which Software Products are Compared
•Procedures▫Established Criteria to which Development and
Control Procedures are Followed•SQA is based on the Following of
Standards and Procedures
66
Techniques•Audit
▫The Major Technique used in SQA▫Perform Product Evaluation and Process
Monitoring▫Performed Routinely throughout the Software
Development Process▫Look at a Process and/or Product in depth and
compare to Established Standards and Procedures
▫Provide an indication of the Quality and Status of the Software Product
67
Benefit of Software Quality Assurance in Projects
• Without SQA, many Software Groups would not reach their release goals/deadlines on time
• Lowers time spent on mundane areas and lets more time be focused on important areas
• Decreases the time from Development to Deployment
• Can help catch errors before they are too costly to fix
SQA Activities
•Document Auditing•Document Reviews•Metrics Calculation and Analysis•Meetings and Discussions
▫Final Reports Submission
68
Reviews
69
Types of Review
•Formal Review•Informal Review
70
Types of ReviewFormal Review• Planning• Documented• Thorough• Focused on a certain purpose
71
Types of ReviewInformal Review• Undocumented• Fast• Few defined procedures• Useful to check that the author is on track
72
73
Why do peer reviews?•To improve quality.•Catches 80% of all errors if done properly.•Catches both coding errors and design
errors.•Training and insurance.
Thank You!!
74