Process Metrics & KPI’s – The vital link to evaluating process effectiveness
Angelo Scangas, President4 Essex Center Drive #3971
Peabody, MA 01960978-430-7611
©2014 QSG, Inc.
©2014 QSG, Inc.
1. Business & Process Improvement2. PDCA Continual Improvement3. Risk Management4. Process Measurement5. Types of Measures6. Best Practices7. Cynosure - Quality & Regulatory Metrics
Agenda
Business Process ImprovementBusiness process improvement (BPI) is a systematic approach to help an organization optimize its underlying processes to achieve more efficient results.
KPIs – Track Business Process Improvement
Source: Wikipedia
Business KPIsKPIs – Track Business Process ImprovementExamples:
1. Product Margins/Profitability2. Customer Satisfaction3. OTD4. Inventory Management5. Safety6. ?????
Process ImprovementProcess improvement is a series of actions taken by a process owner to identify, analyze and improve existing business processes within an organization to meet goals and objectives, such as increasing profits and performance, reducing costs and accelerating schedules. These actions often follow a specific methodology or strategy to increase the likelihood of successful results.
Source: Wikipedia
KPIs and metrics (drivers)
It Helps to Understand Drivers that Influence Performance of a KPI. Which may be other Metrics
and/or other factors.
A Driver Metric (or other factor) may influence more than one KPI.
10/10/14 QSG Meeting Presentation – G. Lane
Internal Business Processes
Innovation processes
Operation Processes
Quality/Regulatoryprocesses
Sales/Marketing processes
Measures
Standard Business and Process Measures
8
Business Process Measures
Measures of Affectivity
On Time Release for Source Inspection
On Time Kit ReleaseCycle Time for Processes A, C, T, R
% of Documentation available at Release
On Time DeliveryOn Time Delivery to MRP
On Time ReleaseOn Time Approval
Customer SatisfactionOn Time Delivery to Contract
EscapesCustomer CARs
Critical Process ManagementCritical Items Identified
Key Characteristics IdentifiedVariability Management in Place
Product Yield
Process YieldsTest Yields
Inspection YieldsRolling Throughput Yields
PDCA
PDCA (plan–do–check–act or plan–do–check–adjust) is an iterative four-step management method used in business for the control and continuous improvement of processes and products. It is also known as the Deming circle or the Shewhartcycle.
Source: Wikipedia
4.1.3 Process ApproachISO/DIS 9001, clause 0.4, Plan-Do-Check-Act cycle
CustomerSatisfaction
Customers& other
relevantinterested
parties
5 Leadership
8 Operations
6 Planning 9 Performanceevaluation
4.1, 4.2, 4.3Establish context,
define relevantinterested parties& scope of QMS
10 Continual Improvement
7 Support Processes
4.4 QMS- GeneralProcess - Approach
OutputsInputs Products &ServicesRequirements
*
*
4.8 Performance evaluation (section 9)
• Organizations need to plan how they’re going to monitor, measure, analyze, and evaluate their QMS, and then
• Monitor, measure, analyze, and evaluate their organization's QMS
• Clause “8.4 Analysis of data” was substituted with clause “9.1.3 Analysis and evaluation”. It’s obvious from sub-clause title that now beside analysis it is required to do evaluation as well.
ISO/DIS 9001, clause 9.1
4.4.1 Actions to address risks and opportunities
ISO/DIS 9001, clause 6.1
One of the key changes in theISO/DIS 9001 standard is toestablish a systematic approachto risk.
By preventing or reducing the undesired effects, the organization becomes proactive. Preventive action is automatic in risk-based approach system.
Instead of being onlypart of the preventiveaction procedure, risk isconsidered thoughoutthe quality managementsystem.
1
3 2
Pane 2 – Why are we here?
$0.00
$50,000.00
$100,000.00
$150,000.00
$200,000.00
$250,000.00
$300,000.00
$350,000.00
1 2 3 4 5 6 7 8 9
Pane 1 – Where are we?
0.00%
1.00%
2.00%
3.00%
4.00%
5.00%
6.00%
Month
Perc
enta
ge o
f Pai
d Sh
ipm
ents
with
Exc
eptio
n
Run Chart Data Exceptions by Month
Month Paid shipments # Exceptions Exceptions Goal
September FY05 886 43 4.85% 2.50%October FY05 1139 32 2.81% 2.50%November FY05 1103 45 4.08% 2.50%December FY05 1346 48 3.57% 2.50%January FY05 993 38 3.83% 2.50%February FY05 856 36 4.21% 2.50%March FY05 1056 44 4.17% 2.50%April FY06 923 33 3.58% 2.50%May FY06 877 13 1.48% 2.50%June FY06 891 14 1.57% 2.50%July FY06 779 9 1.16% 2.50%Aug FY06 923 7 0.76% 2.50%
Pane 3 – How did we get here? Pane 4 – What do we do now?
Action Plan
Action Resp Date Comment
PDCA - Example
13
CustomersProcess
Products or Services
MeasurementProcess
Dealing withthe Future
Dealing withthe Past
Product Controland Disposition
Process Controland Improvement
Use of Measures
Lead Indicators
• Common examples are:– Market share, backlog (book-to-bill ratio),
new product introductions, new product development lead times, product quality, customer satisfaction, employee morale, personnel development, inventory turnover, bad debt ratio, or safety
Lagging Indicators• In contrast to lead indicators, lag indicators
are measures that point to earlier plans and their execution.
• Financial performances are lag indicators.• Many times, financial performances are too
late to affect future products and services.
• Understand Stakeholder needs/requirements – business, customer, regulatory, legal, etc.
• Use value stream mapping to identify key processes/points for measures.
• Develop a process for collecting, recording, and analyzing data.
• Try the measure and adjust as necessary.Remember – A measure is as good as the actions it identifies for improvement
Developing Measures
• Where can we measure the process?
• What are pros and cons of the above locations?
• Where should we start??
Selecting MeasuresOutputs
CustomersProcessProducts or Services
GaugeGauge Gauge
In-Process Final Product Customer Satisfaction
Gauge
Inputs
Measures that do more harm then good
???
Thank you !!
Angelo Scangas, President4 Essex Center Drive #3971
Peabody, MA 01960978-430-7611
©2014 QSG, Inc.
The Power of MetricsHow Data Drives Decisions
Connie HoyJanuary 2015
Why?
• Cynosure uses metrics for 2 reasons:
– Drive Business decisions• Where do we need new employees?• What products need to have reliability improvements?• Do we have the right tools to drive training?
– Measure how we comply with the quality policy
Cynosure, Inc. strives to achieve satisfactionfor our customers and patients through ourinnovative products and services. Cynosure isa leader in the design and manufacture ofhigh quality, reliable products that are safeand effective. Cynosure is committed tocomplying with regulatory requirements andmaintaining and improving the effectivenessof our quality management system.
Quality Policy
Mapped to objectives and metrics
Cynosure, Inc. strives to achieve satisfactionfor our customers and patients through ourinnovative products and services. Cynosure isa leader in the design and manufacture ofhigh quality, reliable products that are safeand effective. Cynosure is committed tocomplying with regulatory requirements andmaintaining and improving the effectivenessof our quality management system.
Satisfied Customers
Service Response Time (All Laser Consoles)
Objective: Respond under 2 Days
Service First-Call Resolution (All Laser Consoles)
84%
84%
85%
85%
86%
86%
87%
87%
88%
88%
89%
Q1 2012 Q1 2013 Q1 2014
1st Call Resolution RateQ1 2012 vs. Q1 2013 vs. Q1 2014 1st Call Resolution Rate
Service Calls per Warranty Laser (Not real data)Q1 2013 YoY Q1 2014
*Cynergy Units under Warranty: 4
How do we use this data?
• Drives the decision on what the sustaining team should work on
• Prioritizes the sustaining projects based on:• Safety• Customer Nuisance• Cost of issue
Satisfied Patients
• MDRs• 18 MDRs submitted to the FDA in 2013
• Of these, 1 was also reported to Canada (MPR)• 16 patient-related• 2 product-related (CAPAs issued)
• 8 MDRs submitted to the FDA in Q1 2014• All patient-related
• Implemented improvements in the reportabilityreview process, including clearer guidelines to ensure reportability determinations are accurate or otherwise justified.
20 MDRs in this period7 due to product issue
13 due to user error and/or serious injury
Monthly MDRs Submitted
High Quality and Reliablility
92.6%96.5%
85.3% 87.6%93.3% 94.5% 93.3%
96.9%
83.8%
71.9% 73.1%
64.9% 67.6%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
50%
60%
70%
80%
90%
100%
Q2 - 12 Q3 - 12 Q4 - 12 Q1 - 13 Q2 - 13 Q3 - 13 Q4 - 13 Q1 - 14
Overall Cynosure Training Status(North American Employees)
Overall Completion Status Complete Within 30 Days
• Note: Began measuring “on-time” rates in Jan 2013. Status is # of completed requirements against total active training requirements. Cycle is % completed “on time” (within 30 days / per due date).
Target = 100%
Material Review Board (MRB) Process (Westford and Fremont, Burlington Dec ‘13 to Present)
Total Disposition Dollars:Q4 2013 = $190,508.30
Total Disposition Dollars:Q1 2014 = $268,496.95
Accolade, Cynergy/Vstar, Elite MPX, PicoSure Lasers (Westford)
6) Process/Product/Supplier PerformanceAcclaim, Apogee/Elite MD, Apogee/Elite+ (Westford/Cogmedix)
Safe
Patient Impact Complaints –not real data
• 63 complaints (2013) vs. 69 complaints (2012)• 38 complaints (Q1 2014)• Improvements to complaint investigation process: QA and Clinical working closely
together to gather and evaluate patient data.
Laser Type 2012 2013 Q1 2014Accolade 1
Affirm 4 2Cynergy 4 1Elite + 2
Elite MD 23 20 6Elite MPX 10 17 7
Pico 1Medlite C6 3 3 1Revlite/SI 1 5Affinity 1
CO2 3SLT II 9 9 5
SmartSkin 1 2SmartXide 1
Adivive 1Icon 1 8
StarLux 500 2 1Vectus 2
8) Customer Feedback
• MDRs• 18 MDRs submitted to the FDA in 2013
• Of these, 1 was also reported to Canada (MPR)• 16 patient-related• 2 product-related (CAPAs issued)
• 8 MDRs submitted to the FDA in Q1 2014• All patient-related
• Implemented improvements in the reportabilityreview process, including clearer guidelines to ensure reportability determinations are accurate or otherwise justified.
20 MDRs in this period7 due to product issue
13 due to user error and/or serious injury
Monthly MDRs Submitted
Compliant
Q4 2013 – Q1 2014: CAPA Distribution to FDA 21 CFR 820 (QSR)(All Sites)
N = 30
End