1
Continuous Improvement
Introduction to
Measure
Module 1.08
Lean Sigma Associates Ltd.
2
Contents
• Why do we need to measure?• What exactly are we measuring?
• Output Indicators• Input Indicators
• How will we measure• Data Types• Data Attributes• Collection methods
3
Objectives
Identify Output and Input Indicators that effect CTQ’s
Understand the types of dataContinuous vs. Discrete
Collecting some data (Check Sheets)
4
Where have we been?
• Key Define Takeaways:– We have identified problems/opportunities– We have launched a project and put together a team– We understand our customer wants/desires– We have translated these wants into a measureable– We have mapped the process
Define Measure Analyse Improve Control
Concepts/Tools:VOCAffinity DiagramCTQ’sSIPOCCharterProcess Map
5
Where are we going?
• Key Measure Takeaways:– We can identify types of data– We have defined the operational definition of
what we measure– We have determined the baseline for how well
we are doing– We have a plan on how we will collect the data
Define Measure Analyse Improve Control
Concepts/Tools:Data TypesOperational DefinitionBaselineData Collection
6
DMAIC Process & Tools
BusinessNeed orProblem
UnderstandCustomer
Want(VOC)
Map &AnalyseProcess
TranslateVOC to CTQ
DetermineOperationalDefinitions
Get Dataand set
Baseline(s)
DetermineSolutions to
Root Causes
PilotSolutions as
needed
Pilot &Evaluate as
needed
ImplementSolutions &
Control
Dig for RootCause of the
Problem
– SIPOC– Charter– SMART
– Surveys– Customer Data
– Process Map– VA/NVA Analysis
– Run Chart– Pareto Chart– Histogram– Fishbone– Is/Is Not
– Brainstorm– Solution Selection Matrix– Fishbone–Tree–5 Why’s
– Risk Analysis
– Affinity Diagram
– Data Attributes– Variation– Sample Size– DCP
– Defining the measure
– Action/ Timing plans
– Standardise
7
Why Do We Need to Measure?
• You don’t know what you don’t know. • Making decisions without data is like testing
detonation fuses with a hammer. It is all up to chance!
NOOOOOOOO!!!!!
Do you like the odds?
8
Data Helps Us . . .
– Separate what we think is happening from what is really happening
– Confirm or disprove preconceived ideas and theories
– Establish a baseline of performance– See the history of the problem over time– Measure the impact of changes on a process– Identify and understand relationships that might
help explain variation– Control a process (monitor process performance)– Avoid “solutions” that don’t solve the real problem
Data converts organisational issues (I think) into a quantitative definable problem (it is)
9
Critical Relationships
VOC CTQ’s Y f (x) f (x)1 The wants
of the customer
The wants of the
customer which can
be measured
The measurable
output of the process
Customer care point
Continuous Improvement care point
The input to the Y
The input to f (x)
Critical path thinking to understand and quantify customer wants
Critical path thinking to understand and quantify the
inputs which deliver the desired output
10
OvertimeScore
Output Input/Process
Y = f ( X1 + X2 + X3 + . . . . . . . . . Xn )
Customer Satisfaction = Front Desk
Courtesy+ Check In
Ease+ Room
Comfort+ Room
ServiceCheck Out
Ease+
Loan Process Cycle Time =
ApplicationData Entry
Time+ +
Credit & Collateral
Check Time
Risk Assessment
Time+ Review &
Approval TimeLoan Service
Time+
Final Score in Basketball
Game=
First QuarterScore
+SecondQuarterScore
+Third
QuarterScore
+Fourth QuarterScore
+
“X” & “Y” Variables
Input measures must directly relate to the Y measure, if they do not you then need to find a more relevant input indicator.
11
Measuring Processes
Time Per Task
In-Process Errors
Labor Hours
Exceptions
How well do these…
…predict these?
X - PREDICTOR (Leading) MEASURES
Y - RESULTS (Lagging)
MEASURES
InputInput OutputOutputProcessProcess•Customer Satisfaction
•Total Defects
•Cycle Time
•Cost Profit
•Arrival Time
•Accuracy
•Cost
•Key Specs
(X) (Y) (X)
12
Categories of Performance
• Developing Input, Process and Output metrics around the Voice of the Customer (VOC) and Voice of the Business (VOB) process performance needs is a good starting point for determining what to measure
QualityQuality
Product or Service Features, Attributes, Dimensions, Characteristics Relating to the Function of the Product or Service, Reliability, Availability, Taste, Effectiveness - Also Freedom from Defects, Rework or Scrap (Derived Primarily from the Customer - VOC)
CostCostProcess Cost Efficiency, Prices to Consumer (Initial Plus Life Cycle), Repair Costs, Purchase Price, Financing Terms, Depreciation, Residual Value (Derived Primarily from the Business - VOB)
SpeedSpeedLead Times, Delivery Times, Turnaround Times, Setup Times, Cycle Times, Delays (Derived equally from the Customer or the Business – VOC/VOB)
13
Output-Process-Input Measures
Helpful Hints
• Use your SIPOC diagram and sub-process maps to help select measures and ensure “balance.”
• Output measures can be taken before or after the Service is delivered to the patient.– Examples: Errors on a form prior to mailing, # of patient complaints,
etc.
• There are usually more “options” for Process measures than Output or Input measures.
• A team must get Output measure(s) “up-front” to baseline the process.
• Begin at least one Process and/or Input measure early in the project to help get some initial data for Analyse.
14
Translating VOC
• Good customer requirements:– Are specific & measurable (and the method of measurement is specific)
– Are related directly to an attribute of the product or service
– Don’t have alternatives and don’t bias the design toward a particular approach or technology
– Are complete and unambiguous
– Describe what, not how
“I hate filling out this form!”
The form takes too long to fill out
The form takes less than five minutes to
complete
Voice of Voice of the Customerthe Customer
After Clarifying,After Clarifying,the Key Issue(s) Is...the Key Issue(s) Is... Customer(s) RequirementsCustomer(s) Requirements
15
Exercise: Identify Potential Output Indicators
Objective
Identify a list of potential input and output indicators that evaluate the extent which the process meets CTQs
Instructions:
Review the SIPOC below or the one for your project and determine what input and output indicators would tie to a customer requirement Internal/external.
Time: 10 Minutes
16
Continuous
Measured on a continuum
Objective• Time• Money• Weight• Length
Subjective• Satisfaction• Agreement• Extent• Type of error
Discrete
Count or categories
Objective• Count defects• # approved• # of errors• Type of document
Subjective• Yes / No• Categories• Service performance
rating (good, poor)• Satisfaction• Agreement
Types of Data
• Classifying data is important because it will:– Provide a choice of data display and analysis tools
– Provide performance or cause information
– Determine the appropriate control chart to use
– Determine the appropriate method for calculation of the improvement or capability of the process
17
Examples of Data
Both: Customer rating (1=very satis/ 5= very dissatis); day of week (MTWRF), date, time order
Service: Proportion of late applications, incorrect invoices NHS: Proportion of breaches, secondary infections, damaged items, late shipments Both: Proportion of associates absent, incomplete forms
Examples
Continuous:(or “variables”)
Measuring instrument
or a calculation
Service: Elapsed time to complete transaction,average length of phone calls NHS: Elapsed cycle time, drug purity, oxygen flow, weight, length, speed Both: Budget vs. actual (dollars); average customer satisfaction score; amount purchased
Discrete:Percentage orproportion
Count occurrencesand non-occurrences
Type/How Obtained
Discrete:Count Count occurrencesin an area ofopportunity
Discrete:Attribute
Observation
Discrete:OrdinalObservation orRanking
Service: Number of applications, errors, complaints,etc. NHS: Number of computer malfunctions, machine breakdowns, accidents
Service: Type of application, type of request NHS: Type of treatment Both: Type of customer, type of method used (new vs. old), location of activity (city/state)
18
Exercise: Types of Data
Objective: Practice identifying different types of data. This information is important for knowing both how to collect the data and how to analyse it.
Instructions: Label the following with the appropriate type of data. If more than one may apply, describe how. You may work in pairs. Be prepared to share your answers with the whole group.
Time: 10 minutes.
19
Page left intentionally blank
20
Exercise: Answers
Data Type
Lateness of a delivery Continuous1
Defective insterments Discrete: Percentage or Count2
Overdue accounts Continuous3
Amount of recycled materials Continuous
Equipment breakdowns Discrete: Percentage or Count2
Cycle time Continuous
Lost patients Discrete: Percentage or Count2
Errors on reports Discrete: Count
Changes to a schedule or plan Discrete: Count
Percent of a report that needs to Discrete: Percentagebe reworked
21
The road to…...
1) Define (Practical Business Issue)
Problem
S I P O C VOC CTQ’s Cost
Delivery
QualityParts
ServiceSurveyFocus groupsInterviews
Charter
Problem StatementGoal Statement
ID Stakeholders
Business Case
2) Measure
- Data Collection
- Sampling Strategy
Representative
Contextual
Sufficient
Reliable- MSA (Gauge R&R)
Data Collection Plan (DCP)
22
Remember Data Helps Us . .
– Separate what we think is happening from what is really happening
– Confirm or disprove preconceived ideas and theories– Establish a baseline of performance– See the problem over time– Avoid “solutions” that don’t solve the real problem
You do not know what you do not know and making decision on the unknown is paramount to chance at
best!
23
The Importance of Measure
• To understand your process and the variation which can cause your output to fluctuate (Defective), one has to measure the process.
• To understand output variation we need to measure the input variables to find the defect.
• With good reliable data, you can make better decisions than without data or bad data.
Remember, if your reacting to an output you are reacting when it is too late!
24
Asking Questions????
• Asking questions is one of the primary ways of collecting data.
• Asking the right questions guides you to collecting the right data
• Deduction from the right data makes the decision “Elementary”
25
What questions do you need to ask?
It is essential that you have a grasp of the knowledge requirements of your project thus:
1) What do you want to know?
2) How do you want to represent what you want to know, i.e. chart or other visual tool?
3) What type of tool will generate the results you seek?
4) What type of data is needed to fit your tool selection, continuous or discrete?
5) Where can you obtain the required data type?
26
Who to Ask, Where to Look?
• Along with deciding what to ask, one must decide who to ask and where to look. – There may be multiple measurement systems
• People• Data Source
–Is there only one data repository?
Ask enough questions of enough people to understand what is going on
27
Data Attributes
Sufficient Enough data so any patterns you see are likely to be real Does one warm weekend in March indicate the start of Spring
Season?
Representative Full range of actual process conditions is seen in the data Are Saturdays representative of every sales day of the week?
Reliable Actually represents the process of interest collected in a manner that is
repeatable and reproducible If we were to do an inventory count in lighting, how many of us would
get the same counts? What if we each did it twice?
Contextual Collected along with other information about what is happening
throughout the process (who, what, where, when) Are higher sales in spring due to all departments or just some?
Additional information in the Appendix
28
Some thoughts about data
• You have no data 30 samples to set base line
• You have historical data > 100
• Demonstration of improvement will depend on the type of data and the precision you require to demonstrate change
• You are uncertain of the data new or old (Meaning until you have validated it) use it with speculation.
29
Population Vs. Sample• Population
– an entire group of objects that have been made or will be made.
– highly unlikely we can ever know the true population parameters.
• the average time to treat 20,000 sq. ft of grass.
• all registered voters in the U.S.
• Sample– the group of objects actually
measured.
– a sample is usually a subset of the population of interest.
• the average time to treat 20,000 sq. ft of grass today.
• a survey of 1000 voters.
“Population Parameters” = Population mean = Population standard deviation
“Sample Statistics”XX = Sample mean
ss = Sample standard deviation
Sample
Popu
latio
n
30
Population Vs. Sample
Why use samples?
• Sampling is:– Collecting a portion of all the data– Using that portion to draw conclusions (make
inferences)
• Why use samples? Because looking at all the data may be:– Too expensive– Too time-consuming– Destructive (e.g., taste tests)
31
NameAddressTelephoneType of Service NeededMonthly or Quarterly CustomerExisting Customer
Missing Service Request Information
Missed Service Appointments
Week 1 Week 2 Week 3 Total
Wrong Address 6
Traffic 11
Not Scheduled 3
No One Home 5
Running Late 13
Total 15 12 11 38
Tools to Help You Collect Data
Check sheets
• Simple data collection form which helps determine howoften something occurs.
Concentration Diagrams• Pictorial check sheet which
helps you mark where something occurs or the type of problem.
32
• The most important aspect of a concentration diagram is that it lets you quickly see where problems physically appear and how they cluster.
• Advantages include being easy to fill out and not requiring a lot of words.
Concentration Diagrams• A concentration diagram is a data collection form where
you write directly on a picture of an object, form, or work area.
E: Entry missingR: Receipt missingM: “Misc.” not explainedT: “Trans” not explainedA: Arithmetic error
E A E EA E
E E E E EA E E A
E E E E
E
R RR R RR
E E E E E
E E E E R R R E A
M M T MF
E E E E A E A E
Expense ReportName: _Tom Rodgers _______ Week ending ___________ 20__Jan 21 03
Date ProjectCode
Hotel Trans Meals Misc Total Comments
Totals
33
Distribution Check Sheet
Incoming Patients
Time of Day7 AM 7 PM
2 3 5 8 9 18 14 22 15 13 10 8 5 3 1
Totals
34
Review and Transition
• In Identifying Measures we learned:– Identify Output and Input Indicators that effect CTQ’s– Creating an operational definitions for Measure– Understand the types of data and attributes – Data Collection
• In Basic Statistics, we will learn:– To distinguish between two types of data– How to calculate central tendency– Evaluate distribution based on location, shape, spread and consistency– Distinguish the types of variation
35
Appendix
36
Representative
• For conclusions to be valid, the samples must be representative.– The data you collect fairly represents all the data
– No systematic differences between the data you collect and the data you don’t collect
– Every item stands a known and usually equal chance of being included
• Statisticians call this “avoiding bias.”
37
Representative• Being representative in your data collection strategy is one
of the most important aspects of creating a data collection plan.
• If you only collect data on a location in Southern California can you make any inference towards the same process in in Maine?
The idea of representative simply means that the population is fairly represented in your sample
38
Contextual
• Contextual data are collected to provide information on the context in which individual attitudes– behaviour, or other experiences take place
• Contextual makes the primary measure make more sense– offers supporting data which might help in determining the
few x’s which have the greatest influence on the Y output. What was the situation when the primary metric was collected?
• Example: Cycle time is the primary metric, but time of day the process, who collected it, how was it captured, the day of the week, month of the year, and which location are important attributes to the CTQ of service deliver within 2 hours
39
Questionable Data??
• Bias in a sample is the presence or influence of any factor that causes the population or process being sampled to appear different from what it actually is.
40
Biased??
So what are these statistics telling me?
What is wrong with what they are telling
me?
Is there any Bias which has been
introduced?
41
Sufficient
Sufficient
• Enough data so any patterns you see are likely to be real.
• There has to be enough data to make statistical inference
• If we are measuring a monthly measure, how many consecutive months must we have to determine patterns?
– Is it better to have a weekly or daily measure?
– But what does that do to the required sample size?
42
Practicality
• Sampling in some cases can either be too slow, costly, biased or non-contextual which regardless of how many samples you have makes your conclusions invalid.
• Two basic rules:1. At the very minimum to establish a base line you will need
to collect 30 samples
2. Verify the sampling strategy with your Project Lead or Stat Guru prior to collecting any data as even with good work and intention you might collect the information incorrectly or introduce bias
43
Reliability• The last element about data requirements is that it is reliable. Simply put, you have to prove the integrity of the data.
• A Measurement System Analysis (MSA) may be required before you can make any conclusion regarding your data.
• MSA will be covered in the following module ‘MSA Attribute’. However, understand that if you can't prove that you have good data is the same as making decisions with bad or no data at all.
• With an MSA you can actually confirm the size of the issue
44
How Reliable?
• In determining reliability outside of conducting an MSA,
asking the following questions will provide some key
insights as to the reliability of the data.– Do all data collectors have the same operational
definition?
– Is the data collected from the same data pool?
– How is the data transferred?
– Is the data collected for the same time period?
– Is the data collected by the same people over time?
If you cannot answer these questions satisfactorily, it is even more doubtful that your data will be reliable.