© 2011 Crain Communications Inc
Workshop
Tuesday | 4:45 pm | Veranda DE
Jason Ezratty, Managing PartnerBrightfield Strategies LLC
Using Business Analytics and Program Data to Drive Success
© 2011 Crain Communications Inc
© 2011 Crain Communications Inc
“I’ve Been Avoiding Math My Whole Life, Why Start Now?”
Casual Methods Quantitative Methods
Observation “headcount is higher than usual” “headcount is up 28% compared to this month last year,”
Understanding “those guys charge so much more” “those guys charge 46% more than competitors for like job titles”
Inference “we seem to pay more in TX than NY”
“there is no statistical difference between TX and NY pay rates”
Prediction “we are hoping for industry average levels of savings, ~10%”
“measures of rate variance suggest competitive bidding can save 16%”
© 2011 Crain Communications Inc
Common Misuses of Statistics Paint Confusing Pictures
• “Lies, damn lies, & statistics”
• Round number syndrome
• Forests vs Trees
• Under interpretation
• Over interpretation
© 2011 Crain Communications Inc
Understanding & Implementing Analytical Methods is Worth the Journey
I’m able to measure the benefits of my
decisions.
What’s my headcount?
© 2011 Crain Communications Inc
Start at the Beginning, Most Mistakes Happen in Least Sophisticated Data Analysis Steps
• Only deal in trusted data sources
• Data must be comprehensive
• Data must be representative, current
• Know your data type~ Discrete (aka categorical)
~ Continuous
• Condition your data
• Understand how your data distributes across key variables
© 2011 Crain Communications Inc
Sample Output Quality Depends on Quality of Sample
What’s the best way to sample this data?
© 2011 Crain Communications Inc
Directed Plus Random Sampling
ConsultingJob Title
Popularity
Top 15% of managerswith greatest headcount inmost popular job titles
Additional randomsampling of 15%of managers
Manager’s Consultant Headcount
© 2011 Crain Communications Inc
Data Types of Common CW Data Elements
• Number of rejected invoices: __________
• Payroller Pay Rates: __________
• Nonexempt Headcount: __________
• Mark-Up %: __________
• VMS Bug Count, by Severity: __________
• Avg Tenure by Location: __________
• Satisfaction Rating (1-5): __________
• Number of rejected invoices: Discrete
• Payroller Pay Rates: __________
• Nonexempt Headcount: __________
• Mark-Up %: __________
• VMS Bug Count, by Severity: __________
• Avg Tenure by Location: __________
• Satisfaction Rating (1-5): __________
• Number of rejected invoices: Discrete
• Payroller Pay Rates: Continuous
• Nonexempt Headcount: __________
• Mark-Up %: __________
• VMS Bug Count, by Severity: __________
• Avg Tenure by Location: __________
• Satisfaction Rating (1-5): __________
• Number of rejected invoices: Discrete
• Payroller Pay Rates: Continuous
• Nonexempt Headcount: Discrete
• Mark-Up %: __________
• VMS Bug Count, by Severity: __________
• Avg Tenure by Location: __________
• Satisfaction Rating (1-5): __________
• Number of rejected invoices: Discrete
• Payroller Pay Rates: Continuous
• Nonexempt Headcount: Discrete
• Mark-Up %: Continuous
• VMS Bug Count, by Severity: __________
• Avg Tenure by Location: __________
• Satisfaction Rating (1-5): __________
• Number of rejected invoices: Discrete
• Payroller Pay Rates: Continuous
• Nonexempt Headcount: Discrete
• Mark-Up %: Continuous
• VMS Bug Count, by Severity: Discrete
• Avg Tenure by Location: __________
• Satisfaction Rating (1-5): __________
• Number of rejected invoices: Discrete
• Payroller Pay Rates: Continuous
• Nonexempt Headcount: Discrete
• Mark-Up %: Continuous
• VMS Bug Count, by Severity: Discrete
• Avg Tenure by Location: Continuous
• Satisfaction Rating (1-5): __________
• Number of rejected invoices: Discrete
• Payroller Pay Rates: Continuous
• Nonexempt Headcount: Discrete
• Mark-Up %: Continuous
• VMS Bug Count, by Severity: Discrete
• Avg Tenure by Location: Continuous
• Satisfaction Rating (1-5): Discrete
© 2011 Crain Communications Inc
Satisfaction Survey Data - Discrete
1 2 3 4 5 1 2 3 4 5
© 2011 Crain Communications Inc
The PivotTable: Microsoft’s Gift for Discrete Data Analysis
• Fast & simple way to analyze discrete data with “count” field summary type
• PivotCharts aid in rapid visualization of sub-groups
• (Great for continuous data, too)
© 2011 Crain Communications Inc
When Dealing with Continuous Data, Remember: AVERAGES LIE.
vs
AveragedRaw Data
© 2011 Crain Communications Inc
Client-identified vs Recruited Average Bill Rate Per Job Title & Level
RECRUITED
CLI
EN
T-S
OU
RC
ED
even-value line
regression line
© 2011 Crain Communications Inc
Scatter Plot Also Effective with Conditioned Data
C
A
B
© 2011 Crain Communications Inc
Visualizing Central Tendency & Range of Distributions
AB
C
© 2011 Crain Communications Inc
Standard Deviation: Tastes Better than it Looks
© 2011 Crain Communications Inc
Idealized Relationship Between Mean & Standard Deviation
© 2011 Crain Communications Inc
Kurtosis is the Measure of “Spikiness”
© 2011 Crain Communications Inc
Variance is Like the Bread of a Sandwich, Too Much and You Can’t Taste the Meat
Thick Doughy Mess Thin & Crispy
© 2011 Crain Communications Inc
Relationship Between Average & VarianceGROUP A GROUP B GROUP C GROUP D GROUP E
1 2 3 4 52 3 4 5 53 4 5 5 54 5 5 5 55 5 5 5 56 6 6 6 67 6 6 6 68 7 6 6 69 8 7 6 6
10 9 8 7 6
mean 5.5 5.5 5.5 5.5 5.5
standard deviation 3.0 2.2 1.4 0.8 0.5standard deviation (%) 55% 40% 26% 15% 10%
min 1.0 2.0 3.0 4.0 5.0median 5.5 5.5 5.5 5.5 5.5
max 10.0 9.0 8.0 7.0 6.0
© 2011 Crain Communications Inc
Bill Rate Data - Continuous
GROUP A1 GROUP B1 GROUP A2 GROUP B241$ 39$ 52$ 48$ 48$ 42$ 53$ 48$ 53$ 44$ 53$ 49$ 53$ 47$ 54$ 49$ 56$ 48$ 54$ 49$ 58$ 50$ 55$ 49$ 58$ 51$ 55$ 49$ 59$ 53$ 55$ 50$ 59$ 57$ 56$ 50$ 59$ 60$ 57$ 50$
mean 54.40$ 49.10$ 54.40$ 49.10$ standard deviation 5.93$ 6.54$ 1.51$ 0.74$
standard deviation (%) 11% 13% 3% 2%min 41.0 39.0 52.0 48.0
median 57.0 49.0 54.5 49.0max 59.0 60.0 57.0 50.0
NOISY DATA TIGHT DATA
© 2011 Crain Communications Inc
Per Job Title: Bill Rate StDev vs Avg Bill Rate
© 2011 Crain Communications Inc
COLOR CORRESPONDS TO SUPPLIERSCOLOR CORRESPONDS TO SUPPLIERS
© 2011 Crain Communications Inc
Using Querying to Identify Misclassified SOW Workers with a Venn Diagram
greatest likelihoodof SOW misclassification
© 2011 Crain Communications Inc
Beware of False Metrics
• Average Bill Rate per Location~ Actually saw this in a top VMS’s
demo
• Total Number of Suppliers~ Fine as a point in fact but not great
as indicator of performance~ Better to look at supplier sufficiency
per skill per location
• Know your Headcount from your Spend~ Different situations call for either or
both
© 2011 Crain Communications Inc
Any Other Questions?
…feel free to call/email me afterward to speak confidentially: [email protected]
212.448.1843