Five Steps for Analytical Insight

Paul Askew

CONFERENCESEPTEMBER 2014

SHEFFIELD

Five Steps for Analytical Insight - Contents

A. Context

B. Five Steps

C. Key Message

The only statistics you can trust are those you falsified yourself.

Winston Churchill

Statistical thinking will one day be as necessary for efficient citizenship as

the ability to read and write. HG Wells

You cannot ask us to take sides against arithmetic. Winston Churchill

The War Office kept three sets of figures: one to mislead the public,

another to mislead the Cabinet, and a third to mislead itself.

Herbert Asquith

It is by the art of Statistics that law in the social sphere can be ascertained and codified, and certain aspects of the

character of God thereby revealed. The study of statistics is thus a religious service.

Florence Nightingale Smoking is one of the

leading causes of statistics. Fletcher Knebel

If you want to inspire confidence, give plenty of statistics. It does not matter that

they should be accurate, or even intelligible, as long as there is enough of

them. Lewis Carroll

A small error at the outset can lead to great errors in the final conclusions.

Saint Thomas Aquinas

Facts are stubborn, but statistics are more pliable.

Mark Twain

Errors using inadequate data are much less than those using no data at all.

Charles Babbage

Skills for Life Survey 2011 (England)Department for Business Innovation and Skills

% A

dults

at G

CS

E+

Lev

els

The numeracy challenge is big and getting bigger…

7.5m adults with

GCSE+

17m

adults at primary school

level Maths

A Framework for Understanding Statistical Performance

Paul Askew

Policy Gap

Aging Population 2x [36%, actually 16% 65+]

Immigration 2x [31%, actually 13% immigrant]

Religion 5x [24%, actually 5% muslim]

Teenage Pregnancy 25x [15%, actually 0.6% under

15 girls annually]

Benefit Fraud 34x [£24/£100, actually

70p/£100]

RSS & Ipsos Mori

Clustering Illusion

Confirmation Bias

Hindsight Bias

Overconfidence EffectIllusion of

Control

Incentive Super Response Tendency

Endowment Effect

Scarcity Error

Loss Aversion

Cognitive Dissonance

Hyperbolic discounting

The Art of Thinking Clearly, Dobelli

Conjunction Fallacy

Forecast Illusion

Fundamental Attribution Error

Contrast EffectAvailability Bias

Outcome Bias

Operational Delivery

Methodological Leadership

✗ Yes but it’s up compared the same

month last year

✗ Yes but its up for the calendar year so far

✗ Yes but they are reducing faster than

we are this year

✓ Burglary is down compared to last month

✓ Yes but it’s down overall for the financial year to date

✓ Yes but we’re still better than our

neighbours

✓ “Yes but we’re still on target”

Performance Pantomime

Five Steps for Analytical Insight - Contents

A. Context

B. Five Steps

C. Key Message

B. Five Steps

1. Snapshot

2. Trend

3. Benchmark

4. Trajectory

5. Target

1. Snapshot

5. Define

6. Specify

8. Record

7. Collect

9. Enter

10.Process

12. Store

11. Validate

Plan

Implement

Manage

1. Purpose

2. Require-

ments

4. Design

3. Const-raints

Aggregated Data or Real Data

Highs and lowsHigh and low

DecreasingIncreasing Increasing convergence

Decreasing convergence

Three month step Six month stepTwo month step

Smoothed Data – 12 month rolling average

Aggregated Data

Example Real Data

Notes: Real data for 12 months, previous 12 months is exactly the same, to create 12 month rolling average (mean).

This aggregated (averaged) data is derived from any of these underlying raw data

examples.

2. Five Steps

1. Snapshot 2. Trend 3. Benchmark 4. Trajectory 5. Target

Purpose Week National Future Estimate

Requirements Month Regional Intervention

Constraints Quarter Neighbours Impact

Design Six Months Similar

Define Annual Specific

Specify FYTD

Collect

Record

Enter

Process

Validate

Storage

Value

Time

1. Snapshot – we have a number which is important to us

Value

Time

2. Trend – what’s happening over time (s)

Value

Time

3. Benchmark – how this measure compares to others

Value

Time

3. Benchmark-Trend - comparison to others over time

Value

Time

4. Trajectory - the future for our measure

Value

Time

4. Trajectory Benchmark - comparison to others

Value

Time

5. Target Consideration

Five Steps for Analytical Insight - Contents

A. Context

B. Five Steps

C. Key Message

C. Key Message

Framework

Multidimensional

measures - times - comparisons

C. Key Message

42

Five Steps for Analytical Insight

Thank you

CONFERENCESEPTEMBER 2014

SHEFFIELD