Improving decision-making through evidence-based

management

University of Bath taster event – June 15, 2015 - London

Our mission for today:

By the end of this workshop you will be more concerned about the way you typically make decisions as a manager/leader/consultant, so you firmly resolve that, within the next year, you will increase the ‘evidence-base’ of your decisions by at least 25%.

You can do this, of course, by signing up for the executive course ‘Evidence-based decision-making’ at Bath University School of Management.

20 min EBDsc: What is it and why do you need it?

20 min Professional experience & judgment

10 min Organizational data

20 min Scientific evidence: mini case

Agenda

Exercise

Think about a decision you have been involved in making. This decision should be one which: Was reasonably important for your organization Involved spending significant resources Involved several or more people Was made over a period of time (ie. weeks or

months)

Exercise

Discuss with your neighbor (2 min)

What exactly was the problem (or opportunity)?

How many alternative decision options were considered?

How much evidence was used, and from which sources (scientific, organizational, experience, crystal ball?)

Was any attempt made to explicitly evaluate its quality or trustworthiness?

Evidence based decision-making:

What is it?

Evidence-based decision making

Central Premise:

Decisions should be based on a

combination of critical thinking and

the ‘best available evidence‘.

Evidence?

outcome of scientific research,

organizational facts & figures,

benchmarking, best practices,

personal experience

All managers and leaders

base their decisions on

‘evidence’

But…many managers and

leaders pay little or no

attention to the quality of the

evidence they base their

decisions on

Trust me, 20 years of management experience

SO ...

Teach managers/leaders

how to critically evaluate the

validity, and generalizability of

the evidence and help them

find ‘the best available’

evidence

Evidence based decision

Professional experience and

judgment

Organizational data, facts and figures

Stakeholders’ values and concerns

Scientific research

outcomes

AskAcquire

AppraiseAggregate

ApplyAssess

diagnosis intervention

Evidence based practice:

Where does it come from?

McMaster University Medical School, Canada

Medicine: Founding fathers

David Sackett Gordon Guyatt

How it all started

1. Ask: translate a practical issue into an answerable question

2. Acquire: systematically search for and retrieve the evidence

3. Appraise: critically judge the trustworthiness of the evidence

4. Apply: incorporate the evidence into the decision-making process

5. Assess: evaluate the outcome of the decision taken

5 steps of EBmed

Evidence-Based Practice

1991Medicine

1998Education

2000Social care, public policy

Nursing, Criminal justice,

Policing, Architecture, Conservation

2010Management

Evidence-Based Practice

Evidence-Based Practice

Evidence-Based Practice

Evidence-Based Practice

Evidence-based decision-making =

the use of critically appraised evidence from multiple sources to

increase the likelihood of a favourable outcome

Focus on the decision making processThink in terms of probability

“Where the evidence is strong, we should act on it.

Where the evidence is suggestive, we should consider it.

Where the evidence is weak, we should build the knowledge

to support better decisions in the future.”

Jeffrey Zients, acting director of the Office of Management and Budget and President Obama’s Economic Advisor

Evidence-Based Decision-Making

Why do we need it?

1. Incompetent people benefit more from feedback than

highly competent people.

2. Task conflict improves work group performance while

relational conflict harms it.

3. Encouraging employees to participate in decision

making is more effective for improving organizational

performance than setting performance goals.

True or false?

How evidence-based are we?

959 (US) + 626 (Dutch) HR professionals

35 statements, based on an extensive body of

evidence

true / false / uncertain

HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008)

Outcome: not better than random chance

Evidence-based decision making

Professional experience and

judgment

Organizational data, facts and figures

Stakeholders’ values and concerns

Scientific research

outcomes

AskAcquire

AppraiseAggregate

ApplyAssess

Thinking critical about

professional experience and judgment

Discuss with your neighbor

Why is a physician’s clinical experience,

as a rule, more trustworthy than

a manager’s professional experience?

60595857565453525150494847464544434241403938373635343332313029282726252423222120191817161514131211109876543210

Developing expertise

1. A sufficiently regular, predictable environment

2. Opportunities to learn regularities through prolonged

practice and feedback

The management domain is not highly favorable to expertise!

Bounded rationality

Bounded rationality / prospect theory

System 1

Fast Intuitive, associative heuristics & biases

System 2

Slow (lazy) Deliberate, Reasoning Rational

dominant

System 1: short cuts

Shepard’s tables

System 1: short cuts

System 1: dominant, cause necessary to survive

95%

Seeing order in randomness Overconfidence bias Halo effect False consensus effect Group think Self serving attribution bias Sunk cost fallacy Cognitive dissonance reduction

System 1: cognitive errors

Confirmation bias Authority bias Small numbers fallacy In-group bias Recall bias Anchoring bias Availability bias

1. Pattern recognition

2. Confirmation-bias

3. Groupthink

Cognitive errors that mess up decision making

We are predisposed to see order, pattern and causal relations in the world.

Patternicity: The tendency to find meaningful patterns in both meaningful and meaningless noise.

Error 1: pattern recognition

We are pattern seeking primates: association learning

Bias 1: pattern recognition

Points of impact of V-1 bombs in London

Points of impact of V-1 bombs in London

A Type I error or a false positive, is believing a pattern is real when it is not (finding a non existent pattern)

A Type II error or a false negative, is not believing a pattern is real when it is (not recognizing a real pattern)

Dr. Michael Shermer (Director of the Skeptics Society)

Error 1: pattern recognition

A Type I error or a false positive: believe that the rustle in the grass is a dangerous predator when it is just the wind (low cost)

Error 1: pattern recognition

A Type II error or a false negative: believe that the rustle in the grass is just the wind when it is a dangerous predator (high cost)

Error 1: pattern recognition

A Type I error or a false positive: believe that the rustle in the grass is a dangerous predator when it is just the wind (low cost)

A Type II error or a false negative: believe that the rustle in the grass is just the wind when it is a dangerous predator (high cost)

DEFAULT

Error 1: pattern recognition

superstitious rituals

superstitious rituals

more stress = more prone to type 1 errors

Error 1: pattern recognition

Error 1: pattern recognition

1. Pattern recognition

2. Confirmation-bias

3. Groupthink

Cognitive errors

We are predisposed to selectively

search for or interpret information in

ways that confirms our existing beliefs,

expectations and assumptions, and

ignore information to the contrary.

In other words, we “see what we want to see”

2. Confirmation bias

Example

You may believe that astrology actually

works. As a result of confirmation bias

you’ll remember only those instances

when when the prediction in the astrology

column came true and forget the majority

of the cases when the prediction was very

wrong. As a result you will continue to

believe astrology has some base in reality

2. Confirmation bias

Confirmation bias

Pattern recognition

Error 2: confirmation bias

McKinsey (1997 case study / 2001 book)

McKinsey: case study

War on Talent

1. Pattern recognition

2. Confirmation-bias

3. Groupthink

Errors

Groupthink:

Groupthink is a psychological phenomenon that occurs within a group of people, in which the desire for harmony or conformity in the group results in an incorrect or irrational decision

Error 3: Groupthink

Error 3: Groupthink

Error 3: Groupthink

Lean Management / Lean Six Sigma

Self steering / autonomous teams

Agile working / New World of Working

Talent management

Employee engagement

Groupthink?

“I’ve been studying judgment for 45 years, and I’m no better than when I started. I make extreme predictions. I’m over-

confident. I fall for every one of the biases.”

Bounded rationality

EBDsc = Safeguard

Pattern

recognition

Confirmation bias

Small numbers fallacy

Halo effect

Selection bias

20 min EBDsc: What is it and why do you need it?

20 min Professional experience & judgment

10 min Organizational data

20 min Scientific evidence: mini case

Agenda

Evidence based decision

Professional experience and

judgment

Organizational data, facts and figures

Stakeholders’ values and concerns

Scientific research

outcomes

AskAcquire

AppraiseAggregate

ApplyAssess

diagnosis intervention

Evidence-based decision making

Professional experience and

judgment

Stakeholders’ values and concerns

Scientific research

outcomes

AskAcquire

AppraiseAggregate

ApplyAssess

Organizational data, facts and figures

1. financial data (cash flow, solvability)

2. business outcomes (ROI, market share)

3. customer/client impact (customer satisfaction)

4. performance indicators (occupancy rate, failure frequency)

5. HR metrics (absenteeism, employee engagement)

6. marketing intelligence (brand awareness, customer feedback)

7. ‘soft’ data (organizational culture, trust in senior management, leadership style, commitment)

8. data from benchmarking

Types organizational evidence

Organizational facts and figures

Examples

Trends

People operate with beliefs &

biases. To the extent you can

reduce both and replace them

with data, you gain a clear

competitive advantage

Laszlo Bock (CHRO Google)

Organizational data

Organizational facts and figures

Organizational facts and figures

Can your organization correlate/regress

level of education

years of experience

productivity?

customer satisfaction?

failure frequency?

employee satisfaction?

employee turnover?

absenteeism?

+

In the next weeks, before you make a decision, ask yourself:

What exactly is the problem?

What is the evidence available?

Where does it come from (multiple sources?)

How trustworthy is it?

Identify the many other factors, rather than evidence, that shape managerial decisions such as cognitive biases, politics, power, management fads and the ideas of management gurus

Distinguish between the four different types of evidence used in evidence-based management

Identify differences between your own and your organisation’s decision-making and an evidence-based management approach to decision-making

Understand practical ways in which your and your organisation’s decision-making can become more evidence-based and the potential benefits and costs that may result.

Learn how to critically evaluate the trustworthiness of evidence (including assumed best practices, organisational data, findings from research etc.)

At the end of the course you will be able to