1
Making Decisions Better
…….Evidence-informed decision making……how to feel comfortable with not knowing everything
….. working with our human nature, not against it
Jonathan UnderhillAssociate Director, Medicines Evidence
NICE Medicines and Prescribing Centre
www.nice.org.uk/[email protected]
2
What I want to talk about
How we make decisions
How to manage information overload
How to keep up to date
3
What I want to talk about
How we make decisions
How to manage information overload
How to keep up to date
4
Humans make decisions by……
Small number of variables
+
Allocate value to those variables
+
Time frame
=
DECISION
HOW?HOW?
5
Allocating value to those variables
• Brief reading
• Talking to other people
6
7
8
DiagnosisDiagnosis RxRx
9
Can this approach let you down???
10
It is most likely that Steve is a ……It is most likely that Steve is a ……
1.1. FarmerFarmer
2.2. PharmacistPharmacist
3.3. Disc jockeyDisc jockey
4.4. LibrarianLibrarian
5.5. Member of ParliamentMember of Parliament
Steve is very shy and withdrawn, invariably helpful, but with little interest in people. He has a need for order and structure and a passion for detail
11
How we acquire and use information
• Where did you get the information from to make that decision about Noah and the sheep?
• If you had had time, what would you have done to make sure you had the right answer?
12
Information and decision making
• Most decisions are based on what we think is the evidence, not what we know is the evidence
• No one has time to appraise all of the evidence on everything, and even if that were possible the human brain can’t recall and compute it, and certainly not in a 10 minute primary care consultation
• We use brief reading and talking to other people as our information sources
• We often use patterns to make a diagnosis
• We create mindlines ( = patterns) of what to do in common situations
13
14
How is knowledge managed in primary care?Gabbay and le May BMJ 2004; 329: 1013 – 6.
• Not once was a guideline read
• Expert computer systems rarely used (never in real time)
• Shortcuts to evidence – free magazines
– network of trusted colleagues (rarely if ever questioned)
– Pharma reps – considerable scepticism (but not without influence)
– Pharmaceutical adviser – highly trusted source.
“Clinicians rarely accessed, appraised, and used explicit evidence directly from research or other formal sources; rare exceptions were where they might consult such sources after dealing with a case that had particularly challenged them.”
15
“Instead, they relied on what we have called "mindlines,"
collectively reinforced, internalised tacit guidelines, which
were informed by brief reading, but mainly by their interactions with each other and with opinion leaders, patients, and pharmaceutical representatives and by other sources of largely tacit knowledge that built on their early training and their own and their colleagues' experience.”
161616
• Eastern Iowa, 103 family doctors.
• If you ask doctors, they say they need information about once a week.
• But if you debrief them, they raise about 2 questions for every three patients
• Answers to most questions were not immediately pursued.
• Doctors spent an average of less than 2 minutes pursuing an answer, and they used readily available print and human resources.
• Only two questions (out of over 1100) led to a formal literature search.
Information habits of doctorsEly JW,et al. BMJ 1999; 319: 358-361
Covell DG et al. Ann Intern Med 1985; 103: 596-9)
17
Information and decision making
• Most decisions are based on what we think is the evidence, not what we know is the evidence
• No one has time to appraise all of the evidence on everything, and even if that were possible the human brain can’t recall and compute it, and certainly not in a 10 minute primary care consultation
• We use brief reading and talking to other people as our information sources
• We often use patterns to make a diagnosis
• We create mindlines ( = patterns) of what to do in common situations
18
52 cognitive biases
• Anchoring bias – early salient feature
• Ascertainment bias – thinking shaped by prior expectation
• Availability bias – recent experience dominates evidence
• Bandwagon effect – we do it this way here
• Omission bias – natural disease progression preferred to those occuring due to action of physician
• Sutton’s slip – going for the obvious
• Gambler’s fallacy – I’ve seen 3 recently; this can’t be a fourth
• Search satisfycing – found one thing, ignore others
• Vertical line failure – routine repetitive tasks leading to thinking in silo
• Blind spot bias – other people are susceptible to these biases but I am not
19
What I want to talk about
How we make decisions
How to manage information overload
How to keep up to date
20
Information Management
21
More reading?
• Potential journals 10,000
• Potential new articles per week 40,000
• Even if 97% are not relevant (no POOs) 1,200
• Time to read each article 15minutes
• 10h a day, 6 days a week = 240 articles.
• So at the end of the first week you are about 4 weeks behind in your reading.
• At the end of the first month, you are 4 months behind in your reading.
• And at the end of the first year you are almost 5 years behind in your reading.
22
Effect of Exercise on Pain in Knee OA Roddy E, et al. Ann Rheum Dis 2005; 64: 544-8
23
How can we keep up?Sackett D et al BMJ 1996;312:71-72
“The difficulties that clinicians face in keeping abreast of all the medical advances reported in primary journals are obvious from a comparison of the time required for reading
for general medicine, enough to examine 19 articles per day, 365 days per year
with the time available well under an hour a week by British medical consultants, even on
self reports.”
How to best use use your Golden Hour?
24
What I want to talk about
How we make decisions
How to manage information overload
How to keep up to date
25
“Better is possible.It does not take genius, it takes diligence, it takes a clarity of purpose, it takes ingenuity, it takes a willingness to try.”
www.gawande.com/
So is there a better way????
26
What do we know about how people make decisions?
• Behavioural economics and cognitive psychology:
– Bounded rationality (Herbert Simon 1978)
– Dual process theory (Dan Kahneman 2002)
– Most decisions are informed by brief reading and talking to other people
27
How can work with this?
• We all need a system for keeping up to date:
– Hunting: find the best possible answer to a specific question and recognise it as such, quickly and efficiently.
– Foraging: be alerted to new, important, relevant, valid information that requires a change in practice
– Hot synching: update your brain once or twice a year on the 30-40 conditions you see most frequently
28
Pre-digested sources of evidence from trusted sources:Public-sector ethos
Published methodology of how producedTranslation of evidence into practiceContext of the rest of the evidence
29
Finding the ‘best answer’, first timeSlawson DC and Shaughnessy AF
Cochrane LibraryNICE etc
EBM DTB MeReC
“Ivy League” journals
Clinical EvidenceInfoPOEMs, CKSBestTreatments
Textbooks
Usefulness
Medline, Google scholar
30
Medicines awareness
www.nice.org.uk/mpc
31
Medicines Awareness Service
32
Prescribing support inc.• NICE CKS• Awareness service:
– MAD, MAW, MECs
– Evidence summaries
• British National Formulary:– Book– Web– Apps
• Good practice guidance• Key therapeutic topics for QIPP• NICE Evidence• Pathways• eLearning tools for NICE Guidance
33
BNFwww.nice.org.uk/mpc
• Monthly updates for digital versions
• Book going to once a year• User research• Exploring ways to develop
content and integrate content with NICE CKS, NICE Guidance/pathways etc
34
• Give up on reading primary research:
– You don’t have time and you wont be able to do it
• Design your CPD based on these principles:
– Foraging:• 2-3 bits of key new research, summarised for you and set in the context of the
rest of the evidence (MEC)• Evidence awareness service e.g. NICE MPC Medicines Awareness Daily Weekly
– Hot synching• Be aware when new NICE/SIGN guidance comes out• Take time to digest it, talk to others about it• Ask yourself, “what are the important changes since I last updated my brain?”• ONLY about conditions YOU see commonly
– Hunting:• Much more difficult• Use the information pyramid (NHS Evidence/TRIP)
Best use of a clinician’s ‘Golden Hour’?
35
Cognitive Reflective Test
• The test distinguishes intuitive (system 1) from analytical (system 2) processing….
• …….the ability to resist first response that comes to mind
• Of 3,428 people tested only 17% got all 3 correct
• 33% answered all three incorrectly
Frederick 2002 (MIT)
36
TT
Teaching “Think as well as blink”?
37
Better is possible
• Self-awareness (meta-cognition)
– “the right system at the right time”
• Information management
• Teaching “Think as well as blink”?
38
• Decision = Experiences + Evidencesystem 1 system 2
• Usual practice:
– Decision = Experiences + Evidence
• Making decisions better:
– Decision = Experiences + Evidence
39
If you want more…..www.npc.nhs.uk/evidence
40
www.NPC.nhs.uk
InnovAiT: Autumn / Winter 2009-2010
