Audit: what works and what doesn’t
Dr Jo HayesConsultant in Palliative Medicine
Marie Curie Hospice, PenarthCourse Tutor, Diploma in
Palliative Medicine / Care, Cardiff University
Overview Overview
►What audit isWhat audit is►What audit is notWhat audit is not►Why perform auditWhy perform audit►The audit cycleThe audit cycle►Standard setting Standard setting ►Types of auditTypes of audit►Good and bad Good and bad
Definition of auditDefinition of audit
► Clinical auditClinical audit is a quality improvement is a quality improvement process that seeks to improve patient care process that seeks to improve patient care and outcomes through systematic review of and outcomes through systematic review of care against explicit criteria and care against explicit criteria and implementation of changeimplementation of change
NICE: Principles for best practice in clinical audit. 2002NICE: Principles for best practice in clinical audit. 2002
► Introduced in the UK in the 1980sIntroduced in the UK in the 1980s► Part of the clinical governance framework in Part of the clinical governance framework in
the UKthe UK
History of auditHistory of audit► Crimean War 1850sCrimean War 1850s
soldiers 7 x risk of death soldiers 7 x risk of death from cholera & typhus from cholera & typhus than from battle injuriesthan from battle injuries
► Mortality decreased Mortality decreased dramatically due to:-dramatically due to:- Hygiene measures Hygiene measures Fresh waterFresh water Fresh fruit and vegFresh fruit and veg Hospital equipmentHospital equipment
What audit isWhat audit is
► A measure of A measure of compliance with compliance with defined best defined best practicepractice
► Do we know what Do we know what we should be doing?we should be doing?
► Are we doing it?Are we doing it?
What audit is notWhat audit is not
► What should What should we be doing? - we be doing? - researchresearch
► What are we What are we doing? – survey doing? – survey / review etc/ review etc
Audit vs ResearchAudit vs Research
► Tests care given Tests care given against known best against known best practicepractice
► Measures against Measures against standardsstandards
► Normal clinical Normal clinical practicepractice
► No randomisation or No randomisation or placeboplacebo
► Most relevant locallyMost relevant locally
► Creates new Creates new knowledgeknowledge
► Tests hypothesesTests hypotheses
► ExperimentalExperimental► May involve May involve
randomisation or randomisation or placeboplacebo
► GeneralisableGeneralisable
Why perform audit?Why perform audit?
To provide a high quality service and improve To provide a high quality service and improve patient carepatient care
► Compulsory for all UK doctorsCompulsory for all UK doctors► Royal colleges require ongoing departmental Royal colleges require ongoing departmental
audit to be in placed before approving audit to be in placed before approving training poststraining posts
► Meetings are educational Meetings are educational ► Encourage teamworkEncourage teamwork► Protected timeProtected time
The Audit CycleThe Audit Cycle
Set standardsObserve practice
Identify topic / issues and agree importance
Compare practice with standard
Review current practice and implement change
Compare new practice with standard
Reflect / consider new standards
What can you audit ?What can you audit ?
►Structure: premises, staffing, Structure: premises, staffing, equipment equipment
►Process: waiting times, investigationsProcess: waiting times, investigations►Outcomes:Outcomes: survival, improved survival, improved
symptoms, achieving desired place of symptoms, achieving desired place of death, improved quality of life etcdeath, improved quality of life etc
‘‘Not all that can be counted matters and Not all that can be counted matters and not all that matters can be counted.’not all that matters can be counted.’
Audit standards Audit standards
SMART StandardsSMART Standards
► SSpecific (clear, unambiguous and jargon-free)pecific (clear, unambiguous and jargon-free)► MMeasurable – and with a target attached e.g. easurable – and with a target attached e.g.
100% 100% ► AAgreed (by all concerned with delivering that greed (by all concerned with delivering that
aspect of care)aspect of care)► RRelevantelevant► TTheoretically sound (based on evidence heoretically sound (based on evidence
about best practice, reviewed and updated as about best practice, reviewed and updated as new evidence becomes available)new evidence becomes available)
UBHT Central Audit OfficeUBHT Central Audit Office
Writing standards Writing standards
Audit Criteria Audit Criteria
(Standard (Standard statements)statements)
TargeTargett
ExceptionsExceptions Source of Source of EvidenceEvidence
InstrumentInstruments for data s for data collectioncollection
‘‘Patients Patients referredreferred
to the to the communitycommunity
SPCT for urgentSPCT for urgent
review ofreview of
uncontrolleduncontrolled
symptoms symptoms shouldshould
be assessedbe assessed
within 2 days ofwithin 2 days of
referral.’referral.’
100%100% Patient choicePatient choice
Patient Patient outside of outside of community community team remit team remit e.g. hospice / e.g. hospice / acute hospitalacute hospital
All WalesAll Wales
NationalNational
Standards for Standards for
Specialist Specialist
Palliative CarePalliative Care
Cancer ServicesCancer Services
20052005
RetrospectivRetrospective review of e review of patient notes patient notes against against referral referral forms. forms.
UBHT Central Audit OfficeUBHT Central Audit Office
TargetsTargets
► Often set at 100% (or 0%)Often set at 100% (or 0%)
► Exceptions to criteria can allow 100% standard Exceptions to criteria can allow 100% standard ► May set a target below 100% to be realistic for May set a target below 100% to be realistic for
a first audit in an aspect of care where you a first audit in an aspect of care where you know that your team performs badly.know that your team performs badly.
► Figures suggested by research literature could Figures suggested by research literature could be set as a target e.g. percentage deaths at be set as a target e.g. percentage deaths at home home
Identify problem & agree need for Identify problem & agree need for auditaudit
►Team or own ideas of area/issue of Team or own ideas of area/issue of concern or with potential for concern or with potential for improvementimprovement
Staff on surgical ward 10 at Memorial Staff on surgical ward 10 at Memorial Hospital say all their palliative care Hospital say all their palliative care patients are nauseated / GPs report patients are nauseated / GPs report patients discharged with uncontrolled patients discharged with uncontrolled nauseanausea
Determine a standardDetermine a standard
► SMARTSMART► Qualified: existing (search literature) or guided Qualified: existing (search literature) or guided
by experts (ask) or agreed by team (discuss) by experts (ask) or agreed by team (discuss) ► Quantified: standards should have a percentage Quantified: standards should have a percentage
attached attached
90% patients on ward 10 should be asked about 90% patients on ward 10 should be asked about nausea as part of their initial assessment nausea as part of their initial assessment
75% patients should have acceptable relief within 75% patients should have acceptable relief within 48hrs of treatment for nausea48hrs of treatment for nausea
Measure current practiceMeasure current practice
►Retrospective/prospective (time factor)Retrospective/prospective (time factor)►Tools, scales, questionnairesTools, scales, questionnaires
Trawl patients’ notes from ward 10 for Trawl patients’ notes from ward 10 for medical/nursing entry about nausea medical/nursing entry about nausea (bias?) – define time-scale(bias?) – define time-scale
Trawl drug charts for antiemetic (bias?)Trawl drug charts for antiemetic (bias?)
Compare practice with Compare practice with standardstandard
► Simple arithmeticSimple arithmetic► If standard already met, was standard too If standard already met, was standard too
low?low?► Consider extent of likely improvementConsider extent of likely improvement
40% of patients on ward 10 had a record of 40% of patients on ward 10 had a record of enquiry about nausea in their notesenquiry about nausea in their notes
24% of nauseated patients achieved relief 24% of nauseated patients achieved relief within 48hrswithin 48hrs
Implement change & run for a Implement change & run for a timetime
►Team working needed ++Team working needed ++►Education / forms / posters / guidelines Education / forms / posters / guidelines
Barriers/cascading/resistance/ownershipBarriers/cascading/resistance/ownership►Agree time to allow change to take effectAgree time to allow change to take effect
Ward 10: teaching sessions - need to enquire about Ward 10: teaching sessions - need to enquire about nausea & management of nausea/vomiting to nausea & management of nausea/vomiting to nurses and junior drs; simple guidelines nurses and junior drs; simple guidelines flowchart devised and displayedflowchart devised and displayed
Compare modified practice Compare modified practice with standardwith standard
►Repeat survey – same time frame Repeat survey – same time frame as first viewas first view
►Arithmetic comparisonsArithmetic comparisons
63% of patients on ward 10 had a record 63% of patients on ward 10 had a record of enquiry about nausea in their notesof enquiry about nausea in their notes
18% of nauseated patients achieved 18% of nauseated patients achieved relief within 48hrsrelief within 48hrs
Reflect and consider future Reflect and consider future plansplans
►This part is very important in the This part is very important in the project…and in lifeproject…and in life
►Barriers / cascading / resistance / Barriers / cascading / resistance / ownershipownership
Ward 10 - Standards not metWard 10 - Standards not metBetter rate of enquiry about nausea Better rate of enquiry about nausea Guidelines not followed/prescribing failure as Guidelines not followed/prescribing failure as
junior doctors rotated between teaching and junior doctors rotated between teaching and re-auditre-audit
Diploma audit proposal formDiploma audit proposal form
What makes a good audit?What makes a good audit?
►Relevant topic / team ownershipRelevant topic / team ownership►Multi-professional involvementMulti-professional involvement►SMART standardsSMART standards►Good methodologyGood methodology►Change happensChange happens►Change benefits the patients / carers / Change benefits the patients / carers /
staff staff ►Complete the cycleComplete the cycle►Acknowledge the limitationsAcknowledge the limitations
What makes a bad audit?What makes a bad audit?
►Audit for audit’s sakeAudit for audit’s sake►Seen as a chore Seen as a chore ►No standards / survey / researchNo standards / survey / research►Poor data collectionPoor data collection►Unrealistic time framesUnrealistic time frames
► Inadequate time to Inadequate time to do a good jobdo a good job