The Fraser Health Authority (“FH”) authorizes the use, reproduction and/or modification of this publication for purposes other than commercial redistribution. In consideration for this authorization, the user agrees that any unmodified reproduction of this publication shall retain all copyright and proprietary notices. If the user modifies the content of this publication, all FH copyright notices shall be removed, however FH shall be acknowledged as the author of the source publication.
Reproduction or storage of this publication in any form by any means for the purpose of commercial redistribution is strictly prohibited.
This publication is intended to provide general information only, and should not be relied on as providing specific healthcare, legal or other professional advice. The Fraser Health Authority, and every person involved in the creation of this publication, disclaims any warranty, express or implied, as to its accuracy, completeness or currency, and disclaims all liability in respect of any actions, including the results of any actions, taken or not taken in reliance on the information contained herein.
1.1. Learn chart review concepts and Learn chart review concepts and applications of chart review in health applications of chart review in health researchresearch
2.2. Learn best practices in designing and Learn best practices in designing and conducting a retrospective chart reviewconducting a retrospective chart review
9:30 9:30 –– 10:30 10:30 Designing and Conducting Designing and Conducting Chart ReviewsChart Reviews
10:30 10:30 –– 10:45 10:45 BreakBreak
10:45 10:45 –– 12:00 12:00 Designing and Conducting Designing and Conducting Chart Reviews Chart Reviews
55
66
Chart ReviewChart ReviewUtilizing data/information that was Utilizing data/information that was originally collected for a different originally collected for a different original purposes (e.g. administrative, original purposes (e.g. administrative, recordrecord--keeping)keeping)
““secondary datasecondary data””
Applications in:Applications in:ResearchResearchEvaluationEvaluationQuality improvement / Practice auditsQuality improvement / Practice audits
77
Chart Review ConceptsChart Review Concepts
88
EE XX 1:1:
RR EE SS EE AA RR CC HH
99
EE XX 2:2:
EE VV AA LL UU AA TT II OO NN
1010
Research MethodResearch Method
Chart reviewChart review
Survey / questionnairesSurvey / questionnaires
Prospective data collectionProspective data collectionClinical dataClinical data
Use of databasesUse of databases
1111
Experimental studies
Randomised controlled trial
Non-randomised controlled trial
YesNo
Random allocation
Observational studies
Analytical studyDescriptive study
YesNo
Comparison group
Exposure↓
Outcome
Exposure↑
Outcome
Cohort study
Case-control study
Ecologicalstudy
Case series
Cross-sectionalstudy
1 or few
Group level information
Individual level information
Many
Research Method vs. Study DesignResearch Method vs. Study Design
Cost (at FHA Health Records)Cost (at FHA Health Records)Funded studies: $5 per chartFunded studies: $5 per chartUnfunded studies: Unfunded studies: ““Ask nicely/begAsk nicely/beg””
Research study feasibilityResearch study feasibilityReadily available dataReadily available dataAvoid lag time in waiting for health outcomes to occurAvoid lag time in waiting for health outcomes to occurAccess to rare cases or occurrencesAccess to rare cases or occurrences
1313
Limitations of Chart Reviews Limitations of Chart Reviews
External ValidityExternal ValidityGeneralizabilityGeneralizability of of resultsresults
“We (the journal) are dubious about the integrity of
retrospective chart review studies and therefore cannot accept your manuscript for
publication.”
Authors report in a retrospective way about perfusion reactions on
infliximab The retrospective nature of this
work has its flaws.
Methodology does not outline how the retrospective chart review was standardized. Is having a single rheumatologist evaluating the charts and, more particularly, the
relation of the adverse event to the infliximab infusion sufficient? This is suboptimal methodology.
1414
Limitations of Chart Reviews Limitations of Chart Reviews
External ValidityExternal ValidityGeneralizabilityGeneralizability of of resultsresults
Designing your research
Collecting appropriate data, appropriate analysis
Discussion section (of manuscript)
1515
Chart Reviews in Health ResearchChart Reviews in Health Research
Medline Search , February 23, 2011Medline Search , February 23, 2011
1616
Chart Reviews in Health ResearchChart Reviews in Health Research
Efficacy and costEfficacy and cost--effectiveness of a blood salvage effectiveness of a blood salvage system in primary total knee system in primary total knee arthroplastyarthroplasty----aa retrospective retrospective matchmatch--controlled chart reviewcontrolled chart review..
Factors determining the persistence or recurrence of Factors determining the persistence or recurrence of wellwell--differentiated thyroid cancer treated by differentiated thyroid cancer treated by thyroidectomythyroidectomy and/or radioiodine in the Boston and/or radioiodine in the Boston Massachusetts are: Massachusetts are: a retrospective chart reviewa retrospective chart review
Nurse practitioner management of chronic Nurse practitioner management of chronic musculoskeletal pain: musculoskeletal pain: a chart reviewa chart review
1717
……Designing and Designing and Conducting Doing a Chart Conducting Doing a Chart
ReviewReview
1818
Are you ready for a plate of spaghetti?
1919
Framework(sFramework(s) for Chart Reviews) for Chart Reviews99--Step ProcessStep Process
1. Conception1. Conception2. Literature review2. Literature review3. Proposal development3. Proposal development4. Data abstraction 4. Data abstraction instrument organizationinstrument organization5. Abstraction protocols and 5. Abstraction protocols and guidelinesguidelines6. Data abstraction6. Data abstraction7. Sample size justification7. Sample size justification8. Ethics8. Ethics9. Pilot study9. Pilot study
FH FrameworkFH Framework1. Generate idea1. Generate idea2. Conduct literature review2. Conduct literature review3. Refine research question3. Refine research question4. Plan research methodology4. Plan research methodology
5. Create research protocol5. Create research protocol
6. Apply for funding6. Apply for funding7. Apply for ethics 7. Apply for ethics 8. Collect and analyze data8. Collect and analyze data9. Draw conclusions9. Draw conclusions
Gearing. J Can Acad Child Adolesc Psychiatry 2006;15(3):126-134.
2020
Excellent ResourceExcellent Resource……
2121
1. Generate Research Idea1. Generate Research Idea
Clinical observation, curiosityClinical observation, curiosityOperationalizeOperationalize into research questioninto research question
PICO MethodPICO Method• Translate clinical problem (or population/public
health problem) into a structured question and identify the key concepts
• ElementsPatient: Who are you studying?Intervention: What intervention are you studying?Comparison: What will the intervention be compared to?Outcome: What outcome(s) are you interested in?
2222
Group Discussion 1Group Discussion 1
What are your research questions that What are your research questions that may be addressed using chart reviews?may be addressed using chart reviews?
2323
1. Generate Research Idea1. Generate Research Idea
Consider feasibility issuesConsider feasibility issuesAvailability of dataAvailability of data•• Is the information you are seeking available from Is the information you are seeking available from
the medical record and is the chart information the medical record and is the chart information likely to be useful in answering your question?likely to be useful in answering your question?
•• This step is often overlooked or not thoroughly This step is often overlooked or not thoroughly undertaken.undertaken.
Consult with peers and expertsConsult with peers and expertsPreliminary review (1Preliminary review (1--3 charts)3 charts)
2424
2. Conduct Literature Review2. Conduct Literature Review
Identify available evidenceIdentify available evidenceHow is the patient How is the patient populationpopulation defined?defined?What What interventionsinterventions have been studied?have been studied?What have been used as What have been used as comparatorscomparators??What are reported What are reported outcomeoutcome measures?measures?What methodologies have been used?What methodologies have been used?
Takes time butTakes time but……Very important step in the research process.Very important step in the research process.You have resources and supportYou have resources and support
•• DERS DERS –– formulating research questionformulating research question•• FH librarians FH librarians –– executing literature searchexecuting literature search
2525
3. Refine Research Question3. Refine Research Question
Confirmed through literature search and review
2626
2727
4. Plan Research Methodology4. Plan Research Methodology
A.A. Study sample Study sample How many? How? How many? How? B.B. Data abstraction instrument organizationData abstraction instrument organizationC.C. Abstraction protocols and guidelinesAbstraction protocols and guidelinesD.D. Data abstractionData abstractionE.E. Dataset considerationsDataset considerationsF.F. AnalysisAnalysis
2828
4A. Study Sample4A. Study Sample
How many charts will How many charts will you to look at?you to look at?Descriptive studyDescriptive study
Consider Consider •• Sampling methodsSampling methods•• ““Bounds of your studyBounds of your study””
Convenience samplingConvenience samplingCharts selected over specific time frameCharts selected over specific time frameIf rare disease, all pertaining chartsIf rare disease, all pertaining charts
Quota samplingQuota samplingPredetermined number of charts from each Predetermined number of charts from each site, diagnostic determinantsite, diagnostic determinant
Systematic samplingSystematic samplingEvery nEvery nthth chart (by date of diagnosis, by chart (by date of diagnosis, by ordered patient list)ordered patient list)
What charts will you reviewOutpatient charts, hospital charts
Who will review charts/abstract dataWho will review charts/abstract dataResearch assistant/coordinatorResearch assistant/coordinatorStudy investigatorsStudy investigators*Do they need clinical background?**Do they need clinical background?*
3131
4B. Abstraction Protocols & Guidelines4B. Abstraction Protocols & GuidelinesWhat information will you collect?
List data (should have identified when preparing your protocol)Where to find in the chart
How will data be collected / stored?How will data be collected / stored?Data Abstraction ToolData Abstraction Tool
Goal is to be able to address:Goal is to be able to address:How will methods be standardized?How will methods be standardized?
3333
Data Abstraction Tool (DAT)Data Abstraction Tool (DAT)
Best practicesBest practicesHave list of data that you want to collectHave list of data that you want to collectDevelop DAT along with protocol Develop DAT along with protocol Keep DAT items and instructions clear and conciseKeep DAT items and instructions clear and conciseDesign DAT to follow the data flow from the Design DAT to follow the data flow from the perspective of the person completing it, taking into perspective of the person completing it, taking into account the flow and organization of data in the chartaccount the flow and organization of data in the chart
•• Keep chart handy while developing DATKeep chart handy while developing DAT
Design DAT with the primary measures of interest in Design DAT with the primary measures of interest in mind as the main goal of data collection. mind as the main goal of data collection. Design DAT with accompanying data collection Design DAT with accompanying data collection instruction book, and data dictionary.instruction book, and data dictionary.
3434
DAT Design ConsiderationsDAT Design Considerations
Know your chartsKnow your chartsDAT should be designed keeping in mind chartDAT should be designed keeping in mind chartKeep chart handyKeep chart handy
DAT featuresDAT featuresFeasibleFeasible
•• Capable of being completedCapable of being completed
AcceptableAcceptable•• Resulting data is usefulResulting data is useful
ReliableReliable•• Information abstracted is consistentInformation abstracted is consistent
3535
Know your Charts!Know your Charts!
Static DataStatic DataSnapshot in timeSnapshot in timeDemographicDemographicMedical historyMedical history
Evolving DataEvolving DataInformation is collected Information is collected over timeover timeRepeated measurementRepeated measurementVital signsVital signsDaily Medication ordersDaily Medication orders
Cumulative DataCumulative DataCollected over time, but Collected over time, but not linked to specific not linked to specific points in timepoints in timeMedication errorsMedication errors
Single record - single page
Series of single records/pages per time interval
Single page cumulative log linked with time interval
Single page cumulative log
3636
DAT: FeasibleDAT: Feasible
Have logical order of items in DATHave logical order of items in DATMatch with order and type of information in the Match with order and type of information in the chartchart
Consider Consider ‘‘modulesmodules’’ to match chart informationto match chart informationLab values, pharmacyLab values, pharmacy
Goal is to facilitate data entry into DAT and minimize Goal is to facilitate data entry into DAT and minimize errorserrors
3737
DAT: Acceptable DAT: Acceptable
PrePre--screen charts for availability of informationscreen charts for availability of information
Have detailed instructions for recording potentially Have detailed instructions for recording potentially ambiguous informationambiguous information
Distinguish between missing information, not Distinguish between missing information, not appropriate information, or not doneappropriate information, or not done
Goal is to minimize and manage of missing Goal is to minimize and manage of missing informationinformation
3838
DAT: AcceptableDAT: Acceptable
Minimize errorMinimize errorLimit manual entry of numbers or textLimit manual entry of numbers or text•• ““ForceForce”” entryentry
Standardize response optionsStandardize response options•• Use boxes/shapes to enter information Use boxes/shapes to enter information •• Be consistentBe consistent•• Choose ONE OF Choose ONE OF
circled itemscircled itemscheckmarkscheckmarksXX
•• Ensure DAT is easy to readEnsure DAT is easy to read•• Ensure sufficient spaceEnsure sufficient space
3939
What would you use? What would you use?
LDL cholesterol value _____LDL cholesterol value _____
LDL cholesterol value _____mg/LDL cholesterol value _____mg/dLdL
LDL cholesterol value _____mg/LDL cholesterol value _____mg/dLdL or ______or ______mmolmmol/L/L
LDL cholesterol value ______LDL cholesterol value ______ mg/mg/dLdLunit if different ______unit if different ______
4040
Make good use of instructions, cues, and examplesMake good use of instructions, cues, and examples
TextTextSite of injury (e.g. left shoulder)____________Site of injury (e.g. left shoulder)____________
Standardizing DAT ResponsesStandardizing DAT Responses
4141
For categorical variables specify whether it is single For categorical variables specify whether it is single response, multiple response or coded responseresponse, multiple response or coded response
Single response Single response (Check (Check onlyonly one)one)OOYesYes OONoNo
MultipleMultiple--response response (May (May circlecircle more than one)more than one)Diabetes Arthritis Cancer HypertensionDiabetes Arthritis Cancer Hypertension
Assess consistency within the same personAssess consistency within the same person•• To assess drift over timeTo assess drift over time•• To assess errors during learning phaseTo assess errors during learning phase
InterInter--raterrater reliabilityreliabilityDo any two data abstractors record the same Do any two data abstractors record the same information?information?•• Test data abstractors against each otherTest data abstractors against each other•• Test data abstractors against gold standard Test data abstractors against gold standard
exampleexample
4343
DAT Pilot Testing DAT Pilot Testing ForFor DesignDesign
Consider looking at a few chartsConsider looking at a few chartsSee what is documentedSee what is documented•• Will tell you what you can captureWill tell you what you can capture
Get an idea of what the hurdles are?Get an idea of what the hurdles are?•• E.g. details of when antibiotic reached the ward E.g. details of when antibiotic reached the ward
were not availablewere not available
Find out where the information actually isFind out where the information actually is•• i.ei.e what section of the chartwhat section of the chart•• This helps in planning time to complete the workThis helps in planning time to complete the work
4444
DAT Pilot Testing DAT Pilot Testing ForFor ValidityValidity
Once DAT is designed and Codebook is in place Once DAT is designed and Codebook is in place –– test test out the data abstraction and coding process on a few out the data abstraction and coding process on a few charts:charts:
Are key pieces of information missing?Are key pieces of information missing?Does the information translate easily from chart to DAT?Does the information translate easily from chart to DAT?Are there classifications, codes or other information that need Are there classifications, codes or other information that need refining?refining?
A formal pilot study to test the feasibility, reliability and A formal pilot study to test the feasibility, reliability and validity of the data abstraction process is recommended.validity of the data abstraction process is recommended.
Sample size of up to 10 percent of the intended sampleSample size of up to 10 percent of the intended sample
4545
Group ExerciseGroup Exercise
Critique the DATCritique the DAT
4646
Paper vs. Electronic Paper vs. Electronic DATsDATs
Consider pros and cons of each approachConsider pros and cons of each approach
Reference manualReference manualClear instructions for how to collect the Clear instructions for how to collect the required informationrequired informationListing of each variable, location in Listing of each variable, location in chart, method for transcribing from chart, method for transcribing from chart to DATchart to DAT
Framework(sFramework(s) for Chart Reviews) for Chart Reviews99--Step ProcessStep Process
1. Conception1. Conception2. Literature review2. Literature review3. Proposal development3. Proposal development4. Data abstraction 4. Data abstraction instrument organizationinstrument organization5. Abstraction protocols and 5. Abstraction protocols and guidelinesguidelines6. Data abstraction6. Data abstraction7. Sample size justification7. Sample size justification8. Ethics8. Ethics9. Pilot study9. Pilot study
FH FrameworkFH Framework1. Generate idea1. Generate idea2. Conduct literature review2. Conduct literature review3. Refine research question3. Refine research question4. Plan research methodology4. Plan research methodology
5. Create research protocol5. Create research protocol
6. Apply for funding6. Apply for funding7. Apply for ethics 7. Apply for ethics 8. Collect and analyze data8. Collect and analyze data9. Draw conclusions9. Draw conclusions
Gearing. J Can Acad Child Adolesc Psychiatry 2006;15(3):126-134.
5353
7. Apply for Ethics Approval7. Apply for Ethics Approval
Need to complete Department Agreement Need to complete Department Agreement for Providing Researchfor Providing Research--Related Services Related Services (DAR) form(DAR) form
Health records Health records eepartmenteepartmentLink to DAR form:Link to DAR form:http://research.fraserhealth.ca/approvals_%26_ethichttp://research.fraserhealth.ca/approvals_%26_ethic
