Date post: | 17-Jul-2016 |
Category: |
Documents |
Upload: | atika-musfirah |
View: | 35 times |
Download: | 3 times |
Dr. Erica Roebbelen
January 16, 2015
Banda Aceh
CRITICAL APPRAISAL
The process of systematically examining research evidence to assess validity, results and relevance before using it to inform a decision
Why do you think this is important?
WHAT IS CRITICAL APPRAISAL?
Research articlesExpert opinion – from our seniors, from staff
physiciansCase reportsAnecdotes or storiesPopular media
WHAT TYPES OF EVIDENCE DO WE ENCOUNTER?
Case-control study
REVIEW TYPES OF STUDY DESIGN
• Cohort Study• Randomized Control Trial
• Meta-analysis• Qualitative
Studies
Observational or experimental
What types of studies give us the strongest evidence?
ASSESSING EVIDENCE
ASSESSING EVIDENCE
When reading a study, always ask yourself these 4 questions:
Who is the POPULATION or PATIENT CHARACTERISTICS?
What is the INTERVENTION?What is the COMPARISON or CONTROL group?What is the OUTCOME being measured?
Remember… PICOThese are also the steps to take to formulate a
research question
APPROACH TO CRITICAL APPRAISAL
Once you know the basics of the study (the PICO), you need to ask these 3 questions:
1. Are the results of the study valid?
2. What are the results?
3. Will the results help me in caring for my patients?
APPROACH TO CRITICAL APPRAISAL
What is validity? The degree to which the outcome observed in the study can be attributed to the intervention.
ASSESSING VALIDITY
5 questions to ask about the validity of primary studies:1. Were the patients randomized?2. Was the follow-up of patients sufficiently
long enough and complete?3. Were all patients analyzed in the groups to
which they were randomized? (intention to treat analysis)
4. Were the groups treated equally except for the intervention?
5. Were the patients and clinicians kept blind to treatment? (was the study “double blinded”?)
ASSESSING VALIDITY
Per-Protocol Analysis (PP)Strategy of analysis in which only patients who complete the entire study are counted towards the results
Intention to Treat Analysis (ITT)When groups are analyzed exactly as they existed upon randomization (i.e. using data from all patients, including those who did not complete the study)
ANALYSIS
Some more questions to consider:
Were the groups similar at the start of the trial? (think about gender, comorbidities, age, etc.)
Were the appropriate and valid exposure and outcome measures obtained?
Were outcome assessors aware of group allocation?
Was the study conducted in an ethical way?Who funded the study?Do the authors have any conflicts of interest?
ASSESSING VALIDITY
Internal Validity the degree to which the findings of the sample truly represent the findings in the study population
External Validitydegree to which the results of the study can be generalized to other situations or populations
ASSESSING VALIDITY
Once you know the basics of the study (the PICO), you need to ask these 3 questions:
1. Are the results of the study valid?
2. What are the results?
3. Will the results help me in caring for my patients?
APPROACH TO CRITICAL APPRAISAL
What was the impact of the treatment results?
How precise was the estimate of the treatment effect?
What were the confidence intervals and power of the study?
CRITICAL APPRAISAL OF RESULTS
Confidence Interval:provides a range of values within which the true population result (e.g. the mean) lies
frequently reported as 95% CI (e.g. one can be 95% certain that the true value is within this data range)
bounded by the upper and lower confidence limits
A wider confidence interval implies more variance than a tighter confidence interval
A VERY QUICK STATISTICS REVIEW
Power the probability of finding a specified difference to be statistically significant at a given p-value
power increases with an increase in sample size
the probability of a true positive result
A VERY QUICK STATISTICS REVIEW
Relative Risk Reduction (RRR)Absolute Risk Reduction (ARR)Number Needed to Treat (NNT)Number Needed to Harm (NNH)
EFFECTIVENESS OF INTERVENTIONS
EFFECTIVENESS OF A TEST
• SPIN: Specificity – use a SPecific test to rule IN a hypothesis. Specific tests have very few false positives.
• SNOUT: Sensitivity – use a SENsitive test to rule OUT a hypothesis. Sensitive tests have few false negatives.
Once you know the basics of the study (the PICO), you need to ask these 3 questions:
1. Are the results of the study valid?
2. What are the results?
3. Will the results help me in caring for my patients?
APPROACH TO CRITICAL APPRAISAL
are the results clinically significant?can I apply the results to my patient
population?were all clinically important outcomes
considered?are the likely treatment benefits worth the
potential harm and costs?
WILL THE RESULTS HELP ME IN CARING FOR MY PATIENTS?