EVALUATING  After retrieving the literature, you have to

evaluate or critically appraise the evidence for its validity and applicability to your patient and adherence to truth. 

This often requires knowledge of basic statistics and a familiarity with the terminology of EBM …

Example: positive predictive value likelihood ratio number needed to treat (NNT)

Physicians do critical appraisals for cases that they see quite often in their practice

Where as for cases that are less frequently seen they conserve their time by seeking out critical appraisals already done by others such as Cochrane, Best Evidence, etc. 

For those cases that are very infrequently seen in their clinics, they seek advice from a colleague who is an authority

in this specialty.

Levels of Evidence Depends on the Type of Research Methods Used– Strong evidence from at least 1 systematic review of

multiple well-designed RCT.– Strong evidence from at least 1 properly-designed RCT of

appropriate size.– Evidence from well-designed trials without randomization,

or case-control studies.– Evidence from well-designed non-experimental studies

from more than 1 center or research group.– Opinions of respected authorities, based on clinical

evidence, descriptive studies or reports of expert committees.

Evidence Pyramid

– The base has the largest number of literature studies and provides the least strength of evidence.  

– If you do not find an upper level of evidence, go to the next one. 

– Remember that there may be no good evidence to support clinical judgment.

Animal Research/ In Vitro StudiesExpert Opinion

Case Series/Case Reports

Case Control StudiesCohort Studies

RCT

The “best” evidence for therapy question This is found in double-blind

randomized controlled trials

The “best” evidence for diagnosis question This is found in controlled trials of

prospective studies that compare tests with a “gold standard” test

patients suspected of disease

Diagnostic Test & GoldStandard

condition present

condition absent

The “best” evidence for etiology question This is found in cohort studies.

The “best” evidence for prognosis question

This is found in cohort or case control studies

patients

Prognostic Factors

Suffer target outcome

time

Does notSuffer target outcome

CRITICALLY APPRAISE the collected literature according to category

Therapy   When evaluating the literature to answer

a therapy question ask yourself:

Was the study randomized and double blind to prevent bias?

Was follow-up > 80%? Were the groups similar at the start of the trial? Were all enrolled patients included in the

conclusion of the study? Was the study valid?  did the authors answer

the question?

Therapy   When evaluating the literature to answer

a therapy question ask yourself:

Do the results present an unbiased estimate of the treatment effect?

How large is the treatment effect? Will the results help my patient? Were the study patients similar to your

patient? Are the benefits worth the harm and cost?

The features common to evaluation of therapeutic interventions:

Random allocation Single, double or triple blind

methods Placebo RCT

Concepts used in therapy studies:

Number Needed to Treat (NNT) Relative Risk Reduction (RRR) Intention To Treat Analysis

Diagnosis  Diagnostic tests are evaluated in a manner to ascertain

which are more accurate, faster, less expensive, less invasive than existing diagnostic tests. 

Good diagnostic tests must provide positive results when the disease is present, and negative results when the patient does not have the disease. 

In contrast to therapeutic evaluations, all persons involved in a new diagnostic test must receive the test. 

The results are compared with the results of the standard  "gold standard" test.

To evaluate a diagnosis question ask yourself:

Did the authors do a blind comparison with a gold standard?

Did patients in the study undergo both the diagnostic test and the gold standard?

Did the paper describe the method for doing the test?

Were the patients tested similar to your patient? Are the results of the test useful? Did the patient sample include an appropriate

spectrum of patients similar to those found in general practice?

The features used in diagnostic studies:

Sensitivity vs. Specificity Positive and Negative Predictive Value False positive and false negative reaction Likelihood Ratio of a positive and negative

test Receiver operator characteristic curve 

(ROC curve)

Etiology Two methods predominate to assess whether

something causes disease: – Cohort study - the one with the strongest

evidence (persons with exposures are followed in time to assess outcomes and the results are compared with a similar group that does not have the exposure)

– Case-control study - the second method for testing etiology which is more common but has less evidence because of a larger potential for bias

To evaluate a etiology question ask yourself

Were the exposures and outcome measured similarly in both groups (exposed and non exposed patients)?

Was the comparison group similar to the outcome group in all aspects except for the variable in question?

Was follow up sufficiently long and complete?

Terms used in search strategy include     - Cohort studies        - Case control studies     - Follow up studies     - Risk

To evaluate a etiology question

Prognosis  Uses cohort studies to see how the

disease is progressing

To evaluate a prognosis question ask yourself:

Was the patient sample selected to reflect a well-defined point in the course of disease?

Was the follow-up adequate and complete (>80%)?

Was there objective and unbiased outcome criteria used?

Terms used in search strategies: Cohort studies Incidence (number of new cases in a

given period of time) Prevalence (number of current cases

at a specific time) Follow-up studies Disease progression 

To evaluate a prognosis question

Applying the results of a study to individual patients

Once you determine that the study methodology is valid, you must then examine the results and their applicability to your patient.

Therapy

Is my patient so different from those in the study group that the results cannot be applied?

According to the study results how much could my patient benefit from the treatment?

Are the treatment and its consequences consistent with my patient's values and beliefs?

Diagnosis 

Is the test affordable, accurate and available locally?

Can it estimate the pretest probability of the disease in question?

Will the posttest probability affect my management?

Etiology  Can the study results be extrapolated

to my patient? What is my patient's risk for adverse

effects? Can my patient's preferences &

expectations be met by an alternative therapy?

Prognosis   

Is my patient similar to the patients in the study group?

Will the evidence alter my choice of treatment?

Final words to remember:

EBM builds on and reinforces but never replaces your clinical judgment or experience.