Evaluating Medical Literature

Evaluating Evaluating Medical Medical

LiteratureLiterature

Dr. Shounak DasDr. Shounak Das

Aug. 26, 2008Aug. 26, 2008

Problem-Solving Using the Problem-Solving Using the Medical LiteratureMedical Literature

• Searching for the answer – 4 types of Searching for the answer – 4 types of clinical questions:clinical questions:

1.1. TherapyTherapy• determine the effect of a treatmentdetermine the effect of a treatment

2.2. HarmHarm• ascertain the effect of exposure to a harmful agentascertain the effect of exposure to a harmful agent

3.3. DiagnosisDiagnosis• establish the power of an intervention to establish the power of an intervention to

distinguish those with and without the target distinguish those with and without the target conditioncondition

4.4. PrognosisPrognosis• estimating the future course of a patient’s diseaseestimating the future course of a patient’s disease

Problem-Solving Using the Problem-Solving Using the Medical LiteratureMedical Literature

• Sources of evidence:Sources of evidence: PrefilteredPrefiltered

• authors have already accumulated best of authors have already accumulated best of published +/- unpublished evidence (i.e. a published +/- unpublished evidence (i.e. a systematic review)systematic review)

• examples = Best Evidence, Cochrane examples = Best Evidence, Cochrane Library, UpToDate, and Clinical EvidenceLibrary, UpToDate, and Clinical Evidence

UnfilteredUnfiltered• MEDLINE = U.S. National Library of MEDLINE = U.S. National Library of

Medicine database (contains primary Medicine database (contains primary studies and reviews)studies and reviews)

• world wide webworld wide web

Evaluation of Primary Evaluation of Primary StudiesStudies

Assessing Therapies Assessing Therapies

• When using the medical literature to When using the medical literature to answer a clinical question, approach answer a clinical question, approach the study using three discrete steps* :the study using three discrete steps* :

1)1) Are the results of the study valid?Are the results of the study valid?

2)2) What are the results?What are the results?

3)3) How can I apply these results to patient How can I apply these results to patient care?care?

*these 3 steps are applicable to primary studies assessing harm, diagnosis, and prognosis as well as therapies; they are also applicable for the evaluation of systematic reviews

1. Are the Results of the 1. Are the Results of the Study Valid?Study Valid?

• Assessing validity requires Assessing validity requires answering 2 further questions:answering 2 further questions:

a)a) Did experimental and control Did experimental and control groups begin the study with a groups begin the study with a similar prognosis?similar prognosis?

b)b) Did experimental and control Did experimental and control groups retain a similar prognosis groups retain a similar prognosis after the study started?after the study started?

1a. Did Experimental and Control 1a. Did Experimental and Control Groups Begin the Study with a Groups Begin the Study with a

Similar Prognosis?Similar Prognosis?• To determine whether or not the To determine whether or not the

experimental and control groups began experimental and control groups began the study with similar prognoses, we ask the study with similar prognoses, we ask 4 questions:4 questions:

1.1. Were patients randomized?Were patients randomized?

2.2. Was randomization concealed (blinded or Was randomization concealed (blinded or masked)?masked)?

3.3. Were patients analyzed in the groups to which Were patients analyzed in the groups to which they were randomized?they were randomized?

4.4. Were patients in the treatment and control Were patients in the treatment and control groups similar with respect to known groups similar with respect to known prognostic factors?prognostic factors?


Similar Prognosis?Similar Prognosis?

• Were patients randomized?Were patients randomized? randomization is important so that randomization is important so that

the experimental and control the experimental and control groups are matched for both groups are matched for both known and unknown prognostic known and unknown prognostic factorsfactors


Similar Prognosis?Similar Prognosis?• There are many examples of RCT’s There are many examples of RCT’s

contradicting the findings of contradicting the findings of observational studies, i.e.:observational studies, i.e.:

Clinical QuestionObservational Study Findings RCT Findings


Similar Prognosis?Similar Prognosis?• Was randomization concealed (blinded or masked)?Was randomization concealed (blinded or masked)?

Some years ago, a group of Australian investigators Some years ago, a group of Australian investigators undertook a randomized trial of open vs laparoscopic undertook a randomized trial of open vs laparoscopic appendectomy.appendectomy.

At night, the attending surgeon's presence was required At night, the attending surgeon's presence was required for the laparoscopic procedure but not the open one.for the laparoscopic procedure but not the open one.

When an eligible patient appeared, the residents checked When an eligible patient appeared, the residents checked the attending staff and the lineup for the operating room the attending staff and the lineup for the operating room and, depending on the personality of the attending and, depending on the personality of the attending surgeon and the length of the lineup, held the translucent surgeon and the length of the lineup, held the translucent envelopes containing orders up to the light. As soon as envelopes containing orders up to the light. As soon as they found one that dictated an open procedure, they they found one that dictated an open procedure, they opened that envelope.opened that envelope.

If patients who presented at night were sicker than those If patients who presented at night were sicker than those who presented during the day, the residents' behavior who presented during the day, the residents' behavior would bias the results against the open procedure. would bias the results against the open procedure.



• Were patients analyzed in the groups to Were patients analyzed in the groups to which they were randomized?which they were randomized?

This principle of attributing all patients to the This principle of attributing all patients to the group to which they were randomized results in group to which they were randomized results in an an intention-to-treatintention-to-treat analysis, which is analysis analysis, which is analysis of outcomes based on the treatment arm to of outcomes based on the treatment arm to which patients were randomized, rather than which patients were randomized, rather than which treatment they actually received. This which treatment they actually received. This strategy preserves the value of randomization: strategy preserves the value of randomization: prognostic factors that we know about--and those prognostic factors that we know about--and those we do not know about -- will be, on average, we do not know about -- will be, on average, equally distributed in the two groups; and the equally distributed in the two groups; and the effect we see will result simply from the effect we see will result simply from the treatment assigned. treatment assigned.


Similar Prognosis?Similar Prognosis?• Let us assume that the treatment is ineffective Let us assume that the treatment is ineffective

and that the true underlying event rate in both and that the true underlying event rate in both treatment and control patients is 20%. treatment and control patients is 20%.

• If the 20 nonadherent patients are sicker and If the 20 nonadherent patients are sicker and their event rate (60%) is higher, the their event rate (60%) is higher, the nonadherent patients will suffer 12 of the 20 nonadherent patients will suffer 12 of the 20 events destined to occur in the treated patients. events destined to occur in the treated patients.

• If one compares only the adherent patients If one compares only the adherent patients (with an event rate of 8/80, or 10%) with the (with an event rate of 8/80, or 10%) with the control group (event rate 20/100, or 20%), one control group (event rate 20/100, or 20%), one will mistakenly conclude that treatment cuts will mistakenly conclude that treatment cuts the event rate in half. the event rate in half.



20 events

20 events

12 events

8 events

8/80 = 10% vs. 20/100 = 20% 50% RRR in treatment arm

20/100 = 20% vs. 20/100 = 20%0% RRR in treatment arm


Similar Prognosis?Similar Prognosis?• Were patients in the treatment and Were patients in the treatment and

control groups similar with respect to control groups similar with respect to known prognostic factors?known prognostic factors?

Clinicians should look for a display of Clinicians should look for a display of prognostic features of the treatment and prognostic features of the treatment and control patients at the study's control patients at the study's commencement -- the baseline or entry commencement -- the baseline or entry prognostic features. Although we never prognostic features. Although we never will know whether similarity exists for the will know whether similarity exists for the unknown prognostic factors, we are unknown prognostic factors, we are reassured when the known prognostic reassured when the known prognostic factors are well balanced. factors are well balanced.


• Assessing validity requires Assessing validity requires answering 2 further questions:answering 2 further questions:

a)a) Did experimental and control Did experimental and control groups begin the study with a groups begin the study with a similar prognosis?similar prognosis?

b)b) Did experimental and control Did experimental and control groups retain a similar prognosis groups retain a similar prognosis after the study started?after the study started?

1b. Did Experimental and Control Groups 1b. Did Experimental and Control Groups Retain a Similar Prognosis After the Retain a Similar Prognosis After the

Study Started?Study Started?

• To determine whether or not the To determine whether or not the experimental and control groups retain experimental and control groups retain a similar prognosis, we ask 4 questions:a similar prognosis, we ask 4 questions:1.1.Were patients aware of group allocations?Were patients aware of group allocations?

2.2.Were clinicians aware of group allocations?Were clinicians aware of group allocations?

3.3.Were outcome assessors aware of group Were outcome assessors aware of group allocations?allocations?

4.4.Was follow-up complete?Was follow-up complete?



• Were patients aware of group Were patients aware of group allocations?allocations? Patients who take a treatment that they Patients who take a treatment that they

believe is efficacious may feel and perform believe is efficacious may feel and perform better than those who do not, even if the better than those who do not, even if the treatment has no biologic action – the treatment has no biologic action – the placebo effect.placebo effect.

The best way to avoid the placebo effect The best way to avoid the placebo effect skewing the results is to ensure that skewing the results is to ensure that patients are unaware of whether they are patients are unaware of whether they are receiving the experimental treatment. receiving the experimental treatment.



• Were clinicians aware of group allocations?Were clinicians aware of group allocations? If randomization succeeds, treatment and control If randomization succeeds, treatment and control

groups in a study begin with a very similar groups in a study begin with a very similar prognosis. However, randomization provides no prognosis. However, randomization provides no guarantee that the two groups will remain guarantee that the two groups will remain prognostically balanced. Differences in patient prognostically balanced. Differences in patient care other than the intervention (cointerventions) care other than the intervention (cointerventions) under study can bias the results. under study can bias the results.

Effective blinding eliminates the possibility of Effective blinding eliminates the possibility of either conscious or unconscious differential either conscious or unconscious differential administration of effective (co)interventions to administration of effective (co)interventions to treatment and control groups. treatment and control groups.



• Were outcome assessors aware of Were outcome assessors aware of group allocations?group allocations? Unblinded study personnel who are Unblinded study personnel who are

measuring or recording outcomes such measuring or recording outcomes such as physiologic tests, clinical status, or as physiologic tests, clinical status, or quality of life may provide different quality of life may provide different interpretations of marginal findings or interpretations of marginal findings or may offer differential encouragement may offer differential encouragement during performance tests, either one of during performance tests, either one of which can distort results. which can distort results.



• Was follow-up complete?Was follow-up complete? The greater the number of patients who The greater the number of patients who

are lost to follow-up, the more a study's are lost to follow-up, the more a study's validity is potentially compromised. The validity is potentially compromised. The reason is that patients who are lost reason is that patients who are lost often have different prognoses from often have different prognoses from those who are retained. The situation is those who are retained. The situation is completely analogous to the reason for completely analogous to the reason for the necessity for an intention-to-treat the necessity for an intention-to-treat analysis.analysis.




• The final assessment of validity is The final assessment of validity is never a "yes" or "no" decision. never a "yes" or "no" decision. Rather, think of validity as a Rather, think of validity as a continuum ranging from strong continuum ranging from strong studies that are very likely to yield studies that are very likely to yield an accurate estimate of the an accurate estimate of the treatment effect to weak studies treatment effect to weak studies that are very likely to yield a biased that are very likely to yield a biased estimate of effect. estimate of effect.



• When using the medical When using the medical literature to answer a clinical literature to answer a clinical question, approach the study question, approach the study using three discrete steps :using three discrete steps :



3)3) How can I apply these results to How can I apply these results to patient care?patient care?

2. What are the Results?2. What are the Results?

• Once validity of a study is Once validity of a study is assessed, the next step is to assessed, the next step is to analyze the results. This analyze the results. This involves looking at:involves looking at:

a)a) How large was the treatment How large was the treatment effect?effect?

b)b) How precise was the estimate of How precise was the estimate of the treatment effect?the treatment effect?

2a. How Large Was the 2a. How Large Was the Treatment Effect?Treatment Effect?

• Most frequently, randomized clinical trials Most frequently, randomized clinical trials carefully monitor how often patients carefully monitor how often patients experience some adverse event or outcome. experience some adverse event or outcome. Examples of these dichotomous outcomes Examples of these dichotomous outcomes ("yes" or "no" outcomes -- ones that either ("yes" or "no" outcomes -- ones that either happen or do not happen) include cancer happen or do not happen) include cancer recurrence, myocardial infarction, and recurrence, myocardial infarction, and death. Patients either do or do not suffer an death. Patients either do or do not suffer an event, and the article reports the event, and the article reports the proportion of patients who develop such proportion of patients who develop such events. events.


• Even if the outcome is not one of these Even if the outcome is not one of these dichotomous variables, investigators sometimes dichotomous variables, investigators sometimes elect to present the results as if this were the case. elect to present the results as if this were the case. For example, in a study of the use of forced For example, in a study of the use of forced expiratory volume in 1 second (FEVexpiratory volume in 1 second (FEV11) in the ) in the assessment of the efficacy of oral corticosteroids in assessment of the efficacy of oral corticosteroids in patients with chronic stable airflow limitation, patients with chronic stable airflow limitation, investigators defined an event as an improvement investigators defined an event as an improvement in FEVin FEV11 over baseline of more than 20%. over baseline of more than 20%.

• The investigators' choice of the magnitude of The investigators' choice of the magnitude of change required to designate an improvement as change required to designate an improvement as "important" can affect the apparent effectiveness of "important" can affect the apparent effectiveness of the treatment the treatment

2a. How Large Was the Treatment 2a. How Large Was the Treatment Effect?Effect?

• When clinicians consider When clinicians consider the results of clinical the results of clinical trials, they are trials, they are interested in the interested in the association between a association between a treatment and an treatment and an outcome. outcome.

• Consider three different Consider three different treatments that reduce treatments that reduce mortality administered mortality administered to three different to three different populations.populations.

• Although all three treatments reduce the risk of dying by Although all three treatments reduce the risk of dying by a third, this piece of information is not adequate to fully a third, this piece of information is not adequate to fully capture the impact of treatment. Expressing the capture the impact of treatment. Expressing the strength of the association as a relative risk (RR), a strength of the association as a relative risk (RR), a relative risk reduction (RRR), an absolute risk reduction relative risk reduction (RRR), an absolute risk reduction (ARR) or risk difference (RD), an odds ratio (OR), or a (ARR) or risk difference (RD), an odds ratio (OR), or a number needed to treat (NNT) or number needed to number needed to treat (NNT) or number needed to harm (NNH) conveys a variety of different information.harm (NNH) conveys a variety of different information.



28% 45%

Risk

1. In this study, the risk of death in the ligation arm is 18/64 = 28%2. The risk of death in the sclerotherapy arm is 29/65 = 45%

3. The relative risk (RR) of death in the ligation arm is 28%/45% = 63%4. The relative risk reduction (RRR) of death in the ligation arm is (45% - 28%)/45% = 17%/45% = 37%

RR RRR63%37%

•the RRR tells us the proportion of baseline risk that is removed the RRR tells us the proportion of baseline risk that is removed by the therapyby the therapy•using nontechnical language, we would say that ligation using nontechnical language, we would say that ligation decreases the relative risk of death by 37% compared to decreases the relative risk of death by 37% compared to sclerotherapysclerotherapy


(28%)

Risk 28%

45%

ARRNNT

The absolute risk reduction The absolute risk reduction (ARR)(ARR) = 45% – 28% = = 45% – 28% = 17%17% the ARR tells us what proportion of patients are spared the adversethe ARR tells us what proportion of patients are spared the adverse outcome if they receive the experimental therapy, rather than the control therapyoutcome if they receive the experimental therapy, rather than the control therapy

17%

The number needed to treat The number needed to treat (NNT)(NNT) = 1/ARR = = 1/ARR = 66 the NNT is the number of patients one would need to treat to prevent an adverse eventthe NNT is the number of patients one would need to treat to prevent an adverse event

6



(28%)

Risk 28%

45%

Odds OR0.39 0.490.81

The odds of death in the ligation arm is 18/46 = 0.39The odds of death in the sclerotherapy arm is 29/36 = 0.81

The odds ratio (OR) = 0.39 ÷ 0.81 = The odds ratio (OR) = 0.39 ÷ 0.81 = 0.490.49

• the (OR) represents the proportion of patients with the target event divided bythe (OR) represents the proportion of patients with the target event divided by the proportion without the target eventthe proportion without the target event• in most instances in medical investigation, odds and risks are approximately equalin most instances in medical investigation, odds and risks are approximately equal


• Relative Risk and Odds Ratio vs Relative Risk and Odds Ratio vs Absolute Risk Reduction: Why the Absolute Risk Reduction: Why the Fuss?Fuss? distinguishing between OR and RR will distinguishing between OR and RR will

seldom have major importance seldom have major importance we must pay much more attention to we must pay much more attention to

distinguishing between the OR and RR distinguishing between the OR and RR vs the ARR (or its reciprocal the NNT) vs the ARR (or its reciprocal the NNT)


(ARR)

• Forrow and colleagues demonstrated that Forrow and colleagues demonstrated that clinicians were less inclined to treat patients clinicians were less inclined to treat patients after presentation of trial results as the after presentation of trial results as the absolute change in the outcome compared absolute change in the outcome compared with the relative change in the outcome.with the relative change in the outcome.

• The pharmaceutical industry's awareness of The pharmaceutical industry's awareness of this phenomenon may be responsible for their this phenomenon may be responsible for their propensity to present physicians with propensity to present physicians with treatment-associated relative risk reductions. treatment-associated relative risk reductions.

(10%)

(0.5%)

2. What are the Results?2. What are the Results?

• Once validity of a study is Once validity of a study is assessed, the next step is to assessed, the next step is to analyze the results. This analyze the results. This involves looking at:involves looking at:

a)a) How large was the treatment How large was the treatment effect?effect?

b)b) How precise was the estimate of How precise was the estimate of the treatment effect?the treatment effect?

2b. How Precise Was the 2b. How Precise Was the Estimate of the Treatment Estimate of the Treatment

Effect?Effect?• Realistically, the true risk reduction Realistically, the true risk reduction

can never be known. The best we have can never be known. The best we have is the estimate provided by rigorous is the estimate provided by rigorous controlled trials, and the best estimate controlled trials, and the best estimate of the true treatment effect is that of the true treatment effect is that observed in the trial. This estimate is observed in the trial. This estimate is called a point estimate, a single value called a point estimate, a single value calculated from observations of the calculated from observations of the sample that is used to estimate a sample that is used to estimate a population value or parameter.population value or parameter.


Effect?Effect?• Investigators often tell us the Investigators often tell us the

neighborhood within which the true neighborhood within which the true effect likely lies by calculating effect likely lies by calculating confidence intervals.confidence intervals. We usually (though arbitrarily) use the We usually (though arbitrarily) use the

95% confidence interval. 95% confidence interval. You can consider the 95% confidence You can consider the 95% confidence

interval as defining the range that interval as defining the range that includes the true relative risk reduction includes the true relative risk reduction 95% of the time. 95% of the time.

2b. How Precise Was the Estimate 2b. How Precise Was the Estimate of the Treatment Effect?of the Treatment Effect?

• In this hypothetical situation, 2 different-sized studies In this hypothetical situation, 2 different-sized studies with the same RRR of 25% have widely varying with the same RRR of 25% have widely varying confidence intervals.confidence intervals.

• It is evident that the larger the sample size, the It is evident that the larger the sample size, the narrower the confidence interval. narrower the confidence interval.

This study enrolled 1000 patients

This study enrolled 100 patients

(RRR = 0.2595% confidence intervals 0.09 – 0.41)

(RRR = 0.2595% confidence intervals -0.38 – 0.59)


Effect?Effect?• When is the sample size big enough?When is the sample size big enough?

In a positive study -- a study in which the authors In a positive study -- a study in which the authors conclude that the treatment is effective -- if the conclude that the treatment is effective -- if the lowest RRR that is consistent with the study results lowest RRR that is consistent with the study results is still important (that is, it is large enough for you is still important (that is, it is large enough for you to recommend the treatment to the patient), then to recommend the treatment to the patient), then the investigators have enrolled sufficient patients.the investigators have enrolled sufficient patients.

In a negative study -- in which the authors have In a negative study -- in which the authors have concluded that the experimental treatment is no concluded that the experimental treatment is no better than control therapy -- if the RRR at the better than control therapy -- if the RRR at the upper boundary would, if true, be clinically upper boundary would, if true, be clinically important, the study has failed to exclude an important, the study has failed to exclude an important treatment effect (i.e. has not enrolled important treatment effect (i.e. has not enrolled sufficient patients).sufficient patients).



• When using the medical When using the medical literature to answer a clinical literature to answer a clinical question, approach the study question, approach the study using three discrete steps :using three discrete steps :



3)3) How can I apply these results to How can I apply these results to patient care?patient care?

• Before embarking on therapy based Before embarking on therapy based on a study for a particular patient, ask on a study for a particular patient, ask 3 questions:3 questions:

a)a) Were the study patients similar to the Were the study patients similar to the patient in my practice?patient in my practice?

b)b) Were all clinically important outcomes Were all clinically important outcomes considered?considered?

c)c) Are the likely treatment benefits worth Are the likely treatment benefits worth the potential harm and costs?the potential harm and costs?

3. How Can I Apply These 3. How Can I Apply These Results to Patient Care?Results to Patient Care?

3a. Were the study patients 3a. Were the study patients similar to the patient in my similar to the patient in my

practice?practice? If the patient had qualified for enrollment in the If the patient had qualified for enrollment in the

study -- that is, if she had met all inclusion criteria study -- that is, if she had met all inclusion criteria and had violated none of the exclusion criteria -- and had violated none of the exclusion criteria -- you can apply the results with considerable you can apply the results with considerable confidence. Even here, however, there is a confidence. Even here, however, there is a limitation. Treatments are not uniformly effective in limitation. Treatments are not uniformly effective in every individual. Conventional randomized trials every individual. Conventional randomized trials estimate average treatment effects. Applying these estimate average treatment effects. Applying these average effects means that the clinician will likely average effects means that the clinician will likely be exposing some patients to the cost and toxicity of be exposing some patients to the cost and toxicity of the treatment without benefit. the treatment without benefit.

A better approach than rigidly applying the study's A better approach than rigidly applying the study's inclusion and exclusion criteria is to ask whether inclusion and exclusion criteria is to ask whether there is some compelling reason why the results there is some compelling reason why the results should not be applied to the patient. should not be applied to the patient.

3b. Were all clinically important 3b. Were all clinically important outcomes considered (and were outcomes considered (and were

the outcomes that were measured the outcomes that were measured clinically significant)?clinically significant)?

Treatments are indicated when they provide Treatments are indicated when they provide important benefits. Demonstrating that a important benefits. Demonstrating that a bronchodilator produces small increments in forced bronchodilator produces small increments in forced expired volume in patients with chronic airflow expired volume in patients with chronic airflow limitation does not provide a sufficient reason for limitation does not provide a sufficient reason for administering this drug. What is required is evidence administering this drug. What is required is evidence that the treatment improves outcomes that are that the treatment improves outcomes that are important to patients, such as reducing shortness of important to patients, such as reducing shortness of breath during the activities required for daily living. breath during the activities required for daily living. We can consider forced expired volume as a We can consider forced expired volume as a substitute or surrogate outcome. Investigators substitute or surrogate outcome. Investigators choose to substitute these variables for those that choose to substitute these variables for those that patients would consider important, usually because patients would consider important, usually because they would have had to enroll many more patients they would have had to enroll many more patients and follow them for far longer periods of time. and follow them for far longer periods of time.

3b. Were all clinically 3b. Were all clinically important outcomes important outcomes

considered?considered?

3b. Were all clinically 3b. Were all clinically important outcomes important outcomes

considered?considered? Even when investigators report favorable Even when investigators report favorable

effects of treatment on one clinically effects of treatment on one clinically important outcome, you must consider important outcome, you must consider whether there may be deleterious effects whether there may be deleterious effects on other outcomes. For example a on other outcomes. For example a cancer chemotherapeutic agent may cancer chemotherapeutic agent may lengthen life but decreases its quality. lengthen life but decreases its quality.

3c. Are the likely treatment 3c. Are the likely treatment benefits worth the potential benefits worth the potential

harm and costs?harm and costs? If you can apply the study's results to a If you can apply the study's results to a

patient, and its outcomes are important, patient, and its outcomes are important, the next question concerns whether the the next question concerns whether the probable treatment benefits are worth probable treatment benefits are worth the effort that you and the patient must the effort that you and the patient must put into the enterprise.put into the enterprise.

the NNT can help clinicians judge the the NNT can help clinicians judge the degree of benefit and the degree of harm degree of benefit and the degree of harm patients can expect from therapy.patients can expect from therapy.

3c. Are the likely treatment 3c. Are the likely treatment benefits worth the potential harm benefits worth the potential harm

and costs?and costs? As a result of taking aspirin, patients with As a result of taking aspirin, patients with

hypertension without known coronary artery disease hypertension without known coronary artery disease can expect a reduction of approximately 15% in their can expect a reduction of approximately 15% in their relative risk of cardiovascular related events.relative risk of cardiovascular related events.

For an otherwise low-risk woman with hypertension For an otherwise low-risk woman with hypertension and a baseline risk of cardiovascular related event of and a baseline risk of cardiovascular related event of between 2.5% and 5%, this translates into an NNT of between 2.5% and 5%, this translates into an NNT of approximately 200 during a 5-year period.approximately 200 during a 5-year period.

For every 161 patients treated with aspirin, one would For every 161 patients treated with aspirin, one would experience a major hemorrhage (NNH = 161). experience a major hemorrhage (NNH = 161).

Thus, in 1000 of these patients, aspirin would be Thus, in 1000 of these patients, aspirin would be responsible for preventing five cardiovascular events, responsible for preventing five cardiovascular events, but it would also be responsible for causing but it would also be responsible for causing approximately six serious bleeding episodes. approximately six serious bleeding episodes.

3c. Are the likely treatment 3c. Are the likely treatment benefits worth the potential harm benefits worth the potential harm

and costs?and costs? Trading off benefit and risk requires an Trading off benefit and risk requires an

accurate assessment of medication adverse accurate assessment of medication adverse effects. Although RCTs are the correct effects. Although RCTs are the correct vehicle for reporting commonly occurring vehicle for reporting commonly occurring side effects, reports regularly neglect to side effects, reports regularly neglect to include these outcomes. Clinicians must include these outcomes. Clinicians must often look to other sources of information – often look to other sources of information – often characterized by weaker methodology often characterized by weaker methodology – to obtain an estimate of the adverse – to obtain an estimate of the adverse effects of therapy.effects of therapy.

The preferences or values that determine The preferences or values that determine the correct choice when weighing benefit the correct choice when weighing benefit and risk are those of the individual patient. and risk are those of the individual patient.

Evaluation of Primary StudiesEvaluation of Primary Studies Assessing Therapies Assessing Therapies

• Are the results valid?Are the results valid?

Randomized?Randomized? Blinded?Blinded? Intention to treat analysis?Intention to treat analysis? Follow-up complete?Follow-up complete?

• What are the results?What are the results? How large was the treatment effect (ARR/NNT)?How large was the treatment effect (ARR/NNT)? How precise was the estimate (confidence intervals)?How precise was the estimate (confidence intervals)?

• How can I apply these results to patient care?How can I apply these results to patient care? How is my patient different from the study patient?How is my patient different from the study patient? Was the outcome measured clinically significant (surrogate Was the outcome measured clinically significant (surrogate

outcomes), and were all clinically significant outcomes outcomes), and were all clinically significant outcomes considered?considered?

What is the risk of treating vs. the risk of not treating What is the risk of treating vs. the risk of not treating (benefit vs. harm)?(benefit vs. harm)?

SUMMARY

User’s Guides to the Medical User’s Guides to the Medical LiteratureLiterature

• The “User’s Guides to the Medical The “User’s Guides to the Medical Literature” is a valuable resource. Literature” is a valuable resource. Please consult this for analyzing studies Please consult this for analyzing studies on harm, diagnosis, prognosis, and on harm, diagnosis, prognosis, and systematic reviews. The principles are systematic reviews. The principles are the same as presented today, but the the same as presented today, but the steps are different.steps are different.

• There is a direct link to the “User’s There is a direct link to the “User’s Guides” from the JAMA home page – Guides” from the JAMA home page – www.jama.ama-assn.orgwww.jama.ama-assn.org

Date post:	31-Dec-2015
Category:	Documents
Upload:	raymond-martin
View:	28 times
Download:	0 times

Evaluating Medical Literature

Documents