How to assess a randomized controlled trialin surgery
Achilleas Thoma, MD, MSc; Forough Farrokhyar, PhD; Mohit Bhandari. MD, MSc; Ved Tandan, MD, MSc;for the Evidence-Based Surgery Working Group
C linicians are presented with prob-lems that require evidence. Ran-
domized controlled trials (RCri's) areconsidered tlie optimal study designfor evaluating the effect ot* a newmedical or surgical intervention. Sur-gical RCTs are carried out less oftenthan RCTs of medical interventions,partly because using RCTs to evalu-ate surgical or interx'entional proce-dures is difficult and demands specialconsideration of issues such as blind-ing and the effects of surgeon factor,learning cur\'e and differences in pre-or postprocedural care. Despite theseobstacles, RCTs are becoming morecommon as surgeons in many spe-cialties become acquainted with theproper methodolog}' for health re-search.
The mere description of ii study as"randomized" does not allow clini-cians to infer validity — to base infer-ences on the accuracy of the resultsdrawn from the sample of patients
studied {internal validit\') or the gen-eralizabiiit\' of these results to othersettings (external validity). The pur-pose of this article is to review, with-in a framework of a critical appraisalof surgical RCTs, strategies to inter-pret the results and the issues to con-sider \\ hen applying these results toa clinical practice setting. A clinicalscenario will be used to illustrate howsurgeons might retrieve and evaluateevidence.
Scenario
A 67-year-old accountant who youknow has osteoarthritis in her hipscomes to your office. In the pastyear, the pain in her right hip hasincreased dramaticiilly despite theconservative treatment you have pre-scribed. Pain now prevents her fromwalking for more than 5 minutes,and enjoying recreational activitiessuch as golf
After examination and a review ofher x-rays you conclude that she maybenefit from total replacement of herright hip. You discuss with her thenature of her problem, and the ex-pectations and risks of the surgery.You recommend a "cemcntless" im-plant recently advocated at an ortho-pedic conference you attetidcd.
Your patient recalls 2 neighbourswho were exceptionally satisfied withtheir hip replacements. In their cases,however, cement was used to bondthe metal hip to the bone, "What-ever way they did it/ ' she tells you, "Iwant it done the same way."
You ask her to go home, thinkabout what you told her, and returnin a few weeks for another discus-sion. Realizing that you remain un-certain about the latest evidence onoutcomes favouring cemented versusuncemented arthroplast\\ you decideto search die literature before yourpatient's next appointment.
From the Surgical Outcomes Research Centre, Department of Surgery, St. Joseph's Healthcare and McMaster University.Hamilton. Ont
The Evidence-Based Surgery Working Group members Include: Stuart Architxitd. MD:'^^ Mohif Bhandari MDj^ Charles H. Goldsmitti.PhD:'^ Dennis Hong. MD:^ John D. Miller. MD:'^^ Marko SImunovic. MD. MPH;^^^^ Ved Tandan. MD. MSc;'^^ Achilleas ThomaJohn Urschel. MD:^^ Sytvie Comacchi,
'Department of Surgery. St Joseph's Hospital. fDeparfment of Surgery. tSurglcal Outcomes Research Centre and ^Departmentof Clinicai Epidemiology and Biostatistics, McMaster University, and ^Hamilton Health Sciences, Hamilton Ont.
Accepted for publication Nov. 5. 2003
Correspondence to: Dr Achilleas Thoma. Division of Plastic Surgery. St Joseph's Healthcare. Hamilton. 101-206 James St. S,Hamilton ON L8P 3A9: fax 905 523-0229: athoma@mcmaster. ca
The ideal article addressing the ques-tion would be onc comparing casesof cemented hip arthroplasty withthose not cemented. From yourhome computer you enter Medline,the National Library of Medicine'sPubMed database. In the search fieldyou enter "hip AND cement ANDrandomized" with limits to the last 3years, English articles, studies usinghuman subjects, and clinical trials.
This search yields 10 hits. As youreview the titles of the articles found,"Comparison of total hip arthro-plast\' performed with and withoutcement," a 6-page article by Laupacisand colleagues published in 2002,'attracts your attention. This recentarticle appears to address your partic-ular question, and you expect it togive you the answer.
Evaluating the search
This search strategy is unfortunatelyinsufficient to identify all the RCTsand other sources that could give youa more complete review of topics. Tolearn how to plan an appropriatesearch, refer to die first article of thisseries, by Birch and associates.^
Although various study designsarc used in clinical research, RCTsare generally regarded as the mostscientifically rigorous for evaluatingthe effect of a surgical inten'entionand protecting against selection bias. *Results from obser\'ational studiesmay generally be valid, but their limi-tations are acknowledged.
The tiindamental criticism of ob-servational studies is that unrecog-nized confounding factors may dis-tort the results. Altiiough RCTs willremain a prominent tool in clinicalresearch, the results of a single RCT,like those of a single obsenationalstudy, should be cautiously inter-preted. If an RCT is later determinedto have given wrong answers, evi-dence from other trials and fromwell-designed cohort or case-controlstudies can and should be used to
find the riglit answers. The common-ly held belief that only RCTs producetrust\vorthy results and that obsei va-tional studies are inherendy mi.slead-ing does disser\'ice to patient care,clinical in\estigation and the educa-tion of health care professionals.'
Summary of the appraised article
The Laupacis group's study was de-signed to compare the fixation of aMallory-Head total hip prosthesiswith and without cement. Recruit-ment was slow; the intended samplesize of 300 patients, which had beenplanned in order to attain a studypower of 80%, was reduced to 250.'Of these 250 patients, 124 under-went hip arthroplasties with cementand 126 without cement. The out-comes assessed included mortality,revision arthroplast\', health-relatedquality of life {HRQoL, measuredwith 5 different scales) and a 6-niinute walk test.
The mean time of foUow-up was6.3 years (minimum 0, maximum9.4 yr). Complete follow-up datawere unavailable for 36 patients(14.4%): 31 refi-ised to continue withthe HRQoL follow-up examinations,4 were lost to follovi'-up and 1 wasmistakenly not followed. Overall 35of the patients studied died, at amean postoperative period of 4.2years; but it is unclear how manywere in either group.
More revisions were required inpatients who had a prosthesis withcement (10.5%) than those without(4.5%), but the difference was notstatistically significant. However, pa-tients in the group with cementedimplants had femoral revisions signif-icantly more frequently (9.7%) thanthose with cementless implants (0.8%;P = 0.002). (A femoral revision dealswith the stem of the implant thatgoes inside the length of the bone.)
A comparison of pre- and post-operative HRQoL scores revealedsubstantial improvements across awide array of dimensions of healthstatus and HRQoL in both groups.
particularly during the first 3 monthsafter surgery. During the 7 years offollow-up, HRQoL was maintainedwith minor decline between the firstand seventh years. Mean time trade-off scores, in total, improved from0.29 before to 0.8 after operation,but /> values were not provided.
Our article has tlie same structureas previous items in the evidence-based surger\' series.'""^ The purposeof ours is to help readers appraise thestrengths and weaknesses of RCTs inthe surgical literature. This articlewill apply the principles in the Users'Guides to the critical appraisal of asurgical RCT, as in Box 1.
Are the results of this RCT valid?
Did the investigators take thelearning curve into consideration?
In contrast to drug trials, the surgicalRCT, which compares a "novel" tothe "usual" intervention, has to dealwith the learning curve. The learningcur\e usually refers to the accumu-lated experience on a new procedurethat allows continuous refinement ofpatient selection, operative technique,adjunctive medication and postpro-cedural care. Although some investi-gators suggested that surgical RCTsshould start with die first patient,"'^most investigators disagree. Most be-lieve that it is inappropriate to com-pare a familiar with an unfamiliar sur-gical intervention, as mistakes andadverse outct)mes are more likely tooccur with the unaccustomed pro-cedure, which will bias the resultsagainst the novel intervention.'^''
The report by Laupacis and co-workers' has no indication that a learn-ing curve was considered. We do notknow if the surgeons took a courseon the cementless prosthesis or did afew cases to feel comfortable beforeembarking on tlie RCT. Consideringthat the 2 participating surgeons areexperienced orthopedic specialists, letus assume that the critical part of dieprocedure was performed by themmany times before, and the learning
Can J Surg, Vol, 47. No. 3. June 2004 201
Thoma et al
cur\'e was therefore not an issue. Anymisleading data resulting from thisproblem would bias against the non-cemented prosthesis; thus, if any-thing, the estimate of benefit of anon-cemented prosthesis is conserva-tive. Learning cur\es can be a majorfactor when the interventions beingcompared are quite different in tech-nique, such as open versus laparosco-pic cholecystectomy.
Were patients randomized?
Randomization tends to producecomparable groups; that is, knownand unknown prognostic factors inparticipating patients will, with ran-domization, become evenly balanced
between die study groups."' Becausea patient's, physician's or researcher'spreferences may bias allocation, ran-domization removes tliat potential.
Since bias in the selection of casescan influence outcomes, it is impera-tive that experimental and controlgroups be randomized. The autliorsof the study under scrutiny tell usthat their means of randomizationwas computer-generated, a methodthat is appropriate.
Was randomization conceaied(biinded or masked)?
It is important that those decidingwhether a given patient is eligible tbrthe trial remain unaware of the arm
Box 1. Users' Guides for an article about RCT in surgery
Are the results valid?
Did the 2 surgical groups begin the study with a similar prognosis?
• Did the investigators lake Into consideration the learning curve?
• Were the patients randomized?
• Was randomization conceaied (blinded and masked)?
• Were patients stratified?
• Were patients' data analyzed in the surgical groups to which they wererandomized?
• Were patients in the 2 surgical groups similar with respect to knownprognostic tactors?
Did the surgical groups retain a similar prognosis after the study started?
• Were patients aware of group allocation?
• Were surgeons aware of group allocation?
• Were outcome assessors aware of the surgical group allocation?
• Was fo! low-up complete?
What are the results?
• How large was the treatment effect?
• How precise was the estimate of the treatment effect?
How can I apply the results to patient care?
• Were the study patients similar to my patients?
• Were all clinically important outcomes considered?
• Are the likely treatment benefits worth the potential harm and costs?
This list was modlfed from Users' guides to the rr^edlcal literature: a manual forevidence-based clinical practice, by G. Guyatt and D, Rennle (Chicago: AMAPress; 2002).
of the study to which the patient willbe allocated. If randomization orconcealment tails, they may systema-tically enrol sicker or less sick patientsto either treatment group, leading tobiased results.* ' The randomizationmethod is fault\' if investigators as-sign patients according to even orodd birth year, alternate chart num-bers, and so on, as these strategiesarc prone to selection bias."* For ex-ample, if even/odd birth year is used,the randomization method is uncon-cealed and investigators, consciouslyor subconsciously, may select patientsbased on knowledge of the group towhich they arc being allocated. Inthe Laupacis group's article,' opaqueenvelopes provided the sequence ofthe procedure, which were opened inthe operating room. Provided noone tampered with these envelopes,we can be sadstied to this point tliatin feet concealment in this study wasachieved.
A sealed, opaque envelope camiotbe read without spoiling the seal.Wlien such envelopes are used, theyshould be sequentially numbered tocircumvent repeated opening. As theauthors did not mention any meansthat was used to prevent prematureor repeated opening of their opaqueenvelopes, we must place a questionmark beside concealment as appliedin this study.
Central randomization (offsite ran-domization by an unin\olved party)is the most effective way of ensuringconcealment in surgical trials.
In contrast to pharmaceutical stu-dies where concealment and treat-ment masking can be continued tothe end of the study, in surgical stud-ies concealment can only be main-tained until the surgeon opens theenvelope. From that time forward,the surgeon knows what procedurethe patient will undergo.
Were patients stratified?
Some investigators might want tomake sure, prior to randomization,that any prognostic factors known to
202 J can chir, Vol. 47. N° 3. juin 2004
Randomized controlled trial evaluation
be strongly associated with outcomeevents are c\ cnJy divided bet\\ eeii thestudy groups. This randomizationmethod is called stratified randomiza-tion. 1-iupacis and associates' appliedthis method by stratifring patients bysurgeon (initials RB or CR) and age(under and o\cr 60 years) and b\' sel-ecting a computer generated ran-domization system.
Moreso than in medical studies,individual skills and technique mayhave a major effect on surgical out-comes. In surgical RClTs, stratifica-tion by surgeon should theretbre beconsidered whenever possible.
I jupacis and coauthors' inform usthat they used a stratified randomiza-tion method. Unfortimately, they didnot report whether age or surgeonfactor accounted for any of the ob-ser\ed differences in outcomes (pa-tient sur\ival or need tbr revision ofthe prosthesis).
Were patients analyzed in ttie groupto which they were randomized?
Surgical investigators can corruptrandomization if not all patients re-ceive their assigned surger\'. For ex-ample, if the anesthetist, atter induc-tion of anesthesia, voices concernabout a patient's arrhnhmia or fearsa myocardial infarction, the surgeonmight decide to carry out whicheverprocedure would take less time, evenif it is not the one rand(.)mized to beperformed. Another scenario couldinvoKe a decision by the surgeon touse alternate procedure because ofunexpected tindings during the sur-gery, e.g., poor qualit)' of hone.
Data for such patients might tlienbe dropped trom die analysis. If sur-geons include such poorly destinedpatients in one treatment group anddrop them trom the other, then e\ena suboptimal surgical procedure mayappear to be etiectixe.
Intention-to-trcat analysis avoidsthis potential bias.""'' In essence, theanalysis of outcomes is based on thetreatment arm to \\ hicli patients wererandomi7x'd and not on tlie surgical
treatment received. With this analysismetluid, all known and unknownprognostic factors remain distributedbct\\cen the surgical groups as equal-!)• as the\' were at randomization.
Were patients in the treatment andcontrol groups similar with respectto known prognostic factors?
Since bias in case selection can influ-ence the etfect of the intervention,it is imperative that the experimental and control (more traditionaltreatment) groups begin their treat-ment with same prognosis, i.e.,stage of disease, underlying morbidcondition and age. In the Laupacisstudy,' the authors informed us thatthey included patients with osteo-arthritis (OA) of the hip who wereundergoing a unilateral procedure.They excluded patients who were75 years of age or older. Note thatthe more elderly patients were likelyexcluded because uncemented de-signs can fail in the presence of poorbone quality and unsuitable canalmorphology.
In the article itself, we are notgiven the information necessary' tojudge for ourselves if in fact thegroups are similar. However, an ad-dress was provided for a Web pagewhere this information can be found.Checking this page, we found a listof taetors provided that included theratio of women to men, percentageof patients operated on by one of theauthors, ajid preoperative scores ondie HRQoI. scales used in the study.The distribution between the 2 treat-ment groups of all the prognosticfactors in Box I can therefore be de-termined. Although no statistical an-alyses were presented, no significantclinical difference was apparent.
It would have been beneficial toknow whether patients had the samele\'el of disease; we only know that"severe OA" patients were omitted.Radiographic staging does not al-ways correlate with clinical limita-tions; it may not have been includedfor that reason.
Were patients aware of their groupaiiocation?
It has been recognized in the medi-cal literature tor some time tliat if pa-tients know the treatment and be-lieve it to be etficacious, they tend tofeel better than those who do not,even when the treatment is the same.This has been dubbed the placeboeffect.'" Placebo etVect, which is usu-ally but not necessarily beneficial, isattributable to an expectation thatthe regimen will have an effect —thepower of suggestion.^' Knowing (oreven thinking that they know) thatthey are in the experimental groupcould influence patients to answerQoL questions in a favourable man-ner, exaggerating tiie response in thesurgical arm receiving (in this case)the newer cementless pro.sthesis.
In this particular study, the au-thors made it clear that the patientwas unaware of the t)'pe of prosthesishe or she received. Furthermore, theeventual scar would be identical, as"all patients were operated on by ei-ther surgeon with use of an identicaldirect lateral approach...." However,in surgical trials where the questionof interest compares surgical andnonsurgical interventions, this is fre-quently impossible.
Were clinicians (surgeons) awareof group allocation?
Dit 'erences in patient care other thanthe surgical intervention under con-sideration can bias the results. Forexample, if the surgeon was biasedtoward the cementless prosthesis, heor she might take a little extra timeand pay a bit more attention to de-tails such as hemostasis or placementof the prosthesis, pnxiucing an tner-estimate of its good etfects.
Even use of correct randomiza-tion does not guarantee that 2 surgi-cal groups will remain prognosticallybalanced. In contrast with pharma-ceutical trials where the clinicians donot know whether the patient takesthe experimental drug or the pla-
Can J Surg, Vol. 47, No. 3, June 2004 203
Thoma et al
cebo, in surgery the procedure cannot be concealed from the surgeonexecuting it. Despite the attention"''paid to this problem, it unfortunatelyhas no easy solution.
In the aiticlc under study., becauseno effective blinding of the surgeonwas possible., we just don't knowhow much differential surgical carehas been applied., consciously or un-consciously., to the cementless im-plant group.
Were outcome assessors aware ofgroup allocation?
Even when randomization has suc-cessfully controlled for selection biasand .surgical groups have been keptprognostically balanced, the studycan still introduce bias if the assessorsof outcomes have not been blinded.We define as assessors those who eol-Iect outcome information. If eithersurgical arm of the study receivesmore frequent or thorough measure-ment of outcome or co intervention(additional physiotherapy., for exam-ple), positive interpretation of mar-ginal findings, or a different degree ofencouragement during performancetests, then results can be distorted infavour of the group receiving the ad-ditional attention.-*
The audiors of the appraised RCTinform us that neither the patient northe research assistant who assessed theoutcomes was aware of the type ofprosthesis that had been inserted.Whether this materialized or not, wejust don't icnow. Physiotherapists do-ing assessments in hospitals do haveaccess to patients' charts. Conscious orunconscious perusal of the operativerecord could have revealed the type ofimplant used. We don't know to whatextent die investigators concealed diisinformation on patients' charts.
Was follow-up complete?
A major tlireat to the validity' of anRCT is failure to account for all thepatients at the end of the study. Thegreater the number of patients lost in
this way, die greater the harm doneto the study. Patients who did notreturn for a follow-up appointmentor test may have died or had a badoutcome. It is also possible they mayhave had a very satisfactory outcomeand didn't bother returning for fol-low-up. When the loss rate is large(greater than 10% or so of the studypatients), it affects the study's valid-it}'. There are specific guidelines onthis subject.'"- ^ A large lost-to-follow-up rate reduces the study power.
The (Kcurrence of die event of in-terest among patients is uncertain aftera specified time when follow-up datacollection ends. It is unknown whenor whether the event of interest oc-curred subsequendy. Such study sub-jects are described as "censored" or"lost to follow-up".^"^ Th^e patientsstill contribute to the snidy up to dietime at which their outcome statuswas last known. One way to examinethe data for censoring effect requiresthe use of appropriate anal\tic meth-ods, such as survival analysis (applica-tion of life-table method to data). - *In diis study, the aLidiors have not de-scribed their statistical methods; butwhen the results are reviewed, it isclear that they used survival analysis.
The authors had planned original-ly to recruit 300 patients, to provide80% study power. Study power is anissue decided early in the planningphase; the calculated sample size isthe minimum required to maintainthat power throughout the study.With 250 patients randomized intothe appraised RCT, this study likelybecame underpowered. Furthermore,complete follow-up data were un-available for 36 patients: 31 who re-flised to continue with all of the HRQoL follow up examinations, 4 whowere lost to follow-up and 1 who wasmistakenly not followed.
What are the results?
How targe was ttie treatment effect?
The authors reported that during thefollow-up of the study, revision sur-
ger\' occurred more often with thecemented prostheses than with thosewithout cement (13 patients v. 6, p= 0.11). They inform us that this dif-ference was not statistically signifi-cant; however, cemented implantshad significantly more revisions ofthe femoral component of the pros-thesis (12 V. 1, /)= 0.002). Unfortu-nately, the results were presented in away not easily interpreted by practis-ing surgeons.
From the data the authors pro-vided, we were able to calculate theabsolute risk reduction (ARR), rela-tive risk (RR), relative risk reduction(RRR) and numbers needed to treat(NNT), all shown in Table 1 andTable 2. (Fxplanations of these termsand calculations can be found in thearticle by Urschel and associates^ inthis series.) Notice the subset of val-ues related to the femoral compo-nent of the prosthesis.
The NNT is very usefiji to theclinician. It tells us that ever>' 20cementless prostheses installed by
Table 1
Number (and %) of hip replace-ments needing revision surgery
Cemented Cement-implants less
Potlents randomized 124 126
Revisions required 13 (10.5) 6 (4,8)
Femoral only 12(9.7) 1 (0.8)
Table 2
Catculctians from study dataRevisions
Factor & equation All Femoral
Absolute risk reduction 0.05 0.1ARR = X - y
e.g., tor all subjects:x= 13/124 = 0.10V= 6/126 = 0.05
x-y=0.10-0 ,05^ _ _ = 0.05
Relative risk 0.5 0.08RR = / /x
Relative risk reduction 50% 92%
RRR= Cl-y/x)xlOO
No. needed to treot 20 10NNT= 1/ARR
204 J can chir, Vol. 47, N° 3. Juin 2004
Randomized controlled trial evaluation
surgeons prevents 1 revision surgcn'that would be needed if eementedprostheses had been used. Only 10cementless prostheses need to beperformed to prevent 1 revision of afemoral component.
With regard to the various HRQoL scales that we believe are veryrelevant to surgical studies., the au-thors made a cursory mention in theresults section of their article that "alloutcomes measures had substantiallyimproved by 3 months after sur-gery." HRQoL, the article goes onto report, ""was maintained duringthe 7 years of tbllo\v-up, although itappeared to decrease slightly betweenthe first and seventh years." Thisstatement is insufficient. It can be ex-plained by a tailurc of tlie investiga-tors to power the study adequatelyfor HRQoL scales.
How precise was the estimateof the treatment effect?
The true reduction to the revisionrate caused by use of a cementlessprosthesis can never be known. Thebest estimate we have is that pro-vided by this RCT, called the pointestimate.
The true reduction value liessomewhere in this neighbourhood,and it is unlikely to be precisely cor-rect. The best method to communi-cate this point estimate is to providethe confidence interval (CI). A CI isa set of values within which one canbe confident that the true valuelies.'"- " Although die breadth of theCI chosen is somewhat arbitrar\', byconvention a 95% intenal is used. A95% CI defines the set witliin whichwe expect to find the true effect esti-mate 95% of the time. It becomesnarrower as sample sizes increase.
Cls are important for interpretingstudies \vidi negati\'e results as well aspositi\e. In a positive RCT where theauthors claim that the novel treat-ment is effective, all we have to do isto look at die lower boundary of dieCI. If we believe that this lower-bound effect change is clinically im-
portant, then the indication to adoptthis new surgical intervention isstrong. If on the other hand we donot consider this value clinically im-portant, despite the fact that the pvalue was significant, we may wantfiirther evidence before we adopt it.
Ixjoking at the upper CI bound-ary can help us respond to negativestudies, i.e. those where the p value istoo large to be significant. Wlicn theupper value, if true, shows a resultthat would be clinically important,the study does not exclude an impor-tant treatment effect.'" The authorsof this RCT did not provide a CIvalue of the revision rate, either forthe whole implant or for the femoralcomponent. They did, however, pro-vide a p value for both outcomes.From the p value we can calculate CIourselves.'"
The formula for calculating the95% CI for a proportion (p) is CI =p ± 1.96 X SE, where SE is the stan-dard error. The formula for SE of aproportion is SE = f p X (1 - P)/K]^-
where p is the proportion and n isthe number of patients.
We have calculated 95% Cls fortlie rates provided in the article. Therevision rate in the cemented groupwas 10.5%, with a CI of 5.1-15.9.The revision rate in the cementlessgroup was 5%, witli a CI of 1.2-8.8.The femoral revision rate in the ce-mented group was 10%, with a CI of4.7-15.3. The tcmoral revision ratein the cementless group was 0.8%,with a CI of 0.04-3.95.
With regard to the various HRQoL scales, the authors did not pro-vide an estimate of the change in thescore or its CI. They did provide, ona Web page, the raw data fi'om whichwe can calculate the pre- to post-operative score change.
How can I apply the results topatient care?
Were the study patients similarto the patient(s) In my practice?
Before we accept the RCT's findings
and apply them to our scenario pa-tient, we need to decide whether sheis similar to the patients who partici-pated in the RCT.
The patients in any RCT may bequite different from our ov\ n becausethe inclusion/exclusion criteria usedin a trial are usually stringent. Eor ex-ample, the RCT may have includedmale patients only, whereas our pa-tient is female; our patient may havecomorbid conditions not present inRCT participants, and so on. Onemust also consider that RCT reportaveraged the treatment effects intheir suidy patients, when some indi-viduals undoubtedly benefited morethan others, and some may not havebenefited at all.
If we look at the inclusion and ex-clusion criteria set by the investiga-tors of the study scrutinized in ourarticle, we find that indeed, our pa-tient meets their inclusion criteria.We can therefore be confident thattheir results apply to our patient aswell. Although the average age oftheir study participants was younger(mean 64 yr), upon closer examina-tion of the article we see that a sub-group analysis of the results was per-formed by age, for those 60 years oryounger and those 60-75 years old.The RCT findings still hold for theolder subgroup. Our patient is simi-lar in age to the patients in the study,and hence we are willing to apply theRCT's findings to her.
Were all clinically importantoutcomes considered?
Surgical intenentions are indicatedonly if they produce clinically impor-tant changes to the patient. A smallphysiological change in some variableis not a justification to submit the pa-tient to surgery. Eor example, a posi-tive change of 10° in the range ofmotion (RoM) of the knee or hipwith a prosthesis is not justificationfor surgerv', unless tliis change leadsto a substantial change in the pa-dent's quality of life.
In their ardcle, Laupacis and co-
Can J Surg, Vol. 47, No. 3, June 2004 205
Thoma et al
authors mentioned that they assessed3 different outcomes.' First, theyused mortality as an important out-come. Unfortunately, they do notstate the death rate in each group,but only that 35 of all their random-ized patients died.
The second outcome they studiedwas revisional surgerv' of the hip. Aship prostheses are associated withcomplications downstream, this is animportant outcome to consider —and indeed, it was. Revision of theimplant submits a patient to addi-tional surgery with weil-recognizedpotential for perioperative morbidity.Revisional surgeries also add expen-ses to the health care system, whichmatter from a societal perspective.The investigators unfortunately didnot "piggyback" an economic analy-sis onto their RCT report.
The third outcome was the assess-ment of HRQoL. With 5 differentscales they demonstrated that scoresimproved substantially after implant.Although it was lefi unclear whetherthe cementless prosthesis outdid theother in this respect, both groups'rise in HRQoL scores appears tohave been similar during the studyperiod.
Are the likely treatment benefitsworth the potential harm and costs?
Before we decide to use the study'sfindings to persuade our patient, it isimportant to consider whether theprobable benefits that we expect thepatient to derive are worth the effort.Any surgery carries risks, which caninclude anesthetic complications, sur-gical wound infection, pneumoniaand deep-vein thrombosis.
There is evidence in the literaturegoing back to die 1960s that intro-duction of a hip prosthesis improvesHRQoL in patients afflicted withOA. The RCT we have been examin-ing suggests that in padents youngerthan 75 years of age without severeOA, infectious arthritis or previouship/knee arthroplasty, cementlessarthroplasty results less offcn in revi-
sional surgery while exhibiting equiv-alent risk of mortality and improve-ments in HRQoL.
Trading off benefit and risk re-quires that we examine carefiilly therisks of surgery, not only the initialone but also any revisions down-stream. If we believe die findings ofthe study that the cementless implantis less likely to require later revision,we may want to consider using thisrather than the cemented one.
Costs also need to be considered.Our universal health care system cov-ers the surger\' costs; as well, thereare indirect costs borne by the pa-tient and her family to diink of. If weare convinced that the cementlessprosthesis is le.ss likely to require revi-sion in the fijture, there may be com-pelling reason to advocate this novelintervention.
Resolution of the scenario
After going carefully over the evi-dence presented in the publishedmanuscript and the Web page, weare convinced that our patient willbenefit from either prosthesis interms of pain control, abilitv' to walkand general improvement in herquality of lite. If she chooses die ce-mentless prosthesis, she is more likelyto avert revision of her prosthesisdownstream. Although you recog-nize that tlie results of tliis study maynot be generalizable to all cementedand uncemented prostheses, the oneused in your institiLtion is the sameas that used by Laupacis and col-leagues.' Therefore, you recommendtlic cemendess prostliesis.
Discussion
Although this article discussed theappraisal of a single RCT that ad-dresses a single clinical question, thereader should be aware that to graspthe clinical question fiilly, one shouldalso review other pertinent literaturethat may include additional RCTsand systematic reviews.
Before accepting RCT findings, it
is important to comprehend the im-portant elements of the study, in par-ticular its design and conduct, andthe analyses and interpretation of theresLilts. Altliough an RCT is consid-ered the best study design to providean estimate of the effect of a surgicalintervention, a study that is method-ologically poorly executed may over-estimate die true effect. The RCT re-port must convey to the reader whytiie study was undertaken, how it wasconducted and how it was analyzed;and it is necessarv' that these steps betransparent to tlie reader.
There is evidence in tlie literaturethat reporting randomization inade-quately can lead to bias in the effectof interventions.""' Because of prob-lems associated with the reporting ofRCTs, an international group of clin-ical trialists, statisticians, epidemiolo-gists and journal editors publishedthe Consolidated Standards of Re-porting Trials (CONSORT) state-ment" to help investigators and au-thors. It includes a checklist and flowdiagrams incorporating the crucial el-ements of an RCT. The CONSORTstatement has been revised since theoriginal publication to clarify specificchecklist items.'-
In the past tew years, as more sur-geons have trained in or been ex-posed to health research methodol-ogy, there has been an increase inpublished RCTs dealing with surgi-Ci.ll problems. Unfortunately, surgicaltrials, in contrast to medical trials, areplagued by unique methodologicalproblems. Although the issue of ran-domization is straightforward, that ofconcealment remains problematic.Although investigators and partici-pants in a medical study cannot tell atreatment pill from a placebo rightup to study completion, concealmentof treatment allocation group disap-pears once a surgeon opens an enve-lope or telephones the randomiza-tion centre.
Differential surgical care may biasresults while the patient is in the op-erating room or during the follow-up visits, especially if the surgeon is
206 J can chir, Vol. 47. N" 3. juin 2004
Randomized controlled trial evaluation
biased in favour of one or the otherprocedure.
.Surgical interventions may includeadditional procedures that may inHu-ence the postoperative results andoverestimate the effect. Standardiza-tion of a surgical iiitcrvcntion is a dif-ficult problem.
As a large number of patients maybe required to give the study statisti-cal power, an RCT may require mul-tiple surgeons (with different degreesof e.xperience) and multiple centres(with different cultures). The analvsisneeds to take these variables into con-sideration, making large RC'T studiescomplex.
As novel surgical interventions areintroduced, learning curves need tobe taken into account.""" An RCTdesign should be considered onlywhen the surgeons participating areequally capable of performing thenovel and comparator interventions.Failure to control for the learningcurve may underestimate the effectsize of the novel intervention.
The sine qua non for undertakingan RCn" is a state of equipoise.'"'"Surgical equipoise is present whensurgeons are truly ambivalent as towhich of 2 readily available surgicalinterventions is superior. Further-more, clinical equip<»se reflects am-bivalence within the practice com-munit)'. This ethical considerationwas addressed by the authors of thereport under consideration in an in-direct way. In the introduction oftheir article thev- state that "cementdisease" may be obviated by the useof the cemendess prosthesis, as initialreports were encouraging.
Assessors of outcomes can beblinded if the surgical interventionsunder consideration have similar in-cisions and hence, similar scars. Con-cealment may faii if assessors such asphysiotherapists, who usually per-form QoL assessments and RoMmeasurements, have access to pa-tients'" charts and operative records.Prudent surgical investigators de-scribe what steps have been taken toavoid bias in outcome assessment.
Since publication of the CONSOKT
statement, the quality of reporting inpublished R(!Ts has improved con-siderabh. Surgeons shouid make aneffort to implement the steps men-tioned in this .statement vv hen repott-ing a surgical RC'T. Finally, when ap-praising a surgical RCT, surgeons arcencouraged to be critical and use sys-tematically the Users' Guides listedin Bcjx 1.
Competing interests: Ndiic declared.
References
1. Laupacis A, Bourne R, Rorabeck C, FcenyD, Tupwcll, I\ Wong C. Comparison oftotal hip .inhrnplasr\' pertormcd with andv v i t l i i J i i i L c i i i c i i i : a i . i n d o i n i / c d t r i a l . /
Bom-Joint Stirs Am 2Q02;S^H)):lii2i-S.
2. Birch DW. Eady A, Robertson D, DcPauw .S, Tandaii V. Users' giiidc to thesurreal literature: how to pcrtbmi .i literacure search. Can J Sifrij 2003;46:136 41.
3. Coditz GA, Milltr IN, Mostcller F. Howstudy design afix:cts outcomes in compari-sons oi' therapy. I. Medical. Stat Med
4. Schuliz KF, C'haJtncrs 1, Hayes RJ. AltmanDG. Empirical evidence of bias: dimen-sions of methoiiologjcal qualitx' associated«ith estimates ot treatment ejects in con-tmUcd trials./AWX1995;273:408-l2.
5. (loncato I, Shah N, Horwitz RI. Random-ized controlled trials, obscrx'aiional smdies,and the hierarchy of research designs. N
6. .Archibald S, Bhandari M, Thoma A.Users" guide to tlie surgical literature: hem-to use an article about a diagnostic test.
7. Un.chol i n . Goldsmith CH. Tandan VR,Miller II). Users' guide to the surgkal lit-erature: how to use an article evaluatingsurgical intcr\'ention. C'flw/.SMrff2001;44:9S 100.
8. Thoma A, TandanV, Sprague S. Users'guide to tlic surgical literature: how to usean article on economic analysis. Can }
litemture: how to use an article reportingpopulation-based volume-outcome rela-tionships in surgery. Can J Surff 2002;45:109-15.
10. Guyatt G. Rennie D. Users'ifuides to themedical Uterntiire: a manual for reidcnce-based clinical practice. Chicago: AMAPress; 2002.
11. Chalmers TC. Randomization of the firstpatient. Med Clin Siirtli Am 1975;59:1035-8.
12. Parikli L>. Chagla L. )ohnson M, Ixjwe D,McCulloeh P. D2 gastrcctomy: lessonsfrom a prospective audit of the learning
rJSurj! 1996;83:l5y5 9.
13. Boncnkamp JJ, Songim I, Hermans I,Sasako M, Welsaart K. Plukker H M, et al.Randomized comparison of niorbidit\' andmortality after Dl and D2 di.s.section forgastric cancer in Dutch patients. Lancet1995;345:745 8.
14. Ramsay CR, Grant AM. Wallace SA,Ciarthwaite PH, Monk AF. Russell IT.Statistical assessment of the learning curvesof health technologie;.. Health TcchnolAsscssimX =^.\-79.
15. Mohammed MA, Cheng KK, Rouse A.Marshall T. Bristol, Shipman and C!linicalgovernance: Shewhart's forgotten lessons.
;357:463-7.
16. In: Friedman I.M. Furberg CD. DeMetsDl., editors. Fundamentals of clinical tri-als. 3rd ed. New York: Springer Verlag;1998. p. 43 6.
17. Moher D, Pham R, Jones A, Cook D).Jadad AR, Moher M. et al. Does qualit)' ofreports of randomized trials aftcct esri-mates of intenention efficac)' re[v>rted inmeta-anal^s? j:iincrt 1998;352:609-13.
18. ChalmeR TC, Cclano P, Sacks HS, SmithH Ir. Bias in treatment assignment in con-trolled clinical trials. N Enjjl J Med 1983;309:135861.
19. HoUis S, Campbell F. What is meant byintention to treat analysis? Sur\ry of pub-lished randomized controlled trials. Br.Wfrf/1999;319:670-4.
20. Kaptchuk TI. Powertu! placebo: the darkside of the randomized controlled trial.
9. Hong D. Tandjn VR. Goldsmith CH.Simunovic M. Uscn,' guide to the surgical
21. Last IM. A dictiimaiy of epidemiology.New York: Oxford University Press; 1995.
Can J Surg. Vol. 47. No. 3. June 2004 207
Thoma et al
22. Van Dtr Linden W. Pitfalls in randomizedtrials. Surgery I980;7:258-62,
23. McCulloch P, Taylor I, Sasako M, LovcttB, Griffin D. Randomized trials in surgery:problems and possible solutions. Br Med}2002;324:1448-=ll.
24. Guyatt GH, Pugslcy SO, Sullivan MI,Thompson PJ, Bcrman L, Jtmes NL, ct al.Effect of encouragement on walking testperformance. 7>wfljf I984;39:818 22.
25. Harrcll FE Jr. Rej^ression modelling strate-ffles, with application to linear models, logis-tic regression, and survival analysis. NewYot^: Springer Verlag; 2001.
26. Elwooti M. Critical appraisal of epidemio-logical studies and cltnical trials. NewYork: Oxford University Press; 1998,
27. Last IM. A dictionary of epidemiolo/fy.New York: Oxford LIniversin' Press; 2001.
29. Attman DG, Gore SM, Gardner MI,Pocock SL Statistical guidelijies for contri-butors to medical journals. In: GardnerMI, Altman DG, editors. Statisties withconfidence: confidence intervals and statis-tical jfuidelines. London, UK: BMI Books;1989. p, 83-100.
30. Sackett DL, Haynes RB, Guyatt GH,Tugwell P. Clinical epidemiobffy: a basicscience for clinical medicine. 2nd ed. Bos-ton: Little Brown; 1991,
31. Begg CB, Cbo MK, Basrwood S, HortonR, Moher D, Olkin I, ct al. Improving the
of reporting of randomized con-trolled trials: the CONSORT statement.JAMA 1996;276:637-9.
32. Mohcr D, Schulz Ki-', Altman DG. TheCONSORT statement: revised recom-mendations for improving the quality' ofreports of parallel group randomized trials.BMCMedResMethodonm\\\.2.
33. Tcstori M, Bartoformei M, Grana C,Mczzetti M, Chinol M, Mazzarol G, et al.Sentinel node localization in primarj' mel-anoma: learning curve and results. Melan-oma Res 1999 .,9:587-93.
34. Schafer A, The ethics of the randomizedclinical trial. /V£w^//M.'rf 1982;307:719-24.
