8/2/2019 Inter Reliability

1/3

Interrater Reliability of a C l in ica l Sca le of RigidityLINDA R. VAN DILLENandKATHRYN E. ROACH

The purposes of this study were 1) to describe a clinical scale of rigidity andtesting procedure for use in patients with Parkinson's disease and 2) to examinethe scale's interrater reliability. Twenty subjects (3 women, 17 men; age = 64years, s = 16.3) participated in the study. Criteria for participation were 1)diagnosis of Parkinson's disease, 2) physician-documented rigidity, 3) ability tofollow one-step verbal directions, and 4) ability to attain at least 75% of thestandard passive-range-of-motion measurements of the elbow, forearm, and wristof the tested upper extremity. Each of two raters used a standardized set ofinstructions and test procedures. The degree of rigidity was assessed using afour-point scale ranging from 0 (absent) to 3 (severe). The observed agreementbetween raters was 16 out of 20 trials. A Cohen's weighted Kappa was used toanalyze the data (Kw = .636, p = .20). Factors were identified that may havecontributed to the discrepancy between agreement and the agreement beyondchance.Key Words: Parkinson disease; Tests and measurements, range of motion; Upperextremity, general.

Accurate examination of signs, symptoms, and functional disability of patients with Parkinson's disease is essential in monitoring the progression of thedisease, determinin g th e effects of medication regimens, and directing physicaland surgical treatment.1- 6 Two approaches have been developed and usedto assess the clinical features frequentlyexhibited in the p atient with Parkinson 'sdisease: 1) qualitative observer ratingscales for use by the clinician and 2)quantitative assessment with the use ofelectronic instrumentation.1- 4 Observerrating scales typically define specific cri

teria operationally for each of the clinical features to be assessed and identifythe conditions under which the featureswill be tested.The clinical scales most frequentlyused in Parkinson's disease studies arethe Columbia University Rating Scale,7the Northwestern University DisabilityScale,5 Hoehn and Yahr's staging forParkinson's disease,8 and Webster's Parkinson's disease rating scale.9 Two instruments include the measurement ofrigidity: 1) the Columbia Scale7 and 2)Webster's Parkinson's disease ratingscale.9 Both scales define criteria forclinically assessing the degree of rigidityin a pa tient's trunk or limbs at any givenpoint in time. TheColumbia Scale defines criteria for five degrees of rigidity:absent, slight, mild, marked, and severe.7 Webster's scale defines criteria forfour degrees of rigidity: absent, mild,moderate, and severe.9Researchers have stated that the advantage of clinical methods ofassessingparkinsonian signs and symptoms arespecificity, sensitivity, and low cost.1,6The disadvantage of clinical methodscan be in the lack of consistency between observers or within an observerusing the rating scale.1,610 This disadvantage is particularly im portan t if clinical decisions about patient treatmentand management are to be determined from the findings of a clinical scale. 11,12(pp53-68) Of the above-mentioned scales that measure signs an dsymptoms of Parkinson's disease, inter

rater reliability has been reported forselected clinical features of the Columbia Scale.10 The clinical features evaluated were bradykinesia, gait disturbance, postural abnormality, and tremor.Rigidity, although part of the ColumbiaScale, was not evaluated. The samestudy also reported interrater reliabilityfor Hoehn and Yahr's disability staging.10 The Northwestern University Disability Scale5 was examined for interrater reliability, but the study did not include the measurement of rigidity. Noreliability studies have been reported forWebster's Parkinson's disease ratingscale.9Because of the minimal attentiongiven to the reliability of clinical ratingscales for Parkinson's disease, we decided to test the interrater reliability ofthe rigidity scale used in our clinic. Wedeveloped our own clinical rating scaleto more precisely identify operationalcriteria and testing procedures for therating of rigidity tha t we felt w ere lacking in the above-mentioned instruments. We defined the scale based onthe description of rigidity by Ada ms an dVictor,13 who defined rigidity as a hy-pertonus characterized by a sustainedinvoluntary m uscle contraction that affects both flexor and extensor muscles.Our rigidity scale was developed as p artof a composite assessment used withpatients with Parkinson's disease. Thecomposite assessment is currently beingused to docum ent the baseline status ofselected clinical features and to monitor

L . VanDillen , MHS-PT, is Instructor, Programin Physical Therapy, Washington University, POBox 80 83,6 060 S Euclid Ave, St. Louis, MO 63110,an d a doctoral candidate, Department of Psychology, Washington University. Address correspondence to 4355 Maryland Ave, #127, St. Louis , MO63108 (USA) .K. Roach, MHS-PT, is Research Coordinator,Health Services Research Division, Edward J. HinesVeterans Administration Hospital, Fifth Ave andRoosevelt Rd, Hines, IL 60141 , and a doctoralstudent, Department of Epidemiology, School ofPublic Health, University of Illinois at the MedicalCenter, Chicago, IL.Ms. Van Dillen and Ms. Roach were graduatestudents at Washington University Medical School,St. Louis, M O, when this study was conducted. Thisstudy was completed in partial fulfillment of therequirements for Ms. V an Dillen's and Ms. Roach'sMaster of Health Science in Physical Therapy de-gree, Washington University M edical School.This article wasadapted from a presentation atthe Sixtieth Annual Conference of the AmericanPhysical Therapy Association, Las Vegas, NV, J une17-21 , 1984.This article wassubmitted March 25, 1988, andwas accepted May 23, 1988. Potential Conflict ofInterest: 5.

Volume 68 / Number 11 , November 1988 1679

8/2/2019 Inter Reliability

2/3

changes of status throughout a day asmedications are adjusted.14The purposes of this study were 1) todescribe the clinical rating scale and testing procedure we use to assess rigidityin patients with P arkinson's disease and2) to examine the interrater reliabilityof the rigidity scale when used with patients with Parkinson's disease. We expected that the two clinicians participating in the study would be reliable intheir measurement of rigidity using theclinical scale described.METHODS u b j e c t s

Twenty patients (3 women, 17 men)treated in the Movement DisordersClinic and acute care service of a majoruniversity-based hospital gave their informed consent to participate in thisstudy. Subjects' mean age was 64 years(s = 16.3). Each subject was diagnosedby a neurologist as having Parkinson'sdisease with rigidity as p art of the clinical syndrome. Criteria for study participation were 1) physician-documentedrigidity; 2) ability t o follow simple one-step verbal directions; 3) ability to attainat least 75% of the standard passive-range-of-motion (PROM) measurements of the elbow, forearm, and wristof the tested upper extremity (UE ); and4) diagnosis of Parkinson's disease.R a t in g S c a l e

Th e degree of rigidity was assessed bya defined measurement scale we developed. The scale was based on a reviewof the literature and on our own clinicalexperience. The four-point rating scalewas defined as follows:1. 0 (Absent)Normal muscle tone ispresent. No resistance to passivemovem ent is detected.2. 1 (Slight)Resistance to passivemovement can be detected, but theresistance is mild and inconsistentthroughout the PROM .3. 2 (Moderate)Resistance to passivemovement is detected consistentlythroughout the PROM, but fullavailable PROM is easily obtained.4. 3 (Severe)Resistance to passivemovement requires maximal effortby the rater to obtain the full PRO M,or full PR OM cannot be attained.Procedure

Two physical therapists (L.V.D. andK.E.R.) each used a standardized set of

instructions and test movements thatthey had developed. After agreeing onthe procedure for rating and the operational definitions for rigidity, a practicesession was conducted. During this session, the raters became familiar with theuse of the rating scale and practiced thestudy protocol on healthy individualsand patients with Parkinson's disease.Each subject's UE PROM was assessed in the supine position before rating the degree of rigidity. Each subjectwas then rated in the sitting positionwith the back supported. The UE to berated was chosen on a random basis.Identical verbal instructions were givento each subject and include d the following: The first rater would be m oving thesubject's arm, and the subject was torelax, allowing the tester to hold the a rmcompletely, and neither resist nor assistin the movement. The rater thengrasped the subject's elbow with onehand so that the subject's upper arm wassupported by the rater's forearm. Withher other hand , the rater grasped aroundthe dorsum of the subject's hand andthen performed a series of randommovem ents of the extremity. The m ovements consisted of varying degrees ofelbow flexion and extension, forearmpronation and supination, and wristflexion and extension. Each joint wastaken through the full available PROMat least once during a test. The randommovements were performed at varyingspeeds within an individual test, witheach test lasting no longer than one m inute. The degree of rigidity detected wasthen recorded by the rater on a datasheet. The same procedu re was repeatedby the second rater immediately afterthe first rater. N either rater w as allowedto observe the other rate r during testing.Raters alternated being first and secondin the testing procedure u ntil all subjectswere assessed. Data rem ained confidential until completion of the study.

D a ta A n a ly s isWe analyzed agreement between thetwo raters with Cohen's weightedKappa.15

RESULTSThe Table shows the agreement between raters. The raters agreed in 16 out

of 20 trials (80%) and never disagreedby more th an a single level on the ra tingscale. Of the 4 disagreements, neitherrater consistently graded lower than theother rater. A weighted K appa value of.636 (p = .20) was obtained.DISCUSSION

The results of this study indicate ahigh percentage of observed agreement(80%) between raters in estimating thedegree of rigidity detected in the subjects' UE. The results of the statisticalanalysis with Cohen's weightedKappa,15 however, showed that the levelof reliability achieved when agreementcaused by chance was considered was.636 and was not significant at the levelof .05. The discrepancy between the percentage of observed agreement and theobtained weighted K appa value can partially be explained by the nonuniformdistribution of our sample and the relatively small samp le size. As stated by M .J. Strubbe (unpublished report), "Theconsequence of deviating from auniform distribution is to increase theinstability of the Kappa statistic." Themajority of our subjects (90%) wereclassified in the 1 or 2 rigidity category(Table). The effect of such a nonuniform distribution is to increase the variance of the weighted Kappa, thus diminishing its strength as an estimate ofreliability. The small sample size (N =20) we used may also have been a contributing factor to the discrepancy observed. Because the sample size was

TABLEInterrater Agreement8 Betwe en Therapists Rating De gr ee of Rigidity in Upper Extremi-tie s of S ubject s with Parkinson's Disea se (N = 20)Sca le Ca te g oryb

01Rater 2 23Sum

00(0)0000

127(4)1010

Rater 12

019(5)010

30000(0)0

S um2810020

a Kw = .636, p = .20; number in parentheses indicates number of agreements expected onthe hypothesis of chance a ssociation.b O = absent, 1 = slight, 2 = moderate, 3 = severe.

1680 PHYSICAL THERAPY

8/2/2019 Inter Reliability

3/3

RESEARCHsmall and the number of subjects wasused in the calcu lation of the p robabilityvalue, an increase in sample size alonemay improve the level of significance.An increase in the sample size, if selected to represent the actual spectrumof patient types, should also improvethe uniformity of the distribution andpotentially improve the stability of theweighted Kappa as an indication ofreliability. 12(pp638-640)We a ttribute the observed agreementin this study to the raters' involvementin the development of the scale and totheir practice in using the rating scalewith patients.16 We also attribute thehigh degree of reliability to our attemptto operationally identify the testing procedure and define the rating scale asprecisely as possible. Although both theColumbia Scale7 and Webster's scale9include the rating of rigidity, we foundboth scales to be lacking in definitionand standardization.As discussed previously, testing wasperformed at the wrist, forearm, andelbow. Both raters noted during thestudy that, in some instances, the degreeof rigidity varied among these joints,making rigidity difficult to estimate forthe entire UE. Upon completion of thestudy, this difficulty was discussed. Wedecided that ou r rating in these instanceswas based most frequently on the greatest amou nt of resistance encoun tered.

Schwab statedIt is well known that hypertonia variesfrom day to day in each patient andeven from hour to hour. A host ofvariable and unknown factorsanxiety, tension, volitional effort, alteredalertness, and relaxationchangesthe level of the hypertonia, therebyconfounding our efforts to measure

it quantitatively and estimate itclinically.2Although testing was performed in astandard method and within a shorttime frame, the actual degree of rigiditymay have varied somewhat between rating attempts. This variance may havecontributed to some of the disagreementbetween the raters in rating subjects inthis study.The definition of "moderate" rigidityin the scale we used encompassed a widerange of hypertonia. If the grade of moderate could be separated into two distinct ratings, the sensitivity of the scalecould potentially be increased. Thischange could increase the overall reliability of the scale but would requirefurther testing.CONCLUSION

Rigidity is a sign frequently identifiedin patients with Parkinson's disease.Along with other clinical signs andsymptoms of Parkinson's disease, rigidity is monitored to determine the natural history of the disease and to establishthe efficacy of treatment and its effecton function. Standardized and reliablemeas uremen t of clinical signs and symptoms, including rigidity, therefore, isnecessary. We developed and tested aclinical scale of rigidity and found thatour observed agreement was 80%.When statistical analysis was performed,we found that our agreement beyondchance was 63.6% and was not significant. W e have discussed the factors th atmay have contributed to this discrepancy. Further clinical investigations ofthe rigidity scale should increase thenumber and spectrum of subjects used.

REFERENCES1. Marsden CD, Schachter M: Assessment ofextrapyramidal diso rders. Br J Clin Pharmacol11:129-151,19812. Schwab RS : Problems in the clinical estimationof rigidity (hypertonia). Clin Pharmacol Ther5:942-946, 19643. Brumlik J, Boshes B: Quantitation of muscletone in normals and Parkinsonism. Arch Neurol4:399-406,19614. Larsen TA, Calne S, Calne DB: Assessment of

Parkinson's disease. Clin Neuropharmacol7:165-169,19845. Canter GJ, La Torre R, Mier M: Method forevaluating disability in patients with Parkinson's d isease. J Nerv Ment Dis 122:143 -147,19616. Ward CD, Sanes JN, Dambrosia JM, et al:Methods for evaluating treatment in Parkinson's disease. In Fahn S, et al (eds): Experimental Therapeutics of Movement Disorders.New York, NY, Raven Pr ess, 1983 , vol 37, pp1-77. Montgomery GK: Parkinson's disease and theColumbia scale. Neurology 3 4:55 7-55 8,19 848. Hoehn MM, Yahr MD: Parkinsonism: Onset,progression and mortality. Neurology 5:427-442,19679. Webster DD: Critical analysis of the disabilityin Parkinson's disease. Modern Treatment5:257-282, 196810. Montgomery GK, Reynolds NC, Warren RM:Qualitative assessment of Parkinson's disease: Study of reliability and data reductionwith an Abbreviated Columbia Scale. Clin Neuropharmacol 8:83-92, 198511. Rothstein JM: Measurement and clinical practice: Theory and application. In Rothstein JM(ed): Measurement in Physical Therapy: Clinicsin Physical Therapy. New York, NY, ChurchillLivingstone Inc, 1985, vol 7, pp 1-4612. Feinstein AR: Clinical Epidemiology: The Architecture of Clinical Res earch. Philadelphia, PA,W B Saunders Co, 1984, pp 53-68 , 638-64013. Adams RD, Victor M: Abnormalities of movement and posture due to disease of the extrapyramidal motor sy stem . In Adams RD, VictorM: Principles of Neurology, ed 2. New York,NY, McGraw-Hill Book Co, 1981, p 5314. Van Dillen LR, Nuessen J, Montgomery E, etal: A description of an all-day Parkins on's evaluation. Abstract. Phys Ther 68:86 4,198 815. Cohen J:A coefficient of agreement for nominalscales. Educational and Psychological Measurement 20(1 ):37-46,1 96016. Garraway WM, Gore AS, Prescott RJ, et al:Observer variation in the clinical assessmentof stroke. Age Ageing 5:233 -240 ,1976

Volume 68 / Number 11 , November 1988 1681