3/27/17  

1  

VALID ASSESSMENTS WHAT DO THE APA STANDARDS AND THE AMA GUIDES TELL US?

! Arthur D Williams, PhD PSY6047 ! williamsart35@gmail.com !  Diplomate, American Board of Professional

Neuropsychology

!  Fellow, National Academy of Neuropsychology

!  Fellow, American College of Forensic Psychology

!  Licensed: CA, WA. Registered: BC

!  Former Assistant Professor: University of Washington and Pepperdine University

AMA Guides to the Evaluation of Permanent Impairment (6th Edition)

! used in workers' compensation systems, federal systems, automobile casualty and personal injury cases to rate impairment, not disability.

!  The Standard in Impairment Evaluation

Impairment and Disability

!  Impairment is a significant deviation, loss, or loss of use of any body structure or body function in an individual with a health condition, disorder or disease.

!  Disability is activity limitations and/or participation restrictions in an individual with a health condition, disorder or disease.

Do Not Rely Solely on Self-Report

! Subjective complaints that are not clinically verifiable are generally not ratable under the Guides.

3/27/17  

2  

Purely Subjective Tests

! It is well-appreciated that tests touted as being ‘objective tests’ such as the Beck Depression Inventory, but which are scored solely by the patient, are purely subjective.

Neuropsychological Testing

! The Halstead Reitan Test Battery…is the best standardized neuropsychological test battery.

Mild Brain Injury

!  Special mention should be made of mild traumatic brain injury (MTBI), which has been the subject of extensive research in the last ten to twenty years. In contrast to previously held belief, the symptoms of mild traumatic brain injury generally resolve in days to weeks, and leave the patient with no impairment.

! Patients with persistent postconcussive syndrome generally have noninjury related factors which complicate their clinical course. Postconcussive syndrome is a relatively rare sequelae of MTBI, seen in 1-5% of all MTBI patients.

AMA Guides to Disease and Injury Causation (2nd Edition)

DSM

! “If a legal claim of causation of mental illness is going to be made, the associated mental health evaluation should comply with the modern standards for mental illness diagnosis as documented in the American Psychiatric Association’s diagnostic manual…and its companion texts….

 

3/27/17  

3  

! Evaluators who stray from that system are violating professional standards and risk entering into a realm of absolute chaos where there are no standards whatsoever….There is no credible justification for introducing total chaos into the legal system….

The Following Characteristics

In order to be credible, clinical or forensic conclusions or opinions that are offered in support of a legal claim of mental illness must include the following characteristics:

Treating vs Forensic Roles

!  Treating clinicians should not become involved in forensic issues (such as causation) involving their patients. Forensic evaluators should not provide treatment for forensic examinees. Treating and forensic roles should be completely and permanently separated for any individual case.

 

Diagnostic Standards or Scientific Findings

!  For diagnosis, claims must be guided by diagnostic standards or scientific findings. Idiosyncratic definitions of a claimed mental illness, and idiosyncratic approaches to diagnosis, must be avoided. If diagnostic standards are not adhered to, the burden of proof is placed on the person who strays from those standards to reference credible scientific support for his or her avoidance of the standards….

History

!  The evaluation…must involve consideration of the examinee’s mental health history and general medical history, including a review of prior general medical records, records of mental health care, and other pertinent records. The evaluator should recommend that he or she be given an opportunity to review such records for the examinee’s entire life. No such evaluation is credible without a thorough investigation of the examinee’s past….

Do Not Rely Solely on Self-Report

!  Such claims/conclusions/opinions will not be sufficient if they rely exclusively on reports from an examinee….

 

!  The evaluator should not rely on idiosyncratic criteria, subjective reports and impressions, or exclusively on clinical experience in assessing psychological impairment.

3/27/17  

4  

Personality Disorders

! An associated evaluation is deficient if it does not include an evaluation for personality disorders….

 

!  Structured Clinical Interview for DSM-5

!  Structured Clinical Interview for DSM-5 for Personality Disorders

Testing

! Scientifically validated psychological testing should be utilized to enhance the objectivity, thoroughness, and credibility of the evaluation process….

No Conjoint Interview

! The evaluator should not interview the examinee or any collateral informant when both are present (each interview should take place in the absence of the other interviewee)….

Do Not Rely Solely on Self-Report

The extremely unreliable nature of reports from claimants/plaintiffs should be highlighted. Such unreliable reports cannot credibly serve as a primary basis for an evaluator’s conclusions….

AMA Causation (2008)

! Research findings have indicated that reliance on an examinee’s self-reported history is always a less-than-credible practice because of a tendency for examinees to underreport preexisting symptoms, and because of a tendency for reported histories to be influenced by an examinee’s beliefs about the causes of his or her current problem.

Post Hoc Ergo Propter Hoc

! Logical Fallacy

! Simply because symptoms or behaviours occurred after an event that they were caused by that event.

3/27/17  

5  

Depression and Work

! Research findings have reliably indicated that work is a protective agent in regard to [Major Depressive Disorder]. It is not a risk factor and therefore not a cause….

! Work has repeatedly been demonstrated to have an inverse relationship to MDD. …Genetics reportedly plays a dominant etiological role, and childhood experience also appears to play a ‘compelling’ role….

Depression and Causation

!  The chronic nature of MDD is implied by scientific findings that indicate that an individual who has had 1 major depressive episode any point in his or her life is probably going to experience, or has already experienced, other major depressive episodes.

! In most cases, an MDD-like presentation is going to be a chronic, recurring pattern rather than a focused response to a specific experience. Subsequently, it becomes extremely difficult to justify a claim of recent causation for any single demonstration of a major depressive episode….

PTSD

! Scientific findings have established preexisting mental illness as a dominant risk factor for PTSD-like presentations. For example, research findings have indicated that PTSD-like presentations ‘almost always develop in the context of other (preexisting) mental disorders.

Personality Disorders

! The central role of personality disorder presentations in all types of medical-legal claims is another relatively simple example of the importance of considering the entire clinical presentation.

! Evaluation for a personality disorder is a standard part of the diagnostic system for mental illness.

3/27/17  

6  

AMA Guides to the Evaluation of Work Ability and Return to Work (2nd Edition)

! Consensus Statements of the

! Canadian Medical Association,

!  the American College of Occupational and Environmental Medicine, and

!  the American Academy of Orthopaedic Surgeons

AMA Return to Work

! All three documents strongly recommend that physicians return patients to their usual work roles as soon as possible.

! Prolonged absence from one’s normal roles, including absence from the workplace, is detrimental to a person’s mental, physical, and social well-being. Physicians should therefore encourage a patient’s return to function and work as soon as possible after an illness or injury.

Standard of Care

Standards for Educational and Psychological Testing

! All applicable standards should be met by all tests and in all test uses unless a sound professional reason is available to show why a standard is not relevant or technically feasible in a particular case (page 1).

Validity is…

! the most fundamental consideration in developing and evaluating tests….Evidence of the validity of a given interpretation of test scores for a specified use is a necessary condition for the justifiable use of the test….

! Identifying the propositions implied by a proposed test interpretation can be facilitated by considering rival hypotheses that may challenge the proposed interpretation….as in all scientific endeavors, the quality of the evidence is paramount….

3/27/17  

7  

! The test user is ultimately responsible for evaluating the evidence in the particular setting in which the test is to be used….Support is needed for the propositions underlying each interpretation for a specific use.

Norms and Samples

!  It is important that the norms be based on a technically sound, representative sample of test takers of sufficient size.

! Mitrushina et al. (2005): “[A] sample size of 50 is typically viewed as adequate….”

WAIS-IV and Mild Brain Injury

!  Two studies with small samples <50 subjects

! Dikmen et al. is model

Cut Scores

! Partition the distribution into categories which are the basis for interpreting test results

! What is impairment?

! 16%

! 5%

! 2%

! Are average scores impaired?

!  False negatives versus false positives

Standards for Validity

! 1.3: If validity for some…interpretation for a given use has not been evaluated…, that fact should be made clear and potential users should be strongly cautioned about making unsupported interpretations.

! 1.4: If a test score is interpreted for a given use that has not been validated, it is incumbent on the user to justify the new interpretation for that use, providing a rationale and collecting new evidence, if necessary.

3/27/17  

8  

Conners’ CPT-II

! “The instrument is helpful when a diagnosis of ADHD is being considered.”

!  In both clinical and nonclinical contexts, the CPT II can be used as a relatively quick screening device to determine whether further investigation might be required.

CPT-II

!  Diagnostic conclusions cannot be made based on a screening test.

!  223 people in the Neurologically Impaired Adult Clinical Sample

!  18-34: 75 !  35-55: 88 !  50+: 60 !  29% were “postconcussive” and only 6% had

“concussion with brief loss of consciousness.”

Interpretations

! 1.14: When interpretations of subscores, score differences, or profiles is suggested, the rationale and relevant evidence in support of such interpretation should be provided.

Test of Premorbid Functioning

! Average score

! Prediction interval: 68-130

Discrepancy Analysis

! WAIS-III/WMS-III

!  [Clinicians] are disadvantaged by the limited clinical data yet available. The validation of [discrepancy analysis] with these instruments calls for the generation of data across a broad range of clinical circumstances and conditions, and there remains much work to be done”

! Concern about false positives

Construct-Irrelevant Components

!  The degree to which test scores are affected by processes that are extraneous to the test’s intended purpose

3/27/17  

9  

Reading Test

! Material too far above or below the level to be tested

! Emotional reaction to the test content

!  Familiarity with the subject matter

! Writing skill needed to compose a response

Standardized Administration

! 3.4: [Users] should adhere to standardized test administration, scoring and security protocols so that test scores will reflect the construct(s) being assessed and will not be unduly influenced in the testing process.

Translating a Test

! 3.12: When a test is translated and adapted from one language to another,… [users] are responsible for describing the methods used in establishing the adequacy of the adaptation and documenting empirical or logical evidence for the validity of test score interpretations for intended use.

Language

! A test should be administered in the language that is most relevant and appropriate to the test purpose.

!  Linguistic and cultural characteristics

! Relative language proficiencies of examinees who speak multiple languages

!  Test takers should be tested in the language in which they are most proficient

Establishing Relative Language Proficiencies

!  Such procedures may range from self-identification to formal language proficiency testing.

! Determination of a test taker’s most proficient language for test administration does not automatically guarantee validity of score inferences for the intended use.

Use of Interpreters

! Unless a test has been standardized and normed with the use of interpreters, their use may need to be viewed as an alteration that could change the measurement of the intended construct.

!  Introduction of a third party during testing

! Modification of the standardized protocol

3/27/17  

10  

Interpretation

! 9.0: Test users are responsible for knowing the validity evidence in support of the intended interpretations of scores on tests that they use, from test selection through the use of scores, as well as common positive and negative consequences of test use.