Date post: | 20-Dec-2015 |
Category: |
Documents |
View: | 254 times |
Download: | 5 times |
Chapter 5
Instrument Selection, Administration, Scoring, and Communicating Results
Selection of an Assessment Instrument
Identify possible alternatives Educational Testing Services may be
useful Mental Measurement Yearbook (MMY)
reviews many tests and is available at Morris or reviews can be ordered on-line for a fee.
Client’s lives can be adversely affected by selecting and using a faulty instrument
Evaluating an Instrument Test Purpose Instrument
Development Appropriate Norm
Group or Criterion Reliability
Validity Bias Interpretation and
Scoring User Qualifications
Test purpose
Does instrument meet counseling needs?
Manual may state purpose Purpose may not relate to name Purpose may not be met because of
construction problems
Instrument development
Construction of items crucial Look for detailed item analysis data Face validity
Selection of norm group
Is norming group appropriate for client? Age Gender Ethnicity Socioeconomic representation Educational level Geographic location
Selection of criterion
How did developers determine criterion? Standards from professional
organizations Reviews by leading researchers Studies of curriculum
Manual must supply enough data to be convincing
Reliability
No clear guidelines for what constitutes “good” reliability
Type of instrument affects how reliability should be viewed
Can be estimated in different ways and manuals will often provide range of coefficients calculated using different methods
Validity
Look for preponderence of evidence Manual must provide enough
information to judge GO TO THE LITERATURE!
Bias
Test bias in context of multicultural issues Differential item functioning (item
difficulty differs for ethnic groups) Reliability coefficients may differ
ethnically Construct validity may be compromised
as the construct may be culturally based
Interpretation and scoring materials
Manual must provide adequate description of what scores mean
Hand vs. machine scoring
User qualifications Level A: manual is sufficient (Holland’s Self-
Directed Search) Level B: master’s degree or equivalent
training (Myers-Briggs, Strong Interest Inventory)
Level C: doctoral level or equivalent training (Wechslers, Binet, MMPI-2)
Code of Fair Testing Practices in Education ACA Code of Ethics, Section E
Practical issues
Cost Time
Evaluating instruments
Form in text, page 95 Instrument evaluation paper
Administering Assessment Instruments
Pre-read administration materials Follow standardized procedures
Administrator Effects
Expectancy (pygmalion effect) Some data support minor effects
Examiner/Examinee Relationship
Some studies support impact of relationship for kids and even more so for lower SES kids
Standardized test manuals usually address relationship boundaries
Examiner race
Research equivocal
Scoring
Hand scoring Clients can score some assessments
(SDS) but may be clinical reason not to Some tests require multiple templates
Computer scoring and interpretation Few to no errors Counselors ethically bound to know
integrity of service and steps taken to develop scoring and interpretation programs
Performance and Authentic Assessment Scoring
Associated with achievement testing (rather than grammar test, write business letter)
Goal is to see if knowledge can be applied
Big issue is objectivity of scoring
Performance and Authentic Assessment Scoring Guidelines
Assessment has specific focus
Scoring plan is based on observable qualities
Scoring is designed to reflect the intended target
Setting for assessment is appropriate
Checklist or rating scales are used
Scoring procedures have been field tested before they are used
Communicating Results
Must know manual information Optimize power of test (It’s useful in
these ways for these reasons) rather than allowing client to maximize (test speaks truth)
Use effective counseling skills Develop multiple methods of
explaining in “plain” English
Communicating Results (cont’d)
Use descriptive terms rather than numerical scores and tie them to the reason for the assessment
Put results in context of other client information
Involve client in interpretation and ask for feedback often
Communicating Results (cont’d)
Limitations are discussed in non-technical terms
Encourage client to ask questions Summarize results to iterate and
stress important points
Communicating Results to Parents
Be empathic Monitor parents’ reactions so child
does not internalize as blame Provide therapeutic environment