Educational Assessment of Students7e
Susan M. Brookhart & Anthony J. Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc. All Rights Reserved
Chapter 17
Interpreting Norm-Referenced Scores
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #1
• A referencing framework is a structure used to compare a student’s performance to something external to the assessment in order to interpret performance.
A norm-referencing framework interprets a student’s assessment performance by comparing it to the performance of a well-defined group of other students who have taken the same assessment.
A criterion-referencing framework interprets a student’s performance according to the kinds of performances a student can do in a domain.
A standards-referenced framework combines elements of both.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #2
• Use normative information to describe student strengths, weaknesses, and progress.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #3
• Test publishers may provide norm-referenced scores based on information from several different norm groups.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Norm Groups
• Local norm group: students in the same grade in the same school district
schools/districts should have scores
publishers may offer scores
• National norm group: intended to be representative of students in the country
test publishers use different norming procedures
• Special norm group
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Norm Groups
• School averages norms: ranked tabulation of the average (mean) score from each school building in a national sample of schools
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Guidelines for Using Publishers’ Norms
• Make sure the norm group is:
relevant
representative
recent
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #4
• Different types of norm-referenced scores are constructed to serve different purposes.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #5
• The percentile rank tells the percentage of the students in a norm group who have scored lower than the raw score in question.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Percentile Ranks: Advantages
• Easily understood.
• Clearly reflect norm-referencing.
• Permit a person’s performance to be compared to a variety of norm groups.
• Can be used to compare a student’s relative standing in each of several achievement or ability areas.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Percentile Ranks: Limitations
• Can be confused with percentage correct scores.
• Can be confused with some other types of two-digit derived scores.
• Do not form an equal-interval scale.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #6
• A linear standard score tells how far a raw score is from the mean of the norm group, expressing the distance in standard deviation units.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Z Scores
• Communicate students’ norm-referenced achievement expressed as a distance away from the mean.
if Z = -1.5, the student’s score is 1.5 standard deviations below the average score.
• Can be used for norm-referenced comparison of raw scores with different metrics
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
SS Scores
• Tells the location of a raw score in a distribution having a mean of 50 and a standard deviation of 10.
transformation of z score
if Z = -1.5, SS = 35
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #7
• A normal distribution is a mathematical model (an equation) based on the mean and standard deviation of a set of scores.
Normal curves are smooth, continuous, symmetrical, and bell shaped.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #8
• Normalized standard scores are based on transforming raw scores on an assessment to make them fit a normal distribution.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Normalized z -scores
• z-scores that have percentile ranks corresponding to what we would expect in a normal distribution
Raw Score Percentile rank Normalized
standard (zn)
Linear
standard (z)
36 98 2.05 2.43
33 96 1.75 1.64
15 4 -1.75 -3.09
14 2 -2.05 -3.36
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Normalized T-scores
• Tell the location of a raw score in a normal distribution having a mean of 50 and a standard deviation of 10. Joey’s T -score is 40, which means he is one
standard deviation below the mean of the norm group, and his percentile rank is approximately 16.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Deviation IQ Scores
• Tells the location of a raw score in a normal distribution having a mean of 100 and a standard deviation of 15 or 16.
Meghan has DIQ = 116, which means she has scored one standard deviation above the mean of her age group and the percentile rank of her score is 84.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Stanines
• Tells the location of a raw score in a specific segment of a normal distribution.
Blake’s stanine on the spelling subtest of the standardized test was 3, which means that his raw score was in the lower 20% of the norm group. Specifically, his percentile rank was between 11 and 22.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
SAT-Scores
• Historically, a normalized standard score from a distribution that has a mean of 500 and a standard deviation of 100.
No longer use this computation, but the current scores on the 200-800 scale can be compared across administrations
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Normal Curve Equivalents
• Normalized standard score with a mean of 50 and a standard deviation of 21.06.
Primary value is evaluating gains from various educational programs that use different publishers’ tests
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #9
• Developmental and educational growth scales are norm-referenced scores that can be used to chart educational development or progress.
extended normalized standard score scale
grade-equivalent score scale
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #10
• An extended normalized standard score tells the location of a raw score on a scale that is anchored to a lower grade reference group.
based on extended z scale OR
based on item response theory
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #11
• A grade-equivalent score tells the grade level at which a raw score is average.
Useful for reporting educational development.
Provided by test publisher.
• GE is the median score (sometimes mean score) in each grade’s norm group.
• A third grader’s GE score of 5.7 on a mathematics test covering third-grade content does not mean that this student should be placed in fifth-grade math.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Things to Keep in Mind When Interpreting Grade Equivalents
• In some subject areas, students’ performance drops over the summer months.
• The meaning of grade-equivalent scores for a subject depends very much on the subject matter.
• Grade-equivalent scores do not necessarily indicate mastery of the material.
• The more closely the test items match the material emphasized in the classroom before the test was administered, the more likely the students will score well above grade level.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Things to Keep in Mind When Interpreting Grade Equivalents
• Grade equivalents from different tests cannot be interchanged.
• Grade equivalents for different subjects cannot be compared.
• Grade equivalents do not indicate “normal” growth.
• The grade-equivalent score scale does not have a one-to-one correspondence with the number of questions a student answers correctly on a test.
• Grade mean equivalents are also problematic.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Key Concept #12
• Five guidelines for score interpretation will serve you well:
Look for patterns in scores.
Seek explanations for the patterns.
Don’t expect many surprises.
Don’t overinterpret small differences.
Use evidence from other assessments to clarify interpretations.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Typical Misunderstandings
• The grade-equivalent score tells which grade the student should be in.
THEY DO NOT.
• The percentile rank and percent-correct scores mean the same thing.
THEY DO NOT.
• The percentile rank norm group consists of only the students in a particular classroom.
IT DOES NOT.
• “Average” is the standard to beat.
IT IS NOT.
Educational Assessment of Students, 7eBrookhart & Nitko
Copyright © 2015, 2012, 2009 by Pearson Education, Inc.
All Rights Reserved
Typical Misunderstandings
• Small changes in percentile ranks over time are meaningful.
THEY ARE NOT.
• Percent-correct scores below 70 are failing.
THEY ARE (USUALLY) NOT.
• If you get a perfect score, your percentile rank must be 99.
IT MAY NOT BE.