31
1 NWEA MAP Measures, Metrics and Cut- Points for Alternative Education Campuses Colorado Department of Education July 10, 2012
1
NWEA MAP Measures, Metrics and Cut-Points for Alternative
Education Campuses
Colorado Department of EducationJuly 10, 2012
2
Agenda
• Purpose• Selection of Measures Process & Considerations• Metrics and Cut-Points for NWEA MAP Achievement
Results• Metrics and Cut-Points for NWEA MAP Growth Results• Calculating NWEA MAP Growth Results• Timeline
3
Purpose
• Explain methodologies for calculating and reporting NWEA MAP assessment results for accountability purposes on the AEC SPF report (metrics and cut-points).
• Answer questions related to NWEA MAP assessment results for accountability purposes on the AEC SPF report.
4
Selection of Measures Process• AECs submitted the Selection of Measures Form July 2.
– http://www.cde.state.co.us/Accountability/Downloads/AECSelectionOfAccountabilityMeasures.xlsx
• There are a set of state required AEC measures:– Achievement: CSAP/TCAP % proficient/advanced– Growth: CSAP/TCAP Median Growth Percentiles– Postsecondary: Completion rate, Dropout rate, ACT composite– Engagement: Average daily attendance, Truancy rate
• Districts may select additional measures, subject to CDE approval.
5
Considerations for Measures• The three most recent years of data will be included.• AECs will receive indicator ratings similar to the
traditional SPF:– Does not meet, approaching, meets, or exceeds AEC norms
• Each indicator/sub-indicator:– Uses adjusted AEC cut-points relative to each other– Typically uses either normative cuts of 40th/60th/90th percentile
or head counts of 40%/60%/90%• Where there are common measures across districts,
common metrics and cut-points will be used.
6
Considerations for Overall Results
• Based on the indicator ratings, AECs receive an overall point total and plan type.
• AECs engage in the same UIP process as all other schools, and have the same consequences for performance.– Note: There is a modified UIP template for AECs to reflect the
modified performance indicators.
7
Metrics and Cut-Points for MAP Academic Achievement (Status) Results
Metrics Cut-Points
% of students scoring at grade level on spring assessment • Exceeds – at/above 90%
• Meets – below 90% but at/above 60%• Approaching – below 60% but at/above 40%• Does Not Meet – below 40%
% of students increasing at least one grade level between pre- and post-test administrations
Of students continuously enrolled for at least 8 weeks
8
Metrics and Cut-Points for NWEA MAP Academic Growth Results
Metrics Cut-Points
Targets based on seasonal assessment windows:• Fall to Winter Growth• Winter to Spring Growth• Fall to Spring Growth
• Exceeds – at/above 90%
• Meets – below 90% but at/above 60%
• Approaching – below 60% but at/above 40%
• Does Not Meet – below 40%
Targets based on 8-week enrollment intervals:• 8 weeks• 16 weeks• 24 weeks, etc.
% of students achieving target growth from one test administration to the next
9
Calculating NWEA MAP Growth Results
• The following slides have been adapted from documents written by Dr. Jody Ernst:– NWEA Use Guidance (07.05.12)
– NWEA AEC Research Findings to Inform How to Best Articulate the Assessments Use for Purposes of AEC Accountability
– http://www.cde.state.co.us/cdedocs/OPP/NWEAAECWriteUp.pdf
10
Calculating NWEA MAP Growth Results
• The results and norms from these studies are all based on Colorado AEC MAP data from 2008-2010
• Insufficient data were available to calculate Colorado AEC growth norms for Science, but Reading, Mathematics and Language Usage norms have been published
11
NWEA MAP Characteristics• Grades K-11 (or students with grade level
equivalents of K-11)• Reading, Mathematics, Language Usage, Science*• Fall, Winter, Spring assessment windows• Student-level results are RIT scores
* As noted previously, Science growth targets are not available
12
Pulling NWEA MAP Data• Export NWEA data for all subjects in which most
students in a grade take the assessment
• Include 3 years of data or as many years as are available if your school only recently started giving NWEA MAP
13
Formatting NWEA MAP Data• For each student, compute the difference, in
weeks, between each testing occasion (i.e., fall to winter, winter to spring, and/or fall to spring)
• Only results based on assessments that were at least eight weeks apart may be used.
14
Calculating each Student’s Pre-test Grade Equivalent
• Using the Fall/Winter/Spring RIT Ranges and Fall/Winter/Spring RIT Grade Equivalents, compute the grade level equivalent for each student’s pre-test RIT score
• The following tables show the RIT ranges and corresponding grade equivalents for the Fall MAP administration
15
For example, if a student receives a RIT score of 219, they have a Reading level equivalent to that of an 8th grader (regardless of their assigned grade). This value is used as their pre-test grade equivalent.
16
17
18
Calculating RIT Growth• Calculate the observed amount of pre-test to post-test growth for each student according to the following:
– Fall to Winter = Winter RIT – Fall RIT– Winter to Spring = Spring RIT – Winter RIT– Fall to Spring = Spring RIT – Fall RIT
19
Calculating Target RIT Growth using Seasonal Assessment Windows
• If most students are consistently enrolled in your AEC and take the MAP assessment at seasonal intervals, the easiest option for finding a student’s growth target is to use the tables previously published by Jody Ernst.
• For each student, use the assessments that give the longest window between pre- and post-test administrations.
20
Calculating Target RIT Growth using Seasonal Assessment Windows
• Using a student’s pre-test grade equivalent and the duration between pre- and post-test administrations, determine the appropriate growth target for that student
• The following tables show the RIT growth targets for each combination of starting grade equivalent and test window.
21
For example, the student at an 8th grade equivalent for Fall (RIT score of 219), is expected to increase their Spring RIT score by 3 points (to a RIT of 222). For AEC reporting purposes, 3 RIT points is this student’s target growth.
22
23
24
Calculating Target RIT Growth using 8-week Enrollment Windows
• If students enroll in your AEC throughout the year and for radically differing lengths of time and take the MAP assessment as a pre- and post-test at irregular intervals, consider using growth targets based on 8-week enrollment windows.
• For each student, use the assessments that give the longest window between pre- and post-test administrations.
25
Calculating Target RIT Growth using 8-week Enrollment Windows
• CDE is currently working to calculate growth targets for by pre-test grade equivalent for 8 week intervals.
• These results will be available within the next week or two and will enable AEC’s opting into NWEA and the 8-week interval method to complete their Selection of Measures forms.
26
Determine if Target Growth Achieved
• Determine whether each student met their growth target by comparing the actual growth obtained to their target growth.
• If a student’s observed growth was greater than or equal to their target growth, the student MET their target.
• If a student’s observed growth was less than their target growth, the student DID NOT MEET their target.
27
Aggregating at the AEC Level
• Compute the percentage of students meeting their target growth
Number of students that met their target growth (being sure to use only one growth result per student)
Total number of students with pre- and post-test records (using one record per student)
28
Completing Selection of Measures Form
• Fill in your AEC’s aggregated percentage of students meeting their target growth• Fill in your AEC’s total number of students with pre- and post-test records (denominator)
To receive the indicator rating, the percent of students achieving their target growth was:
Exceeds AEC norms at or above 90 percent of students
Meets AEC norms below 90 percent but at or above 60 percent of students
Approaching AEC norms below 60 percent but at or above 40 percent of students
Does Not Meet AEC norms below 40 percent of students
Cut-point values for percent of students achieving target growth were:
Cut-point value for Exceeds 90.0
Cut-point value for Meets 60.0
Cut-point value for Approaching 40.0
Your school's actual percent of students achieving target growth was:
Percent achieving target 66.7
Total student count (if N ≥ 16) 27
YES
Use
mea
sure
?
OPT
ION
AL
NWEA MAPS Reading
29
Assigning a Growth Rating
• CDE will assign a rating to your AEC’s reported NWEA MAP growth• This calculation is done separately for each of the three subject areas by comparing the percentage of students meeting their target growth against the following state cut-points:
Cut-Points
• Exceeds – at/above 90%• Meets – below 90% but at/above 60%• Approaching – below 60% but at/above 40%• Does Not Meet – below 40%
30
Process/Timeline
July 25: NWEA Results to CDE
31
Resources• Alyssa Pearson, Executive Director
– [email protected] , 303-866-6855• Marie Huchton
– [email protected] , 303-866-6203• Donna Morganstern
– [email protected] , 303-866-6209• Somoh Supharukchinda
– [email protected] , 303-866-6778• http://www.cde.state.co.us/Accountability/
StateAccountabilityAECs.asp
