Student tests are hand scored by classroom teachers trained and calibrated using standard protocols.

Students in grades 2 through 10th/11th grade are administered performance exams (5 apprentice tasks per exam).

District scoring leaders are trained in using task specific rubrics

Student results are collected, analyzed, and reported by an independent data contractor.

Random sample of student papers are audited and rescored by SJSU math & CS students. (Two reader correlation >0.95)

Performance Exams40,000 – 70,000 students per year since 1999

1

22

MARS Scoring• The MARS tasks use a point-scoring rubric. • Each task is assigned a point total corresponding to the

complexity of the task and the proportional amount of time that the average student would spend on the task in relation to the entire exam.

• The task points are then allocated among its parts. – Some points are assigned to how the students approach the

problem, – Majority of points to the core of the performance– A few points to evidence that, beyond finding a correct

solution, students demonstrate the ability to justify or generalize their solutions.

– In practice, usually points are assigned to different sections of a multi-part question.

33

Use of MARS Reports• The combination of PA tasks and weighted rubrics

provides a detailed picture of student performance– how students approached the different tasks– a description of common misconceptions – evidence of what students understand.

• Reports – include student work samples at each grade showing the

range of students’ approaches, successes, challenges.– provide implications for instruction: specific suggestions and

ideas for teachers as a result of examining students’ strengths and the areas where more learning experiences are required.

Longer MAC experience Higher Scores

Grade/ Course

Percentage Proficient Students with

Non-MAC Teachers

Percentage Proficient Students with MAC Teachers

6 42 64

7 29 59

8 15 25

Algebra I 52 70

4

Large Scale Assessment

Percent Proficient on MARS and LSA

5

  2005 2006 2007 2008 2009 2010 2011

Grade 2   65.6 67.7 63.6 64.8 72.1 72.5

Grade 3 57.0 63.6 62.8 57.3 66.1 61.3 66.3

Grade 4   57.9 56.0 58.4 67.4 57.6 65.8

Grade 5 49.0 51.6 52.1 51.1 53.6 47.9 64.4

Grade 6   52.5 49.0 47.6 46.7 36.3 39.6

Grade 7 42.0 32.6 42.7 43.9 33.7 48.6 23.5

Grade 8   25.5 18.1 12.1 6.0 25.5 17.0

Algebra 1   35.2 36.7 37.9 35.6 43.2 42.0

Spring 2011 Trends Grade to Grade

Grade 2 MARS Below MARS At or ^ TotalCST Below 13.7% 6.5% 20.2%CST AT or ^ 7.3% 72.5% 79.8%Totals 21.0% 79.0% 100.0%

Grade 2 MARS 1 MARS 2 MARS 3 MARS 4 Total

Far Below 1.0% 0.6% 0.1% 0.0% 1.7%

Below Basic 1.9% 4.1% 1.1% 0.1% 7.2%

Basic 0.8% 5.3% 4.6% 0.6% 11.3%

Proficient 0.4% 5.1% 16.2% 6.5% 28.2%

Advanced 0.2% 1.6% 15.2% 34.6% 51.6%

Total 4.3% 16.7% 37.2% 41.8% 100.0%

6

Spring 2011 Trends Grade to Grade

Grade 6 MARS Below MARS At or ^ TotalCST Below 34.7% 3.8% 38.5%CST AT or ^ 21.7% 39.6% 61.3%Totals 56.4% 43.4% 99.8%

Grade 5 MARS Below MARS At or ^ TotalCST Below 17.3% 6.0% 29.7%CST AT or ^ 12.4% 64.4% 70.4%Totals 23.3% 76.8% 100.1%

Grade 4 MARS Below MARS At or ^ TotalCST Below 15.6% 5.8% 21.4%CST AT or ^ 12.9% 65.8% 78.7%Totals 28.5% 71.6% 100.1%

Grade 3 MARS Below MARS At or ^ TotalCST Below 16.4% 4.5% 20.9%CST AT or ^ 12.7% 66.3% 79.0%Totals 29.1% 70.8% 99.9%

7

Spring 2011 Trends Grade to Grade

Course 2 MARS Below MARS At or ^ TotalCST Below 15.4% 0.0% 15.4%CST AT or ^ 36.0% 48.7% 84.7%Totals 51.4% 48.7% 100.1%

Course 1 MARS Below MARS At or ^ TotalCST Below 31.9% 4.1% 36.0%CST AT or ^ 21.5% 42.0% 63.5%Totals 53.4% 46.1% 99.5%

Grade 8 MARS Below MARS At or ^ TotalCST Below 55.1% 2.8% 57.9%CST AT or ^ 25.0% 17.0% 42.0%Totals 80.1% 19.8% 99.9%

Grade 7 MARS Below MARS At or ^ TotalCST Below 38.1% 0.4% 38.5%CST AT or ^ 38.1% 23.5% 61.6%Totals 76.2% 23.9% 100.1%

8

8th Grade Geometry California’s Highest Achieving Students

Geometry MARS Below

MARS At or Above Total

CST Below 15.3% 0.0% 15.3%CST AT or Above 36.0% 48.7% 84.7%

Totals 51.3% 48.7% 100%

9

1010

Summary of Results• These findings represent what has been consistently

found across the years, even with an increasing number of students and teachers. – In 1999, 21 school districts, 462 teachers, and 23,128 students

were involved in the study. – The project reached the largest number of students in 2004

with 81,075 students and 1622 teachers in 28 districts. – When funding for the project needed to be supported solely by

district funds, the number of participating districts, teachers, and schools decreased. In 2011, 28 districts participated with 38,538 students.

1111

Implications• MARS tasks represent the assessment consortia’s

descriptions of must-needed performance tasks• Better understanding the data from these assessments

and their relationship to improved teaching practices and student learning is paramount for informing the pragmatic decisions that will be made around what types of mathematics tasks assess deeper student learning and reflect improved teaching practices.

• It is NOT just the assessment – it’s the use of the information via support and coaching of teachers.

1212

Limitations and Next Steps• This paper is the first in a series studying the

usefulness of MARS resources for teachers and their students, and thus provides only a sneak peek at the many layers of data to be studied.

• This work used simple exploratory analyses and comparisons, and does not utilize the methods for studying longitudinal nested data, which will provide a much clearer view of longer-term impact, following students and teachers over the years, controlling for previous performance and other trends that cannot be seen by simple year-to-year splicing of data.

1313

Intensive Vs. General Coaching

Score CodeIntensive Formative

Assessment

General Coaching

Grade 6 MARS 69 45

Grade 7 MARS 45 28

Grade 8 MARS 38 20

Grade 6 statewide assessment 65 50

Grade 7 statewide assessment 59 48

Grade 8 statewide assessment 48 35

1414

Summary of Results• A focused intervention around formative assessment

using MARS tasks and data shows that students with such teachers have even further gains on both performance and multiple-choice assessments.

• This finding is even true with the ever-increasing number of students and teachers involved every year, indicating that the process is scalable and sustainable.