Post on 30-Dec-2015
description
transcript
Mathematics Performance TasksApplying a Program Logic Model to a
Professional Development Series
California Educational Research AssociationDecember 6, 2013
Sally J. Bennett-Schmidt, Ed.D.Assessment Director
San Diego County Office of Education
Overview of the Session
•Context of the Professional Learning Experience•Summary of the Professional Development Series•Application of the Program Logic Model▫Vision & Current Situation▫Resources & Activities▫Outputs▫Outcomes: Short-, Intermediate-, Long-Term
• Learnings and Next Steps
Context of the Project
•San Diego County Office of Education▫Support 42 school districts + county-run programs▫Curriculum, instruction, and assessment staff working
collaboratively to support districts in implementation of the common core state standards
•San Diego County School Districts▫Along a continuum in implementation of CCSS▫Substantial needs in mathematics, both content
knowledge and instructional approaches
The Professional Development Series:Mathematics Performance Tasks & Formative Assessment
•Purpose: Support district efforts in the use of performance tasks to maximize student achievement
•Key Activities▫Examine assessment in the light of the new Common Core
State Standards and work being explored by the Smarter Balanced Assessment Consortium
▫Experience performance tasks along with scoring protocols▫Explore the formative use of performance task results
including student re-engagement▫ Incorporate formative assessment practices in districts’
ongoing work
The Professional Development Series
•Days 1-2: August 2012▫Audience: District teams of 5-7 members (district and
site administrators, mathematics teacher leaders)▫Facilitated by David Foster from the Silicon Valley
Mathematics Initiative▫Focus:
Understand the new assessment context under the common core standards
Experience several performance tasks with scoring and analysis protocols
Begin discussion of the formative use of performance tasks and student re-engagement
The Professional Development Series
•Day 3: September – October 2012▫Audience: Expanded district teams with additional
classroom teachers▫Facilitated by SDCOE Mathematics & Assessment
Staff▫Focus:
Select an extended response mathematics task to use with students
Anticipate student successes and challenges Plan for embedding the task in instruction
The Professional Development Series
•Day 4: December 2012 – January 2013▫Audience: Expanded district teams with additional
classroom teachers▫Facilitated by SDCOE Mathematics & Assessment
Staff▫Focus:
Score student work from the performance task Analyze student performance for mathematical
understanding and misunderstanding Plan a re-engagement lesson
The Program Logic Model
•Context for using a Logic Model▫Learning and Leadership Services Division began
exploring tools to help us better determine the impact of our work on district systems, instructional leadership, teacher practice, and student learning.
▫Logic model introduced as a tool to help us frame our services within an outcome orientation For planning For program evaluation
▫Logic model completed via a series of guiding questions
The Program Logic Model
1. What is our vision and our current situation?2. Who do we serve?3. What end result do we desire – how will the “rules
of the game” change?4. What new behaviors will we see?5. What will change right away?6. What do we do?7. What will we count?8. What will we need to make it happen?9. How will we know we’re on the right track?
The Program Logic ModelSituation / Vision: Question 1Target Audience: Question 2
Assumptions, Environmental / External Factors
Resources
Question 8:What will
we need to make it
happen?
OutcomesStrategies Activities
Question 6:What do we do?
Outputs
Question 7:What will we count?
Short-Term
Question 5: What will
change right away?
Intermediate
Question 4: What new behaviors
will we see?
Long-Term
Question 3: How will the rules of the
game change?
Indicators
Question 9: How will we know we’re on track?
Vision
•Mathematics instruction will reflect the common core instructional shifts, and will embed high-quality tasks and formative assessment strategies to gauge students’ conceptual understanding and re-engage students in mathematical thinking.
Current Situation
•Current classroom instructional practices generally reflect training aligned to curricular materials and prior state content standards.
•Teachers have had limited professional learning and experience with embedding high-quality, open-ended mathematics tasks and formative assessment into instructional practice.
•Student achievement in mathematics has improved, but continues to show weaker performance in middle and high school grade levels and courses.
Target Audience
•Mathematics teacher leaders•Site administrators•District curriculum / instruction / assessment
administrators
Resources
•SDCOE Mathematics and Assessment staff (managers and support staff)
•Time for collaborative planning and delivery of professional development
•Funding to contract for SVMI member resources•Funding for costs associated with professional
development sessions
Activities
•Four-part mathematics performance task and formative assessment professional development series
•Contract for SDCOE and small districts in the county to access SVMI resources
Output and Indicators•Number of professional development sessions▫1 four-part series; 9 total sessions (Day 3 offered four times;
Day 4 offered three times)•Number of participating teachers and administrators▫135 participants from 14 districts and 4 independent charters
•Number of teachers that select and administer a task from SVMI MARS or other resource▫88% of participating classroom teachers administered a task
•Number of teachers that score and analyze student work▫76% brought back student work to score and analyze
Short-term Outcomes and Indicators
• Increased awareness of how mathematics performance tasks and formative assessment can improve instruction and student learning▫94% reported better understanding of assessment in
relation to CCSS▫78% reported better understanding of analyzing student
work to identify students’ mathematical understanding
Short-term Outcomes and Indicators
• Increased awareness of where to find mathematics performance tasks, rubrics, and scored student papers▫78% reported better understanding of how to find and
access identified websites to select performance tasks• Increased awareness of how to select and administer a
performance task, and score student work using a rubric with training & standardizing papers▫89% reported the PD helped them plan to administer a
performance task with their students▫86% reported the PD helped them score and analyze
their students’ work
Intermediate Outcomes & Indicators
• Teachers try out performance tasks in classrooms, and score and analyze student work▫88% tried out a performance task▫76% scored and analyzed student work
• Teachers use student work for professional conversation▫100% of teachers who brought back work engaged in
professional conversation on Day 4▫Proposed use of teacher learning logs to document continuing
practice• Teachers share student work and instructional practices
through PLCs▫Proposed use of PLC logs and observations at a sample of sites
Long-term Outcomes & Indicators•Teachers and students engage in rigorous, coherent,
and focused mathematics learning experiences•Classroom practice reflects high-quality mathematics
instruction and assessment•PLCs are focused on student learning and instruction•Student achievement in mathematics improves▫Teacher learning logs and classroom observations▫PLC logs and observations▫District / Site and Smarter Balanced assessment results
Learnings & Next Steps
•What we learned about supporting teacher use of mathematics tasks and formative assessment strategies▫More time was needed on re-engagement portion of
mathematics performance task & formative assessment work
▫Teachers need continued support to embed tasks and formative assessment strategies into ongoing instructional practice
▫School site structures need to support teacher collaboration around selecting tasks, analyzing student work, and re-engaging students in mathematics learning
Learnings & Next Steps
•What we learned about collecting “evidence of impact”▫We were able to collect evidence of short-term
outcomes in terms of increased awareness▫We were able to collect limited evidence of
intermediate outcomes in terms of changing practice▫We need a structured plan and resources (human,
time) to collect evidence of intermediate and longer-term outcomes
Learnings & Next Steps
•What we learned about the program logic model▫The approach helped us better define our service(s)
and our desired outcomes▫The tool helped us differentiate short, intermediate,
and long-term outcomes and identify appropriate evidence to collect to assess impact
▫The process helped us understand that, to assess long-term outcomes / impact, we need a multi-year project and ongoing data collection methods
Contact Information
•Sally J. Bennett-Schmidt, Assessment Director▫San Diego County Office of Education▫858-292-3688▫sbennettschmidt@sdcoe.net