Date post: | 05-Jan-2016 |
Category: |
Documents |
Upload: | christian-berry |
View: | 214 times |
Download: | 0 times |
CaMSP Cohort 8 OrientationCaMSP Cohort 8 Orientation
Cohort 8
State and Local Evaluation
Overview, Reporting Requirements, and Attendance Database
February 23, 2011
California Department of Education
Mikala L. Rahn, Ph.D
Public Works, Inc.
Who Are We?Who Are We?
Public Works, Inc. a nonprofit in Pasadena dedicated to working with communities, government, schools and parents by providing services and resources to educate and inform children, youth and families. Our work is in three areas:
• Education Reform
• Workforce Development
• Intervention/Prevention
Our TeamOur Team
Project Director:
Mikala L. Rahn, PhD.
Team: Patty O’Driscoll, Velette Bozeman, Albert Chen, Scott Phelps, James Maloney
Multiple Evaluation Multiple Evaluation RequirementsRequirements
1) Statewide Evaluation
2) Local Evaluation
3) Federal Reporting
Levels of EvaluationLevels of Evaluation
Central versus local
• CDE/Public Works, Inc.
• CDE/Federal
• Partnership/ Local evaluators
Statewide Evaluation Statewide Evaluation Research QuestionsResearch Questions
1. How have the Partnerships ensured that all students have access to, are prepared for, and are encouraged to participate and succeed in challenging and advanced mathematics and science courses?
2. How have the Partnerships enhanced the quality of the mathematics and science teacher workforce?
3. What evidence-based outcomes from the Partnerships contribute to our understanding of how students effectively learn mathematics and science?
Key FeaturesKey Features
Features of the programs reviewed:
• Partnership driven
• Teacher quality
• Challenging courses and curricula
• Evidence-based design and outcomes
• Institutional change and sustainability
Balanced Evaluation Approach in Balanced Evaluation Approach in the Statewide Evaluationthe Statewide Evaluation
Process measures focus on how a program is implemented and the outcome measures focus on the results of the program or intervention.
Public Works, Inc. uses both quantitative and qualitative data collection methods.
Qualitative AnalysisQualitative Analysis
• Qualitative research including site visits to Cohort 8 partnerships and selected additional partnerships from previous cohorts in Spring 2012. • Site visits include interviews, focus groups and observations to gain general patterns across partnerships for a statewide picture, and best practices discovered throughout the state. •Telephone interviews in years you are not visited (Spring 2013).
Quantitative Data CollectedQuantitative Data Collected
• Teacher Database: Incorporates PAIF data on all teachers in participating districts and attendance in professional development for all participating teachers (ongoing)
• Partner Survey (Winter)• Participating Teacher Survey (Spring)• Student rosters of comparison and
treatment teachers for statewide study/CST Data (first in Summer-Fall 2012)
Teacher outcomes examined Teacher outcomes examined in state evaluationin state evaluation
TeachersNumber and characteristics of teachers who
participate in professional development; satisfaction with training
• Demographics/Years teaching• Hours of Training• Qualifications/Assignments• Treatment vs. Comparison Student
Performance on CSTs
Student outcomes examined Student outcomes examined in state evaluationin state evaluation
Students
• Improved student academic achievement on state mathematics and science assessment across the state and at the partnership level
Local EvaluationLocal Evaluation
Two Goals:
• Fulfill state evaluation requirements
• Fulfill commitments made in Local Evaluation Plan
Internal or External evaluator (not a training provider) qualified to conduct a research study
Local EvaluationLocal Evaluation
Fulfill commitments in CaMSP proposal:• Pre- and post- assessments of teacher content knowledge• Matched control/comparison group of teachers• Local assessment data/CSTs • Teacher observation, portfolio, lesson study, video• Surveys• Qualitative/case studies• Value added/comparison/treatment methodologies
Most important for local: Teacher knowledge and instructional strategies & measuring student knowledge with local assessments. Also, the completion of an evaluation report to attache to the March CDE YTD report and another for the fall Federal Report.
Measuring the UnmeasurableMeasuring the Unmeasurable
• Capturing your story….
• Narrative/journaling professional growth from the teacher’s perspective
• Demonstration lessons/video
• Teacher portfolios
• Performance tasks from students
How do we document our work in math and science?
Being Clear on TermsBeing Clear on Terms
What is your model for…• Coaching
• Lesson study
• Hands-on
• Classroom follow-up
• Mentoring
Bottom Line on Local Bottom Line on Local EvaluationEvaluation
• Documentation is crucial• High profile at the state-level: outcomes
matter!• Impact: How does professional
development translate into improved classroom practice?
• Demonstrate best practices in content delivery/pedagogy within state content
Federal Reporting Cohort 8Federal Reporting Cohort 8
Completion of APR is partnership and local evaluator responsibility
PW provides the Student GPRA data and technical assistance to partnerships
Cohort 8 First APR: PW Data Collection for APR Begins July 15, 2012
First Annual Performance Report (APR) due to CDE October 15, 2012
Block it now!
Narrative, Project Profile, & Local Evaluation Report are RequiredDue for Cohort 8 from CDE to ED by
November 30, 2012
PW State Evaluation Work Plan: PW State Evaluation Work Plan: Cohort 8Cohort 8
• Contacts/Partnership Profiles on CaMSP site (March 2011) • Conduct site visits, observations, and phone interviews
(Spring 2012)• Administer Partner Survey (Nov. 2011)• Web-based Database (ongoing)• Toolbox of instruments (ongoing)• Administer teacher survey (Feb-March 2012)• Determine list of treatment/control teachers based on
attendance through June 30, 2012 by July 15, 2012• Create matched comparison list of teachers by Aug 15, 2012• Collect rosters of students of treatment and comparison
teachers from partnership districts by September 15, 2012• Match to 2012 CSTs and provide student GPRA # to
partnerships by October 15, 2012
Next StepsNext Steps
• After Lunch: Demonstration of CaMSP Website and PW Attendance and Teacher Participation Database
• CDE provides updated PAIF (working from 08-09); CDE provides statewide CST data
• Determine who should get access to the database
• Input attendance data into Web system frequently (used by CDE for YTD’s!!)
• Closeout attendance data every June 30th by July 15th
• All professional development activities are expected to be logged as soon as possible following the activity.
Web-based Teacher Web-based Teacher Database RemindersDatabase Reminders
• Demographic teacher data is updated once per year from statewide PAIF (October).
• Input training data as you go.
• Database information will be reviewed by CDE as part of the YTD process.
To Do ListTo Do List
• Provide Updated contact information to CDE and PW (including database contacts)
• Review and approve PW partnership profile for CaMSP Website
• Work with Public Works, Inc. to begin data entry of attendance for periodic reports
Web-based CommunicationWeb-based Communication
www.publicworksinc.org
• Contacts (check)
• Profiles (check)
• Administrative (use)
• Database (use)
Contact InformationContact Information
• Mikala Rahn, [email protected]
• Patty O’Driscoll, [email protected]
• Albert Chen, [email protected]
• Velette Bozeman, [email protected]
• Phone: (626) 564-9890