10705ZZ4211 Pennsylvania Benchmarks Data Analysis and Discovery (Root Cause Analysis)

transcript

10705 ZZ4211

Pennsylvania

BenchmarksData Analysis and Discovery

(Root Cause Analysis)

Pennsylvania

BenchmarksData Analysis and Discovery

(Root Cause Analysis)

Component 1:

Data Analysis and Discovery

Component 1:

Data Analysis and Discovery

Data DialoguesData Dialogues

PRIMARY QUESTIONSHow many proficient?

SECONDARY QUESTIONSStrengths/Weakness

TERTIARY QUESTIONSWhy/Discovery/Root Causes

Primary QuestionsPrimary Questions

Primary questions are those that closely match your most essential goals.

They tend to be very, very simple.

Analyzing Proficiency ReportsPrimary Questions

• How many students are proficient in reading/math?

• How many students in this subgroup are proficient?

• How many students do you need in that subgroup to make AYP?

• How many more quarters until the PSSA is given?

• Who are the students just below proficiency?

• How many points do these students need to move up to be proficient? How many points per quarter?

Member Center Primary Question Tools

• Proficiency Reports (Under Testing Center – Review/Enter Scores)

• RP 3508: Predicted State Results (Under Reports – School Reports)

• RP 3299: Student Data Sheets (Under Reports – School Reports)

• RP 3434: 4Sight Proficiency Projections (Must have entered last year’s AYP percentages.)

Planning For and Monitoring AYP

From the Member Center Menu Select: ReportsSchool Reports AYP Reports

• RP-3533 Measures• RP-3379 Projections by Grade • RP-3434 4Sight Proficient Projections

Caution: These reports are only as accurate as the student data in the Member Center.

Secondary QuestionsSecondary Questions

Secondary questions discover what strengths and weaknesses are according to student learning.

They tend to be more specific.

Analyzing Subscale Data Secondary Questions

• What are the strengths identified on the graph?

• What are the areas for concern?

• If the graph is flat, are the students proficient readers or close to nonreaders?

Member Center Secondary Question Tools

• Subscale Reports (Under Testing Center – Review/Enter Scores)

• RP 3304: Subscale Test Results (Under Reports – School Reports)

• RP 3474: Individual Student Test Results (Under Reports – School Reports – Prints a report for each individual student)

Tertiary QuestionsTertiary Questions

Tertiary questions ask why. They seek to discover root causes for the identified

strengths or areas of concern.

Tertiary Question ProbesTertiary Question Probes

• Why do our students not perform according to state standards?

• Why do our students not know what they should know?

• Why are the students unable to use what they know?

• Why are our subgroups performing below?

StudentDemographics

Attendance

Subgroups

Transiency

Suspensions

Curriculum

Alignment

Balance

Instruction and Preparation

Alignment

Cycle of Effective Instruction

Quality of Program Implementation

Classroom Management

Professional Development

School Processes

Assessment andRegrouping

InterventionResources

TutoringSolutions TeamSchool Programs

Teacher Selection

Scheduling

Leadership

OrganizationalCulture

Beliefs

Values

External Factors

Parent Involvement

Health

Systems Approach to Discovery/Root Causes

Determine Root Causes Related to Targeted

Area of Concern

Determine Root Causes Related to Targeted

Area of Concern

• Attendance• Subgroups

• Ethnicity• Gender• Language• Special

Education• Transiency• Suspensions

Student Demographics

• Alignment• Balance• Rigor

Curriculum

• Alignment• Cycle of Effective Instruction

• Active Instruction: Teach, Model, Guide Practice

• Partner/Team Practice• Individual Mastery and

Accountability• Quality of Program Implementation

• Classroom Management• Professional Development

Instruction and Preparation

• Assessment and Regrouping• Intervention Resources

• Tutoring• School Programs

• Teacher Selection• Scheduling• Leadership

• Principal• Leadership Team• Literacy/Math Coaches

School Processes

• Beliefs• Values• Artifacts• Common Language • Celebrations

Organizational Culture

• Parental Involvement• Health• Socioeconomic Status• Shelter • Nutrition• Parent Educational Level • Community Crime Rate

External Factors

Data Analysis • Identify Proximity to Goal(s).• Identify Areas of Concern • Develop Target(s).Discovery• Brainstorm Root Causes.• Prioritize and verify Root Causes with data.• Select the Root Causes that are most important and

doable. Keep the number low.Solutions• Design interventions to address identified Root Causes.• Evaluate results and readjust.

Steps for Data Analysis, Discovery,

and Solution Identification

Data Analysis Identify Proximity to Goal(s).Identify Areas of Concern Develop Target(s).

DiscoveryBrainstorm Root Causes.Prioritize and verify Root Causes with data.Select the Root Causes that are most important and doable. Keep the number low.

SolutionsDesign interventions to address identified Root Causes.Evaluate results and readjust.

Member Center and 4Sight Discovery Tools

• RP 3491: 4Sight Item Analysis Report: (Under Reports – School Reports)

• 4Sight Pennsylvania Benchmark Reporting Categories: (Admin. Guide – PA standards condensed version)

• 4Sight Pennsylvania Benchmark Answer and Alignment Guide: (Admin. Guide – Reporting Category, PA Standard, Assessment Anchor)

Root Cause Analysis: Clarification

Clarification

Basal uses primarilynarrative text

No explicit instruction in clarification at the word

or paragraph level

Inadequate time spentOn instructing how to read expository text

Lack of PD on clarification with expository text

No consistent readingblock.

MS have classroom management issuesNo school wide measurable goals on clarification

Teachers and tutors not aligned

Leadership not monitoring

Attendance lowHomework return lowHomework not aligned

Parents unaware of school focusand child’s skill level

doable. Keep the number low.Solutions• Design interventions to address identified Root Causes.• Evaluate results and readjust.

Verifying Root CausesVerifying Root Causes

• Consult additional records/data (such as attendance).

• Review curriculum guides.• Analyze student assignments.• Conduct classroom walk-throughs.• Do a school artifact inventory.• Review office referral records. • Etc.

Data Does Not Fix Problems

“Disaggregating data is not a problem-solving strategy. Rather it’s a problem-finding strategy.”

— Lezotte (1999)

doable. Keep the number low.Solutions• Design interventions to address identified Root

Causes.• Evaluate results and readjust.

Sample Improvement PlanSample Improvement PlanData Analysis

Instructional Planning FormPlans for to (dates)

Student Group: All students

Target/Objective: All students will use clarification strategies when reading independently narrative and expository text.

Success Indicator: Students will score 90% or higher on basic comprehension when reading

material at an independent level.

Action Steps Resources Facilitator TimelineInitial Training on teaching clarification strategies.

TrainerTeacher Guides Student Practice Materials & Videos

Reading Coach August 15

Initial coaching on teaching clarification strategies.

TrainerClarification strategies use dataCompleted lesson plans.

Reading Coach Sept. 30

Designing Improvement Plans

(1 year or 1 quarter)

Designing Improvement Plans

(1 year or 1 quarter)

• Turn Primary Root Causes into Action Steps• Improvement Plans Include:

1. Target(s)2. Action Steps 3. Resources Needed4. Facilitator/Point Person for Each Step5. Timeline 6. Success Indicator(s)

Component 2:

Using 4Sight to Inform Professional Development

Component 2:

Using 4Sight to Inform Professional Development

Results-Based Professional Development

• Identifies desired improvement in student achievement.

• Identifies a plan to help teachers achieve these results with their students.

• Gives teachers an opportunity to gain new knowledge and behaviors.

Traditional vs. Results-Based Professional

Development

Traditional Results-Based

Opportunities Responsibilities

You Might Want to Go Professional Responsibility

Staff Desires Student Needs

Focus: Good Things to Do

Focus: Needs of Students

Decision by Feelings Decision by Data

Development

Traditional Results-BasedMany Topics Addressed

One Topic in Depth

Reactive to Change Proactive to Change

Primarily Workshops Workshops with Implementation

Evaluated Based on Satisfaction

Evaluated Based on Student Data

Awareness, Knowledge

Skill Attainment

Passive Participants Participants Partners in Process

Key ComponentsKey Components

• Knowledge/Theory • Modeling Demonstration • Low-Risk Practice • Feedback • On-the-Job Practice and

Feedback

Knowledge/TheoryKnowledge/Theory

What: rationale and description of techniques and uses

How: readings, lectures, videos, discussions

Value: increase awareness and conceptual understanding

Note: Used alone it rarely results in skill acquisition

nor does it transfer to the classroom.

Modeling/DemonstrationModeling/Demonstration

What: enactment of strategy or skill

How: simulation with students/participants

Value: mastery of knowledge

related to skill

Note: Modeling alone seldom results in transfer to the classroom.

Low-Risk PracticeLow-Risk Practice

What: nonthreatening experience with new skill

How: simulation with a small group

Value: begin to acquire new skill/strategy

Note: Low-risk practice is a way to develop competence.

Feedback on Low-Risk PracticeFeedback on Low-Risk Practice

What: provides for reflection on implementation

How: self-reflection or provided by peers/coaches

Value: increased awareness of one’s skills

Note: Low-risk practice changes behaviors over the long term.

On-the-Job PracticeOn-the-Job Practice

What: self-analyze and make adjustments for improvement

How: uses consultants, peers, supervisors, and trainers

Value: builds support, collegiality, and commitment

Note: On-the-job practice is necessary for internalization of learning.

Professional-Development Formats

• External Consultants • Internal Consultants • Observation of Others • Collaborative Groups • Literature

External ConsultantsExternal Consultants

Who: trainers/consultants from intervention and curriculum providers, department of education staff, university staff

Plus: bring in outside knowledge and strategies

Delta: not part of the day-to-day implementation

Internal ConsultantsInternal Consultants

Who: district curriculum specialists, coaches, administrators, master teachers

Plus: work in district every day Delta: may be limited in bringing

new skills and strategies to the district

Observing OthersObserving Others

Who: peers observing peers Plus: maximizes demonstration,

modeling, and on-the-job practice with feedback

Delta: requires courage

Collaborative Peer GroupsCollaborative Peer Groups

Who: peer groups who meet regularly to learn and hold each other accountable

Plus: allows staff to be fully active participants in their own professional development

Delta: finding time is difficult and the temptation to conduct housekeeping chores in meetings is great

Professional-Development Format Jigsaw

1. Summarize information from reading. 2. For what component of results-based

professional development could this format be used effectively?

3. What other supports would need to be provided along with professional development in this format to ensure teacher success and improve student achievement?

Professional Development Based on Student Needs

• What student needs does the data identify?

• What are the long-term needs? • Are there different needs for different student groups?

• How should student needs be prioritized?

Planning Professional Development

Data-Analysis Findings

Gap Between Desired Outcomeand Current Status

Areas of Concern

Component

Action StepDate

Responsible

Knowledge

Modeling

Low-Risk Practice

Feedback

On-the-Job Practice with Feedback

10705ZZ4211 Pennsylvania Benchmarks Data Analysis and Discovery (Root Cause Analysis)

Documents