Denver Public Schools

UIP Qualitative AnalysisDistrict Level

February 2014

Table of Contents

I. Major Findings II. Top 10 Lists (Areas of Focus/Improvem

ent StrategiesIII. Professional Development IV. High/Low Growth School Strategies V. Appendix

A. Comprehensive Strategy List B. Definitions/Examples of Common Strategies C. Methodology/Limitations

2

Annual UIP Reports

A qualitative and quantitative UIP report are produced annually by ARE.

District Level Qualitative Analysis: • The top 10 major improvement strategies (MIS) (e.g., LEAP, data

driven practices, etc.) schools are focusing on in their 2013-14 UIPs.*• The content areas and/or disaggregated groups schools are focusing

on.• Specific “professional development” areas schools are focusing on.• Specific areas of focus for schools with the HIGHEST AND LOWEST

TCAP growth by content area

Network Level Qualitative Analysis: • The top 5 major improvement strategy areas of focus by network.• The specific “professional development” areas schools are focusing on.

3* Based on submitted UIPs as of January 2014

Definitions for the Subsequent Slides

4

Strategies

•Areas of Focus that describe the specific approach to improve student performance.

•Examples: Instruction, Curriculum, Collaboration, Culture/Climate

Content Areas

•Areas of Focus that describe the content areas that will be the focus of the strategy (if applicable).

•Math, Science, Social Studies, Reading, and/or Writing

Disaggregated Groups

•Areas of Focus that describe the subgroup that will be the focus of the strategy (if applicable).

•Examples: Special Education Students, English Language Learners, Minority Students

Level of Focus(Data not

analyzed at this level)

•Areas of Focus that describe the Education Levels that will be the focus of the strategy (if applicable).

•School-wide/System or Grade Level Focus

Areas of Focus

Everything a school listed as part of a major

improvement strategy

UIP Qualitative Analysis: Major Findings In 2013-14, DPS UIPs had a more diverse array of major improvement strategy

themes than in previous years. A total of 66 areas of focus (e.g., progress monitoring, ACT, scheduling) were identified among all DPS schools.* This is over a 40% increase.

5*See Appendix A for a list of all of the major improvement strategy areas of focus and % change.**Parent Engagement was embedded in Community Engagement last year.

Strategy% DPS Schools

2013-14Change from

2012-2013

Most Common New Areas of Focus

Parent Engagement**

24% New Code

Rigor 21% New Code

Coaching 9% New Code

Academic Language 7% New Code Of all the UIP focus areas, there were the 3 themes stated by significantly more schools this year than last year, with a year over year change of more than 15%.

Focus Areas with Greatest Increase

Data Driven Practices

48% + 15%

Student Engagement

16% + 15%

Collaboration 18% + 16%

The top 10 areas of focus are similar to last year with minor shifts among the percentage of schools that included them in their MIS. (A focus area includes all strategies, content areas, subgroups, and ed. levels.)

Two areas of focus fell out of the top 10 major improvement strategies this year:

Progress Monitoring and Differentiated Instruction

Two areas of focus moved into the top 10: RtI/Intervention and Curriculum

6

UIP Qualitative Analysis: Major Findings Cont’d

UIP Qualitative Analysis: Major Findings Cont’d

7

We will use frameworks, coaching, feedback and

professional development to support

educator growth.

We will implementnew standards using

rigorous and culturally relevant curriculum,

instruction and assessments.

Over 30% of schools are

using PD as a major

improvement strategy.

Over half of our schools are focused on

Reading/Literacy and Instruction.One quarter on

Curriculum.

An intensified focus this year on

Standards and Rigor.

Two of the district’s priorities* were among the top 10 major improvement strategies with 31% of schools focused on PD and about a quarter of schools focused on curriculum, assessments and CCSS.

ELL students and ELA strategies are no longer in the top ten UIP

strategies.

* The District Priority of “Differentiate Support” is not

measurable using the UIP Major

Improvement Strategies.

Most Common Words in DPS Major Improvement Strategies

8

% of DPS schools with UIP Focus Areas – Top 10

Progress Monitoring and Differentiated Instruction are no longer in the top 10 focus areas.

*CCSS were coded with ELGs and WIDA in 2013. **Curriculum refers to a change or implementation of any variety of curriculum as a means to address a root cause/issue.

Rank UIP Focus Area% DPS

Schools # DPS

Schools Ranked

Last Year1 Instruction 60% 108 42 Reading / Literacy 52% 93 13 School-wide/System Improvement 48% 86 3

4Data Driven Practices/Data

Inquiry/Inquiry Cycle48% 86 8

5 Math 38% 69 56 Writing 38% 68 27 Professional Development 31% 55 7

8 RtI/Intervention 30% 54 14 (top 10 in 2012)

9Common Core State Standards

(CCSSs)28% 50 9*

10 Curriculum** 25% 45 13

NEW!

NEW!

% of DPS schools with UIP Strategies– Top 10

RankUIP Major Improvement

Strategies

% DPS School

s

# DPS School

s Ranked Last Year1 Instruction 60% 108 1

2Data Driven Practices/Data

Inquiry/Inquiry Cycle48% 86 4

3 Professional Development 31% 55 34 RtI/Intervention 30% 54 10

5Common Core State Standards

(CCSSs)28% 50 5*

6 Curriculum 25% 45 97 Assessment 24% 43 88 Parent Engagement 24% 43 New** 9 Rigor 21% 37 New10 Differentiated Instruction 20% 36 6

Progress Monitoring is no longer in the top 10 after it was the #2 strategy in 2013. It is #11 this year.

ELL Students/Strategies fell from #7 to #12.

10

NEW!

*CCSS were coded with ELGs and WIDA in 2013. **In 2013, Parent Engagement was coded with Community Engagement, which ranked 13

NEW!

% of DPS schools with UIP Content Areas

Overall Rank UIP Content Areas

% DPS Schools

2014

% ΔFrom 2013

# DPS Schools

#2 Reading / Literacy 52% -7 pts. 93

#5 Math 38% -8 pts. 69

#6 Writing 38% -13 pts. 68

#44 Science 6% No Δ 5

11

Reading ranked #1 again this year (3 years running). Math outranked Writing this year, falling more than 13%. This

returns Writing to 2012 rates, before it increased 16% and was included by 51% of schools last year.

All content areas decreased in the percentage of schools stating it as an area of focus. 32% (57 schools) did not specify a content area in any of their MISs.

Additionally, Cross Content Area Linkage (e.g., applying a single content’s concepts into all classes) was included by 14 schools (8%) this year, compared to just 2 schools last year.

% of DPS schools with UIP Disaggregated Groups

Last year, it was noted that given our achievement gaps throughout the district, schools should consider how we might place more emphasis on disaggregated student groups.

– Positively, FRL students went from being a focus of the Major Improvement Strategies in 0 schools to 7 schools.

– Yet, as the focus on Minority, FRL, and Gender increased, the focus on ELLs and SPED students declined.

Overall Rank UIP Sub-Groups

% DPS School

s

% From Last Year

# DPS School

s #16 ELL 19% -6 pts. 35#35 Minority 4% +2 pts. 8

#40 FRL 4% +4 pts. 7#43 SPED 3% -4 pts. 6#52 Gender 2% +1 pts. 4 12

Top Professional Development Topics

Data-Analysis to Inform Instruction• Differentiation• Assessments and Assessment Data

Effective Instructional Strategies / New Models for Teaching and Planning• Backward Planning, Understanding by Design,

Guided Reading, Workshop Model, among others• Instructional best practices• Rigorous instruction

English Language Learners

*Book studies were often used to guide topics13

High Growth/Low Growth Analysis

-Schools within the following high growth analysis were selected based on the following criteria:

Two schools with the highest MGPs per content area

One school ranked blue or green on the DPS SPF

One school ranked yellow or lower on the DPS SPF

-Schools within the following low growth analysis were selected based on the same criteria above except they were:

Two schools with the lowest MGPs per content area

14

Schools with Highest TCAP Growth

Content Area

School (SPF rating color)

MGP

Reading

KIPP Collegiate HS 80

Colfax Elementary 68

Writing

DSST: College View 82

North High School 61.5

Math

DSST: College View 85

North High School 61.5

15*Blue=“Distinguished”, Green=“Meets Expectations”, Yellow=“Accredited on Watch”

DPS Schools with Highest TCAP MGPs

Similarities among high growth schools:

16

DPS Schools with Highest TCAP MGPs

Similarities among high growth schools:

17

DPS Schools with Highest TCAP MGPs

Similarities among high growth schools:

18

NEW!

NEW!

Schools with the Lowest TCAP MGPs

Compared to Schools w/ Highest Growth:

At Infancy in Progress Monitoring

Lack of a clear progress monitoring plan and schedule OR just beginning to develop

At Infancy in Standards Alignment/Setting Expectations

Beginning to align to high standards (i.e., CCSS) and determine grade level expectations and proficiency levels

Teachers Not Heavily Involved in Progress Monitoring

Focus of data use is mostly at the leadership level, generally NOT at teacher levelWhen teachers review data—acting upon the data is largely absent

Data Reviewed Less Often

Frequency of data analysis varied and less frequent than high growth schools

Rigor & High Expectations are Inconsistent Across Classrooms

Rigorous instruction, curriculum and assessments are largely absent from classrooms. While some teachers have high expectations for students, many do not.

19

Good news: The low growth schools are beginning to implement someof the high growth school strategies to address the areas above.

APPENDIX

APPENDIX A: All UIP Major Improvement Strategies Codes District-Wide

Rank UIP Focus Area

% DPS School

s

# DPS School

s

Prior Year Rank

YOY % Chang

e1 Strategy- Instruction 60% 108 4 10%2 Content Area- Reading / Literacy 52% 93 1 -7%3 Ed Level- School-wide/System Improvement 48% 86 3 -2%

4Strategy- Data Driven Practices/Data Inquiry/Inquiry

Cycle48% 86

8 15%5 Content Area- Math 38% 69 5 -8%6 Content Area- Writing 38% 68 2 -13%7 Strategy- Professional Development 31% 55 7 -5%8 Strategy- RtI/Intervention 30% 54 14 8%9 Strategy- Common Core State Standards (CCSSs) 28% 50 9*

10 Strategy- Curriculum 25% 45 13 1%11 Strategy- Assessment 24% 43 12 0%12 Strategy- Parent Engagement 24% 43 New13 Strategy- Rigor 21% 37 New14 Strategy- Differentiated Instruction 20% 36 10 -8%15 Strategy- Progress Monitoring 20% 36 6 -18%16 Subgroup- ELL Students/ELL Strategies 19% 35 11 -6%17 Strategy- Common Expectations / Understanding 19% 34 16 1%18 Strategy- Collaboration 18% 32 32 16%19 Strategy- Culture/Climate 18% 32 18 5%20 Strategy- College & Career Readiness 17% 31 26 12%21 Strategy- Student Engagement 16% 29 35 15%22 Strategy- Scheduling 11% 19 15 -9%23 Strategy- Community Engagement 10% 18 17 -6%24 Strategy- Coaching (staff) 9% 17 New25 Strategy- Attendance 8% 15 27 4%26 Strategy- Cross Content Area Linkage 8% 14 37 7%27 Strategy- Academic Language 7% 13 New28 Ed Level- Grade Level Focus 7% 12 19 -5%29 Strategy- Behavior 6% 11 24 -1%30 Strategy- Program Fidelity 6% 10 28 3%

* (CCSS Standards were coded with ELGs and WIDA in 2013)

Rank UIP Focus Area

% DPS Schools

# DPS Schools

Prior Year Rank

YOY % Change

31 Strategy- ACT/ACT Prep 5% 9 20 -3%32 Strategy- Diversity/Cultural Responsiveness/Equity 5% 9 36 4%33 Strategy- Essential Learning Goals (ELGs) 5% 9 New34 Strategy- LEAP 4% 8 23 -3%35 Subgroup- Minority Students 4% 8 30 2%36 Other- Critical-Thinking / Problem-Solving 4% 8 New37 Other- Feedback to Students 4% 8 New38 Strategy- Teacher Evaluation (Not LEAP Specific) 4% 7 New39 Strategy- On-Track/Credit Recovery 4% 7 31 2%40 Subgroup- FRL Students 4% 7 44 4%41 Strategy- AP/IB 3% 6 29 0%42 Strategy- Social-Emotional 3% 6 New43 Subgroup- SPED Students 3% 6 22 -4%44 Content Area- Science 3% 5 25 -3%45 Strategy- Scope and Sequence 3% 5 New46 Strategy- Short-Cycle Assessments (SCAN) 3% 5 New47 Strategy- WIDA Standards 3% 5 New48 Content Area- Social Studies 2% 4 42 1%49 Strategy- Colorado Academic Standards (CAS) 2% 4 New50 Strategy- Staffing Models 2% 4 New51 Strategy- Teacher Qualifications 2% 4 New52 Subgroup- Focus on Gender 2% 4 33 1%53 Strategy- Capacity Building 2% 3 21 -6%54 Strategy- Concurrent Enrollment 2% 3 40 1%55 Strategy- Dropout Rates/Retention Strategies 2% 3 New56 Other- High Expectations 2% 3 New57 Strategy- AVID Strategies 1% 2 39 0%58 Strategy- Blended Learning 1% 2 New59 Strategy- Staff Retention 1% 2 New60 Subgroup- GT/HGT 1% 2 43 0%61 Other- Community Partner 1% 2 New62 Strategy- Grad Rates/Completion Strategies 1% 1 38 0%63 Strategy- Program Evaluation 1% 1 41 0%64 Strategy- Turnaround Partner 1% 1 34 0%65 Other- Ascent 1% 1 New66 Other- Next Generation Science Standards [NGSS] 1% 1 New

APPENDIX B: Definitions/Examples* of Common MISs

MIS Definition or Example

Reading/Literacy -Focus on reading strategies related to instructional strategies, curriculum and interventions-Provide targeted and differentiated instruction with specific focus on individuals students’ needs in reading.

School-wide/System Improvement

-Ex: “Implement a school-wide writing strategy” -Ex: “Initiate school-wide intervention plan”

Progress Monitoring

-Ex:” Create and implement progress monitoring systems designed to ensure on-track graduation for all students”-Ex: “Refine our system of progress monitoring for all content areas and our data cycle”

Data Driven Practices

-Ex: “Use data team process to drive instructional decisions to ensure student achievement”

CCSS Standards -Ex: “Begin to unpack the CCSS, map the standards to curriculum, and see how expectations change across grade levels”-Ex: “Teachers will be provided with effective collaboration time to review data, plan reading and writing instruction based on the CCSS in Literacy.

Common Understanding/ Expectations

-Ex: “Through collaboration, grade level expectations will be agreed upon and shared with all stakeholders”-Ex: “Improve writing instruction for all students by developing a shared understanding of writing expectations”

23* Examples were tagged for the MIS listed on the left but could have been tagged for other MISs as well.

Sample: UIP Sample: 180 Total UIPs submitted by January 2013*

Method: Major Improvement Strategy Theme Choices

56 codes (possible themes) were developed by ARE Team before coding began, taking into consideration district priorities, and adding new codes where necessary. 12 codes were changed/added from prior year.

During coding process, ARE Team discovered additional common themes, and these were added/revised for a final set of 66 possible codes.

Major Improvement Strategy Coding ARE Team reviewed and coded all major improvement strategies

(MISs) listed in the submitted UIPs. Analysis did not include adjustments made to UIPs after Jan 2013

(some schools could have updated/changed major improvement strategies after the Jan 2013 submission)

All codes were analyzed and ranked. 24*Based on 2013-14 UIPs (not made public until May 2014).

APPENDIX C: UIP Qualitative Analysis: Methodology

Professional Development Coding Many schools indicated PD as a major improvement

strategy. When schools provided specifics on the type(s) of PD, that information was captured and reported.

For network analysis only: In most cases, each item on the list of PD areas were mentioned by one to a few schools. Highlighted in blue are the areas where most schools focused their PD.

Schools with High and Low Growth The major improvement strategies, action steps and

targets/interim measures of schools with the highest (and lowest) TCAP MGPs were analyzed and summarized.

25

UIP Qualitative Analysis: Methodology (Cont.)

Analysis Limitations: The way in which some DPS school major improvement strategies

were written provided a limited view of the actual strategies being implemented schools. There is a need for schools to be more precise in the language in the UIP considering the multiple audiences reviewing these plans.

– Consider the short major improvement strategies below:“Common Expectations” or “Parent Engagement” or“Implement comprehensive attendance initiative”

– Other schools provide more information, for a better MIS:“Improve student achievement in Reading, Writing and Math through

consistent instructional practices.”“Develop argumentative writing across content areas as a core

academic skill.”

At their best, schools identify the specific approach to be used to achieve their desired results, at the appropriate magnitude given the overall performance for the school, in response to the identified root cause.

“Implement school-wide daily collaborative planning in order to drive student achievement via improved use of formative data analysis and improved instruction.”

26

UIP Qualitative Analysis: Methodology (Cont.)

Content Area Coding Limitations: The analysis found that almost one-third of schools did not focus on

any specific content area in any of their Major Improvement Strategies.* However, schools may be focusing on a content area, but did not mention it specifically in their major improvement strategies.

– Consider the major improvement strategy below. Schools may explicitly mention that all content areas will benefit from this strategy being implemented, and have appropriately identified strategies that are broader and address issues in the overall system given the magnitude of overall performance for the school.

“Through classroom observations and consistent feedback, improve Tier 1 instruction and intervention strategies in all content areas.”

27

UIP Qualitative Analysis: Methodology (Cont.)

* See the Content Area Analysis Slide.

Disaggregated Group Coding Limitations: The analysis finds that the percentage of schools focusing on

ELL students/strategies fell by 6 percentage points. However, schools may be focusing on this group but did not

mention them specifically in their major improvement strategies. –Consider the major improvement strategies below. ELL students are not mentioned specifically but could be apart of their strategy.

“We will provide systematic and structured supports for progress monitoring, differentiation and interventions as needed.”

“Use blended learning approach to ensure catch up and close gaps through individualized instruction”

“We will implement and provide structures and support to build consistency in core instruction, including differentiation, progress monitoring and implementing RtI in instruction E-8, with particular focus on students within disaggregated groups.”

28

UIP Qualitative Analysis: Methodology (Cont.)

* See the Subgroup Analysis Slide.