Year One Fuller Report

Since 1997, serving as a catalyst for change in the public schools that educate nine of every ten children in our region, teaming with principals and teachers in targeted schools to ensure that every student is prepared for post-secondary success.

February 20, 2012

Dear Challenge Network Stakeholder,

Just over 18 months ago, Houston A+ Challenge launched an ambitious new initiative aimed at significantly

increasing the number of students in our region’s public middle schools who are truly on track for

success in high school, college, career and civic life. We began our work with a simple, research-based

premise – that schools and families must raise their expectations for all students, because our state’s long-

held definition of “good enough” is simply not good enough to ensure the success of our children and our

democracy in the 21st century.

The timing for this project could not have been more opportune. This spring, for the first time, Texas

schools and students will be faced with new state tests that target post-secondary readiness instead of basic

proficiency. Simultaneously, public schools are dealing with the impact of more than $5 billion in cuts

statewide to education-related spending made during the during the recent 2011 legislative session. As they

navigate this new landscape of higher expectations and fewer resources, our partner districts, principals and

teachers have expressed much gratitude for the extra resources and technical support offered by the Challenge

Network.

Thus, it is with great pleasure that we share with you the external evaluation of our pilot year of work

in Challenge Network schools. The report’s emphasis on quantitative student outcomes – specifically, the

number of students who moved from “proficient” (or basic) to “commended” (or college-ready-track)

performance – was by design, as the Houston A+ Challenge Board of Trustees recognizes the need to

demonstrate early, clear, positive student outcomes in order to justify continued investment from A+, its

funders, and its partner districts.

We are pleased to quote Dr. Ed Fuller, Associate Professor and Director of the Center for Education

Evaluation and Policy Analysis at Penn State University, in reporting that his independent analysis of student

data “strongly suggests that the reform effort is having a positive impact on student performance.” Dr.

Fuller further concludes that “these initial results are very promising, and the reform effort should be

fully supported by school and district personnel as well as funders and policymakers.” Significant

findings include:

Four out of six Challenge Network schools exhibited positive results. Excluding one school

that did not have an A+ Coach for the full year, four of the five schools implementing the reform

performed better than or about as well as expected.

Absolute results were strongest at YES Prep West (YES Prep Public Schools), where more

students reached the college-ready standard than at any other Challenge Network school and any

other campus in the high-performing YES district.

Three schools exhibited very strong relative student performance, meaning A+ Scholars were

statistically significantly more likely to achieve commended status than similar peers in comparison

schools statewide. A+ Scholars at Horace Mann Junior School (Goose Creek CISD) were

more than 10 times more likely than similar students in comparison schools to attain commended

performance; A+ Scholars at YES Prep West were 4.6 times more likely to do so; and A+

Scholars at Caraway Intermediate School (Aldine ISD) were 1.6 times more likely to reach

the higher bar.

A+ Scholars at O’Donnell Middle School (Alief ISD) were also more likely to achieve

commended status than similar students in similar schools statewide, but the measured effect was

FOUNDING BOARD MEMBER

Maconda B. O’Connor, Ph.D.

The Brown Foundation

EXECUTIVE COMMITTEE

CHAIR

Bill Crouch

Continental Airlines, Inc.

PRESIDENT

Gasper Mir, III

MFR, P.C.

TREASURER

Thomas L. Elsenbrook

Alvarez & Marsal Business

Consulting, LLC

SECRETARY

Yava D. Scott

Siebert Brandford

Shank & Co., LLC

BOARD MEMBERS

Susan A. Bischoff

Houston Public

Library Foundation

Jonathan Day

Andrews Kurth LLP

Bolivar Fraga

Neighborhood Centers, Inc.

Roberto Gonzalez

Employment & Training

Centers, Inc.

Shawn Gross

SAJG Investments, Inc.

Scott McClelland

H-E-B

Karol Musher, M.A., CCC-SLP

Texas Children’s Hospital

Mary Nesbitt

Parent Visionaries

Rod Paige, Ph.D.

Chartwell Education Group LLC

David Ruiz

Bank of America

J. Victor Samuels

Victory Packaging, Inc.

J. David Thompson, III

Thompson & Horton LLP

Bobby Tudor

Tudor, Pickering, Holt & Co.

Andrea White

Community Volunteer

ADVISORY BOARD

Leonel Castillo

Mayor's Office (Retired)

Michael Dee

Investment Banker

H. Devon Graham, Jr.

R. E. Smith Interests, Inc.

Jenard M. Gross

Gross Investments

Ann Friedman, Ph. D.

Community Volunteer

Harry M. Reasoner

Vinson & Elkins LLP

Rosie Zamora

Houston Wilderness, Inc.

not as statistically significant. However, O’Donnell students did show strong results in their

individual academic growth over time.

Hoffman Middle School (Aldine ISD) was neutral on most measures. Atascocita Middle School

(Humble ISD) had the worst performance profile on multiple measures. One influencing factor

was a mid-year staffing change at A+, which led to the A+ Coach not being present to coordinate

and drive the reform at AMS for the full school year. This was disruptive to A+’s traction with

teachers and students at AMS, and likely had a negative impact on the school’s ability to reach

higher performance targets.

At least 8 in 10 A+ Scholars surveyed reported that, because of their A+ Coach, they work

harder in class, don’t get into trouble as much, pay attention in class more, do better in class,

understand the teacher better, and feel more confident about school.

These early results from our pilot year are promising but mixed, and we must keep in mind that history and

research have taught us that most reforms take multiple years to bear full and consistent fruit. Education

researcher Michael Fullan defines this as “implementation dip”, or the “inevitable bumpiness and difficulties

encountered as people learn new behaviors and beliefs.”

In conclusion, we are so gratified to work alongside many brave, dedicated, thoughtful teachers and

principals who are rising to the challenge of not only learning, but adopting many new beliefs and

behaviors. We are also extremely thankful for the unwavering confidence of the Brown Foundation,

Houston Endowment and a host of other individuals, foundations and corporations that support Houston A+

Challenge. And none of our work would be possible without the support of forward-thinking leaders at our

Challenge Network partner districts – Aldine, Alief, Goose Creek, Humble and YES Prep – and our

talented initiative partners at Communities In Schools and Region 4 Education Service Center.

But most of all, we are grateful for the nearly 900 A+ Scholars who provide the inspiration for all of our

efforts. Each number that you read in this report relates directly to a child’s hard work, perseverance and

vision for the future. We hope that you will be as inspired as we are by their achievements.

Sincerely,

William K. Crouch Scott A. Van Beck, Ed.D.

President, Board of Trustees Executive Director

Year One Evaluation of the

Houston A+ Challenge

Network of Middle Schools

(2010-11)

Published February 2012

Full Report Available Online at:

www.houstonaplus.org/challenge-network/results

Ed Fuller, Ph.D.

Associate Professor and Director

Center for Education Evaluation and Policy Analysis

Penn State University

http://www.houstonaplus.org/challenge-network/results

Houston A+ Challenge Evaluation Fall 2011 Ed Fuller

2

EXECUTIVE SUMMARY

The Houston A+ Challenge Network consisted of six middle schools in six districts in the Houston metro area. Each school selected a grade level and subject area to focus on, with one school choosing both math and reading. Table ES-1 provides the details on the districts, schools, grade levels, and subject areas involved.

Table ES-1: School Size, Grade Span, and Targeted Grade Levels for 2010-11 School Year

District School Number A+ 2010 Grade Target Subject

Name Name Students Configuration Grade Area

Aldine ISD Caraway 359 05 - 06 5 Reading

Aldine ISD Hoffman 411 07 - 08 7 Mathematics

Alief ISD O'Donnell 625 07 - 08 7 Mathematics

Goose Creek ISD Mann 239 06 - 08 6 Mathematics

Humble ISD Atascocita 567 06 - 08 6 Math/Reading

YES Prep YES Prep 144 06 - 07 7 Reading

The primary goal of the reform was to have students “in the middle”—those who had passed TAKS but not attained commended status—improve enough to achieve commended status. The reform consisted of multiple activities that included the following:

Coaching of principals and school leadership teams;

Coaching of teachers;

Engaging with and supporting parents;

Holding network level events and trainings;

Developing Professional Learning Communities (PLCs) within schools;

Developing shared responsibility among all school staff for student success;

Implementing Assessment FOR Learning;

Adopting rigorous, data informed instruction targeted to higher standards;

Improving learner behavior and academic discipline;

Increasing parental awareness of requirements for post-secondary success; and,

Implementing cross-district and charter sharing of best practices

As with most reform efforts, the Houston A+ Challenge (A+) reform effort had mixed results in Year One. This is typical with almost every reform effort, since these efforts typically ask teachers and administrators to make fairly dramatic changes in how they think and behave. This often causes fear and stress, among other emotions, all of which distract from the focus on teaching and learning. On top of the normal reactions to change were deep budget cuts that resulted in layoffs, cuts in services, and potentially larger class sizes. These factors cannot be discounted when evaluating a reform effort.

With respect to the primary goal of attainment of commended status, Table ES-2 shows mixed results. A+ Scholars in three schools—Caraway, Mann, and YES Prep West—were statistically significantly more likely than their peers in comparison schools to achieve commended status. Although not statistically significant, the results also suggested a possible


3

positive impact on O’Donnell students. There was no statistically significant effect at Hoffman, but results suggested a possible negative impact. Atascocita is a special case, as there was a staff change in A+ Coaches during the fall semester, so the school did not receive a full, uninterrupted year of coaching (as did the other schools). This may have had a negative effect on student achievement. Regardless of the reason, A+ Scholars in Atascocita were statistically significantly less likely than their peers in comparison schools to achieve commended status in mathematics. In reading, the result was not statistically significant, but suggested a possible negative effect.

Table ES-2: Logistic Regression Results for Students in the Middle

Attaining Commended Status for A+ Scholars

School Stat. Odds Interpretation of

Name Sig. Ratio Odds Ratio

Caraway 0.036 1.586 58.6% more likely

Hoffman 0.277 0.656 Not stat sig: less likely

Mann 0.000 10.747 10.7 times more likely

Atascocita-Math 0.046 0.242 75.8% less likely

Atascocita-Reading 0.365 0.687 Not stat sig: less likely

O'Donnell 0.321 1.462 Not stat sig: more likely

YES Prep 0.050 4.658 4.65 times more likely If Atascocita is removed from the analysis because the school did not fully implement the entire reform package, then three of the five schools fully implementing the reform had A+ Scholars who were statistically significantly more likely to achieve commended status than peers in comparison schools. Moreover, one of the remaining two schools had a positive, but not statistically significant effect. These results strongly suggest that the reform effort is having a positive effect on student attainment of commended status. Table ES-3 includes estimates of whether students performed greater than, equal to, or worse than predicted based on prior test scores, economically disadvantaged status, and school demographics and achievement. Standardized residual values around zero indicate the school performed about as expected, while positive standardized residuals indicate that the school performed greater than expected.

Three of the six schools performed greater than predicted: YES Prep West, O’Donnell, and Mann. Two schools performed worse than expected: Caraway and Atascocita. One school—Hoffman—performed about as well as expected.

Table ES-3: Standardized Residuals (in Z-Scores) for All A+ Scholar Groups

District School Target Subject Standardized Effect Name Name Grade Area Residual

Aldine ISD Caraway A+ Scholars 5 Reading -0.313 Negative Goose Creek ISD Mann A+ Scholars 6 Math 0.144 Positive Humble ISD Atascocita A+ Scholars 6 Math -0.448 Negative

Atascocita A+ Scholars 6 Reading -0.297 Negative Aldine ISD Hoffman A+ Scholars 7 Math -0.071 Neutral Alief ISD O'Donnell A+ Scholars 7 Math 0.123 Positive YES Prep West YES Prep A+ Scholars 7 Reading 0.263 Positive


4

Discounting the negative results from Atascocita because the school did not fully implement the reforms, then four of the five schools performed about as well as or greater than expected. Again, this strongly suggests that the reform effort is having a positive impact on student performance. The final important measure of the reform effort is the direct effect on student behavior. To assess this impact, a survey was administered to all A+ Scholars. Three of the six schools had large enough response rates to report findings. Responses for the three schools were collapsed together and reported below in Figure EW-1. Students were asked to respond to eight prompts, each of which started with the phrase: “Because of the A+ Coach …” More than 78% of respondents indicated that their own school-related positive behaviors had increased because of interactions with the A+ Coach. Most impressively, about 88% of the students agreed or strongly agreed that the A+ Coach had made them work harder in class, and 84% of the students believed that they worked harder in class because of the A+ Coach. Although there is no other data to substantiate the perceptions of the students in these three schools, the results do strongly suggest that the reform effort—in particular, the A+ Coach—had positive effects on student behavior.

Figure ES-1: Survey Results for Effects of the Coach over Time

Overall, the data collected and analyzed for the Year One evaluation strongly suggests that the Houston A+ Challenge Network reform effort is having a positive effect on the likelihood of students attaining commended status, student growth as measured through value-added analyses, and student behavior as measured by student self-reports. The effects, however, are unevenly distributed across schools. Additional efforts need to be undertaken to ensure that all of the reform components are fully implemented in each school and additional information needs to be collected to determine the causes of some of the low performance so that the reform can be modified in these schools. In final conclusion, the reform efforts have had a strong Year One start despite not being fully operational until the middle of the academic year. These initial results are very promising, and the reform effort should be fully supported by school and district personnel as well as funders and policymakers.


5

SECTION I: DESCRIPTION OF THE PROGRAM This section describes the organization implementing the reform effort, the program goals, and the theory of change that undergirds the reform effort. A. REFORM ORGANIZATION

The Houston A+ Challenge has a long-standing track record of supporting school reform efforts across the Houston metro area. Houston A+ Challenge serves as a catalyst for change in the public schools that educate nine of every ten children in the Houston region, teaming with principals and teachers in targeted schools to ensure that every student is prepared for post-secondary success. A+ selected six schools in six districts across the Houston metro area as partners in an effort to increase the percentage of students on track to achieve college-readiness in high school.

Unlike many reform efforts, the Houston A+ Challenge Network focuses on students, teachers, principals, other school staff, and parents. Moreover, the focus of the reform is not simply on increasing student test scores, but on: improving student behaviors; teaching students how to learn; enhancing the capacity of teachers; providing socio-emotional support to students; encouraging parents to support school behaviors and desire to attend college; and, enjoining school staff and leaders in the reform process.

B. PROGRAM GOALS

The Theory of Change for this reform effort is based on emerging research that points to the importance of building the capacity of school leaders, teachers, students and parents in order to achieve four outcomes: 1. Increased percentage of A+ Scholar students on track to reach college-readiness

standards in high schools;

2. Increased percentage of all students on track to reach college-readiness standards in high schools;

3. Increased awareness of and intentions to enroll in institutions of higher education for all students; and,

4. A sustained system of success and constant improvement within schools

In this Year One evaluation, the focus is on outcomes one and three. In subsequent years, all four outcomes will be examined in the evaluation. C. THEORY OF CHANGE

The Houston A+ Challenge Theory of Change, as depicted in Figure 1, illustrates the process through which A+ will achieve their articulated goals. A more detailed exposition of each component is provided below. The Theory of Change is divided into five areas: inputs, activities, outputs, intermediate outcomes, and final outcomes.


6

Figure 1. Theory of Change, Houston A+ Challenge Network

1. Inputs

The Houston A+ Challenge Network marshals critical inputs towards the goal of achieving the outcomes outlined above. The inputs include: Coaching for Performance by A+ team; a supportive peer network that promotes learning from collaboration; a research-based focus on and approach towards post-secondary success; district commitments; partnerships with organizations such as Communities in Schools, and funding. 2. Activities

Recognizing that improved student outcomes depends on activating all elements of the education ecosystem, A+ Challenge worked with principals, teachers, parents, the district leadership, and students themselves. A+ Challenge aligned their resources to perform five key program activities:


7

a) Coaching of principals and school leadership teams

Students benefit when schools have a vision, a culture and a plan for ensuring post-secondary success for all. A+ provided coaching to middle school principals and the leadership teams to develop their leadership skills and build and implement a comprehensive, sustainable, research-based action plan for advancing student post-secondary success on their campus.

b) Coaching of teachers

Students benefit when teachers work continuously to improve their craft and have adequate support for doing so. A+ Performance Coaches built teacher capacity through intensive individualized coaching and targeted professional development that occurred throughout the entire school year. The coaching and professional development are based on attention to data, curriculum standards, lesson planning, instructional strategies, classroom observations, assessment design, critical feedback, and repetition of the cycle. Teachers are trained to use formative assessment techniques to make their instruction more rigorous, targeted and informed by data. Daily activities include team planning, co-teaching, demonstrating lessons, and providing constructive feedback to teachers.

c) Supporting and motivating students

At each Challenge Network school, the A+ Performance Coach worked with teachers to identify a cohort of middle-school students who scored between “proficient” and “commended” on TAKS in the previous academic year. These students are “in the middle”—not low-performing, but not high-performing either. Since the students had already met the minimum standards on the state-mandated assessment and the accountability system only rewards the attainment of passing, these students often do not receive the extra help they need to become college-ready. Coaches and teachers spent extra instructional time with these students to ensure that they reach and maintain high levels of performance. The Communities in Schools counselor also worked closely with the coach and teachers to identify any social and emotional needs that struggling students might have and create a plan to address those needs. In addition, students received information on pathways to college – such as preparing for and choosing the best high school coursework and learning about the SATs – as well as invitations to special events such as college visits.

d) Engaging with and supporting parents

Houston A+ Challenge and its partner, Communities In Schools, worked with school staff to get student and families to sign commitments, helped parents understand that ‘passing’ is not enough, and continually engaged families in having higher expectations and support for their students.


8

e) Holding network level events and trainings

The A+ Challenge network level events and trainings enabled peer collaborations and opportunities for learning that will hopefully be sustained beyond the A+ Challenge immediate engagement with schools. These forums provided opportunities for district schools to learn from some of the best practices in charters, as well as for charter schools to learn from best practices in district schools.

4. Outputs

The activities outlined above were designed to deliver seven process outputs that will lead to the attainment of the four intermediate goals which, in turn, will lead to the attainment of the four over-arching goals described above. These seven process outputs include the following activities:

a) Development of Professional Learning Communities (PLCs) within schools

Led by the principal, the staff collaboratively and continuously builds its collective capacity to provide high levels of learning for all students. Studies have found that a shared focus that unifies the work of all members of the organization and a collaborative culture are common characteristics of high-performing organizations, including schools. PLCs are identifiable by six characteristics: A focus on high level of learning for all students; a collaborative school culture; collective enquiry into best practices and current reality; action orientation; a commitment to continuous improvement, and a focus on results.

b) Responsibility for students’ post-secondary success shared by non-teaching positions

Schools will identify and designate a non-teacher leader to be the “School-based Coach” who in Year Two will work closely with the A+ Coach and play the role of A+ Coach with an incoming cohort of approximately 150 students to ensure scalability and sustainability. The School-based Coach drives student achievement gains by building teacher capacity through individualized coaching and targeted professional development based on attention to data, curriculum standards, lesson planning, instructional strategies, classroom observations, assessment design, critical feedback, and repetition of the cycle. The School-based Coach also drives student achievement gains through working with an identified group of middle school students to improve their achievement and likelihood of being on-track to be post-secondary ready by the time they enroll in high school. Finally, the School-based Coach drives student achievement gains by working with students’ families to ensure they understand the importance of having their child be post-secondary ready entering high school.


9

c) Assessment FOR Learning practiced by all teachers

Teachers use Assessment FOR Learning techniques to identify the specific learning needs of students, adapt instruction to meet those needs, and motivate students to engage in the learning process. Teachers also provide continuous, consistent, and descriptive feedback to students. Extensive research has documented the significant impact that the effective use of assessments to support student learning has on student performance, including on standardized assessments.

d) Rigorous, data informed instruction to aim for higher standards

Teachers plan lessons and student interventions based on a rigorous analysis of student data, an awareness of standards, an appreciation of best practices in instruction, and the knowledge of experienced coaches, teachers, and administrators.

e) Improved learner behavior and academic discipline

Students respond to the increased instruction time and the support and motivation they receive both inside and outside classrooms by increasing their level of effort and engagement, thereby leading to improved learner behavior and academic discipline.

f) Parental awareness of requirements for post-secondary success

Parents are aware of what it will take for their children to achieve college-readiness and are fully committed to the goal of ensuring their child’s post-secondary success.

g) Cross-district and traditional-charter district sharing

School districts and charter schools continuously share what they have learned, as

well as best practices, thereby creating a sustained, peer-based support network that collectively strives to ensure that all students within the A+ Challenge Network are on track to reach college-readiness.

5. Intermediate outcomes

The activities will lead to four intermediate outcomes that research shows directly translate into the final desired outcomes:

a) Increased capacity of principal and school-based coaches

Research over the past decade has under-scored the importance of the principal and school leadership team in improving schooling outcomes for students. School leaders improve student outcomes primarily through several key indirect strategies associated with strategically managing the human capital within the school setting and improving the capacity of parents to provide support and assistance to their children.


10

b) Increased teacher capacity and efficacy

A large and growing body of evidence has identified teachers as the single most important school-based factor that influences student outcomes. There is now consensus among both researchers and policymakers that the capacity of both individual teachers and the team of teachers at a school are inextricably linked to the academic improvement of students in the school.

c) Increased student effort and engagement

Experience shows that greater effort and involvement by students directly

translates into improved scholastic performance.

d) Increased parental support and involvement

Every study on the factors affecting the academic achievement of students finds that the home lives of students have the largest and most profound effect on student success. Research on improving outcomes has shown that providing non-academic support to students and increasing the knowledge and skills of parents to engage in helping their students and interacting with the school are associated with improving school outcomes for students.


11

SECTION II: CONTEXT This section provides background and contextual information about the state policy environment and the school characteristics of the schools involved in the reform effort. A. STATE POLICY The relevant state policy context within which this reform was undertaken includes an increased focus on ensuring a greater percentage of students are well-prepared for college or for employment after high school, declining investments in education, decreasing budgets for school districts, and increasing poverty rates for students enrolled in schools. 1. Focus on College and Post-Secondary Readiness Texas has long been a leader in testing and accountability. Texas implemented minimum competency testing in the 1980s with the Texas Assessment of Basic Skills (TABS) and replaced that Texas with a more rigorous test, the Texas Assessment of Academic Skills, or TAAS. Concomitant with the introduction of the TAAS in the early 1990s was the introduction of the Texas school and school district accountability system. The system held schools accountable for three primary outcomes: test scores, dropout rates, and attendance rates. The novel component of the system was that all the data was disaggregated by student subpopulation in a way that a school’s lowest performing subgroup determined the school’s accountability rating. TAAS was replaced with the Texas Assessment of Knowledge and Skills (TAKS) in 2003. Again, the rigor of the new test was greater than the test that it replaced. The accountability system changed minimally, but kept the primary driver of focusing on outcomes by student subpopulation intact. In the late 2000s, an increasing volume of research pointed to the need for schools to focus more on preparing students to be college- and career-ready than on meeting minimum competency standards as indicated by the percentage of students passing state mandated tests. Thus, policymakers, parents, and the business community started to demand that schools increase their focus on college- and career-readiness. Consequently, Texas was one of the first states to adopt college- and career-readiness standards. Further, Texas started reporting a college-readiness indicator on the publicly available Texas Education Agency website. Also at the end of the decade, a number of reports were released that showed that student achievement levels in elementary and middle school could predict college readiness in high school. Further, the reports uncovered that high schools did not have the capacity to take students not well prepared for high school and prepare them for college. In essence, then, the research concluded that middle schools are the last chance for students to get on the college-ready track. Thus, the 2000s saw an increasing amount of pressure on high schools to push students past simple proficiency and towards being college- and career-ready. Middle schools also felt pressure to ensure that students focused not just on meeting proficiency, but on preparing students to be well-prepared for high school and on the college-ready track. The adoption of a new state-mandated assessment also drives higher expectations for student achievement as the new tests measures college-readiness across all grade levels.


12

2. Declining Budgets Texas has never been a state renowned for its investment in K-12 education. Indeed, Texas has always ranked in the bottom tier of states in terms of investment in K-12 education. With the recession of 2008 and the school finance/property tax bill of 2006, the available revenue available for education and other areas plunged dramatically. In the 2010 legislative session, legislators cut $4 million from K_12 expenditures over the 2011-12 and 2012-13 school years. The indecisiveness and lack of planning of the Legislature led to widespread confusion and fear among school districts during the spring of 2011. Many districts adopted widespread layoffs of teachers and central office staff. Many districts have now had to ask for waivers from the mandated 22:1 student-teacher ratio in grades one through three. Thus, the implementation of this reform effort occurs at a time of declining budgets, decrease in resource availability, and an increase in class sizes. These factors could certainly have a negative impact on the efficacy of the reform effort since the schools are being asked to maintain progress with fewer resources. 3. Increasing Poverty and Student Diversity Texas has one of the highest child poverty rates in the country and has seen an explosion in the number of low-income students over the past two decades. The increase has been widespread across the state, but more pronounced in the urban centers and along the Texas-Mexico border. Figure 1: Percentage of Economically Disadvantaged Students in Texas Public Schools

Source: Academic Excellence Indicator System (AEIS), Texas Education Agency (TEA)


13

Figure 2: Percentage of Economically Disadvantaged Students by School Level in Texas Public

Schools (1995, 2000, 2005, and 2010)

Source: AEIS, TEA B. SCHOOLS This section provides information on the school characteristics of the reform schools. This information includes the school grade configuration and enrollment, student characteristics, and teacher characteristics. 1. School Configuration and Size As shown in Table 1, the six schools varied considerably in size and grade span. YES Prep West enrolled only 144 students in the 7th grade, while Atascocita enrolled 1,705 students in grades six through eight. Research generally finds that larger schools have detrimental effects on student achievement and persistence in schools, particularly for lower-performing and low-income students. Thus, some schools were in a more advantageous position to advance student achievement than other schools simply based on student enrollment. Three of the six schools chose to focus on 7th grade mathematics, one school chose to focus on 6th grade mathematics, one school chose to focus on 5th grade reading, and one school chose to focus on 6th grade reading and mathematics. Schools focusing on reading have a more difficult time producing gains in achievement, because research strongly indicates that reading ability is strongly influenced by the home life of students – in particular, poverty and the factors associated with poverty, such as poor nutrition, lack of health care, and inadequate access to books at home and in the community.


14

Table 1: School Size, Grade Span, and Targeted Grade Levels for 2010-11 School Year

District School School Size Tgt 2010 Grade Target Subject Area Target:

Name Name Size Grade Configuration Grade Math Reading

Aldine ISD Caraway 749 359 05 - 06 5 none 5

Aldine ISD Hoffman 808 411 07 - 08 7 7 none

Goose Creek ISD Mann 731 239 06 - 08 6 6 none

Humble ISD Atascocita 1705 567 06 - 08 6 6 6

Alief ISD O'Donnell 1213 625 07 - 08 7 7 none

YES Prep YES Prep 144 144 06 - 07 7 none 7

Source: AEIS, TEA; Evaluation data collected from sites Table 2 documents the overall student enrollment of the A+ Scholar schools, as well as the average enrollment of the A+ Scholar comparison schools. In general, the five schools with longitudinal data on enrollment have had stagnant or slightly declining enrollment trends. Mann Junior High School in particular experienced declining enrollment, with a decrease of almost 300 students from 2006 to 2010. Stable student enrollments tend to make reform changes easier to implement because of staff and student stability, while rapidly increasing or decreasing enrollments tend to make changes more difficult to implement. Hoffman, Mann, and YES Prep West had similar enrollments as comparison schools, while Caraway had a smaller enrollment than comparison schools. In contrast, O’Donnell and Atascocita had student enrollments much larger than comparison schools.

Table 2: Student Enrollment for All Grades, 2006-2010

School Spring of Academic Year Change: Name 2006 2007 2008 2009 2010 2006 to 2010

Caraway Int 820.0 780.0 811.0 782.0 749.0 -71.0 Caraway Comparisons 929.8 933.0 892.3 940.3 977.3 47.5 Hoffman MS 971.0 885.0 881.0 930.0 808.0 -163.0 Hoffman Comparisons 860.8 834.3 862.3 791.8 791.3 -69.5 O'Donnell MS 1263.0 1150.0 1143.0 1206.0 1213.0 -50.0 O'Donnell Comparisons 1128.0 1071.8 1033.8 983.4 965.8 -162.2 Mann JHS 1008.0 994.0 978.0 814.0 731.0 -277.0 Mann Comparisons 773.2 771.0 807.6 795.4 785.0 11.8 Atascocita 1508.0 1652.0 1520.0 1656.0 1705.0 197.0 Atascocita Comparisons 1148.5 1250.8 1249.8 1061.0 1109.0 -39.5 YES Prep West 144.0 na YES Prep West Comparisons 477.0 453.0 497.5 496.5 248.3 -228.7

SOURCE: AEIS, TEA


15

2. School Demographics a) Economically Disadvantaged Students As shown in Table 3, three of the five schools with longitudinal data had very stable percentages of economically disadvantaged students, while both Hoffman and O’Donnell had slight increases in the percentage. All but Atascocita were predominantly low-income schools, with at least 73% of the students identified as economically disadvantaged. Atascocita, alternatively, only had 20% of students identified as economically disadvantaged. By design, comparison schools had similar percentages of economically disadvantaged students as the A+ Scholar schools. For five of the A+ Scholar Schools—Caraway, Hoffman, O’Donnell, Mann and YES Prep West—comparison schools had much greater increases in the percentage of economically disadvantaged students over the last five years. The relative stability in the percentage of economically disadvantaged students in A+ Scholar Schools creates a more advantageous environment for implementing reform.

Table 3: Percentage of Economically Disadvantaged Students, 2006 to 2010


Caraway Int 85.6 87.3 86.7 88.2 86.8 1.2 Caraway Comparisons 69.8 72.9 74.1 73.9 80.5 10.7 Hoffman MS 79.9 84.3 83.3 85.7 84.0 4.1 Hoffman Comparisons 67.6 69.8 71.3 73.6 78.7 11.1 O'Donnell MS 66.9 66.7 62.1 74.3 73.5 6.6 O'Donnell Comparisons 59.8 60.5 59.7 68.6 71.0 11.2 Mann JHS 81.5 74.0 77.3 83.0 81.8 0.3 Mann Comparisons 71.2 71.7 73.0 76.6 80.8 9.6 Atascocita 23.3 21.7 14.9 15.4 20.7 -2.6 Atascocita Comparisons 24.2 25.1 20.5 21.9 23.4 -0.8 YES Prep West 84.7 YES Prep West Comparisons 66.1 73.6 68.7 66.9 86.2 20.2

SOURCE: AEIS, TEA b) Percentage of African-American Students

As shown in Table 4, four of the five A+ Scholar Schools with longitudinal data had declining percentages of African-American students. Only Atascocita, with a slight increase in the percentage of African-American students, did not have a decrease. Given that African-American students tend to have the lowest achievement levels of all student sub-populations, the decline of African-American students might increase the overall student performance of the school. Again, by design, comparison schools had relatively similar percentages of African-American students as A+ Scholar Schools. The comparison schools had stable or slightly declining percentages of African-American students over the past five years.


16

Table 4: Percentage of Economically Disadvantaged Students, 2006 to 2010

School Change: Name 2006 2007 2008 2009 2010 2006 to 2010

Caraway Int 55.9 56.4 54.9 52.8 49.3 -6.6 Caraway Comparisons 47.9 47.0 46.2 47.2 48.0 0.1 Hoffman MS 55.3 53.3 52.3 50.2 43.4 -11.9 Hoffman Comparisons 54.9 52.1 50.6 48.7 47.4 -7.5 O'Donnell MS 47.7 46.8 45.8 40.2 37.9 -9.8 O'Donnell Comparisons 35.4 34.8 34.2 32.9 31.9 -3.6 Mann JHS 20.2 17.8 20.6 13.9 13.5 -6.7 Mann Comparisons 16.2 16.3 15.3 14.6 13.1 -3.1 Atascocita 19.8 20.8 22.6 23.1 22.8 3.0 Atascocita Comparisons 19.0 19.9 18.5 20.3 20.0 1.0 YES Prep West 29.2 YES Prep West Comparisons 21.3 22.6 25.3 23.1 21.0 -0.3

SOURCE: AEIS, TEA

c) Percentage of Hispanic Students

As shown in Table 5, four of the five A+ Scholar schools with longitudinal data had increases in the percentage of Hispanic students. Only Atascocita did not have an increase. By design, the percentage of Hispanic students in A+ Scholar schools and comparison schools were relatively similar.

Table 5: Percentage of Hispanic Students, 2006 to 2010


Caraway Int 42.2 41.2 41.9 43.9 47.4 5.2 Caraway Comparisons 39.7 41.5 43.2 43.3 43.1 3.4 Hoffman MS 41.4 44.0 44.9 47.2 53.7 12.3 Hoffman Comparisons 35.2 38.9 41.0 43.7 46.8 11.7 O'Donnell MS 32.5 35.7 38.0 42.0 42.2 9.7 O'Donnell Comparisons 37.0 38.6 41.7 44.8 46.8 9.8 Mann JHS 63.3 66.6 65.5 71.3 73.6 10.3 Mann Comparisons 63.5 64.8 67.7 70.9 73.2 9.7 Atascocita 22.8 24.2 19.9 20.0 21.1 -1.7 Atascocita Comparisons 18.2 20.5 19.6 20.6 21.2 3.0 YES Prep West 54.2 YES Prep West Comparisons 51.9 54.7 54.5 58.0 72.2 20.3

SOURCE: AEIS, TEA


17

d) Percentage of White Students As shown in Table 6, the percentage of White students was relatively stable at all A+ Scholar schools over the five-year time span. Comparison schools evidenced slight declines in the percentage of White students over the same time period, with the exception of the YES Prep West comparison schools, which had a substantial decline in the percentage of White students—from 23% in 2006 to 5% in 2010.

Table 6: Percentage of White Students, 2006 to 2010


Caraway Int 1.2 1.4 1.6 2.2 2.3 1.1 Caraway Comparisons 8.0 7.2 6.7 5.7 5.1 -2.9 Hoffman MS 2.6 2.1 1.7 1.2 1.2 -1.4 Hoffman Comparisons 8.8 7.6 6.9 6.1 4.7 -4.1 O'Donnell MS 7.0 5.3 4.3 5.3 5.4 -1.6 O'Donnell Comparisons 16.4 15.6 13.2 11.7 10.8 -5.5 Mann JHS 15.1 14.3 12.7 13.3 11.8 -3.3 Mann Comparisons 17.3 16.0 14.2 11.8 11.0 -6.4 Atascocita 52.8 51.2 53.6 52.1 51.3 -1.5 Atascocita Comparisons 50.7 46.6 47.9 46.6 46.2 -4.5 YES Prep West 0.7 YES Prep West Comparisons

23.3 19.9 18.0 17.2 5.2 -18.1

SOURCE: AEIS, TEA e) Percentage of Mobile Students Mobile students are defined as those students who have not been in attendance at the school for at least six of the 18 weeks at the school. The lack of attendance could be for absences or transferring into or out of the school at such a time that would cause the student to miss at least 17% of the instructional days at the school.

As shown in Table 7, the A+ Scholar schools had either stable or slightly declining percentages of mobile students. The same was true for the comparison schools. Three of the six schools had a substantial proportion of students identified as mobile. Indeed, Caraway, Hoffman, and O’Donnell had 25% of students identified as mobile. Mann and Atascocita had moderate amounts of mobile students, with 16% and 9.5%, respectively. YES Prep West has 0% mobility.


18

Table 7: Percentage of Mobile Students, 2006 to 2010


Caraway Int 31.2 35.5 29.8 26.1 25.3 -5.9 Caraway Comparisons 19.4 22.7 21.9 19.2 17.5 -1.9 Hoffman MS 26.9 35.5 32.0 28.2 25.3 -1.6 Hoffman Comparisons 20.7 24.4 25.2 21.0 22.7 2.0 O'Donnell MS 24.9 31.4 26.6 24.9 25.4 0.5 O'Donnell Comparisons 18.6 22.0 19.9 17.9 16.9 -1.7 Mann JHS 22.1 25.0 22.4 16.9 16.3 -5.8 Mann Comparisons 20.0 23.2 21.1 17.4 18.0 -2.0 Atascocita 13.1 13.3 13.1 11.9 9.5 -3.6 Atascocita Comparisons 12.7 17.3 14.2 10.3 10.7 -1.9 YES Prep West 0.0 YES Prep West Comparisons 19.9 22.7 23.0 21.1 15.9 -4.0

SOURCE: AEIS, TEA Mobility is important because research has found a clear and consistent negative

relationship between the percentage of mobile students and achievement. The greater the mobility, the more pronounced the negative effect, because of the greater difficulty in teaching students with different knowledge and skills, as well as the lack of information about the prior achievement and academic strengths and weaknesses of mobile students. f) Percentage of Bilingual Students Bilingual education students are those students enrolled in a bilingual education program. As shown in Table 8, three of the A+ Scholar schools had large increases in the percentage of bilingual students in the last five years—Caraway, Hoffman and O’Donnell. All three schools had increases of approximately 10 percentage points. These increases were much greater than the increases for comparison schools. Mann and Atascocita had stable bilingual education populations, and YES Prep did not have longitudinal data available. For Caraway, Hoffmann, and O’Donnell, the increase was far greater than the approximately 3 percentage points for comparison schools. For Mann and Atascocita, the changes were similar to those of comparison schools. The percentage of bilingual students affects achievement in a number of ways. First, the students taking the Spanish-language version of the TAKS were not included in the student growth analysis. Second, bilingual students are typically more expensive to educate, thus fewer resources may be available for other students.


19

Table 8: Percentage of Bilingual Students, 2006 to 2010


Caraway Int 13.5 13.5 18.5 21.4 23.6 10.1 Caraway Comparisons 12.9 12.2 15.2 15.0 16.3 3.3 Hoffman MS 6.6 5.8 6.7 8.6 15.7 9.1 Hoffman Comparisons 6.9 7.4 7.7 9.3 10.2 3.3 O'Donnell MS 10.8 13.0 16.1 16.9 20.4 9.6 O'Donnell Comparisons 8.9 9.2 10.8 12.5 11.4 2.5 Mann JHS 5.9 6.9 5.5 4.1 3.1 -2.8 Mann Comparisons 11.9 10.7 12.2 13.0 15.0 3.1 Atascocita 3.2 2.6 1.4 1.4 1.9 -1.3 Atascocita Comparisons 2.9 3.1 2.7 2.4 2.5 -0.3 YES Prep West 22.2 YES Prep West Comparisons 10.1 11.3 13.0 10.9 22.1 12.1

SOURCE: AEIS, TEA 3. Teacher Characteristics Teacher characteristics are often associated with student achievement, although the research in this area is far from conclusive. Regardless, the percentage of teachers who are assigned “in-field” is a fairly good representation of whether the teachers in a school are prepared to teach the subject area and grade level to which they are assigned. Research in Texas has shown that such an indicator is, in fact, related to growth in student achievement. As shown in Table 9, the percentage of teachers who were assigned in-field varied substantially across the six schools. Both O’Donnell and YES Prep-West had roughly 50% of teachers assigned in-field, while Caraway has about 88% of teachers assigned in-field. Four of the five A+ Scholar schools with available data had increases in the percentage of teachers assigned in-field, while only Mann had a decrease. All but one set of comparison schools (those for Caraway) had a decrease in the percentage of teachers assigned in-field.

Table 9: Percentage of Teachers Assigned In-Field, 2006 to 2010


Caraway Int 60.7 65.8 67.0 77.1 87.9 27.2 Caraway Comparisons 83.2 74.0 83.5 86.0 86.8 3.6 Hoffman MS 53.2 46.4 55.8 57.0 60.7 7.5 Hoffman Comparisons 71.7 56.2 54.9 54.8 50.5 -21.1 O'Donnell MS 46.7 51.3 43.8 52.6 51.7 5.0 O'Donnell Comparisons 71.9 61.8 49.2 51.5 51.9 -20.0 Mann JHS 82.0 75.4 75.4 72.5 75.6 -6.3 Mann Comparisons 80.9 76.2 77.2 70.8 69.6 -11.3 Atascocita 81.2 81.9 71.3 69.7 74.8 -6.4 Atascocita Comparisons 80.8 78.3 74.0 75.5 72.1 -8.6 YES Prep West 52.8 YES Prep West Comparisons 82.0 93.0 75.6 93.9 62.2 -19.8


20

SOURCE: Who is Teaching in Texas, TEA Recent research has found that teacher turnover has a negative effect on student achievement regardless of the teacher “effectiveness” of the teachers leaving the school. Turnover affects achievement in two primary ways. First, turnover—especially high levels of turnover—results in a loss of information about students. When a teacher leaves the school, the detailed information about the strengths and weaknesses of her students and the instructional strategies that best match each student are lost to the remaining teachers. Thus, the remaining teachers must discover on their own the students’ strengths, weaknesses, and learning preferences. As shown in Table 10, two of the A+ Scholar schools had very high rates of turnover. Indeed, both YES Prep West at 26% and Atascocita at almost 38% had very high levels of turnover. Interestingly, their comparison schools also had high levels of turnover as well. Both O’Donnell and Hoffman had moderate levels of turnover—about 20%—while Mann and Caraway had 16% and 17%, respectively. Caraway had a declining trend in teacher turnover, while Atascocita had a trend of increasing teacher turnover, but only for the last year. The other schools had stable teacher turnover trends or not enough data to have a trend over time. Three schools had lower turnover than comparison schools, while two schools had greater turnover than comparison schools. Only Mann had a turnover rate similar to the rate for comparison schools.

Table 10: Teacher Turnover Rate

School Spring of Academic Year Change: Name 06-07 07-08 08-09 09-10 10-11 2006 to 2010

Caraway Int 38.9 38.2 22.7 25.0 16.4 -22.5 Caraway Comparisons 28.4 23.1 21.7 15.7 23.4 -5.0 Hoffman MS 24.6 21.1 27.5 19.5 20.0 -4.6 Hoffman Comparisons 30.6 29.5 25.5 20.1 35.5 4.9 O'Donnell MS 19.9 36.3 17.6 18.4 19.8 -0.1 O'Donnell Comparisons 22.5 30.0 19.5 16.1 12.9 -9.6 Mann JHS 11.2 20.7 17.6 9.7 17.2 6.0 Mann Comparisons 19.5 18.0 19.7 14.4 16.7 -2.9 Atascocita 21.3 23.7 23.5 21.0 37.5 16.3 Atascocita Comparisons 15.6 23.7 21.7 14.9 26.6 10.9 YES Prep-West 25.9 YES Prep West Comparisons 19.7 37.5 20.7 25.4 33.3 13.7

SOURCE: Teacher Responsibility Files, TEA 4. Other Factors Affecting Implementation The school with the worst performance profile is clearly Atascocita. One reason for the low performance is that the A+ Coach assigned to the school was not present for the full length of time as in the other schools. The inclusion and the disappearance of the A+ Coach was likely to be highly disruptive to the teachers and students and could have had a negative impact on the reform effort at that school.


21

SECTION III: IMPLEMENTATION OF REFORM EFFORTS

Michael Fullan (2001) defines the “implementation dip” as “the inevitable bumpiness and difficulties encountered as people learn new behaviors and beliefs.” Fullan continues on to state:

The implementation dip is literally a dip in performance and confidence as one encounters an innovation that requires new skills and new understandings. All innovations worth their salt call upon people to question and in some respects to change their behavior and their beliefs — even in cases where innovations are pursued voluntarily. The Houston A+ Challenge Network reform effort is certainly a serious reform effort. It

not only requires educators to alter beliefs about the capacity of students in the middle to achieve at high levels, it also requires principals and teachers to behave in dramatically different ways. This change causes increased anxiety, fear, and frustration as well as feelings of being overwhelmed and a strong desire to revert back to beliefs and behaviors that are known and easier to the individual. One consequence of the potential negative emotional effects implementing a serious reform might have on teachers is a decrease in achievement or at least a dampening of the student achievement gains that would have occurred naturally without the reforms. Often, this “implementation dip” in student achievement causes a loss of commitment to the reform, which simply decreases achievement further. This can lead to a vicious cycle of lowering levels of commitment to the reform and concomitant failure to attain desired achievement outcomes. When the reform effort includes efforts to teach students knowledge and skills that are not directly assessed on the test instrument, the desire to de-commit from the reform is even more powerful. In the Challenge Network effort, teachers are being asked to work with students to build their capacity to be self-regulated learners. This is a difficult, but crucial task in preparing students to be truly college-ready. Yet accomplishing this task takes time away from directly preparing students to do well on the state-mandated assessment. As such, the reform effort may initially decrease student achievement, even though ultimately the students will be far better-prepared for high school and college than other students. Thus, it is imperative that the reform leaders ensure that everyone stays committed to the program and lower scores do not cause teachers to ignore the reform effort and revert back to previous behaviors or, worse, choose to teach to the test in order to increase student scores. Leadership is crucial in such a situation, since leadership support and reassurance are desperately needed during the trying times of implementing a significant reform effort.


22

SECTION IV: STUDENT OUTCOMES

This section describes the findings related to student outcomes. The student outcomes in this section include the following:

A. Commended Status; B. Vertical Scale Scores Comparison; and, C. Student Value-Added Growth

These three metrics provide information that is relevant to the goals and objectives of the project using different perspectives on student growth and achievement. The examination of commended status uses two measures. The first is just a simple examination of the percentage of students attaining commended status in the year prior to reform and after year one of the reform. The second perspective uses statistical analyses to determine whether students “in the middle” in reform schools were more or less likely than peers in comparison schools to achieve commended status. Vertical scale score comparison examines the academic growth of students “in the middle” in reform and comparison schools by the prior year’s vertical score. This provides information about the growth of students by lower- or higher-performing students in the middle. Finally, value-added growth analysis examines the overall student growth on TAKS relative to all schools in the state. A. COMMENDED STATUS

The analyses in this section were based on student-level data. Only students enrolled in the same district for two consecutive years and who had a valid score for the subject area in both years were included in the analyses.

1) Attainment of Commended Status

The focus of this reform effort is to elicit large enough academic growth by students “in the middle” so that the students attain commended status. Fuller (2010) found that students achieving commended status by the end of middle school had a very high probability of attaining college-ready status by the end of high school.

This section includes three sub-sections:

a) Change in percent of students commended for all students b) Change in percent of students commended for students “in the middle” by TAKS

scale score; and, c) Probability of a student “in the middle” attaining commended status.

a) Change in Percent of Students Commended for all Students

This section examines the attainment of commended status by A+ Scholars. The section relies on individual student-level data from the TEA. The analyses include all students identified as A+ Scholars, regardless of past performance. In other words, the group of students included those not meeting minimum standards, those meeting minimum standards but not commended status, and those meeting commended status.


23

Extreme caution must be taken when interpreting these results for several reasons. First, the analyses match students across years, thus only students remaining in the same school or district (depending on the grade configuration of the school). Thus, these numbers will be different than those appearing on the TEA website. Second, and more importantly, changes in commended status are driven largely by the distribution of scores around the cut point for commended status. If a relatively large percentage of students scored slightly below the commended status cutoff score, then increasing the percentage of students meeting commended status is much easier than if few students have scores slightly below the commended status cut point. Thus, changes in commended rates do not necessarily reflect the effect of the school on scores. However, the rates do provide valuable information. But the changes in the rates cannot be used to make conclusions about the effect of the school or teachers on student achievement or student growth.

As shown in Table 11, four of the six schools had increases in the percentage of students achieving commended status in 2010 and in 2011. Atascocita essentially remained at the same level of commended status in both 2010 and 2011 while Mann showed a slight decrease. Caraway A+ Scholars had the greatest increase at 7 percentage points, followed closely by Hoffman with 6.5 percentage points.

Table 11: Percentage of Students Attaining Commended Status in Reading in 2010 and 2011 A+ Scholars in A+ Challenge Network Schools

District Campus Academic Year Change

Name Name 2010 2011 10 to 11

ALDINE ISD Caraway A+ Scholars 15.0% 21.9% 7.0

ALDINE ISD Hoffman A+ Scholars 0.7% 7.2% 6.5

ALIEF ISD O'Donnell A+ Scholars 5.6% 9.7% 4.2

GOOSE CREEK ISD Mann A+ Scholars 20.3% 18.8% -1.4

HUMBLE ISD Atascocita A+ Scholars 19.0% 20.7% 1.7


YES PREPARATORY West 50.7% 55.6% 4.8

SOURCE: Student-level data; TEA

The remainder of this section compares each A+ Challenge network school to a set of comparison schools. At least one comparison school was selected from the same district as a way to control for any district effects that may have influenced any changes in achievement. While Caraway A+ Scholars had the greatest increase of all the Network schools, the A+ Scholars had about the same growth as other students in the school and smaller growth than most comparison schools.


24

Table 12: Percentage of Students Attaining Commended Status in Reading from 4th Grade in 2010 to 5th Grade in 2011 for Caraway A+ Scholars

District Campus Grade 4 Grade 5 Change

Name Name 2010 2011 10 to 11

ALDINE ISD Caraway Int 15.2% 21.8% 6.6

ALDINE ISD Caraway A+ Scholars 15.0% 21.9% 7.0

ALDINE ISD Parker Int 23.6% 21.9% -1.7

EVERMAN ISD Powell Int 19.1% 28.5% 9.4

ALIEF ISD Budewig Int 19.6% 32.0% 12.4

ALDINE ISD Houston Academy 22.2% 30.8% 8.6


Both Hoffman and O’Donnell had increases in the percentage of A+ Scholars achieving commended status, while students in comparison schools had a decrease in the percentage of students meeting commended status. Even with the cautions in interpretation described above, these data strongly suggest that A+ Scholars at both Hoffman and O’Donnell outperformed their peers in comparison schools.

Table 13: Percentage of Students Attaining Commended Status

in Mathematics from 6th Grade in 2010 to 7th Grade in 2011 for Hoffman and O’Donnell A+ Scholars


Name Name 2010 2011 10 to 11

ALDINE ISD Hoffman MS 22.6% 13.4% -9.2

ALDINE ISD Hoffman A+ Scholars 0.7% 7.2% 6.5

ALDINE ISD Teague MS 23.9% 21.2% -2.7

EVERMAN ISD Baxter MS 20.6% 9.2% -11.4

DUNCANVILLE ISD Reed MS 30.1% 14.6% -15.5

FORT BEND ISD McCauliffe MS 18.4% 7.7% -10.7

ALIEF ISD O'Donnell MS 30.3% 21.6% -8.7

ALIEF ISD O'Donnell A+ Scholars 5.6% 9.7% 4.2

ALIEF ISD Albright MS 39.3% 20.9% -18.4

ALIEF ISD Holub MS 28.6% 14.6% -14.1

MESQUITE ISD McDonald MS 34.8% 16.8% -18.0

RICHARDSON ISD Liberty JHS 51.6% 28.3% -23.3

PASADENA ISD Beverly Hills Int 28.1% 27.2% -0.9



25

As shown in Table 14, there was very little change in commended rates in mathematics for A+ Scholars at Horace Mann and Atascocita. For Mann, the slight decrease was actually better than most comparison schools, which saw larger decreases than at Mann. Only one comparison school had an increase of all comparison schools. Mann A+ Scholars also had a smaller decrease than the entire Mann Junior High School. This suggests that the A+ Scholars may have outperformed other Mann students.

Table 14: Percentage of Students Attaining Commended Status in Mathematics from 5th Grade in 2010 to 6th Grade in 2011 for Mann A+ Scholars


Name Name 2010 2011 10 to 11

GOOSE CREEK ISD Mann JHS 35.30% 30.60% -4.7

GOOSE CREEK ISD Mann A+ Scholars 20.30% 18.80% -1.4

GOOSE CREEK ISD Baytown JHS 39.30% 32.20% -7

SPRING BRANCH ISD Spring Oaks MS 18.30% 23.20% 5

DEL VALLE ISD Del Valle MS 30.70% 24.50% -6.2

EL PASO ISD Terrace Hills 33.50% 19.90% -13.6

GARLAND ISD Bussey MS 32.00% 17.30% -14.7


As shown in Table 15, A+ Scholars at Atascocita had about the same slight increase in the percentage of commended students in reading as the entire school. The slight increase for the Atascocita A+ Scholars was not much different than for comparison schools with the exception of Dulles Middle School in Fort Bend ISD.

In mathematics, the entire Atascocita school had a monumental decrease in the percentage of commended students—31 percentage points. This means that a large number of students had met the commended level in 5th grade, but did not meet the commended level in 6th grade. A+ Scholars, however, showed no change from 2010 to 2011. This makes sense, in that few A+ Scholars would have met the commended status in the 5th grade. All comparison schools had a large decrease in the percentage of commended students from 2010 to 2011.


26

Table 15: Percentage of Students Attaining Commended Status in Mathematics And Reading from 5th Grade in 2010 to 6th Grade in 2011 for Atascocita A+ Scholars


Name Name 2010 2011 10 to 11

MATHEMATICS

HUMBLE ISD Atascocita 38.1% 39.7% 1.6


HUMBLE ISD Timberwood MS 31.7% 36.9% 5.2

FRISCO ISD Stafford MS 54.0% 54.4% 0.5

KLEIN ISD Schimndewolf Int 39.9% 41.1% 1.1

FORT BEND ISD Dulles MS 45.7% 53.7% 7.9

FRISCO ISD Roach MS 52.3% 55.8% 3.5

READING

HUMBLE ISD Atascocita 53.7% 22.7% -31.0


HUMBLE ISD Timberwood MS 41.0% 30.1% -11.0

FRISCO ISD Stafford MS 64.0% 50.5% -13.6

KLEIN ISD Schimndewolf Int 52.4% 28.2% -24.2

FORT BEND ISD Dulles MS 61.6% 39.6% -22.1

FRISCO ISD Roach MS 62.2% 44.8% -17.4

SOURCE: Student-level data; TEA As shown in Table 16, YES Preparatory West had a similar increase in the percentage of

students attaining commended status from the 6th grade in 2010 to the 7th grade in 2011 as comparison schools. One comparison school—Energized for Excellence in Houston ISD—had a decrease in the percentage of commended students.

Table 16: Percentage of Students Attaining Commended Status in Reading

from 6h Grade in 2010 to 7th Grade in 2011 for YES Preparatory West A+ Scholars

District Campus Grade 6 Grade 7 Change Name Name 2010 2011 10 to 11

YES PREPARATORY West 50.7% 55.6% 4.8 YES PREPARATORY Bray 16.3% 20.6% 4.3 THE EAST AUSTIN The East Austin 12.0% 14.5% 2.4 KIPP Inc Intrepid 26.6% 30.4% 3.8 HOUSTON ISD Energized for Excellence 49.0% 45.2% -3.8



27

b) Change in Percent of Students Commended for Students “In the Middle”

This analysis focuses only on those students “in the middle”—passing the TAKS, but not attaining commended status. Because the starting point for the student markedly impacts the ability to attain commended status, the analysis is by scale score. Each score interval is the equivalent of one question answered correctly. So, working from the highest score at the end of the column, each move up the column marks a one question deviation from passing. Further, the A+ Scholars for each school are included with students from a large comparison group of schools from across the state. This group of comparison schools includes schools with the same grades served as the A+ Scholar schools, as well as with similar student characteristics in terms of students in poverty and student race/ethnicity.

As shown in Table 15, A+ Scholars at Caraway were generally as or more successful at attaining commended status at each scale score than for comparison schools. This was especially true at the top two scale scores of 667 and 691 where 22% of all Caraway students in the middle started.

Table 17: Students in the Middle Attaining Commended Status

by 2010 Mathematics Scale Score for Caraway A+ Scholars

Scale Caraway Comparisons Score N % N % 554 7 28.6% 345 3.2% 555 3 0.0% 36 0.0% 560 8 0.0% 399 3.0% 564 1 0.0% 30 0.0% 570 17 0.0% 451 5.3% 580 12 8.3% 521 6.1% 585 2 0.0% 45 0.0% 591 11 18.2% 607 10.7% 603 13 15.4% 732 14.6% 607 1 0.0% 68 1.5% 616 18 16.7% 763 18.3% 620 2 0.0% 88 2.3% 631 17 23.5% 873 22.6% 633 2 50.0% 97 3.1% 647 24 25.0% 976 30.7% 649 2 0.0% 108 4.6% 666 4 0.0% 95 3.2% 667 18 55.6% 992 38.9% 691 24 54.2% 1084 49.5% All 186 23.7% 8421 21.7%



28

As shown in Table 18, very few of the Atascocita A+ Scholars advance enough to attain commended status. Even of the 26 students just one question shy of commended in 2010 (scale score of 706), less than 8% attained commended status as compared to almost 33% in comparison schools. Of the 45 students scoring between two and five questions below commended in 2010 (scale scores between 657 and 691), only one achieved commended status.

In the same table is shown the results for Mann A+ Scholars. In contrast to Atascocita, the Mann A+ Scholars had greater attainment rates than their peers in comparison schools. This was especially true for students within four questions of commended status in 2010. While only 3 of 60 such students in Atascocita attained commended status, 11 of 40 (27.5%) of such students at Mann attained commended status. In comparison, only 15.7% of such students n the Mann comparison schools attained commended status.


by 2010 Mathematics Scale Score for Atascocita and Mann A+ Scholars

Scale Atascocita A+ Atas Comparisons Mann A+ Mann Comparisons

Score N % N % N % N %

603 5 0.0% 76 0.0% 5 0.0% 204 1.5%

606 6 0.0% 72 1.4% 3 0.0% 253 1.2%

613 4 0.0% 106 1.9% 4 0.0% 254 2.4%

621 8 0.0% 123 2.4% 6 0.0% 326 2.5%

629 3 0.0% 148 2.0% 7 0.0% 328 3.4%

638 6 0.0% 164 3.7% 7 0.0% 325 6.8%

647 4 0.0% 198 6.6% 10 20.0% 363 6.3%

657 11 0.0% 215 6.0% 9 11.1% 409 9.3%

667 9 11.1% 271 8.1% 13 23.1% 425 8.0%

678 17 0.0% 344 15.1% 8 25.0% 499 13.8%

691 8 0.0% 430 18.4% 5 40.0% 529 16.3%

706 26 7.7% 519 32.8% 14 28.6% 607 22.1%

Total 107 2.8% 2670 13.6% 91 15.4% 4547 9.7%



29

As shown in Table 19, the attainment rates for Atascocita A+ Scholars was substantially lower than the attainment rates for comparison schools. Indeed, Atascocita had a greater attainment rate for only one of the nine scale scores.


by 2010 Reading Scale Score for Atascocita A+ Scholars

Scale Atascocita All Comparisons

Score N % N %

620 6 0.0% 124 4.8%

630 4 0.0% 129 3.9%

640 3 0.0% 171 13.5%

651 11 9.1% 221 14.9%

662 11 36.4% 304 15.5%

675 7 0.0% 453 23.6%

689 12 25.0% 473 31.5%

706 10 20.0% 623 43.5%

725 9 33.3% 789 52.3%

Total 74 17.6% 3295 32.0%


As shown in Table 20, YES Prep West had slightly greater attainment rates for those students within two questions of commended status in 2010, but had a 0% attainment rate for all other students. Had even just a few of the 13 students in the 697 to 729 scale score range, YES Prep West would have had a significantly greater attainment rate than comparison schools. However, the final attainment rate was roughly the same for YES Prep West and comparison schools.


by 2010 Reading Scale Score for YES Preparatory West A+ Scholars

Scale YES Prep-West All Comparisons Score N % N % 644 1 0.0% 36 5.6% 645 2 0.0% 42 7.1% 655 2 50.0% 77 7.8% 665 4 25.0% 70 5.7% 676 5 0.0% 78 9.0% 688 6 33.3% 119 6.7% 701 9 33.3% 155 14.8% 715 7 0.0% 184 20.7% 731 10 70.0% 197 25.4% 751 17 52.9% 213 44.6%

Total 63 36.5% 1171 20.2%



30

As shown in Table 21, A+ Scholars at Hoffman had lower attainment rates of commended status than students in comparison schools. Indeed, only 1 of the 17 students within 3 questions of commended status in 2010 actually attained commended status in 2011.

A+ Scholars at O’Donnell had slightly greater attainment rates than students in comparison schools for most of the scale scores close to commended, although none of the four students with a 2010 scale score of 729 attained commended status. Further, none of the students starting at more than six questions away from commended status in 2010 advanced to commended status in 2011.

Table 21: Students in the Middle Attaining Commended Status by 2010 Mathematics Scale Score for Hoffman and O’Donnell + Scholars

Scale Hoffman A+ Comparisons O'Donnell A+ Comparisons

Score N % N % N % N %

637 4 0.0% 328 0.0% 8 0.0% 395 0.0%

641 11 0.0% 364 0.3% 5 0.0% 429 0.2%

648 8 0.0% 373 1.1% 11 0.0% 446 0.4%

656 4 0.0% 411 0.2% 16 0.0% 478 0.2%

663 10 0.0% 456 1.5% 10 0.0% 526 1.5%

671 15 0.0% 467 1.1% 7 14.3% 553 1.4%

679 10 20.0% 488 2.5% 10 10.0% 598 1.8%

688 17 5.9% 536 2.8% 19 5.3% 634 2.2%

697 6 0.0% 508 3.3% 17 5.9% 581 3.8%

707 12 25.0% 585 6.5% 0 671 6.6%

717 20 15.0% 669 5.7% 7 14.3% 760 6.4%

729 4 0.0% 691 10.6% 4 0.0% 782 10.0%

742 9 0.0% 709 12.0% 7 14.3% 814 11.9%

757 4 25.0% 685 19.3% 14 21.4% 819 18.9%

Total 134 7.5% 7270 5.9% 135 6.7% 8486 5.8%

SOURCE: Student-level data; TEA c) Probability of a Student “In the Middle” Attaining Commended Status

This section focused on only those students “in the middle.” In other words, students who

had passed the TAKS, but who had not yet attained commended status. As shown in the section above, the odds of attaining commended status are heavily dependent on how far the students was away from the commended cut score. This makes common sense, as a student who was only one question from attaining commended status has a far easier time making a large enough gain to achieve commended status than a student who was six or seven questions away.

The analyses in these sections relied on logistic regression analysis, which is a statistical approach commonly used when the outcome variable is binary. In this instance, the outcome variable is whether or not a student achieved commended status—yes or no. Several control variables were used in order to assess the independent effect of being enrolled in an A+ Scholar program. These variables included the student’s own prior test score, the student’s economically disadvantaged status, and various combinations of school characteristics such as the percentage


31

of economically disadvantaged students, percentage of Black students, percentage of bilingual students, percentage of mobile students, and percentage of special education students. In some cases, school size was also included. Further, either the number or percentage of a school’s students “in the middle” was included as well. The average scale score for all students in the school was also included. Finally, a binary variable identifying whether a student was an A+ Scholar was included. This variable would indicate whether the students in the A+ Scholar group at that particular school were more or less likely than students in comparison school to attain commended status.

The comparison schools in these analyses include the comparison schools listed throughout this report and also a larger group of schools from across the state with relatively similar student populations.

In the tables below, the statistical significance column indicates if the variable was statistically significantly associated with attaining commended status independent of the effect of all other variables included in the analysis, including the prior year’s test score. The accepted statistical significance level is .05, although a result between .05 and .10 suggests a weaker, but statistically significant relationship.

The effect of being in the A+ Scholar program is indicated by the odds ratio. A result greater than 1.00 indicates a positive relationship while a result less than 1.00 indicates a negative relationship. So, for example, if the result was 1.89, then being an A+ Scholar would be associated with being 89% more likely to attain commended status. Or, if the result was 3.56, a student in the A+ Scholar program would be 3.56 times more likely to attain commended status than other students. Alternatively, if the result was .65, then a student in the A+ Scholar program would be 35% less likely (1.00 – 0.65 = .35) than other students to attain commended status. A number of control variables were included in the analyses. These included:

Prior year’s TAKS score; Economically disadvantaged status of the school; Percentage of students in the school: economically disadvantaged; Percentage of students in the school: African-American Percentage of students in the school: enrolled in bilingual education; Percentage of students in the school: mobile; Percentage of students in the school: special education; Percentage of students in the school: passing previous year, but not commended; and, Average scale score for all students in the school in the prior year.

These variables were included in order to control for other factors that might explain why a student attained commended status other than the reform efforts being implemented in the school. After the inclusion of the independent/control variables, the last variable included in the model was a binary variable indicating if the student was an A+ Scholar in the reform effort. The result for this variable would indicate the effect—positive, negative, or neutral—of the reform effort on the likelihood of the student attaining commended status. Table 22 includes the overall results of the seven separate regression analyses. The column entitled “stat sig” indicates the p-value—or level of statistical significance—of the result. Any value equal to or less than .05 indicates that the finding was statistically significant at the generally accepted p < .05 level. The odds ratio provides the magnitude of the effect. Results


32

greater than zero indicate that A+ Scholars were more likely than other students to attain commended status after controlling for other factors. For example, A+ Scholars in Caraway were 58.6% more likely to attain commended status than peers in comparison schools. For Mann, the result indicates that A+ Scholars were 10.7 times more likely to attain commended status than peers in comparison schools after controlling for other factors. When the odds ratio is less than zero, then students were less likely to attain commended status than peers in comparison schools after controlling for other factors. For example, A+ Scholars in Atascocita were 75.8% less likely than peers in comparison schools to attain commended status in mathematics, after controlling for other factors.

Table 22: Logistic Regression Results for Students in the Middle Attaining Commended Status for A+ Scholars

School Stat. Odds

Name Sig. Ratio

Caraway 0.036 1.586

Hoffman 0.277 0.656

Mann 0.000 10.747

Atascocita-Math 0.046 0.242

Atascocita-Reading 0.365 0.687

O'Donnell 0.321 1.462

YES Prep 0.050 4.658

SOURCE: Student-level data, TEA; Analysis: Ed Fuller A+ Scholars at Caraway were almost 59% more likely than students in comparison

school to attain commended status after controlling for prior scores, economically disadvantaged status, and school characteristics.

A+ Scholars in Atascocita were about 76% less likely than students in comparison school to attain commended status after controlling for prior scores, economically disadvantaged status, and school characteristics. This is consistent with the table from the previous section that showed very few A+ Scholars in Atascocita achieved even the nominal growth needed to attain commended status and that the students had a much lower attainment rate than students in comparison schools.

Mann A+ Scholars were more than 10 times more likely than students in comparison schools to attain commended status after controlling for prior scores, economically disadvantaged status, and school characteristics. Again, this result is consistent with the table in the previous section that revealed Mann A+ Scholars had greater commended attainment rates across the different scale score levels.

There was not a statistically significant result for A+ Scholars in Atascocita with respect to attaining commended status in reading. However, the effect shown in math was negative, which was consistent with the results from the previous section.

There were no statistically significant results for A+ Scholars in either Hoffman or O’Donnell. If the effects had been significant, the effect for Hoffman would have been negative and the effect for O’Donnell would have been positive. Finally, students at YES Prep West were 4.6 times more likely than students in


33

comparison schools to achieve commended status. However, the finding was statistically significant with p = .05. The inclusion of other variables in the model move result in the finding not being statistically significant. While this result is not necessarily robust, it does strongly suggest that YES Prep West students were more likely than students in other schools to attain commended status. B. Vertical Scale Score Progress This section examines the vertical scale score growth of students “in the middle” from 2010 to 2011 for each group of A+ Scholars, students from a set of comparison schools, and students from all other schools. Students “in the middle” is defined as the set of students who passed, but did not attain commended status on the previous year’s test. So, the tables below present the vertical scale score growth from one year to the next for students who passed the previous year’s test, but did not attain commended status. The set of comparison schools were selected from schools across the state that had similar demographics as the target school. In this analysis, the comparison group is larger than the four or five schools in previous analyses. A larger group of schools was selected for the binary regression analysis from the previous section and the same group of schools was applied to this analysis. All other schools include all other schools in the state other than the target school and setoff comparison schools. Finally, only students who met the following requirements were included in the analysis:

had a valid score in both years; took the English version of the TAKS both years; and, took either the TAKS-regular or TAKS-accommodated tests in both years.

As shown in Table 23, most of the Caraway A+ Scholars scale score groups had score

changes that were equal to or greater than the score changes in comparison schools. Only three of the eleven groupings had score changes below the average for comparison schools and these three groups only accounted for only 29% of the students.

Table 23: Vertical Scale Score Growth in Reading from 4th Grade (2010) to 5th Grade (2011) for

Caraway A+ Scholars, Comparison Schools, and All Other Schools

Scale Caraway A+ Comparison Schools All Other Schools Score N Change N Change N Change

554 7 115.6 414 70.8 6491 84.4 560 8 50.6 484 72.3 7635 85.2 570 17 56.5 539 71.5 8947 85.2 580 12 92.0 635 70.9 10283 83.2 591 11 75.7 720 74.1 12128 81.0 603 13 72.2 886 70.6 14039 78.7 616 18 65.7 925 66.7 16265 77.9 631 17 76.4 1053 63.3 18494 75.3 647 24 34.3 1174 64.0 21266 71.5 667 18 70.9 1191 55.5 24597 66.6 691 24 51.8 1280 48.8 27311 59.7

All Scores 169 64.3 9301 64.1 167459 73.8 SOURCE: Student-Level data, TEA; Analysis: Ed Fuller


34

As shown in Table 24, most of the Atascocita A+ Scholars scale score groups in reading had score changes lower than the scale score changes for comparison schools. Indeed, six of the nine groups had score changes below that of comparison schools. Only one scale score group—that with a scale score of 662—had score changes greater than comparison schools. Table 24: Vertical Scale Score Growth in Reading from 5th Grade (2010) to 6th Grade (2011) for

Atascocita A+ Scholars, Comparison Schools, and All Other Schools

Scale Atascocita A+ Comparison Schools All Other Schools Score N Change N Change N Change

620 6 11.2 161 47.3 8127 40.1 630 4 46.8 173 44.5 9773 39.1 640 3 55.3 233 47.3 11506 38.6 651 11 34.7 303 47.8 13990 39.3 662 11 55.5 416 44.6 16554 39.0 675 8 -6.6 588 48.5 19485 38.9 689 12 25.7 614 44.9 22594 38.6 706 10 5.1 792 50.8 25343 39.3 725 9 23.0 1001 51.0 28442 38.8

Total 74 23.8 4294 48.0 158168 38.0

SOURCE: Student-Level data, TEA; Analysis: Ed Fuller

As shown in Table 25, most of the Atascocita A+ Scholars scale score groups in mathematics had score changes lower than the scale score changes for comparison schools. Indeed, 11 of the 12 groups had score changes below that of comparison schools. Only one scale score group—that with a scale score of 667—had score changes greater than comparison schools. These results detail why the A+ Scholars had lower growth and a lower probability of attaining commended status. Table 25: Vertical Scale Score Growth in Mathematics from 5th Grade (2010) to 6th Grade (2011)

for Atascocita A+ Scholars, Comparison Schools, and All Other Schools

Scale Atascocita A+ Comparison Schools All Other Schools Score N Change N Change N Change

603 6 -13.5 105 37.2 5822 37.3 606 6 1.7 101 35.2 6463 39.1 613 4 22.0 147 34.7 7076 38.6 621 8 -4.5 175 36.0 8161 36.6 629 3 10.3 179 29.7 8829 35.5 638 6 -14.2 221 29.5 9907 32.6 647 4 -49.5 263 35.7 10813 33.1 657 11 -6.2 293 26.6 11950 31.3 667 9 46.4 350 25.6 13357 31.6 678 17 -3.5 438 31.5 15217 31.4 691 7 -24.0 544 28.6 16773 30.5 706 26 -15.5 653 34.7 18807 28.8

Total 107 -5.1 3470 31.3 133176 32.8



35

As shown in Table 26, all but one of the scale score groups for Horace Mann A+ Scholars had score changes greater than the scale score changes for comparison schools. Eleven of the 12 scale score groups, in fact, had score changes equal to or greater than that of comparison schools. Only one scale score group—that with a scale score of 638—had score changes lower than comparison schools.

Table 26: Vertical Scale Score Growth in Reading from 5th Grade (2010) to 6th Grade (2011) for

Mann A+ Scholars, Comparison Schools, and All Other Schools

Scale Mann A+ Comparison Schools All Other Schools Score N Change N Change N Change

603 5 50.4 256 36.8 5680 37.2 606 3 40.3 300 35.6 6272 39.1 613 4 43.5 319 31.8 6908 38.8 621 6 27.8 393 34.3 7951 36.6 629 7 43.1 399 28.7 8608 35.7 638 7 17.6 407 33.5 9725 32.5 647 9 58.4 454 30.7 10620 33.1 657 8 36.9 494 26.1 11759 31.3 667 13 64.5 513 26.9 13201 31.6 678 7 46.3 603 22.7 15077 31.7 691 5 55.8 624 22.5 16706 30.7 706 14 39.4 707 19.6 18795 29.2

Total 88 44.9 5469 27.6 131304 32.9


As shown in Table 27, there were mixed results for the A+ Scholars from Hoffmann. Four of the scale score groups had greater score changes than at comparison schools, five groups had similar score changes, and five groups had lower score changes. Students at the lower end of achievement tended to have lower gains than students with greater scores. Students in the middle ranges tend to have the greatest score changes.


36

Table 27: Vertical Scale Score Growth in Mathematics from 6th Grade (2010) to 7th Grade (2011) for Hoffman A+ Scholars, Comparison Schools, and All Other Schools

Scale Hoffman A+ Comparison Schools All Other Schools Score N Change N Change N Change 637 4 14.8 386 34.7 5231 39.5 641 8 15.4 431 31.3 5816 41.1 648 8 52.3 440 33.0 6451 39.9 656 4 15.3 458 29.2 6764 36.9 663 10 23.3 515 28.8 7377 36.2 671 15 23.7 560 25.0 8051 34.2 679 10 52.8 560 24.2 8720 32.5 688 17 41.4 616 20.5 9473 30.8 697 6 4.7 589 20.3 9835 29.1 707 12 28.7 672 18.1 10659 27.5 717 20 21.0 767 17.1 11362 25.9 729 4 -28.5 776 13.5 12522 25.1 742 9 -2.2 818 6.5 13376 23.1 757 4 8.5 780 5.8 14628 21.3

All Scores 131 24.2 8368 20.0 130266 29.8


As shown in Table 28, all of the O’Donnell A+ Scholar scale score groups had score changes equal to or greater than the scale score changes for comparison schools. Students at the top end of the score distribution (scale score > 688) did particularly well. Table 28: Vertical Scale Score Growth in Mathematics from 6th Grade (2010) to 7th Grade (2011)

for O’Donnell A+ Scholars, Comparison Schools, and All Other Schools

Scale O’Donnell A+ Comparison Schools All Other Schools Score N Change N Change N Change 637 8 31.3 442 35.2 5389 39.1 641 5 41.4 471 29.7 5960 40.7 648 11 32.0 495 32.2 6552 39.7 656 16 33.4 533 28.4 6871 36.7 663 10 37.0 586 30.3 7469 35.7 671 7 54.1 612 24.9 8160 33.9 679 10 35.2 660 23.0 8742 32.4 688 19 22.2 722 19.1 9485 30.8 697 17 27.2 656 19.4 9891 29.1 707 0 na 743 16.1 10662 27.3 717 7 24.9 839 15.5 11336 25.9 729 4 46.8 879 11.4 12498 25.2 742 7 26.7 923 6.3 13295 23.2 757 14 8.2 930 5.9 14488 21.3

All Scores 135 29.6 9491 19.1 130804 29.7



37

As shown in Table 29, all of the YES Prep West A+ Scholar scale score groups had score changes equal to or greater than the scale score changes for comparison schools. The few students at the bottom end of the score distribution (scale score < 676)) did particularly well. Table 29: Vertical Scale Score Growth in Reading from 6th Grade (2010) to 7th Grade (2011) for

YES Preparatory West A+ Scholars, Comparison Schools, and All Other Schools

Scale YES Prep West Comparison Schools All Other Schools Score N Change N Change N Change

644 1 111.0 36 67.7 7626 43.9 645 2 75.0 42 65.3 9126 50.6 655 2 78.5 77 75.2 10726 50.5 665 4 89.0 70 63.0 12481 47.5 676 5 73.0 78 67.9 14582 47.8 688 6 79.0 119 47.8 16981 45.6 701 9 92.1 155 52.0 18969 45.2 715 7 41.7 184 50.9 21700 45.8 731 10 87.5 197 45.2 24700 46.3 751 17 61.6 213 52.3 27476 43.9

Total 63 73.9 1171 54.5 164368 46.3

SOURCE: Student-Level data, TEA; Analysis: Ed Fuller C. Value-Added Growth The value-added growth component of the evaluation focused on all A+ Scholar students, not just students “in the middle.” The results for students in the middle were not significantly different than the results for all students. This analysis was included because students can attain commended status without making particularly large growth. Indeed, a student one question below the cut score for commended status could answer one additional question correctly and move from “not commended” to commended, while a student scoring 10 questions below the commended cut score point could answer an additional nine questions correctly but still be designated as not attaining commended status. In short, this analysis examines whether all A+ Scholars exhibited greater than expected growth. The results are reported using standardized residuals from an ordinary least-squares regression analysis. Standardized residuals communicate whether the students scored lower than, equal to, or greater than regression analysis predicted them to score. The standardized residuals are similar to z-scores, in that positive results indicate greater than expected growth and negative numbers indicate lower than expected growth. Scores close to zero indicate growth that was close to the growth predicted by the regression model. Using regression analysis is important because it can be used to control for factors outside the control of school personnel. In this case, the regression analysis controlled for prior achievement and a host of school demographic variables. Only by using regression analysis can growth be attributed to the efforts of the school personnel rather than to the demographics of the students.


38

1) Methodology

The regression analysis focused only on those students who met the following criteria:

Enrolled consecutively for three years and advancing a grade level each year;

Had a valid score in the subject area of interest for all three years;

Had a score that was not masked to comply with FERPA for three consecutive years in the subject area of interest (only students in small schools or schools with very low numbers of economically disadvantaged or not economically disadvantaged students would have scores that were masked); and,

Enrolled in the same school district for three consecutive years.

The regression analysis controlled for the following variables: Students economically disadvantaged status in 2011;

Percentage of economically disadvantaged students in the school in 2011;

Percentage of mobile students in the school for 2011;

Percentage of bilingual students in the school in 2011;

Percentage of White students in the school for 2011;

Percentage of Asian students in the school for 2011;

Percentage of special education students;

Percentage of students enrolled three consecutive years that were enrolled in the school in 2011;

Percentage of students enrolled in the district three consecutive years for students enrolled in 2011;

School size in 2011;

Average school tests cores in 2010;

Prior test score in 2010;

Prior test score in 2009; and,

2010 test score above a selected cut point at which gains are difficult to make.

The results of the regression analysis are standardized residuals which describe how many standard deviations the schools’ results were above or below the regression line. In other words, the results provide information on (a) whether a school performed worse than, as well as, or better than predicted by the variables included in the regression analysis and (b) the degree to which the school performed worse than, as well as, or better than predicted. The results are reported in Z-scores.

A negative result suggests the school performed worse than predicted, a result close to 0 suggests the school performed about as well as predicted, and a positive score suggests the school performed better than predicted. The greater the distance from 0, the worse (negative Z-score) the school performed and the greater the distance from 0 (positive Z-score), the better the school performed.


39

2) Individual School Results

As shown in Table 30, there were three results in which A+ Scholars performed better than predicted, three results in which A+ Scholars performed worse than predicted, and one result in which A+ Scholars performed about as well as predicted.

Table 30: Standardized Residuals (in Z-Scores) from the Regression Analyses for All A+ Scholar Groups

District School Target Subject Standardized Name Name Grade Area Residual

Aldine ISD Caraway A+ Scholars 5 Reading -0.313 Goose Creek ISD Mann A+ Scholars 6 Math 0.144 Humble ISD Atascocita A+ Scholars 6 Math -0.448 Humble ISD Atascocita A+ Scholars 6 Reading -0.297 Aldine ISD Hoffman A+ Scholars 7 Math -0.071 Alief ISD O'Donnell A+ Scholars 7 Math 0.123 YES Prep West YES Prep A+ Scholars 7 Reading 0.263

A+ scholars in YES Prep, Mann (Humble ISD), and O’Donnell (Alief ISD) all performed

greater than predicted. Hoffman in Aldine ISD performed about as well as predicted. Caraway (Aldine ISD) and Atascocita (Humble ISD) performed worse than predicted. Note that two of the three negative results were from Atascocita. Thus, in terms of schools, there were three positive results, one neutral result, and two negative results. 3) A+ Scholar School and Comparison School Results:

The results for each school and subject area as well as the results for the comparison schools for each school are shown in the Appendix. Note that a different set of comparison schools will have to be selected for YES Prep, since only one of the comparison schools had enough data to generate results.

In 5th grade reading, Caraway (Aldine ISD) A+ Scholars performed worse than predicted

and under-performed compared to students in three of the four comparison schools.

In 6th grade mathematics, Horace Mann (Goose Creek ISD) A+ Scholars performed better than predicted, but performed better than students in only two of the five comparison schools.

In 6th grade mathematics, Atascocita (Humble ISD) A+ Scholars performed far worse

than predicted and under-performed students in all but one of the five comparison schools.

In 6th grade reading, Atascocita (Humble ISD) A+ Scholars performed worse than

predicted and under-performed students in all but one of the five comparison schools.


40

In 7th grade math, Hoffman (Aldine ISD) A+ Scholars performed about as predicted and performed better than students in two comparison schools, about the same as students in one comparison school, and worse than students in the other comparison school.

In 7th grade mathematics, O’Donnell (Alief ISD) A+ Scholars performed better than

predicted and outperformed four of the five comparison schools.

In 7th grade reading, YES Prep West (YES Prep Public Schools) A+ Scholars scored better than predicted and outperformed students in all of the comparison schools.

4) MAP© Test Scores Two schools also participated in an additional assessment—the Measurement of Academic Progress (MAP©)—produced by the Northwest Evaluation Association (NWEA). Having a second test available is important to ensure that the students are actually learning the material rather than simply learning to do well on a particular test. Indeed, since TAKS is high-stakes for schools and often teachers and principals as well, many schools resort to “teaching to the test” which results in the test scores being inaccurate indicators of what students know and can do. A second test that is low-stakes for all involved serves as a check on the accuracy of the scores from the TAKS. Unfortunately, the MAP© test was not taken by all schools. Only two schools had students take the test twice during the 2010-11 school year so that gains could be calculated. But even in these two schools, the “pre-test” was taken in November, so the gains reflect only gains over 6 to 7 months rather than the full 9 months. TAKS, on the other hand, has an entire calendar year between tests. Thus, the MAP© test and TAKS are not directly comparable. However, the MAP© test still serves a very useful purposes and provides excellent information about A+ Scholar students. In the analysis of MAP© test, students are placed into two groups with respect to growth (above and below average) and two groups with respect to proficiency (above and below average). As shown in Table 33, 35.2% of the Mann A+ Scholars had greater than average growth. Most of these students also had greater than average proficiency. In other words, most of the students making above average growth were also above average in terms of proficiency. Almost 65% of students had below average growth. Remember, however, that this is a norm-referenced test and the Mann students only had 6 or 7 months to show growth and are being compared to students across the nation who had up to 10 months to show growth. Thus, it is to be expected that many Mann students would have below average growth. Of those students having below average growth, 50% were above proficient. So, one-half of students with below average growth were achieving proficient status. Overall, just over 60% of all students achieved proficiency. Thus, as a group, A+ Scholars scored above the national average on the MAP© test.


41

Table 31: MAP© Mathematics Results for Mann A+ Scholars

Student Proficiency

Growth Below Above

Below 33.8 31.0

Above 5.6 29.6

As shown in Table 32, almost 33% of the O’Donnell A+ Scholars showed greater than average growth, with the majority of such students achieving greater than average proficiency as well. While about 67% had below average growth, the O’Donnell students had fewer months to show growth than the comparison students. With respect to proficiency, almost 56% achieved proficiency. Thus, as with the Mann A+ Scholars, the group of O’Donnell A+ Scholars performed better than average as compared to other students across the nation.

Table 32: MAP© Mathematics Results for O’Donnell A+ Scholars

Student Proficiency

Growth Below Above

Below 42.9 25.0

Above 8.3 23.8

5) Summary of Student Outcomes

While the results of the analysis of student outcomes are mixed, one must remember that two of the seven results are from Atascocita—a school that did not have an A+ Coach for the entire year of implementation, as did the other schools. If Atascocita is removed from the analysis because it did not fully implement the reform effort, then the results appear much more favorable with respect to the efficacy of the overall reform effort. Indeed, three of the five schools fully implementing the reform had A+ Scholar students that were statistically significantly more likely to achieve commended status than peers in comparison schools, and the other two schools had students that did not perform statistically significantly different than students in comparison schools.

In addition, three of the five schools fully implementing the reform had greater student vertical scale score growth than comparison schools, and the other two schools had vertical scale score growth that was about the same as in comparison schools.

Finally, three of the five schools fully implementing the reform had greater student value-added growth than predicted, while one school performed as predicted and the last school performed lower than predicted.

Thus, the evidence strongly suggests that the A+ Challenge Network reform effort, when fully implemented, had a positive effect on student outcomes.


42

SECTION V: STUDENT SURVEY RESULTS Surveys were administered at some schools during Year One of implementation, and three of the schools had high enough response rates to report results. Overall results are reported below; individual school results are included in the appendix.

The survey covered questions about the student’s teacher and A+ Coach, as well as the student’s perceptions of changes that occurred in the school during the academic year. Survey responses were generally in a six-point likert scale format ranging from “strongly disagree” to “strongly agree”. Percentages equal to or greater than 79.5% are highlighted in green, percentages between 60.0% and 79.4% are shaded in yellow, and percentages less than 60.0% are shaded in red. Even though a number of results are shaded in red, it is important to notice that a majority of students agreed or strongly agreed to each statement.

A. Perceptions of Coach and Coaching Efficacy

As shown in Table 33, very large percentages of students perceived that the coaches had

a profound positive experience on the students. Indeed, at least 78% of students believed the coach had helped them work harder, stay out of trouble, pay more attention in class, perform better in class, better understand the teacher, improved in both reading and math classes, and feel more confident about school. These were all very important outcomes of the reform effort and the coaches appeared to have been quite successful in working with individual students to achieve these outcomes.

Table 33: Survey Results for Effects of the Coach over Time

Strongly Somewhat Strongly Statement Disagree Disagree or Agree

or Disagree Agree or Agree Because of the A+ Coach . . . I work harder in class 2.1% 10.2% 87.7% I don't get into trouble as much 3.6% 9.9% 86.5% I pay attention in class more 3.3% 12.0% 84.7% I do better in class 3.6% 12.4% 83.9% During class, I understand the teacher better 3.1% 14.1% 82.9% I feel more confident about school 4.5% 14.4% 81.1% I have improved in reading or English class 3.1% 17.1% 79.8% I have improved in math class 4.6% 16.9% 78.5%

SOURCE: A+ Scholar Survey of Students, Spring 2011 As shown in Table 34, a majority of students agreed or strongly agreed that the A+ Coach was nice, cared for the students, and was liked by the students. In addition, students were also positive about the behaviors of the coaches. Specifically, students perceived that the coaches helped them to do better in class, talked to students about going to college, and expected students to work hard. As with the first set of questions, this set of questions revealed very positive relationships between coaches and students and positive behaviors of coaches with respect to students.


43

Table 34: Survey Results for Perceptions of the Coach

Strongly Somewhat Strongly

Statement Disagree Disagree or Agree or Disagree Agree or Agree The A+ Coach: helped me do better on tests 4.9% 7.2% 87.9% was nice and easy to work with 4.4% 19.2% 76.4% talked about me going to college 3.3% 24.6% 72.2% helped me do better in class 4.2% 25.4% 70.4% cared about me 8.8% 26.8% 64.4% explained things in an easy to understand way 10.6% 31.6% 57.8% was someone I I liked working with 15.8% 30.1% 54.2% expected me to work hard 14.6% 31.9% 53.5%

SOURCE: A+ Scholar Survey of Students, Spring 2011 B. Perceptions of Teacher As shown in Table 35, students were also generally positive about their teachers and teachers’ behaviors. Over 70% of students thought that the teachers made the class interesting and cared about the students. Over 60% of students believed that the teacher explains things in an easy-to-understand way and reminds students to think about the future.

Slightly less than 50% of students agreed or strongly agreed that the teacher talked to them about going to college and expected students to work hard. Indeed, 20% of students disagreed or strongly disagreed that teachers expected students to work hard.

Finally, and most importantly, almost 61% of students agreed or strongly agreed that the teacher had improved during the semester. While there is no other data to triangulate with this piece of information, the fact that students perceived improvements in teacher practice is an important outcome.

Table 35: Survey Results for Perceptions of Teacher


or Disagree Agree or Agree My teacher . . . makes the class interesting 3.9% 24.7% 71.4% cares about me 8.6% 20.3% 71.1% explains things in an easy-to-understand way 6.4% 28.9% 64.6% reminds me that I need to think about my future. 4.5% 31.2% 64.3% has become a better teacher during this semester 14.1% 25.3% 60.6% talks to me about going to college 13.4% 37.0% 49.6% expects me to work hard 20.6% 32.5% 47.0%

SOURCE: A+ Scholar Survey of Students, Spring 2011


44

C. Perceptions of School Administration

Finally, as shown in Table 36, slightly greater than 50% of students believed that the school administration had increased talk about the importance of going to college and exhibited greater concern about student achievement. Similarly, slightly greater than 50% of students believed that school had become safer for the individual respondent and her or his friends. Almost 75% of students believed that the school had become a “better place to be” over the course of the year.

Table 36: Survey Results


or Disagree Agree or Agree Compared to the beginning of the school year, the school administration: Talks more about the importance of going to college

13.2% 28.6% 58.2%

More concerned with how well I do in school 14.1% 29.9% 56.0% Compared to the beginning of the school year, the school seems to be: a better place for students to be 4.2% 20.8% 74.9% safer for me and my friends 15.8% 30.1% 54.2%


D. Perceptions of School

Students were also asked about how they view attending school. This question had five answers and, unfortunately, some students answered with multiple perceptions. Thus, the sum of responses exceeds 100%.

As shown in Table 37, 14% of respondents said they were bored at school and 9% said they disliked school. Alternatively, 16% reported they were excited about school. Finally, 45% believed that school was good while almost 38% said school was okay.

Table 37: Student Perceptions of School

Statement Percent

I am bored 14.0 I dislike school 9.2 I am excited about school 16.2 School is good 45.4 School is ok 37.5

SOURCE: A+ Scholar Survey of Students, Spring 2011 E. Perceptions of A+ Scholar Reform

Finally, students were asked about their perceptions of the A+ Scholar reform effort. A small number of students provided multiple responses, so the data was cleaned by including only each student’s most positive response.


45

About 40% of the students were unsure of the program or were unaware of the program. This suggests that more discussion needs to occur between the adults and students regarding the reform and the reform strategies. On the other hand, about 55% of students were either interested in the program or excited about the program. This suggests that students who know about the program are likely to view it in a favorable light.

Table 49: Student Perceptions of A+ Scholar Reform

Response Number Percent Don't know about it 83 23.6% Unsure 69 19.6% Don't like 6 1.7% Interested in it 97 27.6% Excited about it 97 27.6% Total 352 100.0%


Summary of Student Survey Results Overall, a large majority of students agreed or strongly agreed that the A+ reform has improved student behaviors. While this data does not have any other data to substantiate the perceptions of the students, the large percentage of students agreeing or strongly agreeing and the consistency in responses across the three schools strongly suggests that the reform model—in particular, the coaching aspect of the model—had very positive effects on student behavior. In addition, students also held favorable perceptions about the coaches, teachers, school administration, and the school overall. Thus, at least from the perspective of the students, the reform appears to have had a large and positive impact on students, teachers, and the culture of the school.


46

SECTION VI: CONCLUSION

Overall, the data collected and analyzed for the first year evaluation strongly suggests that the Houston A+ Challenge Network reform effort is having a positive effect on the likelihood of students attaining commended status, student growth as measured through value-added analyses, and student behavior as measured by student self-reports. The effects, however, are unevenly distributed across schools.

Additional efforts need to be undertaken to ensure that all of the reform components are fully implemented in each school, and additional information needs to be collected to determine the causes of some of the low performance so that the reform can be modified in these schools.

Further, the available data suggest that the coach is the critical element in the reform effort. Thus, great attention must be focused on choosing a high-quality coach and ensuring that the coach provides high-quality support to teachers and students.

In final conclusion, the reform efforts have had a strong Year One start despite not being fully operational until the middle of the academic year. These initial results are very promising, and the reform effort should be fully supported by school and district personnel as well as funders and policymakers

References Fullan, M. (2001). Leading in culture of change. San Francisco: Jossey-Bass.


47

APPENDIX

As shown in Table A-1, A+ Scholars at Caraway were almost 59% more likely than students in comparison school to attain commended status after controlling for prior scores, economically disadvantaged status, and school characteristics.

Table A-1: Logistic Regression Results for Students “In the Middle” Attaining Commended Status in Mathematics for Caraway A+ Scholars

Variable Stat. Odds

Name Sig. Ratio

Student: Scale Score 2010 0.000 1.023 Student: Eco disadv 0.000 0.722 School: % eco dis 0.000 0.980 School: % Black 0.540 1.001 School: % Bilingual 0.000 1.014 School: % Mobile 0.001 0.974 School: % Special Education 0.077 1.025 School: % in Middle 0.223 1.010 School: Avg Scale Score 0.667 0.999 Caraway 0.036 1.586

As shown in Table A-2, A+ Scholars in Atascocita were about 76% less likely than

students in comparison school to attain commended status after controlling for prior scores, economically disadvantaged status, and school characteristics. This is consistent with the table from the previous section that showed very few A+ Scholars at Atascocita achieved even the nominal growth needed to attain commended status and that the students had a much lower attainment rate than students in comparison schools.

Also shown in Table A-2, A+ Scholars at Horace Mann were more than 10 times more likely than students in comparison school to attain commended status after controlling for prior scores, economically disadvantaged status, and school characteristics. Again, this result is consistent with the table in the previous section that revealed A+ Scholars at Mann had greater commended attainment rates across the different scale score levels.


48

Table A-2: Logistic Regression Results for Students “In the Middle” Attaining Commended Status in Mathematics

for Atascocita and Mann A+ Scholars

Atascocita Mann Variable Stat. Odds Stat. Odds

Name Sig. Ratio Sig. Ratio Student: Scale Score 2010 0.000 1.037 0.000 1.026 Student: Eco disadv 0.000 0.585 0.003 0.663 School: % eco dis 0.027 1.025 0.000 0.949 School: % Black 0.000 1.132 0.003 0.971 School: % Bilingual 0.094 0.975 0.000 1.052 School: % Mobile 0.651 1.019 0.799 1.003 School: % Special Education 0.811 1.009 0.200 0.973 School Size 0.949 1.000 0.000 0.992 School Size Squared 0.994 1.000 0.000 1.000 School: # Students in the Middle 0.776 1.001 0.004 0.991 School Effect 0.046 0.242 0.000 10.747

As shown in Table A-3, there was not a statistically significant result for A+ Scholars in

Atascocita with respect to attaining commended status in reading. However, the effect shown in mathematics was negative, which was consistent with the results from the previous section.

Table A-3: Logistic Regression Results for Students “In the Middle” Attaining Commended Status in Reading for Atascocita A+ Scholars

Variable Stat. Odds

Name Sig. Ratio

Student: Scale Score 2010 0.000 1.026 Student: Eco disadv 0.000 0.722 School: % eco dis 0.878 1.002 School: % Black 0.815 1.002 School: % Bilingual 0.055 1.046 School: % Mobile 0.293 1.028 School: % Special Education 0.992 1.000 School Size 0.546 1.001 School Size Squared 0.793 1.000 School: # Students in the Middle 0.134 0.998 Atascocita 0.365 0.687

As shown in Table A-4, there were no statistically significant results for A+ Scholars at either Hoffman or O’Donnell. If the effects had been significant, the effect for Hoffman would have been negative and the effect for O’Donnell would have been positive.


49

Table A-4: Logistic Regression Results for Students “In the Middle” Attaining Commended Status in Reading for Hoffman and O’Donnell A+ Scholars

Hoffman O'Donnell

Variable Stat. Odds Stat. Odds Name Sig. Ratio Sig. Ratio

Student: Scale Score 2010 0.000 1.033 0.000 1.034 Student: Eco disadv 0.065 0.770 0.000 0.648 School: % eco dis 0.947 0.999 0.008 0.977 School: % Black 0.087 0.992 0.037 0.992 School: % Mobile 0.146 1.021 0.818 0.997 School: % Bilingual 0.919 1.001 0.139 1.011 School: % in Middle 0.000 1.105 0.000 1.046 School: Avg Scale Score 0.030 0.993 0.026 0.994 School Effect 0.277 0.656 0.321 1.462

As shown in Table A-5, students at YES Prep West were 4.6 times more likely than students at comparison schools to achieve commended status. However, the finding was statistically significant with p = .05. The inclusion of other variables in the model move result in the finding not being statistically significant. While this result is not necessarily robust, it does strongly suggest that YES Prep West students were more likely than students in other schools to attain commended status.

Table A-5: Logistic Regression Results for Students “In the Middle” Attaining Commended Status in Reading

for YES Prep West A+ Scholars

Variable Stat. Odds

Name Sig. Ratio

Scale Score Gr6 2010 0.000 1.028

Eco disadv 0.012 0.575

Sch: % eco dis 0.454 1.008

Sch: % Black 0.248 0.992

Sch: % Bilingual 0.413 0.987

Sch: % Mobile 0.045 1.029

Sch: % Special Educ 0.821 0.973

Sch: % in Middle 0.438 0.109

Sch: Avg Scale Score 0.529 0.992

YES Prep West 0.050 4.658


50

Table A-6: Student Survey Results for A+ Scholars at Caraway

Statement Strongly Somewhat Somewhat Strongly Disagree Disagree Disagree Agree Agree Agree

Because of the A+ Coach . . . I have improved in reading or

English class 2.5% 1.7% 4.2% 11.0% 43.2% 37.3%

I have improved in math class 0.9% 2.6% 1.7% 9.4% 41.9% 43.6% I feel more confident abt

school 3.3% 0.8% 2.5% 11.7% 47.5% 34.2%

I do better in class 4.2% 1.7% 2.5% 6.7% 45.4% 39.5% I pay attention in class more 4.2% 0.0% 0.0% 10.1% 42.0% 43.7%

I work harder in class 2.5% 0.0% 0.8% 7.5% 39.2% 50.0% During class, I understand the

teacher better 3.4% 0.8% 2.5% 11.8% 43.7% 37.8%

I don't get into trouble as much 2.5% 1.7% 2.5% 6.7% 37.5% 49.2% The A+ Coach . .

explained things in a way that makes them easy to

understand 7.8% 4.3% 12.2% 25.2% 27.0% 23.5%

expected me to work hard 3.4% 5.9% 8.5% 22.9% 39.0% 20.3% was nice and easy to work

with 0.8% 4.2% 3.3% 15.0% 39.2% 37.5%

was someone I liked working with

5.2% 6.9% 3.4% 19.8% 43.1% 21.6%

cared about me 3.5% 3.5% 4.4% 21.9% 39.5% 27.2% helped me do better in class 3.3% 4.2% 5.0% 17.5% 47.5% 22.5% helped me do better on tests 3.4% 1.7% 2.5% 3.4% 36.1% 52.9%

talked about me going to college

4.2% 0.8% 4.2% 19.3% 38.7% 32.8%

My Teacher . . reminds me that I need to

think about my future. 4.2% 0.0% 2.5% 30.0% 39.2% 24.2%

talks to me about going to college

5.1% 2.5% 3.4% 26.3% 47.5% 15.3%

explains things in a way that makes them easy to

understand 3.4% 6.0% 6.0% 20.5% 44.4% 19.7%

expects me to work hard 4.2% 16.1% 10.2% 26.3% 29.7% 13.6% makes the class interesting 2.5% 0.8% 2.5% 21.8% 37.8% 34.5%

cares about me 5.0% 3.4% 7.6% 11.8% 37.0% 35.3% has become a better teacher

during this semester 5.8% 12.5% 5.0% 22.5% 40.8% 13.3%

Compared to the beginning of the school year, the school seems to be a . . . better place for students to be 3.4% 3.4% 3.4% 16.1% 44.9% 28.8%

safer place for me and my friends

5.2% 6.9% 3.4% 19.8% 43.1% 21.6%

Compared to the beginning of the school year, the school seems to . . . Talk more about the

importance of going to college 4.2% 5.9% 10.1% 18.5% 39.5% 21.8%

More concerned with how well I do in school

6.0% 4.3% 11.1% 22.2% 38.5% 17.9%


51

Table A-7: Student Survey Results for A+ Scholars at Hoffman



English class 0.0% 0.0% 2.0% 18.6% 41.2% 38.2%


school 0.0% 1.0% 4.9% 16.5% 44.7% 33.0%



teacher better 0.0% 1.0% 5.0% 11.9% 44.6% 37.6%

I don't get into trouble as much 0.0% 1.0% 14.6% 43.7% 40.8% The A+ Coach . .

explained things in a way that makes them easy to understand

5.2% 3.1% 3.1% 25.0% 33.3% 30.2%

expected me to work hard 2.7% 11.7% 11.7% 19.8% 28.8% 25.2% was nice and easy to work with 2.7% 0.0% 1.8% 16.2% 44.1% 35.1%


7.2% 7.2% 9.0% 28.8% 33.3% 14.4%



0.0% 0.0% 2.9% 31.1% 35.0% 31.1%

My Teacher . . reminds me that I need to think

about my future. 1.0% 1.9% 4.9% 30.1% 35.9% 26.2%


7.1% 6.2% 8.8% 41.6% 30.1% 6.2%

explains things in a way that makes them easy to understand

2.7% 0.9% 9.9% 20.7% 36.9% 28.8%




Compared to the beginning of the school year, the school seems to be a . . .

better place for students to be 0.0% 0.0% 1.9% 26.2% 37.9% 34.0% safer place for me and my

friends 7.2% 7.2% 9.0% 28.8% 33.3% 14.4%




8.8% 7.1% 5.3% 25.7% 29.2% 23.9%


52

Table A-8: Student Survey Results for A+ Scholars at O’Donnell



English class 2.8% 1.9% 0.9% 15.0% 44.9% 34.6%


school 4.5% 3.6% 0.9% 7.3% 54.5% 29.1%



teacher better 2.8% 0.9% 0.0% 11.2% 48.6% 36.4%

I don't get into trouble as much 4.5% 0.9% 0.0% 6.3% 47.7% 40.5% The A+ Coach . .

explained things in a way that makes them easy to understand

5.5% 5.5% 11.9% 16.5% 37.6% 22.9%

expected me to work hard 8.8% 11.5% 10.6% 22.1% 24.8% 22.1% was nice and easy to work with 2.7% 2.7% 5.4% 16.1% 38.4% 34.8%


11.9% 9.2% 10.1% 19.3% 38.5% 11.0%



2.7% 1.8% 1.8% 15.2% 50.9% 27.7%

My Teacher . . reminds me that I need to think

about my future. 5.5% 0.9% 1.8% 24.5% 41.8% 25.5%


10.7% 8.9% 7.1% 24.1% 41.1% 8.0%

explains things in a way that makes them easy to understand

5.3% 0.9% 3.5% 26.3% 35.1% 28.9%




Compared to the beginning of the school year, the school seems to be a . . .

better place for students to be 3.6% 1.8% 0.0% 15.5% 45.5% 33.6% safer place for me and my

friends 11.9% 9.2% 10.1% 19.3% 38.5% 11.0%




9.3% 6.8% 4.2% 21.2% 33.1% 25.4%

Date post:	02-Mar-2016
Category:	Documents
Upload:	houston-a-challenge
View:	216 times
Download:	0 times

Year One Fuller Report

Documents