Accountability: Maximizing Your Performance

Terry WyattExecutive Director, Tri-County SSA

June 2010

Terry Wyatt Executive Director, Tri-County SSA 120 S. Swenson Stamford, Texas 79553 325-773-3637 twyatt@tricty.esc14.net http://www.tricountyspecialed.net/ Updates to this presentation posted there. Free spreadsheet posted there. The example spreadsheets used in this

presentation are available there.

Contact Information

1) BRIEF review of accountability system demands

2) Orientation to free spreadsheet tool operation

3) Extended discussion regarding analysis of your data as it exists and develops over time using this tool.

Session Agenda

Individual ARDC design high quality, appropriate, and challenging instructional programs for each student.

Those programs are appropriately close to the mainstream setting.

Every student is well taught, personally committed, and supported by a home environment. As a result academic progress is made for each student, including passing an appropriately selected statewide assessment.

In a Perfect World . . .

Even though each ARDC acts individually, the resulting patterns of instructional setting, student eligibility characteristics, and statewide assessment participation do not show any tendencies toward data issues that the SEA monitors on behalf of USDE or the state itself.

The district has in place: 1) policies and procedures, 2) operating guidelines, and 3) staff training to develop and maintain the environment which makes the above possible.

Perfect World Continued . . .

Discussions in this setting and more importantly in your district setting need to recognize the critically important nature of the independent role the ARDC must be allowed to play to meet the individual needs of each particular student.

At the same time, the district can and should review data patterns and consider/plan for how the district can affect instruction to help students make academic progress.

Consequently

AEIS – Academic Excellence Indicator System

AYP – Adequate Yearly Progress PBMAS – Performance Based Monitoring &

Analysis System SPP – State Performance Plan Data Quality Measures

Accountability Review

In AEIS and AYP: Students must be enrolled on snapshot day

& Students must be enrolled on test day and

take the test in such a way that it results in the test being scored.

Performance Subgroup

AEIS Special Education All Students

Take Pass Take Pass

TAKS 4 3 184 147TAKS-Acc 5 3 5 3TAKS-M 8 7 8 7TAKS-Alt 3 3 3 3

20 16 200 160Pass % 80% 80%

AEIS Special Education All StudentsTake Pass Take Pass

TAKS 4 1 184 145TAKS-Acc 5 1 5 1TAKS-M 8 5 8 5TAKS-Alt 3 2 3 2

20 9 200 153Pass % 45% 77%

AEIS Special Education All StudentsTake Pass Take Pass

TAKS 1 1 181 145TAKS-Acc 1 1 1 1TAKS-M 15 14 15 14TAKS-Alt 3 3 3 3

20 19 200 163Pass % 95% 82%

AEIS does not measure the SE subgroup. All SE tests will be combined into the one

AEIS test measure. Schools may fall into the trap of thinking

that moving sets of students toward TAKS-M from TAKS and/or TAKS-Acc will tend to increase their over-all passing percentage.

AEIS – Academically Acceptable standard for math and science continues to rise.

All grades, all subjects measured

AEIS Analysis Issues

1% and 2% Caps still apply The higher your district’s percentage of

special education students, the fewer of them fit inside these caps.

Grades 3 – 8 & 10 Reading and Math Minimum size for all subgroups 50 and 10%

up to 200.

AYP

Special Education CAP AppliedTake Pass Take Pass

TAKS 40 30 40 30TAKS-Acc 50 30 50 30TAKS-M 80 70 80 40TAKS-Alt 30 30 30 20

200 160 200 120Pass % 80% 60%Pass Standard Reading 80%

Math 75%

Special Education CAP AppliedTake Pass Take Pass

TAKS 40 30 40 30TAKS-Acc 80 70 80 70TAKS-M 50 40 50 40TAKS-Alt 30 30 30 20

200 170 200 160Pass % 85% 80%Pass Standard Reading 80%

Math 75%

Test Participation Measures SE TAKS Performance by subject

PBMAS

PBMAS Break Points - Test ParticipationLevel = 0 Level = 1 Level = 2 Level = 3

From To

From To

From To From To

3 TAKS/TAKS-A Only 100 50 50 30 30 15 14.9 04 TAKS-M Only 0 20 20 35 35 54.9 55 100

5TAKS-Alt Participation Rate 0 10         10.1 100

New rates for 2009-10. To be applied on PBM reports for fall 2010.

TAKS-Alt above 10.1 will result in an PL-AR, meaning that the TEA will review to

determine any further necessary action.

AYP Special Education Subgroup Performance

AYP SE Subgroup Participation

State Performance Plan

Download the spreadsheet to your computer.

Save the spreadsheet on your computer. Copy the spreadsheet into a working folder

and start entering data. Save your work regularly. Develop a structure for saving your data.

(first run, by date, proposed changes, etc.)

Dimensional Analysis

Start at any time near the beginning of the year.

Instructions tab is worth reading District Summary TAB – Only enter district

name. AYP TAB – Enter district name Student Count TAB – Enter District Name &

Enter district wide count of # of students by grade. This is ALL STUDENTS, all campuses combined.

Entering Information

Enter the names of all special education students in the district who are in grade 3

In the campus column, put their campus number (3 digit)

To enter their test, you put the number in column D, E, F, G, & H. (As appropriate by grade.)

Grade Tab (G3)

1 – TAKS 2 – TAKS-Accommodated 3 – TAKS-M 4 – TAKS-Alt

You cannot enter data into columns I through M. You must enter the above numbers in columns D through H.

The spreadsheet posts the test name for you.

Test Selection Numbers

The spreadsheet automatically:◦ Evaluates the test pattern and highlights – blue for

TAKS-M only and yellow for TAKS/TAKS-Acc only.◦ Places a “1” in the appropriate column TAKS-M only

or TAKS/TAKS-Acc only based on the student’s test pattern.

◦ Counts how many special education student are in the grade based on the number of reading tests that are not blank.

◦ Counts how many TAKS-M takers you have by subject.

◦ Counts how many TAKS-Alt takers you have.

G3 Continued

You may track/chart the decision status of each student.

In other words, has the ARD committee met yet to determine the student’s testing?

Early in the year you will have to estimate what ARDC decisions will be. As time goes, ARDCs will have made testing decisions on more and more kids.

G3 Continued

This tab has reference information from the PBMAS system.

It has the standards which will be used in the fall 2010 PBMAS reports that will be coming.

This data is one of your main references for determining if your testing pattern is where you want/need it to be.

Break Points TAB

On this tab you enter your AEIS scores for all subjects.

You enter the percentage of students who mastered their test, including the use of TPM.

The spreadsheet handles the color coding. If a subgroup is below the minimum size

criteria, and not close, then you would not want to include that number.

Generally, if the subgroup is within 10 students, I include it. Or if you know it will be growing.

8-9 AIES TPM TAB

On this tab you enter your AEIS scores for all subjects.

You enter the percentage of students who mastered their test, using only the base scores.

The spreadsheet handles the color coding. If a subgroup is below the minimum size

criteria, and not close, then you would not want to include that number.

Generally, if the subgroup is within 10 students, I include it. Or if you know it will be growing.

8-9 AIES Base TAB

On this tab you enter your AYP scores for reading and math.

You enter the percentage of students who mastered their test, using only the base scores.

The spreadsheet handles the color coding. If a subgroup is below the minimum size

criteria, and not close, then you would not want to include that number.

Generally, if the subgroup is within 10 students, I include it. Or if you know it will be growing.

8-9 AYP TAB

Data Analysis IssuesI have my data entered, now what does it

mean?

What do my AEIS scores from last year tell me?

Are there any subjects which:◦ Are critically low and must make improvement?◦ Are comfortably high and could stand some SE

TAKS-Acc testers? Keep in mind, what is the critical level of

performance for the district.

Analysis Issues

Our district is committed to efforts to achieve recognized status for the 2010-2011 school year.

In making recognized happen, we insist that the following also happen:◦ We meet AYP.◦ We get no 3’s on PBMAS.◦ We avoid 2’s on PBMAS.

Example Critical Level of Performance Statement

Based on district goals, what are the critical performance areas?◦ Science?◦ Math?◦ PBM TAKS-Acc participation?

Based on past district performance and current district goals, where does the district have some breathing room?◦ Social Studies?◦ Writing?

Critical Performance Areas

Based on our current plan for testing:◦ Does the SE subgroup meet the minimum size

criteria?◦ How many TAKS-M testers do we have and how

does that match up to 2%, 3%, 3.5% counts?◦ How many TAKS-Alt testers do we have and how

does that match up to the 1% count?◦ ROW 34 – What does the spreadsheet tell us with

respect to how our testing matches up with the plan?

Analysis Continued – AYP TAB

Rows 35 & 36 Based on your student population, these rows give

you some numeric reference points:◦ 1% - This is the cap on the number of TAKS-Alt passers

you can have.◦ 2% - This is the cap on the number of TAKS-M passers you

can have.◦ 3% - The is the combined cap on the maximum number of

TAKS-M & TAKS-Alt passers you can have.◦ 3.5% - This is a TW planning number for TAKS-M

participation in reading and in math.◦ 4.5% - This is a TW planning number for the maximum

number of TAKS-M & TAKS-Alt testers to have.

Analysis Continued – AYP TAB

Rows 37 – 47 Column C is a list of how many testers you have

by test and by subject. Column F is very valuable. Cell F38 (45) subtracts

your TAKS-M count from the 3.5% count. The result tells you this:◦ A positive number is space you still have.◦ A negative number means you have exceeded the TW

planning number. Cell F39 (46) subtracts your (Alt+M) count from

the 4.5% number. Meanings are the same as above.

Analysis Continued – AYP TAB

Row 16 tells you how many students your district has testing in all AEIS grades combined.

Row 17 tells you how many students your district has testing in all AYP grades combined.

Things to remember about this tab:◦ This count can stay current with your enrollment

all the way to test day. (Participation counts are all based only on test day.)

Analysis – Student Count Tab

First: Rows (3 – 11, columns B – F) this section details how many TAKS-M testers you have in each grade by subject.

Column G tells how many students are TAKS/TAKS-Acc only students you have by grade.

Column I tells how many TAKS-M only students you have by grade.

Column K tells how many TAKS-Alt students you have by grade.

Column L gives a special education student count by grade.

Column M tells how many students you have marked as having ARD decisions complete.

Column N gives a TAKS-M planning limit by grade.

District Summary TAB

Cell G:16 – Tells the percentage of students who are TAKS/TAKS-Acc only. This percentage matches the PBMAS calculation.

Cell G:18 – Tells the PBM Performance Level (PL) you would receive based on your percentage in G:16.

Cell I:17 – Tells the percentage of students whoare TAKS-M only. This percentage matches the PBMAS calculation.

Cell I:18 – Tells the PBM PL you would receive.

District Summary TAB

Row 19 – 22 A brief bit of data on AYP Numbers Row 19 is your student count from grades 3

– 8 & 10. Row 20 is your 2% cap number. (Max TAKS-

M passers who count.) Row 21 is your count of projected exceeders

by subject. Row 22 tells you if your have unused cap

space.

District Summary TAB

Row 24 – Tells the percent of your tested students who are taking TAKS-Alt. (Column B & K are the same information repeated.)

Row 25 – reports either a “0” or a “3”. PBMAS will return a PL Review if your district is over 10% on TAKS-Alt. The spreadsheet shows this as a 3.

District Summary TAB

Row 31 – This is a way that TEA will look at your data. How do the numbers compare between subjects?

District Summary TAB

Once you have this spreadsheet filled out with real data for your district:◦ Hold a meeting of campus principals and other

district personnel, including some teachers.◦ Review what your data looks like.◦ Consider: Have you met your data goals? Are you

positioned properly to meet your performance goals?

◦ Project the spreadsheet on the wall. Review grade by grade. What changes could be made to improve data?

Review Meetings

Accountability Changes

Supplement to Presentation

Accountability Change Sources March 23, 2010 TAA Letter: Student Assessment

program Reporting Updates for 2010 April 19, 2010 TAA Letter: New testing and

accelerated instruction requirements under the student success initiative

July 8, 2010 TAA: Update on the Implementation of New Assessment and Accountability provisions

July 1, 2010 TAA: Expedited Waiver Related to the Timeline for New Accelerated Instruction Requirements under the Student Success Initiative

March 23 In grades 3-8 for reading and math, the

vertical scale score will be the only score provided on CSR.

6th and 8th grade reading had their vertical scale score for “Met Standard” increased.

2010 Scale Scores by Subject

Reading Math Writing Science Soc. Std.3 483 5004 554 554 21005 620 603 21006 644* 6377 670 670 21008 700* 700 2100 21009 2100 2100

10 2100 2100 2100 210011 2100 2100 2100 2100

March 23 Continued: TPM TPM reported for first time in grade 8 science TPM calculation will include, for the first time,

the prior year score for the student. A projection to TAKS Commended

Performance will be included for all subjects for which TPM is calculated.

Changes have been made to when TPM will be available for students who take TAKS in both English and Spanish across various subjects.

March 23 Continued: TPM TPM will be calculated for TAKS-M Student must take TAKS-M in all applicable

subjects in order for TPM to be calculated.

March 23: TAKS-Alt TAKS-Alt growth measure will produce:

◦ Stage change Change in student’s score from prior year.

◦ On Track Student is “on track” to meet standard at next high

stakes grade based on the stage change the student made from the prior year to the current year.

March 23: EOCEnglish Math Science Soc. Std.English I Alg. I Biology

WorldGeography

English II Geometry Chemistry U. S. History

English III Alg. II Physics World History

Operational with results provided in 2010 similar to 2009

No Results provided from 2010 field tests

Algebra I is the only subject for which passing standards have been set.

April 19, 2010 TAA SSI Grade 3 no longer subject to SSI Grade 5 & 8 Reading and Math are SSI If students do not pass both reading and math,

then student may advance to the next grade only if:◦ Completes all accelerated instruction assigned by GPC◦ GPC determines that the student is likely to perform at

grade level the next year. GPC must consider: 1) recommendations from

student’s teacher, 2) students grades, 3) student’s TAKS scores, and 4) other relevant information

Accelerated Instruction Requirements SSI students must receive AI after each

failure, including summer administration. SSI students who are promoted to next

grade without passing TAKS, must receive AI during the subsequent school year.

Summer AI must be completed before the school year starts for the student to be promoted. (School can seek waiver).

July 8, 2010: Commissioner Scott New tests start in 2011-12. (STAAR) New Accountability System 2012-13 Transition Plan published December 2010 STAAR will test at a greater depth. STAAR will test at a higher level of

complexity. STAAR will have more rigorous items. STAAR will have higher performance

expectations. (passing standards)

Commissioner Scott 3-8 reading and math will be linked from

grade to grade to the performance expectations for Algebra II and English III EOC exams.

5 & 8 science will focus on promoting readiness for EOC science tests.

Writing will require two writing tasks. In most cases, tests will contain more items.

Commissioner Scott Performance standards will be set using

empirical data linking performance across grades to promote readiness for EOC.

Empirical data linking STAAR results to nationally administered assessments will be used by standard setting group.

Performance standards will be reviewed every three years to assure that they maintain a high level of rigor.

Commissioner Scott: 2011 and Beyond The new accountability system (2012-13)

will look very different than the current system.

Defining characteristic will be emphasis on college and career ready performance on STAAR.

Recognized and Exemplary will emphasize higher levels of student performance rather than higher percentages of proficient students. (80% & 30% Commended)

Commissioner Scott: 2010-11 Consistent with new direction of upcoming

accountability system:◦ Recognized and Exemplary will emphasize higher

levels of student performance.◦ Again, likely 80% plus 30% CP for Rec and 90%

plus 30% CP for Exem. (This is not in the letter, but rather what we hear)

Commissioner: TPM 2011 and Beyond Notes that student performance inmproved

in every subject and for every student group.

Notes that this achievement is being overshadowed by criticism of TPM and its impact on accountability.

Consequently, Changes Considered for TPM Suspension of TPM Only use TPM in accountability for districts that

choose to use it. Modify the use of TPM in accountability by using

additional safeguards such as:◦ Applying performance floors,◦ Count each TPM passer as a fraction,◦ Prohibit TPM from being used for any

subgroup/subject in subsequent years,◦ Limiting the number of measures for which TPM can

be used in any one year,◦ Limiting which ratings categories can use TPM.