Performance of All Student Subgroups in Arkansas: Moving

Beyond Achievement Gaps

Dr. Gary RitterJune 12, 2014

Bridging the Gap Symposium

2

Objectives of this Talk

1. Introduce statistics on achievement gaps

• Frame panel discussions

• Focus on effective strategies for educating disadvantaged students

2. Provide some thoughts on moving forward

3. Introduce OEP … I’ll start here

•AR Education Reports

• Policy Briefs

• Report Cards

• Newsletters

• Data Resources

www.officeforeducationpolicy.org/3

4

Accessing Publications through the OEPRefer to menu bar at the top left of the OEP homepage. www.officeforeducationpolicy.org

OEPublications leads to options such as Report Cards, Education Reports and Policy Briefs.

Remember to sign up for our weekly e-mail, OEP Web Links (OWL), to get updated on current education news across the state and nation.

Please e-mail oep@uark.edu to sign up.Also, sign up for the OEP Blog at www.officeforedpolicy.com to receive alerts when the latest OEP Blog posts are published.

5

Refer to menu bar at the top left of the OEP homepage. http://www.officeforeducationpolicy.org/

Click on Arkansas School Data

Accessing Data Resources through the OEP

Arkansas School Data has multiple databases at both school and district levels.

6

Overview

1. Quick Lake View Background

2. Defining and Measuring the Gaps: Simple Task?

3. Description of OEP Report

4. Achievement Gaps in ARa. What is the Magnitude?

b. How have gaps changed over time?

c. How does AR compare to other states?

d. Gaps in other outcomes

5. Moving Forward?

7

AR School Funding Litigation• AR Constitution requires that the state provide a “general,

suitable and efficient system of free public schools.”– 1979 – Alma School District and 10 others file lawsuit over school

funding. Alma v. Dupree (1983) ruling: AR Supreme Court strikes down public school funding formula

– 1984 – AR raises sales tax by 1¢ to increase funding– 1992 – Lake View School District sues state over disparities in

school funding– 1994 – Pulaski County Chancery Court Judge rules in favor of Lake

View, finding finance system violates education adequacy & equity provisions of state Constitution

– 2001 – After Lake View case sent back, PC Judge again declares system inadequate and inequitable

– Nov. 2002 – AR Supreme Court upholds Lake View; mandates changes by January 2004 (new funding formula)

8

Post-Lake View: New Funding Formula

• Act 69, Act 108, Act 57 – funding changes– $5,400 per student in base funding; – Supplementary funding for specialized needs:

• $3,250 per student - alternative learning programs ;• $195 per student - English language learner; • Low income students

– $480 per student in districts (frl < 70%) – $960 per student in districts (70% < frl < 90%) – $1440 per student in districts (frl > 90%)

• $50 per student for professional development

• Key point progressive funding for disadvantaged students

9

Disadvantaged Student Changes?• More disadvantaged districts receiving more– Lowest wealth districts increased by 22%

(High Wealth 10%) – measured by property value– Highest poverty districts increased by 23%

(Low Poverty 19%) – measured by percent FRL

• Targeted funds went to:– Districts with more NSLA students– Districts with more non-white students– Districts with more students struggling on tests– Districts with declining enrollments

10

Per Pupil Expenditures by Poverty Rates

2003-04 2006-07 2012-13$6,000

$7,000

$8,000

$9,000

$10,000

$11,000

$12,000

$13,000

$6,144

$10,330

$6,455

$10,843

$6,807

$11,395

$6,728

$12,154

$7,290

$12,526

Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5

Least Poor Districts

Poorest Districts

Poorest Districts

have $12,526 per

pupil

Least poor have just

over $10,000

11

Per Pupil Expenditures by Percentage of Minority Students

2003-04 2006-07 2012-13$6,000

$7,000

$8,000

$9,000

$10,000

$11,000

$12,000

$13,000

$6,316

$11,469

$6,158

$11,094

$6,154

$11,248

$6,565

$11,654

$7,014

$11,773

Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5

Fewest Minority Students

Most Minority Students

Districts with most

minority students

have $11,773 per pupil.

Highest expenditures

since 2003

12

Defining and Measuring Achievement Gaps

• What?– Test scores– High School graduation rates– College completion

• For whom?– Arkansas: 36% minority students (21% black, 10% Hispanic)– Arkansas: 61% low-income students (frl)

• Compared to what?– Other states?– Gaps from prior years?

13

Measuring Achievement Gaps: Complications?

• Mind the GAP (only?): Often achievement gaps are presented without the context of performance and growth over time– For instance, a media outlet might report that “The GAP

between performance of rich and poor students across Arkansas decreased by 3 percentage points last year.” • Is this positive?• What if the gap was lower simply because the higher

performing group decreased its performance?

14

Measuring Achievement Gaps• The top method

is the most favorable way for a gap to decrease

• But we might also see a similar gap decrease from the bottom example (not nearly as positive)

15

(Gap) Looks can be deceiving …

2000 20100

20

40

60

80

100

7565

50 45

Rich Students Poor Students

2000 20100

20

40

60

80

100

75

96

50

70

Rich Students Poor Students

• School at right saw GAP decrease by 5 pts (from 25 pts in 2000 to only 20 pt gap in 2010)

• School below saw GAP increase by 1 pt. (from 25 pts to 26 pts. )

Both schools started at the same spot in the year 2000 … which school made better progress for all student groups?

16

Different metrics different answers?

• Perhaps better to consider performance and growth of each group over time rather than simply look at gaps.

• More confusion scale score averages versus proficiency levels

• Just viewing % proficient ignores all movement above and below 250.

Below Basic Basic Proficient Advanced

Scale Scores 150 200 235 250 275 325 350 400

Key Takeaway: It’s not straightforward or obvious!

17

• Recently released report for the Commission

• Examines performance of AR students over time

• NAEP and benchmark

• Subgroups and achievement gap analyses

• Visit and download at OEP!

18

Achievement Gaps in Arkansas:How big are the gaps and are they growing?

19

Data

• Student Subgroups– By race (black, Hispanic, white)– By income (free lunch eligible v. not)

• Student Achievement Measured by:– NAEP: “The Nation’s Report Card” (2000 thru 2013)

• Grade 4 and 8• Math and literacy

– Arkansas Benchmark Tests (2005-06 thru 2012-13)• Grade 3- 8 • Math and literacy

20

Gap Report: Many Sights to See

21

NAEP Results for Arkansas, 2013

• Scale Score Differences (proficiency %)

• Differences are substantial (AR SD ~= 80 points)• Black-white gap is largest, then income gap• Math grade 8 gaps largest

Black-White Gap Hispanic-White Gap

Income Gap

Math Grade 4 -23 (-30%) -12 (-16%) -19 (-28%)Math Grade 8 -31 (-25%) -22 (-14%) -25 (-27%)Literacy Grade 4 -26 (-23%) -15 (-14%) -24 (-24%)Literacy Grade 8 -25 (-25%) -13 (-16%) -22 (-24%)

22

NAEP Results for Arkansas, change over time from 2000 to 2013

• As mentioned earlier … scale scores and proficiency percentages can tell different stories

23

NAEP: Over Time• Arkansas growth over time – in scale scores (% proficient in parentheses)

– Math: 2000 to 2013– Literacy: 2002 to 2013

• Greater gains in math than literacy; not obvious whether disadvantaged students are gaining more

*2005 to 2013

Black Hispanic* White Low-Income

Non-Low-Income

Math Grade 4 +29 (+15%) -1 (0%) +21 (+29%) +27 (+23%) +23 (+35%)Math Grade 8 +28 (+7%) +8 (+1%) +18 (+15%) +18 (+9%) +23 (+25%)Literacy Grade 4 +8 (+7%) +7 (+8%) +4 (+5%) +7 (+5%) +6 (+8%)Literacy Grade 8 +6 (+6%) +6 (+8%) +2 (+3%) +3 (+2%) +7 (+9%)

24

NAEP: Grade 4 Math – Scale Scores

• Black students grew more than white;

• Gap decreased over time between black and white students

25

NAEP: Grade 4 Math - % Proficient• Gap between

black and white students increased when measured by % proficiency

• - but decreased when measured by scale scores

• Note: opposing answers both correct!

• Nevertheless, gaps are large.

26

NAEP Math - Summary• In 4th and 8th grade, Arkansas’ subgroups experienced positive

growth in performance between 2000 and 2013 on proficiency levels and scale score points. – Black–white achievement gap grew in respect to the proficiency

percentages• +14 percentage points in 4th grade• +8 percentage points in 8th grade

– However, in respect to scale score points, the gap between black and white students slightly decreased • -8 scale score points in 4th grade• -10 scale score points in 8th grade

• Thus, black students experienced growth, but not large enough to cross NAEP threshold

27

NAEP Math – Low Income Students

• 4th grade and 8th grade: the gap between non-low-income and low-income students widened, in terms of proficiency percentages, from 2000 to 2013– However, in 4th grade, the average scale score for low-

income students (+27 scale score points) increased more than did the average scale scores for non-low-income students (+23 scale score points).

28

NAEP Literacy

• 4th grade and 8th grade: racial achievement gaps slightly decreased over time

• 4th and 8th grade: the gap between non-low-income and low-income students widened,

29

ArkansasBenchmark Data

** allows for greater digging

30

Benchmark Math

• Gap decreased over time

• Ceiling effects related to proficiency rates

31

Benchmark Literacy• Gap

decreased over time

• Ceiling effects

32

Racial Gaps: Percentile Rankings

33

Income Gaps: Percentile Rankings

34

Summary of Gaps in AR

• Regardless of exam:– Black-white gap is the largest, followed by income gap,

followed by Hispanic-white gap – Each of these gaps appear to have decreased over

time (slightly)

• Because proficiency percentages can mask gains at the upper and lower-end, it may be best to consider these questions in terms of performance of the average student on the entire distribution.

35

Summary of Test Performance

Average black student score 33rd %ile

Average low-income student score 41st %ile

Average Hispanic student score 46th %ile

Average white student score 56th %ile

Average non-poor student score 65th %ile

36

How does AR performance and AR achievement gaps compare to US and surrounding states?

37

NAEP Assessment 2013

Arkansas compared to surrounding states

Black Hispanic White Low-Income

Non-Low-Income

Math Grade 4 +1 +3 0 +3 +1Math Grade 8 -5 -1 -1 -1 +1Literacy Grade 4 -1 +3 0 +3 0Literacy Grade 8 -2 -2 -2 0 +2

38

NAEP – from 2000 to 2013Arkansas Scale score growth over time (compared to surrounding states)

– Math: 2000 to 2013– Literacy: 2002 to 2013

– In nearly every AR students have grown more than peers (stronger relative gains in math)

*2005 to 2013

Black Hispanic* White Low-Income

Non-Low-Income

Math Grade 4 +29 (+17) -1 (+1) +21 (+16) +27 (+18) +23 (+17)Math Grade 8 +28 (+19) +8 (+11) +18 (+12) +18 (+17) +23 (+14)Literacy Grade 4 +8 (+8) +7 (+9) +4 (+3) +7 (+4) +6 (+7)Literacy Grade 8 +6 (+4) +6 (+7) +2 (+2) +3 (+4) +7 (+3)

39

What About Other Measures?

2011-12 Arkansas

2011-12 US

Overall 84% 80%Black 78% 69%Hispanic 78% 73%White 87% 86%Low-income

79% 72%

High School Graduation Rates – Gaps?

40

What Next? 5 Implications of Gap Report

1. Technical / Statistical:

• These “gaps” don’t really matter• Don’t focus only on gaps• Gaps can be “closing” in unfavorable ways• Alternative measurement strategy could be

school grade = average of overall scores and subgroup scores– Include a “penalty” for gap and a “bonus” for growth

41

What Next? 5 Implications of Gap Report

2. Philosophical / Political:

• These gaps really matter• Therefore:– Don’t slay the messenger! We have to be able to discuss

these uncomfortable issues.– Yes … test scores are not everything, but they are

meaningful (do any of us believe that these scores are wrong – we’re actually serving all kids well?)

– College grad rates for poorest quartile = 11%; – 82% for kids from top quartile

42

What Next? 5 Implications of Gap Report

3. Political / Political:

• These gaps really matter• Therefore: don’t pick teams!– This matters too much for real students for us to dismiss

strategies based on who developed them.• Traditional educators should learn from charters• Republicans should accept Democrat ideas

– We should focus on the outcomes of the strategy, not the affiliation of the spokesman

– We should tone down the vitriol

43

What Next? 5 Implications of Gap Report

4. Evaluation:

• These gaps really matter• Therefore: it is imperative that we figure out what

works.• Innovate and Evaluate

– Innovate … because we clearly haven’t figured out what works yet

– Evaluate… because we shouldn’t just trust the claims of “we know it works” from well-meaning program staff

– PS: I mean really evaluate!

44

What Next? 5 Implications of Gap Report

5. Take Action Steps:

• These gaps really matter• Therefore: it is imperative that those of us in schools need to

understand the gaps in our schools• Find and scrutinize subgroup level data

– Larger districts… in house– Smaller districts… seek out OEP or others– Do this every year and share the information (silence will not help!)– Then – innovate, and check data again– Conclude with case that shows it can be done

45

• 2010 Report examining high-growth schools in Arkansas

46Arkansas Rogers Grace Hill

0102030405060708090

100

64

76 7566

80 82

2010 SAT-10 NCE

Read Math

2006 2007 2008 2009 20100

102030405060708090

100

59 63

7886

91

5247

63 6581

Benchmark

GH Math GH LitArk Math Ark Lit

Grace Hill Elementary

Arkansas Rogers Grace Hill0%

10%20%30%40%50%60%70%80%90%

100%

Student De-mographics

White BlackHispanic Other Minority

GH FRL: 87% Enroll: 461Ark FRL: 59%

Math Lit Math Lit Math Lit2008 2009 2010

0

10

20

30

40

50

60

70

80

90

100

81

61

90

67

94

87

76

66

85

70

90

78

Hispanic vs. White

Hispanic White

Arkansas

+5-5

Math Lit Math Lit Math Lit2008 2009 2010

0102030405060708090

100

6252

65 6173

6777 73

80 76 78 79

Hispanic White

+5

-3

+4+9

-15 -21 -15 -15 -5 -12

Outperformed all state subpopulations!

Questions?

http://www.officeforeducationpolicy.org/