CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 1
California English Language Development Test
2010–11 Results and Implications for Title III
Accountability
CALIFORNIA DEPARTMENT OF EDUCATIONTom Torlakson, State Superintendent of Public Instruction
EL Accountability Leadership InstituteDecember 5, 2011
Presenters
Lily Roberts Administrator
California Department of Education (CDE)English Language Proficiency Assessments Office
Assessment Development and Administration Division
CALIFORNIA DEPARTMENT OF EDUCATIONTom Torlakson, State Superintendent of Public Instruction
Robert LinquantiProject Director and Senior Researcher
California Comprehensive Center – WestEdEnglish Learner Evaluation and Accountability Support
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 2
TOM TORLAKSONState Superintendent of Public Instruction
Presentation Overview• California English Language
Development Test (CELDT) p ( )Updates
• CELDT Results
• Title III Accountability Results and Implications
3
and Implications
• Key Dates for 2011–12
TOM TORLAKSONState Superintendent of Public Instruction
California English Language Development Test (CELDT)
Update• English Learner Proficiency
Assessments (ELPA) Office in Assessment Development and Administration Division (ADAD)
• CELDT Contract 2012–15CELDT Contract 2012 15
• Legislation Updates
4
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 3
TOM TORLAKSONState Superintendent of Public Instruction
Legislation Updates• Assembly Bill (AB) 124 (Fuentes) requires the
establishment of an advisory committee to align the English language development (ELD) standards to the Common Core State Standards in English-language arts
• Once ELD standards are revised, the following must occur:– Develop new CELDT Blueprints– Align the CELDT to the new standards– Conduct item writing, pilot testing, and rubric
development– Develop new test forms by grade/grade spans
aligned to the revised ELD standards– Conduct a standard setting for all grades 5
TOM TORLAKSONState Superintendent of Public Instruction
Legislative Updates (Cont.)
As required by Senate Bill (SB) 753 (Padilla) theAs required by Senate Bill (SB) 753 (Padilla), the CDE requested written documentation from the U.S. Department of Education whether two significant changes described in SB 753 are permitted by federal law.
1. Permit measurement of annual progress based p gon an expanded testing window. Response pending.
2. Permit “banking” of domain scores. Response –not allowed.
6
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 4
TOM TORLAKSONState Superintendent of Public Instruction
2006–11 Initial Assessment Summary
Results
PerformanceLevel 2006–07 2007–08 2008–09 2009–10 2010–11
Advanced 7.0 7.0 8.0 7.0 7.0Early
Advanced 12.0 13.0 13.0 12.0 12.0
Intermediate 20.0 20.0 20.0 21.0 21.0Early
Percentage of Students at Each Overall Performance Level
7
EarlyIntermediate 20.0 21.0 21.0 21.0 21.0
Beginning 41.0 39.0 39.0 38.0 40.0Number Tested 398,525 369,967 344,571 324,875 320,820
TOM TORLAKSONState Superintendent of Public Instruction
2006–11 Initial and Annual Assessment Summary Results
Performance Level 2006–07 2007–08 2008–09 2009–10 2010–11
Percentage of Students at Each Overall Performance Level
Level
Advanced 7.0 7.0 9.0 9.0 8.0
Early Advanced 22.0 25.0 26.0 27.0 26.0
Intermediate 35.0 35.0 34.0 35.0 36.0
Early Intermediate 18 0 18 0 17 0 16 0 16 0
8
Early Intermediate 18.0 18.0 17.0 16.0 16.0
Beginning 18.0 16.0 14.0 14.0 14.0
Number Tested 1,709,637 1,704,753 1,677,688 1,617,006 1,574,403
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 5
TOM TORLAKSONState Superintendent of Public Instruction
Percentage of ELs Who Met CELDT Criterion for Possible
Reclassification by Grade SpanGrade Span 2006–07 2007–08 2008–09 2009–10* 2010–11 Change From
Previous YearChange From
2006–07
K–2 20.0 23.6 26.6 28.0 27.6 -0.4 7.6
3–5 27.3 31.5 35.9 34.9 33.0 -1.9 5.7
6–8 37.4 42.4 44.8 47.0 45.2 -1.8 7.8
9–12 34.3 36.8 40.4 41.0 36.2 -4.8 1.9
9
All Grades** 29.1 32.8 36.2 36.7 34.5 -2.2 5.4
* Prior to 2009–10, the calculation of the CELDT Criterion for kindergarten and grade one (K–1) only includes listening and speaking scale scores.
** Weighted average across the grade spans within the school year.
TOM TORLAKSONState Superintendent of Public Instruction Percentage of ELs who Percentage of ELs
CELDT Criterion and Reclassification Results
School Year
Percentage of ELs who met the CELDT Criterion
for Possible Reclassification*
Reclassified as Fluent English Proficient (RFEP)
as Reported by the Language Census (R–30)**
2010–11 34.5 11.4
2009–10 36.7 11.6
2008 09 36 2 10 9
10
2008–09 36.2 10.9
2007–08 32.8 9.6
2006–07 29.1 9.2* CELDT performance is one of the four criteria for reclassification.** The percentage for ELs reclassified RFEP is calculated by dividing the number of reclassified
students by the prior year’s EL count then multiplying by 100.
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 6
TOM TORLAKSONState Superintendent of Public Instruction
Robert Linquanti
California Comprehensive Center at WestEd
11Title III Accountability 101
TOM TORLAKSONState Superintendent of Public Instruction
AMAOsThree AMAOs:
• AMAO 1 – Making annual progress in learning English
• AMAO 2 – Attaining English proficiency
12
proficiency
• AMAO 3 – Meeting Adequate Yearly Progress (AYP) for EL subgroup
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 7
TOM TORLAKSONState Superintendent of Public Instruction Previous CELDT Annual Growth
AMAO 1 Progress CriteriaPrevious CELDT
Overall LevelAnnual Growth
TargetBeginning Early Intermediate
Early Intermediate Intermediate
Intermediate Early Advanced
1.
2.
3.
13
Early Advanced or Advanced,and not English Proficient English Proficient
English Proficient
Maintain English Proficient Level
4.
5.
TOM TORLAKSONState Superintendent of Public Instruction
Targets for AMAO 1
row
th
AMAO 1 Targets
10
20
30
40
50
60
70
f Stu
dent
s to
Mee
t Ann
ual G
rTa
rget
s
0
10
Perc
ent o
f
Target 51 51.5 52 48.7 50.1 51.6 53.1 54.6 56 57.5 59
2003-04
2004-05
2005-06
2006-07
2007-08
2008-09
2009-10
2010-11
2011-12
2012-13
2013-14
14
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 8
TOM TORLAKSONState Superintendent of Public Instruction
AMAO 2Attaining English proficiency• Measured by the CELDT• Early Advanced or Advanced level
overall• Domains of listening, speaking,
reading, and writing are at Intermediate level or above*
15
Intermediate level or above
*For K–1, only listening and speaking need to be at Intermediate level or above
TOM TORLAKSONState Superintendent of Public Instruction
AMAO 2 Cohorts• Include all ELs• Two cohorts with different targets:• Two cohorts with different targets:
– ELs who have been in language instruction educational programs for less than 5 years
– ELs who have been in language instruction educational programs for
16
instruction educational programs for 5 years or more
• LEA must meet targets for both cohorts to meet AMAO 2
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 9
TOM TORLAKSONState Superintendent of Public Instruction
AMAO 2 Targets
17
TOM TORLAKSONState Superintendent of Public Instruction
AMAO 3 Meeting AYP for EL subgroup
• Measured by:– California Standards Tests (CSTs)California Standards Tests (CSTs)– California Modified Assessment (CMA)– California Alternate Performance
Assessment (CAPA)– California High School Exit Examination
• EL subgroup for AYP includes reclassified• EL subgroup for AYP includes reclassified fluent English proficient students until they score Proficient or above three times on the CST-English-language arts (ELA) or CMA-ELA.
18
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 10
TOM TORLAKSONState Superintendent of Public Instruction
AMAO 3 (Cont.)
• ELAParticipation ratePercent Proficient or above
• MathematicsParticipation ratePercent Proficient or above
19
TOM TORLAKSONState Superintendent of Public Instruction
AMAO 3: AYP Targets in ELA, Math (Unified School Districts)
2020
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 11
TOM TORLAKSONState Superintendent of Public Instruction
Annual CELDT Scores 2006–2010
7 8 9 9 8100
18 17 15 15 15
39 39 37 38 39
25 28 30 31 29
20
40
60
80Pe
rcen
t of E
Ls Advanced
Early AdvIntermed.Early IntermBeg
21
10 9 8 7 8
17 15 15 15
02006 2007 2008 2009 2010
AMAOs (2007–08)
AMAOs (2008–09)
AMAOs (2009–10)
AMAOs (2010–11)
TOM TORLAKSONState Superintendent of Public Instruction
Title III Accountability Results: Percent ELs Making AMAO 1 Growth by Prior CELDT Level
90100 2006-07 2007-08 2008-09 2009-10 2010-11
58 60
37
45
72
64 62
40
51
74
64 63
43
57
76
6663
41
54
76
60
37
50
72
64
304050607080
% E
Ls m
eetin
g
22
01020
Beginning Early Int. Intermediate EA/A not EP Eng. Prof.(1.) (2.) (3.) (4.) (5.)
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 12
TOM TORLAKSONState Superintendent of Public Instruction
Title III Accountability Results: Percent of LEAs Meeting AMAOs
100
Includes new AMAO 2 cohort definition & targets
73 7469
8277
38
81
37
78
63
22
5145
29
82
2030405060708090
100
Perc
ent o
f LEA
s
23
01020
Met AMAO 1 Met AMAO 2 Met AMAO 32006 2007 2008 2009 2010
Title III Accountability 101
TOM TORLAKSONState Superintendent of Public Instruction
Title III Accountability Results:Percent of LEAs meeting AMAOs
(Cont.)
7790
100
68
54
74
31
77
32
59
15
39
141020304050607080
Perc
ent o
f LEA
s
24
010
Met AMAOs 1 & 2 Met All AMAOs (1, 2, 3)2006 2007 2008 2009 2010
CELDT CELDT & academic tests
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 13
TOM TORLAKSONState Superintendent of Public Instruction
Title III Accountability Results 2010–11: AMAO Year Status
AMAO Year # LEAs/Consortia % LEAs/Consortia
N/A (Met All AMAOs) 95 14%
Year 1 97 14%
Year 2 125 19%
Year 3 67 10%
Year 4 109 16%
25
Year 4+ 182 27%
LEAs and consortia N=673Source: September 23, 2011 update
TOM TORLAKSONState Superintendent of Public Instruction
Why a 27 %-point drop in LEAs meeting AMAO 1?
Some contributors:• Target increased
– 53.1% 54.6%– Accounts for 8
%-pts of drop• 09-10 RFEP rate
• % ELs progressing fell 2-4% at each ELD level
• %ELs progressing fell in each grade
increased to 11.6%– Removes high-
performers from 10-11 CELDT testing
g– 4% fewer overall– 7+% fewer in 9-12– 3-5% fewer in K,
3-5 26
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 14
TOM TORLAKSONState Superintendent of Public Instruction
Accurate, Meaningful Accountability Requires a Stable EL Subgroup
Students Who Began as ELs (English Proficient Learners)
Met state reclasscriteria Current ELs
1-5 yrs in L t ELShould meet
grade-level proficiency;
Count in subgroup to
hold accountable for equity &
access
ystate
schools
Should meet annual ELP &
academic progress goals
Long-term ELs6 or more yrs. in
state Schools% should decrease
annually
Working Group on ELL Policy, 2011 (www.Ellpolicy.org)
TOM TORLAKSONState Superintendent of Public Instruction
CA ELs and Former ELs (RFEP) by Grade
200000
Ever-EL
K-5 6-12 Total
EL 86% 43% 1,435,734
80000
100000
120000
140000
160000
180000
RFEP
EL
RFEP 14% 57% 794,467
Source: CDE DataQuest, 2010-11 CELDT & CST
0
20000
40000
60000
K 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 15
TOM TORLAKSONState Superintendent of Public Instruction
90%
100%
District ELs, Long-Term ELs and Former ELs (RFEP) by grade
20%
30%
40%
50%
60%
70%
80%
RFEPL-T ELEL
K 1 2 3 4 5 6 7 8 9 10 11 12RFEP 0% 0% 13% 25% 38% 46% 57% 61% 64% 58% 62% 69% 70%L-T EL 0% 0% 0% 0% 0% 36% 32% 29% 28% 32% 26% 21% 24%EL 100% 100% 87% 75% 62% 18% 11% 10% 8% 10% 12% 10% 7%
0%
10%
L-T EL:EL Ratio: 2:1 3:1 3:1 3.5:1 3:1 2:1 2:1 3.5:1
Long-term EL: 6 or more years in LEA
45%
18%
37%
TOM TORLAKSONState Superintendent of Public Instruction
The English Learner Subgroup Self-Assessment (ELSSA): Tool for analyzing district-level EL performance patterns
3030
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 16
TOM TORLAKSONState Superintendent of Public Instruction
Which ELs are making progress annually?
3131
Questions to consider:• At which CELDT level do most ELs score? • How well did ELs progress from one level to next?• Which groups of ELs merit further review?
TOM TORLAKSONState Superintendent of Public Instruction
How well are English Learners progressing over time?
32
Questions to consider:• How are ELs in US schools for 4 or fewer years performing?• How are ELs in US schools for 5 or more years performing?• Which groups of ELs merit further review?
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 17
TOM TORLAKSONState Superintendent of Public Instruction
Entry Grade PercentK 66.0%
1 9.2%
2 5.3%
5th Grade (2009-10) Cohort Analysis
66.0% of 5th graders havebeen in LEA since Kindergarten
ELs in LAUSD Since Kindergarten, andPercentage that Reclassified by 5th Grade
Sample Total EL Cohort Analysis: In LEA Since Kindergarten
3 5.2%
4 5.3%
5 5.5%
Unk 3.9%
Total 100.0%
Of those 5th graders who began their
40%
60%
80%
100%
Percentage that Reclassified by 5th Grade
16.4%
31.0%
28.4%
EL
RFEP
EO
IFEP16.4%
31.0%EO
IFEP
Lang Class PercentEO 31.0%
IFEP 16.4%
EL 24.2%
RFEP 28.4%
Total 100.0%
g geducation in LEA: In 2009-10…
52.6% of the 5th graders began in K As EL (EL + RFEP)
2004-05 2009-100%
20% 52.6%
24.2%
Kinder 5th Grade
EL
TOM TORLAKSONState Superintendent of Public Instruction
5th grade students started Kindergarten as ELs
80%
100%
54 0%RFEP
ELs in LEA Since Kinder
ELsBegan in K(2004-05)
0%
20%
40%
60%
46.0%EL
54.0%
5th Grade (2009-10)
46.0% of the students that began as ELs (EL + RFEP) were still ELs by the end of 5th grade
54.0% of the students that began as ELs (EL + RFEP)were RFEPs by 5th grade
By 5th Grade:
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 18
TOM TORLAKSONState Superintendent of Public Instruction
Of current EL 5th graders who began as Kindergartners in LEA:
48% scored Early Adv. or Advancedon 2009-10 CELDT Overall
5th Grade ELs in LEA Since Kinder
CELDT 2010 Percent
Advanced 9 0% on 2009 10 CELDT Overall
45% scored Intermediate on 2009-10 CELDT Overall
7.0% scored Beg. or Early Intermediate on 2009-10 CELDT Overall
Of ELs at Intermediate on the CELDT Overall, % below Intermediate vs Intermediate or above on each domain were:
Advanced 9.0%
Early Advanced 38.8%
Intermediate 45.1%
Early Intermed. 6.1%
Beginning .9%
Total
% below Intermediate vs. Intermediate or above on each domain were:
CELDT 2010 Below Intermediate
Intermediate and Above
Listening 13.9% 86.1%
Speaking 1.1% 98.9%
Reading 35.9% 64.1%
Writing 33.1% 66.9%
TOM TORLAKSONState Superintendent of Public Instruction
Of Current EL 5th graders who began as Kindergartners in LEA:
5th Grade ELs in LEA Since Kinder
CST 2010 Cluster Scores(Grade 5)
Percent correct
Word Analysis 48%
CST 2010 (Grade 5)
Percent
Advanced .6%
Proficient 5.6%
Basic 39.7%
Below Basic 34.3%
F B l B i 19 8% Word Analysis 48%
Reading Comprehension 40%
Lit Response & Analysis 42%
Writing Strategies 46%
Writing Conventions 52%
Far Below Basic 19.8%
Total
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 19
TOM TORLAKSONState Superintendent of Public Instruction
Some Implications• Monitoring EL progress (AMAO 1)
more important than ever– ELs since Kindergarten– Progress much harder in grades 6–12– Target domains below intermediate
L t EL ill b hi hli ht d i
37
• Long-term ELs will be highlighted in AMAO 2 cohort ≥ 5 years – Reclassification will affect results of
this AMAO 2 cohort (possibly AMAO1)37
TOM TORLAKSONState Superintendent of Public Instruction
Some Implications (cont.)
• Must foster academic language development and academic literacy– Increase oral academic language use
with engaging, grade-level group work, practices and routines
38
, p– ELD and all sheltered content
instruction should foster discipline-specific literacy skills
38
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 20
TOM TORLAKSONState Superintendent of Public Instruction
Key Dates• January 2012 — New CELDT Contract starts• February 2012 — Release of AMAO 3 and 2011–12
CELDT Data Review Module (DRM) begins• March 2012 — DRM ends• April 2012 — 2012–13 CELDT STOT workshops
begin • May 2012 — Release of the 2011–12 CELDT AA
results
3939
• June 2012 — Posting of the preliminary 2011–12 Title III Accountability Reports and Information Guide
• July 1, 2012 — 2012–13 CELDT AA and IA windows open
TOM TORLAKSONState Superintendent of Public Instruction
Contact InformationELPA Office
Assessment Development and Administration DivisionPhone: 916–319–0784E-mail: [email protected] page: http://www.cde.ca.gov/ta/tg/el/
Title III AccountabilityAnalysis Measurement and AccountabilityAnalysis, Measurement, and Accountability Reporting DivisionPhone: 916-323-3071E-mail: [email protected] page: http://www.cde.ca.gov/ta/ac/t3/
40
CELDT Results & Title III Accountability Roberts (CDE) & Linquanti (CACC-WestEd)
CDE EL Accountability InstituteSan Diego CA 12/4/2011 21
Questions?
CALIFORNIA DEPARTMENT OF EDUCATIONTom Torlakson, State Superintendent of Public Instruction 41