Post on 06-Jun-2018
transcript
STEMMING ThE TIDE OF
INTERGENERATIONALILLITERACY:
EXECUTIVE SUMMARY
Principal Investigator: James E. Barr, Ed.D.
A TEN -YEAR IMPACT STUDY OF
PRIME TIME FAMILY READING TIME®
STEMMING ThE TIDE OF
INTERGENERATIONAL ILLITERACY: A TEN -YEAR IMPACT STUDY OF PRIME TIME FAMILY READING TIME®
EXECUTIVE SUMMARYPRIME TIME FAMILY READING TIME® (PRIME TIME) was
developed by the Louisiana Endowment for the Humanitiesin 1991 to reach those people who are typically underservedor never-served by humanities organizations, and for whichmost programming is predicated on literacy. After nearly fiveyears of field testing, a turn-key family literacy programtargeting at-risk families with 6-10 year old children wasdeveloped and launched statewide in 1995, and thennationally in 1998. Its objective was to intervene andreverse the persistent statewide pattern ofintergenerational illiteracy by transforming the familyinto a learning environment bonded around the act ofreading together. In so doing, it intended to significantlyimpact and improve student learning.
PREEMPTIVE SOLUTION TO A PROBLEMIn short, PRIME TIME was created as a preemptive
approach to addressing the problem ofintergenerational illiteracy before it became anirreversible problem. The program model that emergedpost pilot phase offers flexibility to adapt to local needs andsettings while maintaining three essential components: 1. PRIME TIME programs are led by highly skilled and trained
team members, including university professors (PRIMETIME scholars), master storytellers (PRIME TIMEstorytellers) as well as host agency staff (teachers,librarians, social workers etc.)
2. PRIME TIME programs combine high quality, humanities-rich literature with the Socratic Method to developinterest in reading and boost critical thinking skills in thewhole family.
3. PRIME TIME programs specifically recruit at-risk familiesand eliminate barriers to participation by offeringtransportation, meals and child care for younger siblings. The success of the turn-key model in recruiting and
retaining target populations is evident from thedemographic summary of this study and in the program’s76% retention rate (based on individual participantattendance) and 120% program attendance growth rate(based on total attendance from session 1 to session 6 or 8.)Additionally, mandatory biannual intensive trainingprovides host agencies with proven strategies for identifyingand recruiting at-risk families, including targeting Title Ischools, Head Start and Early Head Start programs, andother social service agencies that have direct contact with
There is statistical evidence, based on theanalyses of the study, that students whoparticipate in the PRIME TIME programduring their early elementary years, firstthrough fourth grades, performed at ahigher level on grade level content areasof LEAP, iLEAP, and GEE.
Louisiana
Florida
GeorgiaAlabama
Mississippi
ArkansasSouth
Carolina
NorthCarolina
VirginiaWest
Virginia
Kentucky
Ohio
Pennsylvania
New York
Maine
Indiana
Michigan
Wisconsin
Illinois
Minnesota
SouthDakota
Montana
Wyoming
Oregon
California
Utah
Texas
Oklahoma
New Mexico
Colorado
Arizona
Nevada
Idaho
NorthDakota
Iowa
Vermont NH
Conn.Mass.
RI
MDDEL
Tennessee
Washington
53513
3517 3
1
38
44
1
1
1
1 17
71
56
1
2
2
52
15
26
6
34
4
2
2
1
1
13
6
1
7
4103
23
1Alaska
Kansas
Nebraska
Missouri
Puerto Rico
U.S. Virgin Islands
Puerto Rico
Participating States 2010 Active Affiliate States
2010 –11 national expansion award recipient states
Hawaii
1
NJ
target populations. To date, PRIME TIME has beenimplemented in all of Louisiana’s 64 parishes and 38 otherstates. Since 1991, PRIME TIME has enrolled almost40,000 at-risk children and their parents/guardians. In2000, PRIME TIME was introduced as an effective familyliteracy and engagement program in Louisiana’s publicschools. In recognition of its quality and impact, morethan 200 of all statewide programs have beenimplemented in public schools or in collaboration withpublic schools.
EFFECTING MEASURABLE RESULTS The purpose of this “impact study” was to determine
quantitatively the effects of PRIME TIME READING FAMILYTIME program on student achievement. Using a longitudinalanalysis, the study shows that participants engaged in thePRIME TIME program perform at a higher level onachievement tests than non participants, and that the levelof achievement is statistically significant on numerous gradelevel variables as well as above the national norm.Furthermore, while PRIME TIME targets students and their
National Expansion1,083 sites nationally,
1991-2010 (projected)
38,905 participants nationally1991-2010 (projected)
families in the early elementary years, firstthrough 4th grades, this study shows thecontinued impact on PRIME TIME participantstudent performance one to eight years post PRIMETIME engagement, and through the Graduate Exit Exam.The analysis is based on the value-added assessment systemimplemented in Tennessee, Ohio, Louisiana and other states.
The research question asked “What role does the 6-8week PRIME TIME program targeting 6-to-10 year oldstudents and their parents/guardians have on changingstudents’ achievement on the Louisiana EducationAssessment Program (LEAP) 4th and 8th grade assessments,the integrated Louisiana Education Assessment Program(iLEAP) 3rd, 5th, 6th, 7th and 9th grade assessments, and the
Graduation Exit Examination (GEE) 10th
through 12th grade summativeassessments?”. LEAP is a Criterion-
Referenced Test (CRT), which meansstudents’ scores are based on
achievement within a state-prescribed level ofcompetency. iLEAP is aNorm-Referenced Test
(NRT), meaning studentscores are ranked in
comparison with those of theirpeers on a national level. All
of the assessment dataincluded in the study are
sorted by grade andmerged or pooled into
one data set for the 3 yearperiod of 2006-2007, 2007-2008, and 2008-2009 by test type(LEAP, GEE, and iLEAP).
EVALUATION STRATEGYEight grades in West Baton Rouge Parish School District
were studied and analyzed according to the effect of PRIMETIME program on grade-level assessments.1 Evidence ofsustainability as well as the substantial number ofparticipating families and students were major
535 sites1991-2010 (projected)
21,800 participants reached1991-2010 (projected)
Louisiana Expansion
St. Bernard
Plaquemines
Caddo
Bossier
Claiborne
Sabine
Winn
Jackson
Wes
tCa
rrol
lEa
stCa
rrol
lEvangeline
JeffersonDavis
Tangipahoa
Jefferson
Assumption
Lafourche
Washington
St. Tammany
Orleans
Terrebonne
St.James
Livingston
St.HelenaEast
Feliciana
EastBatonRougeWest
BatonRouge
WestFeliciana
Iberville Ascension
VermilionCameron
Calcasieu
Beauregard
Vernon
Rapides
Avoyelles
AllenSt. Landry
AcadiaSt.
Martin
Lafayette
Iberia
St. Mary
PointeCoupee
Catahoula
Conc
ordi
aGrant
La SalleNachitoches
Bienville
Lincoln
Ouachita Richland
Caldwell
Morehouse
Franklin
Madison
Tensas
St. J
ohn
The
Bapt
ist
St.Charles
Web
ster
De SotoRedRiver
Union
37 3
6
1
3
2
1
1
54
2 2
21756
19
8
83
41
33
3
85
7 65
6
20
19 4
8
4
1
2 1214
3 39
24
13
9
21
563
4
8
11
59
277
2
5
1021
2
12
1
GRADE 4LEAP
Final Evaluation
ReportGRADE 5
iLEAP
GRADE 6iLEAP
Generalized Assessment Model For Data Collection
GRADE 3iLEAP
2006-09
2006-09
2006-09
Gender
Ethnic
Ed Class
Econo
Subgroups
Data across 3 years were merged, but randomlysampled by school based on number of participants per school.
Gender
Ethnic
Ed Cla
ss
Econo
Factored out Factored out
CONTROL NON-PARTICIPANTS
TREATMENTPRIME TIME PARTICIPANTS
SAMPLE
GRADE 7iLEAP
GRADE 8LEAP
GRADE 9iLEAP
GRADES 10-12GEE GEE
2006-09
iLEAP
2006-09
LEAP
2006-09
iLEAP
iLEAP
iLEAP
LEAP2006-09
considerations in choosing this school district for the study. TheWest Baton Rouge Parish Library conducted 21 PRIME TIMEprograms since 1996, enrolling almost 1,400 students and theirparents/guardians. A total of 4,350 West Baton Rouge Parishstudent participants and non participants were studied,including repeaters reoccurring each of the three years, and arandom sample of 816 students in the control group, and 816students in the PRIME TIME treatment group. The data werefurther sorted by school, grade, gender, ethnicity, free orreduced lunch (as a measure of economic status), andeducational classification. In general, the profiles of thePRIME TIME participation (treatment) group contained:more participants on free and reduced lunch (economicallydisadvantaged), a higher percentage of students identifiedfor special education, a higher percentage of Blackstudents, and a higher percentage of female students.Students with exceptionalities were factored out of the study.This decision was based on the low participation numbers forstudents with exceptionalities in PRIME TIME, thus failing toprovide a viable sample with which to work. Below is a figureshowing the data collection components for this study.
Generalized Assessment Model For Data Collection
1 The Curriculum Impact Chart is available in Figure 36, page 54. Table 27 on page 53 provides an overview of the findings.
PRIME TIME was created as apreemptive approach to addressingthe problem of intergenerationalilliteracy before it became anirreversible problem.
GENERAL kEY FINDINGS
0
11
23
34
45
0
11
23
34
45
# of Variables
2
42 43 43 2615
28
1
163rd 5th
6th7th
9thGrade
PTControl
0
8
15
23
30
0
8
15
23
30
# of Variables
25
15
21
4th
GradeHigh
L
PTControl
There is statistical evidence, based on the analyses ofthe study, that students who participate in the PRIMETIME program during their early elementary years, firstthrough fourth grades, performed at a higher level ongrade level content areas of LEAP, iLEAP, and GEE.Students who participated had mean performancescores and sub scores that were consistently higher ingrades 3, 4, 5, 6, 7, and high school. This study revealedthat student performance was consistently higher thannational norms in grades 3 and 5 as measured by Norm
Curve Equivalents (NCEs)2 in Reading, Language, andMathematics, and grade 6 in Reading. While no specificreason can be discerned as to why the 8th grade analysisdiffered from that of the other seven grades in the study,the anomaly can be reasonable attributed to a smallersample size of the treatment group than needed for fullyconsistent measurement. Below is a brief summary ofcompiled findings by grade level and the correspondingnumber of variables showing that the PRIME TIMEparticipants scored higher than the Control Group.
n 3rd Grade (PT Target Grade). Ratio of PT variables is higher than Control Group in 42 out of 44 variables. n 4th Grade (PT Target Grade). Ratio of PT variables is higher than Control Group in 25 out of 26 variables.n 5th Grade (1 Year Post PT). Ratio of PT variables is higher than Control Group in 43 out of 43 variables. n 6th Grade (2 Years Post PT). Ratio of PT variables is higher than Control Group in 43 out of 43 variables. n 7th Grade (3 Years Post PT). Ratio of PT variables is higher than Control Group in 26 out of 42 variables.n 8th Grade (4 Years Post PT). Ratio of PT variables is higher than Control Group in 3 out of 26 variables. n 9th Grade (5 Years Post PT). Ratio of PT variables is higher than Control Group in 15 out of 43 variables.n High School (8 Years Post PT). Ratio of PT variables is higher than Control Group in 21 out of 26 variables.
iLEAP Performance Ratios:# of Variables PRIME TIME Participants
Scored Higher vs. Control Group
LEAP Performance Ratios:# of Variables PRIME TIME Participants
Scored Higher vs. Control Group
2 NCE -Norm Curve Equivalent
SPECIFIC kEY FINDINGSPRIME TIME Participants Scored Higher vs. Control Group
55
61.25
67.50
73.75
80
55
61.25
67.50
73.75
80
LEAP Sub Score
75.5
70.568.3
63.1
69.3
61.6
72.9
67.9
LEAP Sub Score Area Sampling
Use Conventions ofLanguage P=.02 Physical Science
P=.05Life Science P=.002
Science & Environment P=.05
PT
Control
40
42
45
47
49
51
54
56
58
40
42
45
47
49
51
54
56
58
Performance Score
55.4
49.9 51.3
45.8
50.8
46.3
53
47.3
NCEs Sampling
Language NormalCurve Equivalent
P=.018MathematicsTotal Normal
Curve EquivalentP=.018
Reading Total Normal Curve Equivalent
P=.035 Survey Core Total Normal Curve Equivalent
P=.006PT Mean N=109Control Mean N=109
3rd Grade NCE Performance
4th Grade Sub Score Data (2006-2009)
3rd GradePRIME TIME Students:n Performed higher on 95.5%
(42 out of 44) areas of theiLEAP
n 17 areas had performancemeasures that werestatistically significantlyhigher
4th GradePRIME TIME Students:n Performed higher on 96%
(25 out of 26) areas of theLEAP
n 5 areas had performancemeasures that werestatistically significantlyhigher
42
44
46
48
50
52
54
56
42
44
46
48
50
52
54
56
Performance Score
50.6
44.8
54.4
50.4
Survey Core Total NormalCurve Equivalent P=.006
Reading Total NormalCurve Equivalent P=.007
NCEs SamplingPT Mean N=113Control Mean N=113
40
42
44
46
48
50
52
54
40
42
44
46
48
50
52
54
Performance Score
50.3
45.2
52.5
48.6Survey Core Total NormalCurve Equivalent P=.034
Reading Total NormalCurve Equivalent P=.042
NCEs SamplingPT Mean N=120Control Mean N=120
5th Grade NCE Performance
6th Grade NCE Performance
SPECIFIC kEY FINDINGSPRIME TIME Participants Scored Higher vs. Control Group
5th GradePRIME TIME Students:n Performed higher on 100%
(43 out of 43) areas of theiLEAP
n 21 areas had performancemeasures that werestatistically significantlyhigher
6th GradePRIME TIME Students:n Performed higher on 100%
(41 out of 41) areas of theiLEAP
n 15 areas had performancemeasures that werestatistically significantlyhigher
40
45
50
55
60
65
70
75
40
45
50
55
60
65
70
75
Sub Score
57.3
54.2
61.259.8
72.7
69.8
49.7
45.3
56.9
53.1
iLEAP Sub Score Area Sampling
Write Competently Use
Conventionsof Language
Locate Select& SymbolizeInformation
Algebra
Geometry
PT Mean N=108Control Mean N=108
45
51.25
57.5
63.75
70
45
51.25
57.5
63.75
70
LEAP Su
b Score
68
59.6
67.4
60
63.5
57.160
52.1
LEAP Sub Score Area Sampling
Number &Number Relations
P=.047Patterns Relations
& Functions P=.027
Science asInquiry P=.10 Life Science
P=.042
PT Mean N=41-62Control Mean N=47-58
7th Grade Sub Score Data (2006-2009)
High School Sub Score Data (2006-2009)
SPECIFIC kEY FINDINGSPRIME TIME Participants Scored Higher vs. Control Group
7th GradePRIME TIME Students:n Performed higher on 62%
(26 out of 42) areas of theiLEAP
High School PRIME TIME Students:n Performed higher on 81%
(21 out of 26) areas of the GEEn 3 areas had performance
measures that were statisticallysignificantly higher
n 2 areas showed probabilitylevels
OThER OBSERVABLE EFFECTS
n Effects by Grade. Students who participated in PRIMETIME outperformed students in the Control Group whodid not participate in PRIME TIME.
n Effects by Gender. Female and male students whoparticipated in PRIME TIME performed higher whencompared to the control group of female and malestudents who did not participate. Female studentsperformed higher than male students in reading;however, male students tended to score higher inscience, mathematics, and social science. In 3rd, 4th and5th grades, female students scored higher, yet in theupper level grades, the male students demonstrated amix of performance over females.
n Effects by Ethnicity. Summary results indicate thatBlack and White students who participated in PRIMETIME performed higher than Black and White students inthe Control Group who had not participated. The Whitestudents performed higher than Black students in everyarea, reflecting historic national academicdemographics.3 However, the Black students who hadparticipated in PRIME TIME scored close to the mean ofthe White students who had not participated.
n Effects by Economic Level. Three grades: 8th, 9th, andhigh school showed significant differences in thedistribution of students between the two study groups.The PRIME TIME group contained a higher number ofparticipants who were classified as “At-Risk” due toeconomic level based on self report of free or reducedlunch status.
MOVING TOWARDS PRIME TIME FULL CIRCLEThis study was conducted by James Edward Barr, Ed.D.,
Professor Emeritus Nicholls State University, where hetaught graduate courses in Research Methods and ProgramEvaluation and served as a graduate faculty member for 20years. His work with the Board of Elementary andSecondary Education (BESE) over the past 15 years inLouisiana helped establish the evaluation process for charterschools in the state. Prior to his work at Nicholls StateUniversity, he served as a psychometrician in the Office ofResearch and Development helping develop Louisiana’sAssessment Program, and later as State Science Supervisorin the Louisiana Department of Education. Dr. Barrcontinues to work as an Evaluation-Research consultant withvarious federal, state, local agencies, universities, and non-profit organizations, such as the National Association ofCharter School Authorizers, the Louisiana Department ofEducation, the National Institute of Health, National ScienceFoundation and the U.S. Department of Education.
These results were derived from the traditional PRIMETIME FAMILY READING TIME 6-8 week family literacy
In general, the profiles of the PRIME TIMEparticipation (treatment) group contained: moreparticipants on free and reduced lunch(economically disadvantaged),a higherpercentage of students identified for specialeducation, a higher percentage of Black students,and a higher percentage of female students.
3 Example of recent annual publication from American College Testing. Each year’s testing provides a profile of student performance across the nation.http://www.act.org/news/data/09/pdf/National2009.pdf
Demographic summary of pooled high school students for 2006-07, 2007-08,2008-09. showing Control and PRIME TIME students sorted by economic status(At-Risk) (X2=8.8, P=.003).
Demographic summary of pooled high school students for 2006-07, 2007-08, 2008-09. showing control and PRIME TIME students sorted by ethnicity(X2=3.60, P=.02).
SAMPLE DEMOGRAPHIC SUMMARY
programs only. This study does not measure the additionalstructures now incorporated into the PRIME TIMEcomprehensive menu of services known as PRIME TIME FullCircle, which includes graduate seminars for teachers andintensive ongoing professional development workshops forparticipating schools (still in development). It also does notmeasure the effect of the bilingual PRIME TIME programs,offered in Spanish/English and French/English, which havebeen developed and implemented statewide.
Because the human consequence is so great, PRIME TIMEwill continue to monitor closely its impact on theeducational progress of its target audience of at-riskchildren and their parents/guardians. Especially in that theadditional dimension of PRIME TIME Full Circle has beendeveloped, it is our hope that in the future we will provesufficiently effective to become unnecessary.
Miranda RestovicDirector, PRIME TIME FAMILY READING TIME®
Michael Sartisky, Ph.D.President/Executive DirectorLouisiana Endowment for the Humanities
PRIME TIME was created as a preemptiveapproach to addressing the problem ofintergenerational illiteracy before itbecame an irreversible problem.
Since 1991 PRIME TIME has enrolled almost 40,000 at-risk childrenand their parents/guardians. In recognition of its quality and impact,more than 200 of all statewide programs have been implemented inpublic schools or in collaboration with public schools.
Louisiana Humanities Center at Turners' Hall938 Lafayette Street • Suite 300 • New Orleans, LA 70113
504.523.4352 • Fax: 504.529.2358 • www.leh.org
The LEH thanks the West Baton Rouge Parish Library and the West Baton Rouge Parish School District for their collaboration on this study.
THIS STUDY WAS MADE POSSIBLE WITH A GRANT FROM