+ All Categories
Home > Documents > huang_MAPB_v5.pdf

huang_MAPB_v5.pdf

Date post: 03-Apr-2018
Category:
Upload: amincu
View: 216 times
Download: 0 times
Share this document with a friend
15
7/27/2019 huang_MAPB_v5.pdf http://slidepdf.com/reader/full/huangmapbv5pdf 1/15 Making Afterschool Programs Better Denise Huang & Ronald Dietel  Policy Brief  No. 11
Transcript
Page 1: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 115

Making Afterschool

Programs BetterDenise Huang amp Ronald Dietel

Policy Brief No 11

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 215

Copyright copy 2011 The Regents o the University o Caliornia

The work reported herein was supported in part by a subcontract rom Southwest Educational Development Laboratory (SEDL

subcontract no PO54604) unded by contract no ED-03-CO-0048 as administered by the US Department o Education

The ndings and opinions expressed in this report are those o the authors and do not necessarily refect the positions or policies o

the National Center or Research on Evaluation Standards and Student Testing (CRESST) the Southwest Educational Development

Laboratory (SEDL) or the US Department o Education

To cite rom this report please use the ollowing as your APA reerence

Huang D amp Dietel R (2011) Making afterschool programs better (CRESST Policy Brie) Los Angeles CA University o Caliornia

The authors thank the ollowing or their assistance with this policy brie Joan Herman or her valuable review and eedback

Tamara Lau or her design and layout and Karisa Peer or her editorial support

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 315

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

QUALITY

AFTERSCHOOL

PROGRAM

EVALUATION

STAFF

LEADERSHIP

PROGRAM

GOALS

Figure 1 CRESST Afterschool Program Quality Model

Goals are clear rigorous and supported across the program in structure and content Funding

is adequate to support goals

Leadership is experienced well-educated has longevity at the current site uses eective

communications sets high expectations and has a bottoms-up management style

Sta is experienced has longevity at current program relates well to students models high

expectations motivates and engages students and works well with leaders colleagues and

parents

Program aligns to the day school provides time or students to study learn and practice

includes motivational activities requently uses technology science and the arts to support

youth development student learning and engagement

Evaluation uses both internal (ormative) and external (summative) methods Evaluative

inormation and data accurately measure goals results are applied to continuous program

improvement

1

2

3

4

5

These ve components work together to produce a high quality aterschool program (Figure 1)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 515 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

The remainder o this brie covers each o these ve

components First we provide common evaluation ndings

across most programs We then provide examples

either best practices or useul observations noted by

the CRESST team We conclude with a short list o

recommendations which could help make aterschool

programs even better

Component 1 Goals ________________________________________________________

ldquoEverything that we do in afterschool we try to make it connected to the real worldhellipOne of our many goals is tomake it relevanthellipmake a connection for students to stayin school go to college a reason to be on a particular

career pathhelliprdquo - Afterschool Project Director

Overview

Setting clear goals and desired outcomes is a cornerstone

o aterschool program success (Bodilly and Beckett 2005)

Virtually all o the programs we studied had established

unambiguous goals and structured their programs to

meet those goals At the same time most programs alsorecognized the importance o considering student voices

when making decisions regarding program activities

and content hence many programs allowed students

to provide input especially in the arts and technology

programs As a result students were engaged and excited

to be in their aterschool programs

Key Evaluation Findings

The best programs had

bull clearly dened goals in a written plan

bull curricular design and specic practices aligned toprogram goals and

bull internal and external evaluations (or urther detailssee Component 5 o this report)

ldquoAchievementrdquo ocused programs oten district-aliated werebull usually more structured than other aterschool

program types

bull stressed improvement o academic perormance

bull hired more certied teachers and

bull maintained a more consistent linkage with the dayschool then less achievement-ocused programs

Goals o many high quality aterschool programs also

had a specic emphasis (eg science technology

homework support community involvement or the

arts)

Examples

The site coordinator at one math program described

program goals as ocused on developing studentsrsquo

academic skills within a specic content area ldquoOur

primary goals are to bring the student to grade levelrdquo

Similarly goals or three o the math programs and our

o the reading programs specically targeted students

who were struggling academically

A primary goal o many arts ocused aterschool

programs was to provide students who otherwisewould have little-to-no exposure to the arts with

quality art experiences Many interviewees reported

using an integrative approach to arts instruction that

could help students who were struggling academically

and personally

Similar to arts all science sta responded to questions

about curricular goals by suggesting a primary interest

in oering positive science experiences to students A

ew went urther to explain that their principle goal

was to improve standardized test scores in science

by ocusing on extending day school instruction into

aterschool

Interview data across seven homework-ocused

programs suggested that the primary goals in

aterschool homework were to complete homework

and increase academic achievement

Nearly all o the technology sta had the goal o

teaching students the mechanics o a broad range

o technology skills an interest in encouraging the

students to use those skills to enhance learning in other

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 615 Making Afterschool Programs Better | Denise Huang and Ron Dietel

content areas and teaching the students a technical skill

that is relevant to their real lives (both now and in the

uture)

In aligning activities to achieve their program goals nearly

all interviewees described the merits o developing a

curriculum that combined academic skill development with

opportunities to explore and encourage studentsrsquo social

development One program director summed up

ldquoOur goal is to help each child to make sure that itrsquos

an aterschool program thatrsquos un but at the sametime itrsquos enriching so they grow and learn every dayso they can take home more knowledgehelliprdquo

Component 2 Leadership ________________________________________________________

ldquoObviously we try to be democratichellipSo one of thethings we try to do here how we want to make thisa great place to work is in fnding great peoplethen giving the people the power and leeway and theaccountability but also the freedom to do what they

think works best and trusting themhelliprdquo

- Afterschool Project Director

Overview

Our studies ound that directors and managers o high

unctioning aterschool programs usually had many years

o experience in aterschool programs Further the leaders

nearly always shared decision-making with their sta across

aterschool goals programs and evaluation

Key Evaluation Findings

The best programs had leaders who

bull articulated a clear program mission vision statementand goals

bull decentralized decision making used a bottoms upleadership style

bull trusted in the knowledge and skills o sta regardingcurriculum and instruction

bull promoted a team culture o positive relationships

requent communication and sta problem-solvingskills

bull insured that instructors and students had adequatematerials and resources and

bull provided all sta with proessional developmentopportunities that improved individual and team skills

The majority o both site coordinators and instructors

said that aterschool instructors had an active voice in

decisions about curriculum and instruction (44 average

on a 50 point scale) and took active roles in program

leadership and decision-making (41 average)

Administrators consistently described the value o

starsquos content-specic skills and expertise as a result

curriculum development was a group process in which

leaders gave sta members a strong voice in designing

instruction

Leaders and sta across high-quality programs

maintained good relationships with the day school

personnel However ew o them had established

ormal communication systems or that purpose

0

10

20

30

40

50

60

70

80

90

100

Never Rarely Sometimes Frequently Regularly

P e r c e n t a g e o f I n s t r u c t o r s

2217 19

14

28

Figure 2 Frequency of communication between afterschool program and day-school staff

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 715 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

Figure 3 Educational levels of afterschool staff

1Currently in High School

19High School GraduateGED

9Associatersquos Degree

24Masterrsquos Degree

47Bachelorrsquos Degree

(as indicated in Figure 2) with less than 50 o

aterschool sta reporting that they had regular or

requent communication with day school teachers (see

Recommendations section)

Examples

One site coordinator expressed appreciation or the sta

by stating

ldquoThese are proessional adultshellipand they are the

best ones to implement the curriculumhellip Theyrsquoreright there with the studentshellipthey know whattheir levels are and their abilitiesrdquo

An arts program instructor explained

ldquoMy experience has been that the aterschoolteachers propose something that wersquod like to dowith the kids aterschool and [the director] then

just talks to us about what our plans are Wehellipreport to her in terms o lesson plans and how weincorporate standards and benchmarks but a lot oreedom is really given to us We teach what wersquorecomortable teaching and what wersquore passionateaboutrdquo

A project director at a technology program said that his

starsquos latitude was evenly balanced by a strong level o

accountability or their curricular choices

ldquoEvery quarter they have to come back to us andtell us how theyrsquore doing They report back asto what is going on at theirhellipprograms In termso actual decision-making and setting goals anddeciding what wersquore going to do thatrsquos much moreo a bottom-up processrdquo

Component 3 Staff _____________________________________

ldquoWe recognized that the other role we (staff) have to play is to get children engaged in the learning process So itrsquos not just about completing the homework but itrsquosabout finding ways outside of the school to get theminterested excited feeling confident and to build theirself-esteem so that they want to come back the next day

and try a little harderrdquo - Afterschool Staff Instructor

Overview We ound that high quality programs recruited qualied

sta and created collegial environments supporting their

programsrsquo missions Aterschool leaders were able to retain

sta and achieve lower turnover rates than other programs

because sta elt respected supported autonomous and

condent in their ability to reach their students In turn

sta and students constructed positive relationships with

each other characterized by warmth and mutual respect

Sta was oten role a model or students creating a norm

o high expectations appropriate student behavior good

school attendance eective work habits and positiveattitudes towards learning

Key Evaluation Findings

Sta rom quality programs generally had high

educational levels Forty-seven percent o all sta had

Bachelorrsquos degrees twenty-our percent had a Masterrsquos

degree (see Figure 3)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 815 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Aterschool sta in quality programs usually had

substantial aterschool experience Sixty-six percent

had more than three years o work experience in

aterschool twenty-three percent had six years or more

o experience

Program sta at high quality sites also tended to

have low turnover The majority o the sta had

been at their current site or three years or more (see

Recommendations section)Positive relationships and interactions between the sta

and the students were observed in virtually all high

quality programs particularly in terms o expectations

or student perormance disciplinary issues and

democracy

All interviewees reported having some orm o

technology-related proessional development available in

their aterschool program

Most proessional development opportunities were or

helping students with reading and math applying state

standards to the curriculum and connecting with theday school Fewer opportunities were oered on topics

o English learners special needs students evaluation

and assessment (see Recommendations section)

A higher percentage o site coordinators reported

receiving proessional development in most categories

(other than working with English language learners)

compared to instructors who reported receiving the

lowest percentage o proessional development in

program and sel-evaluation (see Recommendations

section)

Although sta generally ound proessional developmentuseul only 26 had regular (2-4) opportunities each

year About 50 o the homework program sta

reported that no proessional development was oered

to them (see Recommendations section)

Examples

Nearly all quality aterschool programs approached

decision-making in a democratic ashion One program

allowed students to oer input on where they would

like to conduct their service learning projects others

considered studentsrsquo activities choices and made great

eorts to include them in their program decision-making

Arts- science- and technology-ocused aterschool

programs tended to give students more autonomy

and input into programs compared to homework or

academic ocused programs One arts program sta

stated that student inputs were ldquoalways o great value

student interests had a vast impact on art curriculum

contentrdquo Consequently students took ownership in

their learning and remained engaged

Component 4 Staff ____________________________________________________

ldquoThese are kids who have never been exposed to

anythingmdashour goal is to open their minds to new

things and to show that they can do ithellipWe want a

well-balanced well-rounded program with a lot of

different things to offer to the childrenrdquo - Afterschool

Site Coordinator

Study ndings revealed curricular similarities and

dierences across all programs a majority o which

oered three or more activities each day Most

programs included homework help or tutoring but

other activities ranged rom academic (eg math

reading writing science) to enrichment (eg arts

and crats cooking gardening health and nutrition

cultural activities computers) and recreation (eg

sports dance drill team outdoor games) The

requency and duration o instruction oered by the

programs are provided in Table 1

These ndings suggest that students were receiving

adequate aterschool time or learning and skills

practice Observation reports across the programs also

indicated that students appeared to be mostly engaged

and attentive and enjoyed the activities Virtually all

programs provided substantial time or recreational

and crats activities keeping students engaged while

exercising other parts o their brains

Key Evaluation Findings

The majority o programs employed unique and

innovative strategies to engage students in theaterschool setting placing a particular emphasis on

making learning un

Technology programs reported the most requent

use o research-based practices whereas reading

programs reported the lowest requency

Technology science arts and homework programs

appeared to be more ocused on developing

higher order thinking skills whereas reading and

math programs were more ocused on direct skills

instruction

Most programs shared similar methods odisseminating inormation to parents as well as

a means o encouraging their involvement in the

aterschool programs

Parents were very satised with the programs both

in terms o positive changes in their childrenrsquos

behaviors and attitudes and in general program

unctioning They elt that the sta cared about and

respected their children They also reported that

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 915

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 2: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 215

Copyright copy 2011 The Regents o the University o Caliornia

The work reported herein was supported in part by a subcontract rom Southwest Educational Development Laboratory (SEDL

subcontract no PO54604) unded by contract no ED-03-CO-0048 as administered by the US Department o Education

The ndings and opinions expressed in this report are those o the authors and do not necessarily refect the positions or policies o

the National Center or Research on Evaluation Standards and Student Testing (CRESST) the Southwest Educational Development

Laboratory (SEDL) or the US Department o Education

To cite rom this report please use the ollowing as your APA reerence

Huang D amp Dietel R (2011) Making afterschool programs better (CRESST Policy Brie) Los Angeles CA University o Caliornia

The authors thank the ollowing or their assistance with this policy brie Joan Herman or her valuable review and eedback

Tamara Lau or her design and layout and Karisa Peer or her editorial support

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 315

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

QUALITY

AFTERSCHOOL

PROGRAM

EVALUATION

STAFF

LEADERSHIP

PROGRAM

GOALS

Figure 1 CRESST Afterschool Program Quality Model

Goals are clear rigorous and supported across the program in structure and content Funding

is adequate to support goals

Leadership is experienced well-educated has longevity at the current site uses eective

communications sets high expectations and has a bottoms-up management style

Sta is experienced has longevity at current program relates well to students models high

expectations motivates and engages students and works well with leaders colleagues and

parents

Program aligns to the day school provides time or students to study learn and practice

includes motivational activities requently uses technology science and the arts to support

youth development student learning and engagement

Evaluation uses both internal (ormative) and external (summative) methods Evaluative

inormation and data accurately measure goals results are applied to continuous program

improvement

1

2

3

4

5

These ve components work together to produce a high quality aterschool program (Figure 1)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 515 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

The remainder o this brie covers each o these ve

components First we provide common evaluation ndings

across most programs We then provide examples

either best practices or useul observations noted by

the CRESST team We conclude with a short list o

recommendations which could help make aterschool

programs even better

Component 1 Goals ________________________________________________________

ldquoEverything that we do in afterschool we try to make it connected to the real worldhellipOne of our many goals is tomake it relevanthellipmake a connection for students to stayin school go to college a reason to be on a particular

career pathhelliprdquo - Afterschool Project Director

Overview

Setting clear goals and desired outcomes is a cornerstone

o aterschool program success (Bodilly and Beckett 2005)

Virtually all o the programs we studied had established

unambiguous goals and structured their programs to

meet those goals At the same time most programs alsorecognized the importance o considering student voices

when making decisions regarding program activities

and content hence many programs allowed students

to provide input especially in the arts and technology

programs As a result students were engaged and excited

to be in their aterschool programs

Key Evaluation Findings

The best programs had

bull clearly dened goals in a written plan

bull curricular design and specic practices aligned toprogram goals and

bull internal and external evaluations (or urther detailssee Component 5 o this report)

ldquoAchievementrdquo ocused programs oten district-aliated werebull usually more structured than other aterschool

program types

bull stressed improvement o academic perormance

bull hired more certied teachers and

bull maintained a more consistent linkage with the dayschool then less achievement-ocused programs

Goals o many high quality aterschool programs also

had a specic emphasis (eg science technology

homework support community involvement or the

arts)

Examples

The site coordinator at one math program described

program goals as ocused on developing studentsrsquo

academic skills within a specic content area ldquoOur

primary goals are to bring the student to grade levelrdquo

Similarly goals or three o the math programs and our

o the reading programs specically targeted students

who were struggling academically

A primary goal o many arts ocused aterschool

programs was to provide students who otherwisewould have little-to-no exposure to the arts with

quality art experiences Many interviewees reported

using an integrative approach to arts instruction that

could help students who were struggling academically

and personally

Similar to arts all science sta responded to questions

about curricular goals by suggesting a primary interest

in oering positive science experiences to students A

ew went urther to explain that their principle goal

was to improve standardized test scores in science

by ocusing on extending day school instruction into

aterschool

Interview data across seven homework-ocused

programs suggested that the primary goals in

aterschool homework were to complete homework

and increase academic achievement

Nearly all o the technology sta had the goal o

teaching students the mechanics o a broad range

o technology skills an interest in encouraging the

students to use those skills to enhance learning in other

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 615 Making Afterschool Programs Better | Denise Huang and Ron Dietel

content areas and teaching the students a technical skill

that is relevant to their real lives (both now and in the

uture)

In aligning activities to achieve their program goals nearly

all interviewees described the merits o developing a

curriculum that combined academic skill development with

opportunities to explore and encourage studentsrsquo social

development One program director summed up

ldquoOur goal is to help each child to make sure that itrsquos

an aterschool program thatrsquos un but at the sametime itrsquos enriching so they grow and learn every dayso they can take home more knowledgehelliprdquo

Component 2 Leadership ________________________________________________________

ldquoObviously we try to be democratichellipSo one of thethings we try to do here how we want to make thisa great place to work is in fnding great peoplethen giving the people the power and leeway and theaccountability but also the freedom to do what they

think works best and trusting themhelliprdquo

- Afterschool Project Director

Overview

Our studies ound that directors and managers o high

unctioning aterschool programs usually had many years

o experience in aterschool programs Further the leaders

nearly always shared decision-making with their sta across

aterschool goals programs and evaluation

Key Evaluation Findings

The best programs had leaders who

bull articulated a clear program mission vision statementand goals

bull decentralized decision making used a bottoms upleadership style

bull trusted in the knowledge and skills o sta regardingcurriculum and instruction

bull promoted a team culture o positive relationships

requent communication and sta problem-solvingskills

bull insured that instructors and students had adequatematerials and resources and

bull provided all sta with proessional developmentopportunities that improved individual and team skills

The majority o both site coordinators and instructors

said that aterschool instructors had an active voice in

decisions about curriculum and instruction (44 average

on a 50 point scale) and took active roles in program

leadership and decision-making (41 average)

Administrators consistently described the value o

starsquos content-specic skills and expertise as a result

curriculum development was a group process in which

leaders gave sta members a strong voice in designing

instruction

Leaders and sta across high-quality programs

maintained good relationships with the day school

personnel However ew o them had established

ormal communication systems or that purpose

0

10

20

30

40

50

60

70

80

90

100

Never Rarely Sometimes Frequently Regularly

P e r c e n t a g e o f I n s t r u c t o r s

2217 19

14

28

Figure 2 Frequency of communication between afterschool program and day-school staff

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 715 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

Figure 3 Educational levels of afterschool staff

1Currently in High School

19High School GraduateGED

9Associatersquos Degree

24Masterrsquos Degree

47Bachelorrsquos Degree

(as indicated in Figure 2) with less than 50 o

aterschool sta reporting that they had regular or

requent communication with day school teachers (see

Recommendations section)

Examples

One site coordinator expressed appreciation or the sta

by stating

ldquoThese are proessional adultshellipand they are the

best ones to implement the curriculumhellip Theyrsquoreright there with the studentshellipthey know whattheir levels are and their abilitiesrdquo

An arts program instructor explained

ldquoMy experience has been that the aterschoolteachers propose something that wersquod like to dowith the kids aterschool and [the director] then

just talks to us about what our plans are Wehellipreport to her in terms o lesson plans and how weincorporate standards and benchmarks but a lot oreedom is really given to us We teach what wersquorecomortable teaching and what wersquore passionateaboutrdquo

A project director at a technology program said that his

starsquos latitude was evenly balanced by a strong level o

accountability or their curricular choices

ldquoEvery quarter they have to come back to us andtell us how theyrsquore doing They report back asto what is going on at theirhellipprograms In termso actual decision-making and setting goals anddeciding what wersquore going to do thatrsquos much moreo a bottom-up processrdquo

Component 3 Staff _____________________________________

ldquoWe recognized that the other role we (staff) have to play is to get children engaged in the learning process So itrsquos not just about completing the homework but itrsquosabout finding ways outside of the school to get theminterested excited feeling confident and to build theirself-esteem so that they want to come back the next day

and try a little harderrdquo - Afterschool Staff Instructor

Overview We ound that high quality programs recruited qualied

sta and created collegial environments supporting their

programsrsquo missions Aterschool leaders were able to retain

sta and achieve lower turnover rates than other programs

because sta elt respected supported autonomous and

condent in their ability to reach their students In turn

sta and students constructed positive relationships with

each other characterized by warmth and mutual respect

Sta was oten role a model or students creating a norm

o high expectations appropriate student behavior good

school attendance eective work habits and positiveattitudes towards learning

Key Evaluation Findings

Sta rom quality programs generally had high

educational levels Forty-seven percent o all sta had

Bachelorrsquos degrees twenty-our percent had a Masterrsquos

degree (see Figure 3)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 815 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Aterschool sta in quality programs usually had

substantial aterschool experience Sixty-six percent

had more than three years o work experience in

aterschool twenty-three percent had six years or more

o experience

Program sta at high quality sites also tended to

have low turnover The majority o the sta had

been at their current site or three years or more (see

Recommendations section)Positive relationships and interactions between the sta

and the students were observed in virtually all high

quality programs particularly in terms o expectations

or student perormance disciplinary issues and

democracy

All interviewees reported having some orm o

technology-related proessional development available in

their aterschool program

Most proessional development opportunities were or

helping students with reading and math applying state

standards to the curriculum and connecting with theday school Fewer opportunities were oered on topics

o English learners special needs students evaluation

and assessment (see Recommendations section)

A higher percentage o site coordinators reported

receiving proessional development in most categories

(other than working with English language learners)

compared to instructors who reported receiving the

lowest percentage o proessional development in

program and sel-evaluation (see Recommendations

section)

Although sta generally ound proessional developmentuseul only 26 had regular (2-4) opportunities each

year About 50 o the homework program sta

reported that no proessional development was oered

to them (see Recommendations section)

Examples

Nearly all quality aterschool programs approached

decision-making in a democratic ashion One program

allowed students to oer input on where they would

like to conduct their service learning projects others

considered studentsrsquo activities choices and made great

eorts to include them in their program decision-making

Arts- science- and technology-ocused aterschool

programs tended to give students more autonomy

and input into programs compared to homework or

academic ocused programs One arts program sta

stated that student inputs were ldquoalways o great value

student interests had a vast impact on art curriculum

contentrdquo Consequently students took ownership in

their learning and remained engaged

Component 4 Staff ____________________________________________________

ldquoThese are kids who have never been exposed to

anythingmdashour goal is to open their minds to new

things and to show that they can do ithellipWe want a

well-balanced well-rounded program with a lot of

different things to offer to the childrenrdquo - Afterschool

Site Coordinator

Study ndings revealed curricular similarities and

dierences across all programs a majority o which

oered three or more activities each day Most

programs included homework help or tutoring but

other activities ranged rom academic (eg math

reading writing science) to enrichment (eg arts

and crats cooking gardening health and nutrition

cultural activities computers) and recreation (eg

sports dance drill team outdoor games) The

requency and duration o instruction oered by the

programs are provided in Table 1

These ndings suggest that students were receiving

adequate aterschool time or learning and skills

practice Observation reports across the programs also

indicated that students appeared to be mostly engaged

and attentive and enjoyed the activities Virtually all

programs provided substantial time or recreational

and crats activities keeping students engaged while

exercising other parts o their brains

Key Evaluation Findings

The majority o programs employed unique and

innovative strategies to engage students in theaterschool setting placing a particular emphasis on

making learning un

Technology programs reported the most requent

use o research-based practices whereas reading

programs reported the lowest requency

Technology science arts and homework programs

appeared to be more ocused on developing

higher order thinking skills whereas reading and

math programs were more ocused on direct skills

instruction

Most programs shared similar methods odisseminating inormation to parents as well as

a means o encouraging their involvement in the

aterschool programs

Parents were very satised with the programs both

in terms o positive changes in their childrenrsquos

behaviors and attitudes and in general program

unctioning They elt that the sta cared about and

respected their children They also reported that

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 915

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 3: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 315

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

QUALITY

AFTERSCHOOL

PROGRAM

EVALUATION

STAFF

LEADERSHIP

PROGRAM

GOALS

Figure 1 CRESST Afterschool Program Quality Model

Goals are clear rigorous and supported across the program in structure and content Funding

is adequate to support goals

Leadership is experienced well-educated has longevity at the current site uses eective

communications sets high expectations and has a bottoms-up management style

Sta is experienced has longevity at current program relates well to students models high

expectations motivates and engages students and works well with leaders colleagues and

parents

Program aligns to the day school provides time or students to study learn and practice

includes motivational activities requently uses technology science and the arts to support

youth development student learning and engagement

Evaluation uses both internal (ormative) and external (summative) methods Evaluative

inormation and data accurately measure goals results are applied to continuous program

improvement

1

2

3

4

5

These ve components work together to produce a high quality aterschool program (Figure 1)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 515 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

The remainder o this brie covers each o these ve

components First we provide common evaluation ndings

across most programs We then provide examples

either best practices or useul observations noted by

the CRESST team We conclude with a short list o

recommendations which could help make aterschool

programs even better

Component 1 Goals ________________________________________________________

ldquoEverything that we do in afterschool we try to make it connected to the real worldhellipOne of our many goals is tomake it relevanthellipmake a connection for students to stayin school go to college a reason to be on a particular

career pathhelliprdquo - Afterschool Project Director

Overview

Setting clear goals and desired outcomes is a cornerstone

o aterschool program success (Bodilly and Beckett 2005)

Virtually all o the programs we studied had established

unambiguous goals and structured their programs to

meet those goals At the same time most programs alsorecognized the importance o considering student voices

when making decisions regarding program activities

and content hence many programs allowed students

to provide input especially in the arts and technology

programs As a result students were engaged and excited

to be in their aterschool programs

Key Evaluation Findings

The best programs had

bull clearly dened goals in a written plan

bull curricular design and specic practices aligned toprogram goals and

bull internal and external evaluations (or urther detailssee Component 5 o this report)

ldquoAchievementrdquo ocused programs oten district-aliated werebull usually more structured than other aterschool

program types

bull stressed improvement o academic perormance

bull hired more certied teachers and

bull maintained a more consistent linkage with the dayschool then less achievement-ocused programs

Goals o many high quality aterschool programs also

had a specic emphasis (eg science technology

homework support community involvement or the

arts)

Examples

The site coordinator at one math program described

program goals as ocused on developing studentsrsquo

academic skills within a specic content area ldquoOur

primary goals are to bring the student to grade levelrdquo

Similarly goals or three o the math programs and our

o the reading programs specically targeted students

who were struggling academically

A primary goal o many arts ocused aterschool

programs was to provide students who otherwisewould have little-to-no exposure to the arts with

quality art experiences Many interviewees reported

using an integrative approach to arts instruction that

could help students who were struggling academically

and personally

Similar to arts all science sta responded to questions

about curricular goals by suggesting a primary interest

in oering positive science experiences to students A

ew went urther to explain that their principle goal

was to improve standardized test scores in science

by ocusing on extending day school instruction into

aterschool

Interview data across seven homework-ocused

programs suggested that the primary goals in

aterschool homework were to complete homework

and increase academic achievement

Nearly all o the technology sta had the goal o

teaching students the mechanics o a broad range

o technology skills an interest in encouraging the

students to use those skills to enhance learning in other

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 615 Making Afterschool Programs Better | Denise Huang and Ron Dietel

content areas and teaching the students a technical skill

that is relevant to their real lives (both now and in the

uture)

In aligning activities to achieve their program goals nearly

all interviewees described the merits o developing a

curriculum that combined academic skill development with

opportunities to explore and encourage studentsrsquo social

development One program director summed up

ldquoOur goal is to help each child to make sure that itrsquos

an aterschool program thatrsquos un but at the sametime itrsquos enriching so they grow and learn every dayso they can take home more knowledgehelliprdquo

Component 2 Leadership ________________________________________________________

ldquoObviously we try to be democratichellipSo one of thethings we try to do here how we want to make thisa great place to work is in fnding great peoplethen giving the people the power and leeway and theaccountability but also the freedom to do what they

think works best and trusting themhelliprdquo

- Afterschool Project Director

Overview

Our studies ound that directors and managers o high

unctioning aterschool programs usually had many years

o experience in aterschool programs Further the leaders

nearly always shared decision-making with their sta across

aterschool goals programs and evaluation

Key Evaluation Findings

The best programs had leaders who

bull articulated a clear program mission vision statementand goals

bull decentralized decision making used a bottoms upleadership style

bull trusted in the knowledge and skills o sta regardingcurriculum and instruction

bull promoted a team culture o positive relationships

requent communication and sta problem-solvingskills

bull insured that instructors and students had adequatematerials and resources and

bull provided all sta with proessional developmentopportunities that improved individual and team skills

The majority o both site coordinators and instructors

said that aterschool instructors had an active voice in

decisions about curriculum and instruction (44 average

on a 50 point scale) and took active roles in program

leadership and decision-making (41 average)

Administrators consistently described the value o

starsquos content-specic skills and expertise as a result

curriculum development was a group process in which

leaders gave sta members a strong voice in designing

instruction

Leaders and sta across high-quality programs

maintained good relationships with the day school

personnel However ew o them had established

ormal communication systems or that purpose

0

10

20

30

40

50

60

70

80

90

100

Never Rarely Sometimes Frequently Regularly

P e r c e n t a g e o f I n s t r u c t o r s

2217 19

14

28

Figure 2 Frequency of communication between afterschool program and day-school staff

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 715 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

Figure 3 Educational levels of afterschool staff

1Currently in High School

19High School GraduateGED

9Associatersquos Degree

24Masterrsquos Degree

47Bachelorrsquos Degree

(as indicated in Figure 2) with less than 50 o

aterschool sta reporting that they had regular or

requent communication with day school teachers (see

Recommendations section)

Examples

One site coordinator expressed appreciation or the sta

by stating

ldquoThese are proessional adultshellipand they are the

best ones to implement the curriculumhellip Theyrsquoreright there with the studentshellipthey know whattheir levels are and their abilitiesrdquo

An arts program instructor explained

ldquoMy experience has been that the aterschoolteachers propose something that wersquod like to dowith the kids aterschool and [the director] then

just talks to us about what our plans are Wehellipreport to her in terms o lesson plans and how weincorporate standards and benchmarks but a lot oreedom is really given to us We teach what wersquorecomortable teaching and what wersquore passionateaboutrdquo

A project director at a technology program said that his

starsquos latitude was evenly balanced by a strong level o

accountability or their curricular choices

ldquoEvery quarter they have to come back to us andtell us how theyrsquore doing They report back asto what is going on at theirhellipprograms In termso actual decision-making and setting goals anddeciding what wersquore going to do thatrsquos much moreo a bottom-up processrdquo

Component 3 Staff _____________________________________

ldquoWe recognized that the other role we (staff) have to play is to get children engaged in the learning process So itrsquos not just about completing the homework but itrsquosabout finding ways outside of the school to get theminterested excited feeling confident and to build theirself-esteem so that they want to come back the next day

and try a little harderrdquo - Afterschool Staff Instructor

Overview We ound that high quality programs recruited qualied

sta and created collegial environments supporting their

programsrsquo missions Aterschool leaders were able to retain

sta and achieve lower turnover rates than other programs

because sta elt respected supported autonomous and

condent in their ability to reach their students In turn

sta and students constructed positive relationships with

each other characterized by warmth and mutual respect

Sta was oten role a model or students creating a norm

o high expectations appropriate student behavior good

school attendance eective work habits and positiveattitudes towards learning

Key Evaluation Findings

Sta rom quality programs generally had high

educational levels Forty-seven percent o all sta had

Bachelorrsquos degrees twenty-our percent had a Masterrsquos

degree (see Figure 3)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 815 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Aterschool sta in quality programs usually had

substantial aterschool experience Sixty-six percent

had more than three years o work experience in

aterschool twenty-three percent had six years or more

o experience

Program sta at high quality sites also tended to

have low turnover The majority o the sta had

been at their current site or three years or more (see

Recommendations section)Positive relationships and interactions between the sta

and the students were observed in virtually all high

quality programs particularly in terms o expectations

or student perormance disciplinary issues and

democracy

All interviewees reported having some orm o

technology-related proessional development available in

their aterschool program

Most proessional development opportunities were or

helping students with reading and math applying state

standards to the curriculum and connecting with theday school Fewer opportunities were oered on topics

o English learners special needs students evaluation

and assessment (see Recommendations section)

A higher percentage o site coordinators reported

receiving proessional development in most categories

(other than working with English language learners)

compared to instructors who reported receiving the

lowest percentage o proessional development in

program and sel-evaluation (see Recommendations

section)

Although sta generally ound proessional developmentuseul only 26 had regular (2-4) opportunities each

year About 50 o the homework program sta

reported that no proessional development was oered

to them (see Recommendations section)

Examples

Nearly all quality aterschool programs approached

decision-making in a democratic ashion One program

allowed students to oer input on where they would

like to conduct their service learning projects others

considered studentsrsquo activities choices and made great

eorts to include them in their program decision-making

Arts- science- and technology-ocused aterschool

programs tended to give students more autonomy

and input into programs compared to homework or

academic ocused programs One arts program sta

stated that student inputs were ldquoalways o great value

student interests had a vast impact on art curriculum

contentrdquo Consequently students took ownership in

their learning and remained engaged

Component 4 Staff ____________________________________________________

ldquoThese are kids who have never been exposed to

anythingmdashour goal is to open their minds to new

things and to show that they can do ithellipWe want a

well-balanced well-rounded program with a lot of

different things to offer to the childrenrdquo - Afterschool

Site Coordinator

Study ndings revealed curricular similarities and

dierences across all programs a majority o which

oered three or more activities each day Most

programs included homework help or tutoring but

other activities ranged rom academic (eg math

reading writing science) to enrichment (eg arts

and crats cooking gardening health and nutrition

cultural activities computers) and recreation (eg

sports dance drill team outdoor games) The

requency and duration o instruction oered by the

programs are provided in Table 1

These ndings suggest that students were receiving

adequate aterschool time or learning and skills

practice Observation reports across the programs also

indicated that students appeared to be mostly engaged

and attentive and enjoyed the activities Virtually all

programs provided substantial time or recreational

and crats activities keeping students engaged while

exercising other parts o their brains

Key Evaluation Findings

The majority o programs employed unique and

innovative strategies to engage students in theaterschool setting placing a particular emphasis on

making learning un

Technology programs reported the most requent

use o research-based practices whereas reading

programs reported the lowest requency

Technology science arts and homework programs

appeared to be more ocused on developing

higher order thinking skills whereas reading and

math programs were more ocused on direct skills

instruction

Most programs shared similar methods odisseminating inormation to parents as well as

a means o encouraging their involvement in the

aterschool programs

Parents were very satised with the programs both

in terms o positive changes in their childrenrsquos

behaviors and attitudes and in general program

unctioning They elt that the sta cared about and

respected their children They also reported that

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 915

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 4: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

QUALITY

AFTERSCHOOL

PROGRAM

EVALUATION

STAFF

LEADERSHIP

PROGRAM

GOALS

Figure 1 CRESST Afterschool Program Quality Model

Goals are clear rigorous and supported across the program in structure and content Funding

is adequate to support goals

Leadership is experienced well-educated has longevity at the current site uses eective

communications sets high expectations and has a bottoms-up management style

Sta is experienced has longevity at current program relates well to students models high

expectations motivates and engages students and works well with leaders colleagues and

parents

Program aligns to the day school provides time or students to study learn and practice

includes motivational activities requently uses technology science and the arts to support

youth development student learning and engagement

Evaluation uses both internal (ormative) and external (summative) methods Evaluative

inormation and data accurately measure goals results are applied to continuous program

improvement

1

2

3

4

5

These ve components work together to produce a high quality aterschool program (Figure 1)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 515 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

The remainder o this brie covers each o these ve

components First we provide common evaluation ndings

across most programs We then provide examples

either best practices or useul observations noted by

the CRESST team We conclude with a short list o

recommendations which could help make aterschool

programs even better

Component 1 Goals ________________________________________________________

ldquoEverything that we do in afterschool we try to make it connected to the real worldhellipOne of our many goals is tomake it relevanthellipmake a connection for students to stayin school go to college a reason to be on a particular

career pathhelliprdquo - Afterschool Project Director

Overview

Setting clear goals and desired outcomes is a cornerstone

o aterschool program success (Bodilly and Beckett 2005)

Virtually all o the programs we studied had established

unambiguous goals and structured their programs to

meet those goals At the same time most programs alsorecognized the importance o considering student voices

when making decisions regarding program activities

and content hence many programs allowed students

to provide input especially in the arts and technology

programs As a result students were engaged and excited

to be in their aterschool programs

Key Evaluation Findings

The best programs had

bull clearly dened goals in a written plan

bull curricular design and specic practices aligned toprogram goals and

bull internal and external evaluations (or urther detailssee Component 5 o this report)

ldquoAchievementrdquo ocused programs oten district-aliated werebull usually more structured than other aterschool

program types

bull stressed improvement o academic perormance

bull hired more certied teachers and

bull maintained a more consistent linkage with the dayschool then less achievement-ocused programs

Goals o many high quality aterschool programs also

had a specic emphasis (eg science technology

homework support community involvement or the

arts)

Examples

The site coordinator at one math program described

program goals as ocused on developing studentsrsquo

academic skills within a specic content area ldquoOur

primary goals are to bring the student to grade levelrdquo

Similarly goals or three o the math programs and our

o the reading programs specically targeted students

who were struggling academically

A primary goal o many arts ocused aterschool

programs was to provide students who otherwisewould have little-to-no exposure to the arts with

quality art experiences Many interviewees reported

using an integrative approach to arts instruction that

could help students who were struggling academically

and personally

Similar to arts all science sta responded to questions

about curricular goals by suggesting a primary interest

in oering positive science experiences to students A

ew went urther to explain that their principle goal

was to improve standardized test scores in science

by ocusing on extending day school instruction into

aterschool

Interview data across seven homework-ocused

programs suggested that the primary goals in

aterschool homework were to complete homework

and increase academic achievement

Nearly all o the technology sta had the goal o

teaching students the mechanics o a broad range

o technology skills an interest in encouraging the

students to use those skills to enhance learning in other

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 615 Making Afterschool Programs Better | Denise Huang and Ron Dietel

content areas and teaching the students a technical skill

that is relevant to their real lives (both now and in the

uture)

In aligning activities to achieve their program goals nearly

all interviewees described the merits o developing a

curriculum that combined academic skill development with

opportunities to explore and encourage studentsrsquo social

development One program director summed up

ldquoOur goal is to help each child to make sure that itrsquos

an aterschool program thatrsquos un but at the sametime itrsquos enriching so they grow and learn every dayso they can take home more knowledgehelliprdquo

Component 2 Leadership ________________________________________________________

ldquoObviously we try to be democratichellipSo one of thethings we try to do here how we want to make thisa great place to work is in fnding great peoplethen giving the people the power and leeway and theaccountability but also the freedom to do what they

think works best and trusting themhelliprdquo

- Afterschool Project Director

Overview

Our studies ound that directors and managers o high

unctioning aterschool programs usually had many years

o experience in aterschool programs Further the leaders

nearly always shared decision-making with their sta across

aterschool goals programs and evaluation

Key Evaluation Findings

The best programs had leaders who

bull articulated a clear program mission vision statementand goals

bull decentralized decision making used a bottoms upleadership style

bull trusted in the knowledge and skills o sta regardingcurriculum and instruction

bull promoted a team culture o positive relationships

requent communication and sta problem-solvingskills

bull insured that instructors and students had adequatematerials and resources and

bull provided all sta with proessional developmentopportunities that improved individual and team skills

The majority o both site coordinators and instructors

said that aterschool instructors had an active voice in

decisions about curriculum and instruction (44 average

on a 50 point scale) and took active roles in program

leadership and decision-making (41 average)

Administrators consistently described the value o

starsquos content-specic skills and expertise as a result

curriculum development was a group process in which

leaders gave sta members a strong voice in designing

instruction

Leaders and sta across high-quality programs

maintained good relationships with the day school

personnel However ew o them had established

ormal communication systems or that purpose

0

10

20

30

40

50

60

70

80

90

100

Never Rarely Sometimes Frequently Regularly

P e r c e n t a g e o f I n s t r u c t o r s

2217 19

14

28

Figure 2 Frequency of communication between afterschool program and day-school staff

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 715 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

Figure 3 Educational levels of afterschool staff

1Currently in High School

19High School GraduateGED

9Associatersquos Degree

24Masterrsquos Degree

47Bachelorrsquos Degree

(as indicated in Figure 2) with less than 50 o

aterschool sta reporting that they had regular or

requent communication with day school teachers (see

Recommendations section)

Examples

One site coordinator expressed appreciation or the sta

by stating

ldquoThese are proessional adultshellipand they are the

best ones to implement the curriculumhellip Theyrsquoreright there with the studentshellipthey know whattheir levels are and their abilitiesrdquo

An arts program instructor explained

ldquoMy experience has been that the aterschoolteachers propose something that wersquod like to dowith the kids aterschool and [the director] then

just talks to us about what our plans are Wehellipreport to her in terms o lesson plans and how weincorporate standards and benchmarks but a lot oreedom is really given to us We teach what wersquorecomortable teaching and what wersquore passionateaboutrdquo

A project director at a technology program said that his

starsquos latitude was evenly balanced by a strong level o

accountability or their curricular choices

ldquoEvery quarter they have to come back to us andtell us how theyrsquore doing They report back asto what is going on at theirhellipprograms In termso actual decision-making and setting goals anddeciding what wersquore going to do thatrsquos much moreo a bottom-up processrdquo

Component 3 Staff _____________________________________

ldquoWe recognized that the other role we (staff) have to play is to get children engaged in the learning process So itrsquos not just about completing the homework but itrsquosabout finding ways outside of the school to get theminterested excited feeling confident and to build theirself-esteem so that they want to come back the next day

and try a little harderrdquo - Afterschool Staff Instructor

Overview We ound that high quality programs recruited qualied

sta and created collegial environments supporting their

programsrsquo missions Aterschool leaders were able to retain

sta and achieve lower turnover rates than other programs

because sta elt respected supported autonomous and

condent in their ability to reach their students In turn

sta and students constructed positive relationships with

each other characterized by warmth and mutual respect

Sta was oten role a model or students creating a norm

o high expectations appropriate student behavior good

school attendance eective work habits and positiveattitudes towards learning

Key Evaluation Findings

Sta rom quality programs generally had high

educational levels Forty-seven percent o all sta had

Bachelorrsquos degrees twenty-our percent had a Masterrsquos

degree (see Figure 3)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 815 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Aterschool sta in quality programs usually had

substantial aterschool experience Sixty-six percent

had more than three years o work experience in

aterschool twenty-three percent had six years or more

o experience

Program sta at high quality sites also tended to

have low turnover The majority o the sta had

been at their current site or three years or more (see

Recommendations section)Positive relationships and interactions between the sta

and the students were observed in virtually all high

quality programs particularly in terms o expectations

or student perormance disciplinary issues and

democracy

All interviewees reported having some orm o

technology-related proessional development available in

their aterschool program

Most proessional development opportunities were or

helping students with reading and math applying state

standards to the curriculum and connecting with theday school Fewer opportunities were oered on topics

o English learners special needs students evaluation

and assessment (see Recommendations section)

A higher percentage o site coordinators reported

receiving proessional development in most categories

(other than working with English language learners)

compared to instructors who reported receiving the

lowest percentage o proessional development in

program and sel-evaluation (see Recommendations

section)

Although sta generally ound proessional developmentuseul only 26 had regular (2-4) opportunities each

year About 50 o the homework program sta

reported that no proessional development was oered

to them (see Recommendations section)

Examples

Nearly all quality aterschool programs approached

decision-making in a democratic ashion One program

allowed students to oer input on where they would

like to conduct their service learning projects others

considered studentsrsquo activities choices and made great

eorts to include them in their program decision-making

Arts- science- and technology-ocused aterschool

programs tended to give students more autonomy

and input into programs compared to homework or

academic ocused programs One arts program sta

stated that student inputs were ldquoalways o great value

student interests had a vast impact on art curriculum

contentrdquo Consequently students took ownership in

their learning and remained engaged

Component 4 Staff ____________________________________________________

ldquoThese are kids who have never been exposed to

anythingmdashour goal is to open their minds to new

things and to show that they can do ithellipWe want a

well-balanced well-rounded program with a lot of

different things to offer to the childrenrdquo - Afterschool

Site Coordinator

Study ndings revealed curricular similarities and

dierences across all programs a majority o which

oered three or more activities each day Most

programs included homework help or tutoring but

other activities ranged rom academic (eg math

reading writing science) to enrichment (eg arts

and crats cooking gardening health and nutrition

cultural activities computers) and recreation (eg

sports dance drill team outdoor games) The

requency and duration o instruction oered by the

programs are provided in Table 1

These ndings suggest that students were receiving

adequate aterschool time or learning and skills

practice Observation reports across the programs also

indicated that students appeared to be mostly engaged

and attentive and enjoyed the activities Virtually all

programs provided substantial time or recreational

and crats activities keeping students engaged while

exercising other parts o their brains

Key Evaluation Findings

The majority o programs employed unique and

innovative strategies to engage students in theaterschool setting placing a particular emphasis on

making learning un

Technology programs reported the most requent

use o research-based practices whereas reading

programs reported the lowest requency

Technology science arts and homework programs

appeared to be more ocused on developing

higher order thinking skills whereas reading and

math programs were more ocused on direct skills

instruction

Most programs shared similar methods odisseminating inormation to parents as well as

a means o encouraging their involvement in the

aterschool programs

Parents were very satised with the programs both

in terms o positive changes in their childrenrsquos

behaviors and attitudes and in general program

unctioning They elt that the sta cared about and

respected their children They also reported that

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 915

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 5: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 515 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

The remainder o this brie covers each o these ve

components First we provide common evaluation ndings

across most programs We then provide examples

either best practices or useul observations noted by

the CRESST team We conclude with a short list o

recommendations which could help make aterschool

programs even better

Component 1 Goals ________________________________________________________

ldquoEverything that we do in afterschool we try to make it connected to the real worldhellipOne of our many goals is tomake it relevanthellipmake a connection for students to stayin school go to college a reason to be on a particular

career pathhelliprdquo - Afterschool Project Director

Overview

Setting clear goals and desired outcomes is a cornerstone

o aterschool program success (Bodilly and Beckett 2005)

Virtually all o the programs we studied had established

unambiguous goals and structured their programs to

meet those goals At the same time most programs alsorecognized the importance o considering student voices

when making decisions regarding program activities

and content hence many programs allowed students

to provide input especially in the arts and technology

programs As a result students were engaged and excited

to be in their aterschool programs

Key Evaluation Findings

The best programs had

bull clearly dened goals in a written plan

bull curricular design and specic practices aligned toprogram goals and

bull internal and external evaluations (or urther detailssee Component 5 o this report)

ldquoAchievementrdquo ocused programs oten district-aliated werebull usually more structured than other aterschool

program types

bull stressed improvement o academic perormance

bull hired more certied teachers and

bull maintained a more consistent linkage with the dayschool then less achievement-ocused programs

Goals o many high quality aterschool programs also

had a specic emphasis (eg science technology

homework support community involvement or the

arts)

Examples

The site coordinator at one math program described

program goals as ocused on developing studentsrsquo

academic skills within a specic content area ldquoOur

primary goals are to bring the student to grade levelrdquo

Similarly goals or three o the math programs and our

o the reading programs specically targeted students

who were struggling academically

A primary goal o many arts ocused aterschool

programs was to provide students who otherwisewould have little-to-no exposure to the arts with

quality art experiences Many interviewees reported

using an integrative approach to arts instruction that

could help students who were struggling academically

and personally

Similar to arts all science sta responded to questions

about curricular goals by suggesting a primary interest

in oering positive science experiences to students A

ew went urther to explain that their principle goal

was to improve standardized test scores in science

by ocusing on extending day school instruction into

aterschool

Interview data across seven homework-ocused

programs suggested that the primary goals in

aterschool homework were to complete homework

and increase academic achievement

Nearly all o the technology sta had the goal o

teaching students the mechanics o a broad range

o technology skills an interest in encouraging the

students to use those skills to enhance learning in other

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 615 Making Afterschool Programs Better | Denise Huang and Ron Dietel

content areas and teaching the students a technical skill

that is relevant to their real lives (both now and in the

uture)

In aligning activities to achieve their program goals nearly

all interviewees described the merits o developing a

curriculum that combined academic skill development with

opportunities to explore and encourage studentsrsquo social

development One program director summed up

ldquoOur goal is to help each child to make sure that itrsquos

an aterschool program thatrsquos un but at the sametime itrsquos enriching so they grow and learn every dayso they can take home more knowledgehelliprdquo

Component 2 Leadership ________________________________________________________

ldquoObviously we try to be democratichellipSo one of thethings we try to do here how we want to make thisa great place to work is in fnding great peoplethen giving the people the power and leeway and theaccountability but also the freedom to do what they

think works best and trusting themhelliprdquo

- Afterschool Project Director

Overview

Our studies ound that directors and managers o high

unctioning aterschool programs usually had many years

o experience in aterschool programs Further the leaders

nearly always shared decision-making with their sta across

aterschool goals programs and evaluation

Key Evaluation Findings

The best programs had leaders who

bull articulated a clear program mission vision statementand goals

bull decentralized decision making used a bottoms upleadership style

bull trusted in the knowledge and skills o sta regardingcurriculum and instruction

bull promoted a team culture o positive relationships

requent communication and sta problem-solvingskills

bull insured that instructors and students had adequatematerials and resources and

bull provided all sta with proessional developmentopportunities that improved individual and team skills

The majority o both site coordinators and instructors

said that aterschool instructors had an active voice in

decisions about curriculum and instruction (44 average

on a 50 point scale) and took active roles in program

leadership and decision-making (41 average)

Administrators consistently described the value o

starsquos content-specic skills and expertise as a result

curriculum development was a group process in which

leaders gave sta members a strong voice in designing

instruction

Leaders and sta across high-quality programs

maintained good relationships with the day school

personnel However ew o them had established

ormal communication systems or that purpose

0

10

20

30

40

50

60

70

80

90

100

Never Rarely Sometimes Frequently Regularly

P e r c e n t a g e o f I n s t r u c t o r s

2217 19

14

28

Figure 2 Frequency of communication between afterschool program and day-school staff

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 715 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

Figure 3 Educational levels of afterschool staff

1Currently in High School

19High School GraduateGED

9Associatersquos Degree

24Masterrsquos Degree

47Bachelorrsquos Degree

(as indicated in Figure 2) with less than 50 o

aterschool sta reporting that they had regular or

requent communication with day school teachers (see

Recommendations section)

Examples

One site coordinator expressed appreciation or the sta

by stating

ldquoThese are proessional adultshellipand they are the

best ones to implement the curriculumhellip Theyrsquoreright there with the studentshellipthey know whattheir levels are and their abilitiesrdquo

An arts program instructor explained

ldquoMy experience has been that the aterschoolteachers propose something that wersquod like to dowith the kids aterschool and [the director] then

just talks to us about what our plans are Wehellipreport to her in terms o lesson plans and how weincorporate standards and benchmarks but a lot oreedom is really given to us We teach what wersquorecomortable teaching and what wersquore passionateaboutrdquo

A project director at a technology program said that his

starsquos latitude was evenly balanced by a strong level o

accountability or their curricular choices

ldquoEvery quarter they have to come back to us andtell us how theyrsquore doing They report back asto what is going on at theirhellipprograms In termso actual decision-making and setting goals anddeciding what wersquore going to do thatrsquos much moreo a bottom-up processrdquo

Component 3 Staff _____________________________________

ldquoWe recognized that the other role we (staff) have to play is to get children engaged in the learning process So itrsquos not just about completing the homework but itrsquosabout finding ways outside of the school to get theminterested excited feeling confident and to build theirself-esteem so that they want to come back the next day

and try a little harderrdquo - Afterschool Staff Instructor

Overview We ound that high quality programs recruited qualied

sta and created collegial environments supporting their

programsrsquo missions Aterschool leaders were able to retain

sta and achieve lower turnover rates than other programs

because sta elt respected supported autonomous and

condent in their ability to reach their students In turn

sta and students constructed positive relationships with

each other characterized by warmth and mutual respect

Sta was oten role a model or students creating a norm

o high expectations appropriate student behavior good

school attendance eective work habits and positiveattitudes towards learning

Key Evaluation Findings

Sta rom quality programs generally had high

educational levels Forty-seven percent o all sta had

Bachelorrsquos degrees twenty-our percent had a Masterrsquos

degree (see Figure 3)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 815 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Aterschool sta in quality programs usually had

substantial aterschool experience Sixty-six percent

had more than three years o work experience in

aterschool twenty-three percent had six years or more

o experience

Program sta at high quality sites also tended to

have low turnover The majority o the sta had

been at their current site or three years or more (see

Recommendations section)Positive relationships and interactions between the sta

and the students were observed in virtually all high

quality programs particularly in terms o expectations

or student perormance disciplinary issues and

democracy

All interviewees reported having some orm o

technology-related proessional development available in

their aterschool program

Most proessional development opportunities were or

helping students with reading and math applying state

standards to the curriculum and connecting with theday school Fewer opportunities were oered on topics

o English learners special needs students evaluation

and assessment (see Recommendations section)

A higher percentage o site coordinators reported

receiving proessional development in most categories

(other than working with English language learners)

compared to instructors who reported receiving the

lowest percentage o proessional development in

program and sel-evaluation (see Recommendations

section)

Although sta generally ound proessional developmentuseul only 26 had regular (2-4) opportunities each

year About 50 o the homework program sta

reported that no proessional development was oered

to them (see Recommendations section)

Examples

Nearly all quality aterschool programs approached

decision-making in a democratic ashion One program

allowed students to oer input on where they would

like to conduct their service learning projects others

considered studentsrsquo activities choices and made great

eorts to include them in their program decision-making

Arts- science- and technology-ocused aterschool

programs tended to give students more autonomy

and input into programs compared to homework or

academic ocused programs One arts program sta

stated that student inputs were ldquoalways o great value

student interests had a vast impact on art curriculum

contentrdquo Consequently students took ownership in

their learning and remained engaged

Component 4 Staff ____________________________________________________

ldquoThese are kids who have never been exposed to

anythingmdashour goal is to open their minds to new

things and to show that they can do ithellipWe want a

well-balanced well-rounded program with a lot of

different things to offer to the childrenrdquo - Afterschool

Site Coordinator

Study ndings revealed curricular similarities and

dierences across all programs a majority o which

oered three or more activities each day Most

programs included homework help or tutoring but

other activities ranged rom academic (eg math

reading writing science) to enrichment (eg arts

and crats cooking gardening health and nutrition

cultural activities computers) and recreation (eg

sports dance drill team outdoor games) The

requency and duration o instruction oered by the

programs are provided in Table 1

These ndings suggest that students were receiving

adequate aterschool time or learning and skills

practice Observation reports across the programs also

indicated that students appeared to be mostly engaged

and attentive and enjoyed the activities Virtually all

programs provided substantial time or recreational

and crats activities keeping students engaged while

exercising other parts o their brains

Key Evaluation Findings

The majority o programs employed unique and

innovative strategies to engage students in theaterschool setting placing a particular emphasis on

making learning un

Technology programs reported the most requent

use o research-based practices whereas reading

programs reported the lowest requency

Technology science arts and homework programs

appeared to be more ocused on developing

higher order thinking skills whereas reading and

math programs were more ocused on direct skills

instruction

Most programs shared similar methods odisseminating inormation to parents as well as

a means o encouraging their involvement in the

aterschool programs

Parents were very satised with the programs both

in terms o positive changes in their childrenrsquos

behaviors and attitudes and in general program

unctioning They elt that the sta cared about and

respected their children They also reported that

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 915

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 6: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 615 Making Afterschool Programs Better | Denise Huang and Ron Dietel

content areas and teaching the students a technical skill

that is relevant to their real lives (both now and in the

uture)

In aligning activities to achieve their program goals nearly

all interviewees described the merits o developing a

curriculum that combined academic skill development with

opportunities to explore and encourage studentsrsquo social

development One program director summed up

ldquoOur goal is to help each child to make sure that itrsquos

an aterschool program thatrsquos un but at the sametime itrsquos enriching so they grow and learn every dayso they can take home more knowledgehelliprdquo

Component 2 Leadership ________________________________________________________

ldquoObviously we try to be democratichellipSo one of thethings we try to do here how we want to make thisa great place to work is in fnding great peoplethen giving the people the power and leeway and theaccountability but also the freedom to do what they

think works best and trusting themhelliprdquo

- Afterschool Project Director

Overview

Our studies ound that directors and managers o high

unctioning aterschool programs usually had many years

o experience in aterschool programs Further the leaders

nearly always shared decision-making with their sta across

aterschool goals programs and evaluation

Key Evaluation Findings

The best programs had leaders who

bull articulated a clear program mission vision statementand goals

bull decentralized decision making used a bottoms upleadership style

bull trusted in the knowledge and skills o sta regardingcurriculum and instruction

bull promoted a team culture o positive relationships

requent communication and sta problem-solvingskills

bull insured that instructors and students had adequatematerials and resources and

bull provided all sta with proessional developmentopportunities that improved individual and team skills

The majority o both site coordinators and instructors

said that aterschool instructors had an active voice in

decisions about curriculum and instruction (44 average

on a 50 point scale) and took active roles in program

leadership and decision-making (41 average)

Administrators consistently described the value o

starsquos content-specic skills and expertise as a result

curriculum development was a group process in which

leaders gave sta members a strong voice in designing

instruction

Leaders and sta across high-quality programs

maintained good relationships with the day school

personnel However ew o them had established

ormal communication systems or that purpose

0

10

20

30

40

50

60

70

80

90

100

Never Rarely Sometimes Frequently Regularly

P e r c e n t a g e o f I n s t r u c t o r s

2217 19

14

28

Figure 2 Frequency of communication between afterschool program and day-school staff

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 715 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

Figure 3 Educational levels of afterschool staff

1Currently in High School

19High School GraduateGED

9Associatersquos Degree

24Masterrsquos Degree

47Bachelorrsquos Degree

(as indicated in Figure 2) with less than 50 o

aterschool sta reporting that they had regular or

requent communication with day school teachers (see

Recommendations section)

Examples

One site coordinator expressed appreciation or the sta

by stating

ldquoThese are proessional adultshellipand they are the

best ones to implement the curriculumhellip Theyrsquoreright there with the studentshellipthey know whattheir levels are and their abilitiesrdquo

An arts program instructor explained

ldquoMy experience has been that the aterschoolteachers propose something that wersquod like to dowith the kids aterschool and [the director] then

just talks to us about what our plans are Wehellipreport to her in terms o lesson plans and how weincorporate standards and benchmarks but a lot oreedom is really given to us We teach what wersquorecomortable teaching and what wersquore passionateaboutrdquo

A project director at a technology program said that his

starsquos latitude was evenly balanced by a strong level o

accountability or their curricular choices

ldquoEvery quarter they have to come back to us andtell us how theyrsquore doing They report back asto what is going on at theirhellipprograms In termso actual decision-making and setting goals anddeciding what wersquore going to do thatrsquos much moreo a bottom-up processrdquo

Component 3 Staff _____________________________________

ldquoWe recognized that the other role we (staff) have to play is to get children engaged in the learning process So itrsquos not just about completing the homework but itrsquosabout finding ways outside of the school to get theminterested excited feeling confident and to build theirself-esteem so that they want to come back the next day

and try a little harderrdquo - Afterschool Staff Instructor

Overview We ound that high quality programs recruited qualied

sta and created collegial environments supporting their

programsrsquo missions Aterschool leaders were able to retain

sta and achieve lower turnover rates than other programs

because sta elt respected supported autonomous and

condent in their ability to reach their students In turn

sta and students constructed positive relationships with

each other characterized by warmth and mutual respect

Sta was oten role a model or students creating a norm

o high expectations appropriate student behavior good

school attendance eective work habits and positiveattitudes towards learning

Key Evaluation Findings

Sta rom quality programs generally had high

educational levels Forty-seven percent o all sta had

Bachelorrsquos degrees twenty-our percent had a Masterrsquos

degree (see Figure 3)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 815 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Aterschool sta in quality programs usually had

substantial aterschool experience Sixty-six percent

had more than three years o work experience in

aterschool twenty-three percent had six years or more

o experience

Program sta at high quality sites also tended to

have low turnover The majority o the sta had

been at their current site or three years or more (see

Recommendations section)Positive relationships and interactions between the sta

and the students were observed in virtually all high

quality programs particularly in terms o expectations

or student perormance disciplinary issues and

democracy

All interviewees reported having some orm o

technology-related proessional development available in

their aterschool program

Most proessional development opportunities were or

helping students with reading and math applying state

standards to the curriculum and connecting with theday school Fewer opportunities were oered on topics

o English learners special needs students evaluation

and assessment (see Recommendations section)

A higher percentage o site coordinators reported

receiving proessional development in most categories

(other than working with English language learners)

compared to instructors who reported receiving the

lowest percentage o proessional development in

program and sel-evaluation (see Recommendations

section)

Although sta generally ound proessional developmentuseul only 26 had regular (2-4) opportunities each

year About 50 o the homework program sta

reported that no proessional development was oered

to them (see Recommendations section)

Examples

Nearly all quality aterschool programs approached

decision-making in a democratic ashion One program

allowed students to oer input on where they would

like to conduct their service learning projects others

considered studentsrsquo activities choices and made great

eorts to include them in their program decision-making

Arts- science- and technology-ocused aterschool

programs tended to give students more autonomy

and input into programs compared to homework or

academic ocused programs One arts program sta

stated that student inputs were ldquoalways o great value

student interests had a vast impact on art curriculum

contentrdquo Consequently students took ownership in

their learning and remained engaged

Component 4 Staff ____________________________________________________

ldquoThese are kids who have never been exposed to

anythingmdashour goal is to open their minds to new

things and to show that they can do ithellipWe want a

well-balanced well-rounded program with a lot of

different things to offer to the childrenrdquo - Afterschool

Site Coordinator

Study ndings revealed curricular similarities and

dierences across all programs a majority o which

oered three or more activities each day Most

programs included homework help or tutoring but

other activities ranged rom academic (eg math

reading writing science) to enrichment (eg arts

and crats cooking gardening health and nutrition

cultural activities computers) and recreation (eg

sports dance drill team outdoor games) The

requency and duration o instruction oered by the

programs are provided in Table 1

These ndings suggest that students were receiving

adequate aterschool time or learning and skills

practice Observation reports across the programs also

indicated that students appeared to be mostly engaged

and attentive and enjoyed the activities Virtually all

programs provided substantial time or recreational

and crats activities keeping students engaged while

exercising other parts o their brains

Key Evaluation Findings

The majority o programs employed unique and

innovative strategies to engage students in theaterschool setting placing a particular emphasis on

making learning un

Technology programs reported the most requent

use o research-based practices whereas reading

programs reported the lowest requency

Technology science arts and homework programs

appeared to be more ocused on developing

higher order thinking skills whereas reading and

math programs were more ocused on direct skills

instruction

Most programs shared similar methods odisseminating inormation to parents as well as

a means o encouraging their involvement in the

aterschool programs

Parents were very satised with the programs both

in terms o positive changes in their childrenrsquos

behaviors and attitudes and in general program

unctioning They elt that the sta cared about and

respected their children They also reported that

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 915

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 7: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 715 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

Figure 3 Educational levels of afterschool staff

1Currently in High School

19High School GraduateGED

9Associatersquos Degree

24Masterrsquos Degree

47Bachelorrsquos Degree

(as indicated in Figure 2) with less than 50 o

aterschool sta reporting that they had regular or

requent communication with day school teachers (see

Recommendations section)

Examples

One site coordinator expressed appreciation or the sta

by stating

ldquoThese are proessional adultshellipand they are the

best ones to implement the curriculumhellip Theyrsquoreright there with the studentshellipthey know whattheir levels are and their abilitiesrdquo

An arts program instructor explained

ldquoMy experience has been that the aterschoolteachers propose something that wersquod like to dowith the kids aterschool and [the director] then

just talks to us about what our plans are Wehellipreport to her in terms o lesson plans and how weincorporate standards and benchmarks but a lot oreedom is really given to us We teach what wersquorecomortable teaching and what wersquore passionateaboutrdquo

A project director at a technology program said that his

starsquos latitude was evenly balanced by a strong level o

accountability or their curricular choices

ldquoEvery quarter they have to come back to us andtell us how theyrsquore doing They report back asto what is going on at theirhellipprograms In termso actual decision-making and setting goals anddeciding what wersquore going to do thatrsquos much moreo a bottom-up processrdquo

Component 3 Staff _____________________________________

ldquoWe recognized that the other role we (staff) have to play is to get children engaged in the learning process So itrsquos not just about completing the homework but itrsquosabout finding ways outside of the school to get theminterested excited feeling confident and to build theirself-esteem so that they want to come back the next day

and try a little harderrdquo - Afterschool Staff Instructor

Overview We ound that high quality programs recruited qualied

sta and created collegial environments supporting their

programsrsquo missions Aterschool leaders were able to retain

sta and achieve lower turnover rates than other programs

because sta elt respected supported autonomous and

condent in their ability to reach their students In turn

sta and students constructed positive relationships with

each other characterized by warmth and mutual respect

Sta was oten role a model or students creating a norm

o high expectations appropriate student behavior good

school attendance eective work habits and positiveattitudes towards learning

Key Evaluation Findings

Sta rom quality programs generally had high

educational levels Forty-seven percent o all sta had

Bachelorrsquos degrees twenty-our percent had a Masterrsquos

degree (see Figure 3)

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 815 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Aterschool sta in quality programs usually had

substantial aterschool experience Sixty-six percent

had more than three years o work experience in

aterschool twenty-three percent had six years or more

o experience

Program sta at high quality sites also tended to

have low turnover The majority o the sta had

been at their current site or three years or more (see

Recommendations section)Positive relationships and interactions between the sta

and the students were observed in virtually all high

quality programs particularly in terms o expectations

or student perormance disciplinary issues and

democracy

All interviewees reported having some orm o

technology-related proessional development available in

their aterschool program

Most proessional development opportunities were or

helping students with reading and math applying state

standards to the curriculum and connecting with theday school Fewer opportunities were oered on topics

o English learners special needs students evaluation

and assessment (see Recommendations section)

A higher percentage o site coordinators reported

receiving proessional development in most categories

(other than working with English language learners)

compared to instructors who reported receiving the

lowest percentage o proessional development in

program and sel-evaluation (see Recommendations

section)

Although sta generally ound proessional developmentuseul only 26 had regular (2-4) opportunities each

year About 50 o the homework program sta

reported that no proessional development was oered

to them (see Recommendations section)

Examples

Nearly all quality aterschool programs approached

decision-making in a democratic ashion One program

allowed students to oer input on where they would

like to conduct their service learning projects others

considered studentsrsquo activities choices and made great

eorts to include them in their program decision-making

Arts- science- and technology-ocused aterschool

programs tended to give students more autonomy

and input into programs compared to homework or

academic ocused programs One arts program sta

stated that student inputs were ldquoalways o great value

student interests had a vast impact on art curriculum

contentrdquo Consequently students took ownership in

their learning and remained engaged

Component 4 Staff ____________________________________________________

ldquoThese are kids who have never been exposed to

anythingmdashour goal is to open their minds to new

things and to show that they can do ithellipWe want a

well-balanced well-rounded program with a lot of

different things to offer to the childrenrdquo - Afterschool

Site Coordinator

Study ndings revealed curricular similarities and

dierences across all programs a majority o which

oered three or more activities each day Most

programs included homework help or tutoring but

other activities ranged rom academic (eg math

reading writing science) to enrichment (eg arts

and crats cooking gardening health and nutrition

cultural activities computers) and recreation (eg

sports dance drill team outdoor games) The

requency and duration o instruction oered by the

programs are provided in Table 1

These ndings suggest that students were receiving

adequate aterschool time or learning and skills

practice Observation reports across the programs also

indicated that students appeared to be mostly engaged

and attentive and enjoyed the activities Virtually all

programs provided substantial time or recreational

and crats activities keeping students engaged while

exercising other parts o their brains

Key Evaluation Findings

The majority o programs employed unique and

innovative strategies to engage students in theaterschool setting placing a particular emphasis on

making learning un

Technology programs reported the most requent

use o research-based practices whereas reading

programs reported the lowest requency

Technology science arts and homework programs

appeared to be more ocused on developing

higher order thinking skills whereas reading and

math programs were more ocused on direct skills

instruction

Most programs shared similar methods odisseminating inormation to parents as well as

a means o encouraging their involvement in the

aterschool programs

Parents were very satised with the programs both

in terms o positive changes in their childrenrsquos

behaviors and attitudes and in general program

unctioning They elt that the sta cared about and

respected their children They also reported that

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 915

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 8: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 815 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Aterschool sta in quality programs usually had

substantial aterschool experience Sixty-six percent

had more than three years o work experience in

aterschool twenty-three percent had six years or more

o experience

Program sta at high quality sites also tended to

have low turnover The majority o the sta had

been at their current site or three years or more (see

Recommendations section)Positive relationships and interactions between the sta

and the students were observed in virtually all high

quality programs particularly in terms o expectations

or student perormance disciplinary issues and

democracy

All interviewees reported having some orm o

technology-related proessional development available in

their aterschool program

Most proessional development opportunities were or

helping students with reading and math applying state

standards to the curriculum and connecting with theday school Fewer opportunities were oered on topics

o English learners special needs students evaluation

and assessment (see Recommendations section)

A higher percentage o site coordinators reported

receiving proessional development in most categories

(other than working with English language learners)

compared to instructors who reported receiving the

lowest percentage o proessional development in

program and sel-evaluation (see Recommendations

section)

Although sta generally ound proessional developmentuseul only 26 had regular (2-4) opportunities each

year About 50 o the homework program sta

reported that no proessional development was oered

to them (see Recommendations section)

Examples

Nearly all quality aterschool programs approached

decision-making in a democratic ashion One program

allowed students to oer input on where they would

like to conduct their service learning projects others

considered studentsrsquo activities choices and made great

eorts to include them in their program decision-making

Arts- science- and technology-ocused aterschool

programs tended to give students more autonomy

and input into programs compared to homework or

academic ocused programs One arts program sta

stated that student inputs were ldquoalways o great value

student interests had a vast impact on art curriculum

contentrdquo Consequently students took ownership in

their learning and remained engaged

Component 4 Staff ____________________________________________________

ldquoThese are kids who have never been exposed to

anythingmdashour goal is to open their minds to new

things and to show that they can do ithellipWe want a

well-balanced well-rounded program with a lot of

different things to offer to the childrenrdquo - Afterschool

Site Coordinator

Study ndings revealed curricular similarities and

dierences across all programs a majority o which

oered three or more activities each day Most

programs included homework help or tutoring but

other activities ranged rom academic (eg math

reading writing science) to enrichment (eg arts

and crats cooking gardening health and nutrition

cultural activities computers) and recreation (eg

sports dance drill team outdoor games) The

requency and duration o instruction oered by the

programs are provided in Table 1

These ndings suggest that students were receiving

adequate aterschool time or learning and skills

practice Observation reports across the programs also

indicated that students appeared to be mostly engaged

and attentive and enjoyed the activities Virtually all

programs provided substantial time or recreational

and crats activities keeping students engaged while

exercising other parts o their brains

Key Evaluation Findings

The majority o programs employed unique and

innovative strategies to engage students in theaterschool setting placing a particular emphasis on

making learning un

Technology programs reported the most requent

use o research-based practices whereas reading

programs reported the lowest requency

Technology science arts and homework programs

appeared to be more ocused on developing

higher order thinking skills whereas reading and

math programs were more ocused on direct skills

instruction

Most programs shared similar methods odisseminating inormation to parents as well as

a means o encouraging their involvement in the

aterschool programs

Parents were very satised with the programs both

in terms o positive changes in their childrenrsquos

behaviors and attitudes and in general program

unctioning They elt that the sta cared about and

respected their children They also reported that

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 915

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 9: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 915

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 10: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1015 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Many sites provided character development The site

coordinator at a reading aterschool program described

two character programs that they oered students

The girls get ldquoSmart Girlsrdquo which is also characterbuilding but it deals with those lie changes duringpuberty taking care o your body in terms ohygiene and what does it mean to be a woman insociety For the boys itrsquos ldquoPassport to Manhoodrdquo

Another site coordinator or a math aterschool programmentioned a program called Character Developmentwhich ocused on teaching students values such ashonesty respect responsibility and caring

Community involvement was common in the programs

such as making get well cards or patients in the

hospitals taking trips to nursing homes to sing to the

senior citizens recycling or community beautication

eorts and cultivating community gardens One

aterschool program worked with a local charitable

organization not only to restore a public garden but

also to hike take horticulture classes go rowing and

swim in the lake

Community members oten volunteered in the ater-

school programs oten a result o aliations with local

universities and high schools who requently tutored

students Boy and girl scouts churches and boys

and girls club members also volunteered Aterschool

program sta requently invited science experts rom

the community to visit their programs and share their

own experiences o practical real-world applications or

science As one project director explained

ldquoWhat makes it unique [at our program] is we

have so much community involvement in teachingsciencehellipWersquove really tried to get experts in thefeld to come inhellipI donrsquot think that there is anyprogram that has more community involvement inteaching students science than oursrdquo

Component 5 Evaluation ________________________________________________________

When you look at their assessments on the [statetest]hellipthey werenrsquot measuring up with other statesand because of that we had to go back and revisit ourcurriculum to see where we were falling through the

cracks - Afterschool Program Principal

Overview

As outlined in the CRESST aterschool model ongoing

evaluation is necessary to measure program perormance

and make continuous improvement While evaluation data

serves many stakeholders including students parents and

aterschool stamdash unding agencies (who are making a

growing investment in aterschool programs) increasingly

want to know i their outlays are making measureable

improvements

We ound that nearly all aterschool programs used

internal evaluation to identiy program strengths and

weaknesses Internal evaluation oten called ormativeevaluation was usually done by the programrsquos own sta or

sta within its unding agencies governance structure The

stakes or consequences were usually low ndash with program

improvement the key goal An external evaluation

on the other hand typically had higher consequences

or programs and was nearly always conducted by a

disinterested third party In some cases accreditation or

even program continuance may be decided by an unbiased

outside ldquoexternalrdquo evaluation expert

Interview and survey responses across our studies indicated

that even though rigorous examination o data was rarenearly all o the aterschool programs conducted internal

or external evaluations o their programs

Key Evaluation Findings

All but two programs reported having conducted

internal evaluations Evaluation varied rom inormal

conversations between aterschool sta day school

sta and parents to a ormal administration o surveys

to students parents sta and tracking o test scores

grades and attendance records (see Recommendations

section)

Responses rom interviewees suggested that many o

the programs were evaluated externally sometimes

by an evaluation organization experienced in program

evaluation Interviewees consistently indicated that most

o the evaluations were o the entire program

External evaluation methodologies typically included

pre-post testing or classroom evaluations comparison

groups surveys ocus groups observational

assessments or a combination o methods

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 11: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1115

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 12: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1215 National Center for Research on Evaluation Standards amp Student Testing | Full Policy Brief No 11 | Summer 2011

ldquo

Recommendations ________________________________________________________________________________________________________________________

Our ndings point to the contributions o goals leadership sta programs and evaluation to high quality aterschool

programs Nevertheless we noticed consistent areas o weaknesses in even the best aterschool programs We oer the

ollowing research-based recommendations that we believe will help improve any aterschool program

Staff Stability

Our studyrsquos results indicated that high unctioning programs tend to have low sta turnover rates Over 60

o the sta had between 1ndash7 years o experience at the current site and over 30 o the sta had over 4 yearso experiences at the current site Sta stability is important or relationship building and a basis or students to

build trust positive attitudes and ecacy toward learning

Recommendation Funders and aterschool administrators should consider incentives or building sta retention rang-

ing rom educational opportunities (eg tuition grants) to ldquooutstanding aterschool teams or teachersrdquo nominated and

selected by parents teachers and students A pay scale incentive or years o service and a possible career ladder may also

improve sta retention

Collaboration with the Day School

In our studies all o the programs maintained positive relationships with the day school However despite

the importance o this relationship too ew programs had strategic systems established that supported and

strengthened this connection Recommendation Formal agreements and written plans (ideally in early stages) should address day school

collaboration Time or day school teachers and aterschool sta to meet and plan lessons together plus a communicationssystem (eg homework log between day school teachers and aterschool sta) should be included in both school andaterschool plans Funding agencies should budget additional resources or aterschool programs that will acilitate linkagessuch as shared proessional development sta retreats or workshops that jointly support students

Space and Technology

Many programs relied on access to common space such as an auditorium or a classroom shared with day

school teachers which oten caused logistical problems and sometimes prevented planned activities rom taking

place Furthermore some programs expressed diculty with not having consistent access to classrooms A

site coordinator illuminated the problem ldquoI would say physical space would be denitely a big thing with ourprogramhellipThat is probably one o the hardest things to work with just because every 6 weeks we are readjusting

the classroom to new classroom seating charts new areas in which the students can and cannot gordquo

Recommendation Aterschool space issues should be addressed early in the planning process and reviewed each

year or adequacy Technology too especially with shared equipment should also be addressed recognizing the growing role

that technology plays in both learning and recreational activities

ldquoFormal agreementsand written plans

ideally in early stagesshould address dayschool collaboration

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 13: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1315 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Professional Development

We ound that proessional development was not regularly oered at all program sites and when oered

participation rates were oten low Moreover project directors and site coordinators appeared to attend

conerences and workshops more requently than program sta

Recommendation Because proessional development has a strong correlation with sta ecacy instructional quality and

student outcomes (Ingvarson Meiers amp Beavis 2005) ederal and state policies guiding the development and operations o

aterschool programs should provide additional guidelines for sufcient quality professional development for all after -

school staff especially at the instructor level

Recommendation Aterschool programs should include proessional development in their written plansThose

plans should address unding issues or substitutes or example I there is a ldquotraining-o-the-trainerrdquo approach at the sites

written guidelines should include evaluation to ensure proessional development delity When planning the yearly calendar

proessional development or all sta should be included Topics should address program evaluation assessment and data use

plus needs o English learners and students with disabilities

Content and Curriculum

Our study ndings showed that although most program sta were aware o the standards within their specic

content areas they were less knowledgeable about the links between the standards and successul instructional

practices Proessional development can help address this need which is increasingly important with most statersquos

recent adoptions o the common core standards

Recommendation Proessional development should help all program sta expand their knowledge o content

standards and instructional methods or aligning those standards to instruction Program goals should include content and

curriculum enhancement or sta as a key purpose plus implementation and evaluation methods

Parent Involvement

Although the research literature continuously stresses the importance o parent involvement in infuencing

childrenrsquos academic outcomes our study repeatedly ound that parents though very satised with the programs

were generally not involved in attending events or volunteering in aterschool programs

Recommendation Despite the time constraints amilies ace aterschool leaders should continue to include parent in-

volvement as a central program goal oering specic late aternoon or early evening times or parent involved events as

well as using both social networks and websites to support positive communication between parents and sta Parents should

be included in an active program committee or evaluation team when possible thereby enhancing parent involvement and

contributions to the aterschool program Open houses and parent-teacher conerences can acilitate parent participation Home

visits and amily assistance can urther solidiy the relationships between the aterschool and its amilies

Evaluation

The contributions o eective internal and external evaluations must not be overlooked Equally important is the

use o the ndings or specic program improvements

Recommendation Federal state and local policies should address and provide unding or systematic evalua-

tion o all programs Evaluation should ideally include internal ormative evaluation as well as annual or biennial external

summative evaluations Multiple perspectives should be sought when gathering evaluation data including parent student andcommunity input Evaluation results should span accountability needs as well as guide continuous program improvement To be

eective all evaluations should be in written ormat

Recommendation For internal evaluation program directors and site coordinators need to clearly defne the pur-

poses o evaluations in their goals Sel-evaluation tools can be used to understand sta proessional development needs

sta utilization o research-based activities and sta knowledge o standards-based curriculum Using these evaluation results

program directors can implement changes allocate resources and design proessional development opportunities to urther

sta expertise in needed areas

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 14: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1415 Making Afterschool Programs Better | Denise Huang and Ron Dietel

Recommendation External evaluation should ocus on student outcomes ideally using student data rom the regular

day school level that includes both perormance and attendance inormation It is crucial that external evaluation results are

provided in a written ormat so that comparisons may be made over periods o time In order or evaluation data either internal

or external to be used eectively and lead to program improvement results must be well communicated to all stakeholders and

a system created or monitoring evaluation usage

Conclusion ___________________________________________________________________________________________________________________

Even at a time o austere ederal state and local unding ndash we highly encourage policy makers aterschool managers and

school districts to ully budget aterschool programs that will provide high quality leaders and sta Proessional development

and external evaluation should also be included in both budgets and program planning As one program director said

ldquoI think itrsquos very important that everybody understand the power that can come rom an aterschool program in aecting

change not only cultural change on campus but also individually in particular lives The sense o accomplishment that comes

rom being in one o our programs where they have more reedom to explore and have more hands-on experiences

is prooundrdquo

Resources

Links to Various CRESST Afterschool Evaluations

bull Examining Practices of Staff Recruitment and Retention in Four High-Functioning Afterschool ProgramsExtended Study rom the National Aterschool Partnership Report

bull Examining the Relationship between LArsquos BEST Program Attendance and Academic Achievement of LArsquos BEST Students

bull Identication of Key Indicators of Quality in Afterschool Programs

bull A Circle of Learning Children and Adults Growing Together in LArsquos BEST

bull What Works Common Practices in High Functioning Afterschool Programs Across the Nation in MathReading Science Arts Technology and Homework--A Study by the National Partnership

bull The Afterschool Hours Examining the Relationship between Afterschool Staff-Based Social Capital andStudent Engagement in LArsquos BEST

bull The Afterschool Experience in Salsa Sabor y Salud

bull Exploring the Relationships between LArsquos BEST Program Attendance and Cognitive Gains of LArsquos BESTStudents

bull Preparing Students for the 21st Century Exploring the Effect of Afterschool Participation on Studentsrsquo

Collaboration Skills Oral Communication Skills and Sel-Ecacy

bull Exploring the Intellectual Social and Organizational Capitals at LArsquos BEST

Link to the Afterschool Toolkit

bull SEDL - National Center for Quality Afterschool Afterschool Training Toolkit

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515

Page 15: huang_MAPB_v5.pdf

7272019 huang_MAPB_v5pdf

httpslidepdfcomreaderfullhuangmapbv5pdf 1515


Recommended