Migrant Education Program Evaluation and Program …mathematics enrichment activities, support...

2018-19

Prepared for: Office of English Learning and Migrant Education Indiana Department of Education 115 West Washington Street Indianapolis, IN 46204 (317) 232-0558

Prepared by: META Associates www.metaassociates.com (406) 855-2594

December 2019

Migrant Education

Program

Evaluation and Program

Improvement

GLOSSARY OF ACRONYMS

CNA Comprehensive Needs Assessment COE Certificate of Eligibility CSPR Consolidated State Performance Report ECE Early Childhood Education EDEN Education Data Exchange Network EL English Learner ELA English Language Arts ESEA Elementary and Secondary Education Act ESSA Every Student Succeeds Act of 2015 ESL English as a Second Language IEP Individual Education Plan IDOE Indiana Department of Education ID&R Identification and Recruitment ILEARN Indiana Learning Evaluation Assessment Readiness Network IMEP Indiana Migrant Education Program IMP Individual Migrant Education Plan IN Indiana ISTE International Society of Technology in Education LOA Local Operating Agency MEP Migrant Education Program MPO Measurable Program Outcome MRC Migrant Regional Center MSIX Migrant Student Information Exchange OME Office of Migrant Education OSY Out-of-School Youth PAC Parent Advisory Council PD Professional Development PFS Priority for Services PK Pre-Kindergarten QSI Quality of Strategy Implementation RSY Regular School Year SDP Service Delivery Plan SEA State Education Agency STEAM Science, Technology, Engineering, Art, and Mathematics STEM Science, Technology, Engineering, and Mathematics WIDA World-class Instructional Design and Assessment

TABLE OF CONTENTS

Page Executive Summary ................................................................................................................. 1 Methodology............................................................................................................................. 4 Evaluation Context................................................................................................................... 8

Program Description ............................................................................................................... 8

State Education Agency Monitoring Process .........................................................................11

Program Implementation and Support Services ...................................................................12 Implementation of Strategies Summary .................................................................................12

Student Services ...................................................................................................................16

Support Services ...................................................................................................................20

Professional Development .....................................................................................................21

Parent Involvement................................................................................................................22

Program Results .....................................................................................................................24 State Performance Indicators ................................................................................................24

Proficiency in English Language Arts ....................................................................................26

Proficiency in Mathematics ....................................................................................................27

School Readiness .................................................................................................................30

High School Graduation and Services for Secondary Youth ..................................................31

Implications and Recommendations .....................................................................................33 Progress Made on Previous Recommendations ....................................................................33

Recommendations for 2019-20..............................................................................................33

Appendix: Survey Comments

TABLE OF EXHIBITS Exhibit 1 Summary of MPO Results ........................................................................................... 1

Exhibit 2 Indiana MEP Regional Map ......................................................................................... 8

Exhibit 3 Total Eligible Migratory Children 2009-10 to 2017-18 .................................................. 9

Exhibit 4 Indiana Migratory Student Demographics ...................................................................10

Exhibit 5 Mean QSI Ratings of Strategy Implementation ...........................................................12

Exhibit 6 Participation in MEP Services During 2017-18 ...........................................................17

Exhibit 7 Migratory Students Served by Type of Service ...........................................................17

Exhibit 8 Staff Ratings of MEP Services ....................................................................................18

Exhibit 9 Parent Ratings of MEP Services .................................................................................19

Exhibit 10 Percent of Parents Indicating a Service as Most Useful ............................................20

Exhibit 11 Support Services Provided .......................................................................................21

Exhibit 12 Professional Development and Technical Assistance Provided ................................21

Exhibit 13 Parent Activities, PACs, and Meetings ......................................................................23

Exhibit 14 Parent Ratings of Materials and Information Provided at Parent Meetings ...............23

Exhibit 15 Proficiency Targets for Migratory Students through 2023 .........................................24

Exhibit 16 Baseline Proficiency on the ILEARN ELA Assessment for Migratory and Non-Migratory Students 2018-19 .....................................................................................25

Exhibit 17 Baseline Proficiency on the ILEARN Mathematics Assessment for Migratory and Non-Migratory Students 2018-19 .............................................................................25

Exhibit 18 Results on Summer Reading Assessments by Grade ..............................................26

Exhibit 19 Results on Summer Reading Assessments by PFS Status ......................................27

Exhibit 20 Results on Summer Mathematics Assessments by Grade ........................................27

Exhibit 21 Results on Summer Mathematics Assessments by PFS Status ................................28

Exhibit 22 Gains on the Digital Literacy Assessment by Grade .................................................29

Exhibit 23 Gains on the Digital Literacy Assessment by PFS Status .........................................30

Exhibit 24 Percent of Preschool Migratory Children Proficient on a School Readiness Assessment .............................................................................................................30

Exhibit 25 Percent of Preschool Migratory Children Proficient on a School Readiness Assessment .............................................................................................................31

Exhibit 26 Secondary Student Progress Toward Individual Learning Objectives .......................32

Exhibit 27 Results on OSY Consortium Life Skills Lessons .......................................................32

2018-19 Evaluation of the Indiana Migrant Education Program 1

Executive Summary This report presents documentation, results, conclusions, and recommendations on the delivery of Indiana Migrant Education Program (IMEP) services for the 2018-19 reporting period. The IMEP Service Delivery Plan (SDP) update was completed in spring 2019, and services in the summer 2019 program were implemented under the

new SDP. The 2018-19 regular school year was implemented under the previous SDP. Administered by the Indiana Department of Education (IDOE), the MEP assists schools in helping migratory children and youth who may be negatively impacted by frequent migration and interrupted schooling meet state expectations for achievement. Services are designed to facilitate continuity of instruction to students who migrate between Indiana and other states and within Indiana. During 2018-19, 1,782 migratory students (ages birth through 21) were identified in Indiana, a 5% decrease from the previous year. Of the 1,651 eligible migratory students ages 3-21, 95% participated in MEP services during the performance period. Services included tutoring/instructional support, summer school programs, reading and mathematics enrichment activities, support services, career education, and technology instruction. Regions also provided evidence-based activities to involve parents in the education of their children. During 2018-19, Indiana updated the statewide SDP. The plan was finalized in February 2019 and implemented beginning with summer 2019 programs. Most Indiana students are present only for the summer months, so the State chose to focus the summer 2019 implementation and results evaluation on the new strategies and MPOs. Progress made toward measurable program outcomes (MPOs) in the areas of English language arts (ELA) and mathematics, school readiness, and secondary/out-of-school youth (OSY) achievement is detailed in the Results section of this report. Exhibit 1 displays a summary of the progress toward the MPOs. Of note is that all six applicable MPOs (100%) were met in 2018-19. MPO 3B addressing the provision of services to preschool students will be reported in 2019-20.

Exhibit 1 Summary of MPO Results

English Language Arts (ELA) MPO met?

Evidence

1A) By the end of the 2018-19 reporting period, migratory students in grades K-8 receiving MEP instructional services in ELA for 30 sessions or three weeks of summer programming will achieve a statistically significant gain (p<.05) on a curriculum-based assessment.

Yes 78% of students made gains at the .001 level


Mathematics MPO met?

Evidence

2A) By the end of the 2018-19 reporting period, migratory students in grades K-8 receiving MEP instructional services in math for 30 sessions or three weeks of summer programming will achieve a statistically significant gain (p<.05) on a curriculum-based assessment.

Yes 83% of students made gains at the .001 level

2B) By the end of the 2018-19 reporting period, 75% of K-8 migratory students participating in at least three weeks of MEP summer programming will demonstrate proficiency or grow by 5% on an IMEP Digital Literacy Assessment aligned to International Society of Technology in Education (ISTE) standards.

Yes 87% gained or were proficient

School Readiness MPO met?

Evidence

3A) By the end of the 2018-19 reporting period, 75% of migratory children ages 3-5 who are not proficient on the IMEP Kindergarten School Readiness Checklist will receive site-based or home-based services in a MEP-funded regular school year or summer program for ten sessions and will make one level progress on three skills.

Yes 100% gained 1 or more levels

3B) By the end of the 2019-20 reporting period, 70% of migratory preschool students ages 3-5 will receive instructional services from existing preschool programs or MEP-funded programs.

N/A To be reported in 2019-20

High School Graduation and Services for Secondary Youth MPO met?

Evidence

4A) By the end of the 2018-19 reporting period, 80% of secondary-aged migratory students enrolled in MEP supplemental instructional services for five sessions in RSY or SSY will meet objectives set in their Individual Migrant Education Plan (IMP), which may include academic skills, life skills, English skills, college and career readiness, alternative programming, or other activities

Yes 100% met objectives

4B) By the end of the 2018-19 reporting period, 75% of migratory students enrolled in the MEP for one session and assessed with OSY Consortium materials will score 80% or higher.

Yes 100% scored 80% or higher

Other key findings/trends revealed in the 2018-19 evaluation follow.

• Inter/intrastate collaboration resulted in enhanced services to migratory students. Local projects collaborated with community and school programs; agencies serving farmworkers; community agencies that provide food, clothing, and shelter; and adult education programs.

• All three regions completed the Quality of Strategy Implementation (QSI) rubric to determine the level of implementation of each of the strategies in the Indiana MEP SDP. Eight of the nine strategies (89%) were rated at the “succeeding” (considered proficient) or “exceeding” levels, and the mean rating for all strategies combined was 4.1 out of 5.0.

• There are substantial gaps in the percentage of migratory and non-migratory students scoring proficient or advanced on 2019 State ELA and Mathematics Assessments, with non-migratory students outperforming migratory students in every grade level.

Findings of the 2018-19 evaluation show that the Indiana MEP made substantial progress toward meeting its MPOs and implementing high quality programming designed to ameliorate the effects of mobility on student learning and achievement. The Indiana MEP offered


individualized, needs-based, student-centered services to migratory students/youth that improved their learning and academic skills and helped them earn high school credits. In addition, parents were provided services to improve their skills and increase their involvement in their child’s education; MEP staff were trained to better serve the unique needs of migratory students/youth and their parents; and community resources and programs helped support migratory students/youth by providing instructional and support services.


Methodology In 1966, Congress included language in the Elementary and Secondary Education Act (ESEA) to help the children of migratory farmworkers and established the Office of Migrant Education (OME). The ESEA was reauthorized as the Every Student Succeeds Act (ESSA) of 2015. Currently, state MEPs provide supplemental instruction and support

services to children of migratory workers and fishers. These programs must comply with federal mandates as specified in Title I, Part C of the ESSA. The goal of state MEPs is to design programs to help migratory children overcome educational disruption, cultural and language barriers, social isolation, health-related problems, and other factors inhibiting them from doing well in school and making the transition to postsecondary education or employment [Section 1301(5)]. A migratory child is defined as a child or youth, from birth to age 21, who made a qualifying move in the preceding 36 months as a migratory agricultural worker or migratory fisher; or with, or to join, a parent or spouse who is a migratory agricultural worker or migratory fisher [Section 1309(3)(A)–(B)]. The ESEA requires Title I, Part C recipients to provide comprehensive services through the coordination of and collaboration with locally and federally funded programs. Supplementary MEP funds must be used to meet the identified needs of migratory children as well as meet the intent and purpose of the MEP. These migrant funds must supplement and not supplant other local and state funding. Indiana has established high academic standards for all students and holds the Indiana public education system accountable for providing all students with a high-quality education that enables them to achieve their full potential. The Indiana standards support Title I, Part C, section 1301 of the ESEA for the Education of Migratory Children to ensure that migratory children have the opportunity to meet the same challenging State academic standards that all children are expected to meet. States are required to evaluate the effectiveness of the MEP and to provide guidance to their local projects on how to conduct local evaluations. In its most recent Non-Regulatory Guidance (March 2017), OME indicates that evaluations allow state education agencies (SEAs) and local operating agencies (LOAs) to:

1. determine whether the program is effective and document its impact on migratory children;

2. improve program planning by comparing the effectiveness of different types of interventions;

3. determine the degree to which projects are implemented as planned and identify problems that are encountered in program implementation; and

4. identify areas in which children may need different MEP services.


To achieve these results, OME requires that SEAs conduct an evaluation that examines both program implementation and performance results. Program implementation evaluation is recommended every two or three years, and progress toward the performance results (State Performance Indicators 1 and 5 and MPOs) is required to be reported annually. In evaluating program implementation, Indiana identified the following implementation evaluation questions in the SDP.

► What was the average gain on ELA and math assessments for disaggregated groups (regular school year [RSY] and summer, priority for services [PFS]and non-PFS, grade level, etc.)?

► What percentage of identified K-8 migratory students received ELA and mathematics instruction for 30 sessions or three weeks of summer programming?

► What percentage of migratory students were assessed with the Digital Literacy Assessment by Region?

► What was the average gain on the Digital Literacy Assessment for disaggregated groups (RSY and summer, PFS and non-PFS, grade level, etc.)?

► What percentage of migratory children were assessed with the IMEP Kindergarten School Readiness Checklist?

► How many parents participated in parent education programs? ► How many opportunities for parent education were provided by region? ► What materials did parents receive? ► What training did parents receive? ► What goals were included in IMPs? ► Which model (school-based, home-based, or technology-assisted model) was employed

by region? ► What percentage of migratory students met their IMP objectives disaggregated by group

(PFS and non-PFS, model used, etc.)? ► What instructional services were provided to migratory OSY? ► What percentage of OSY received a certificate of high school completion? ► What percentage of migratory OSY were assessed with OSY Consortium materials?

In looking at performance results, OME requires that a program’s actual performance be compared to “measurable outcomes established by the MEP and state’s performance targets, particularly for those students who have priority for service.” The following results evaluation questions address the MPOs as listed in the SDP.

► What was the average gain on ELA and mathematics assessments of students receiving 30 sessions or three weeks of summer programming? Was the average gain statistically significant (p<.05)?

► What percentage of students demonstrated proficiency or 5% growth between the pre- and post-tests on the Digital Literacy Assessment?

► What percentage of migratory children are proficient on the IMEP Kindergarten School Readiness Checklist?


► What percentage of migratory preschool children participated in community-based programs?

► What percentage of migratory preschool children attended a migrant-funded early childhood education program?

► What percentage of migratory students enrolled in 5 or more sessions met their objectives set on their IMP?

► What percentage of migratory students that were assessed with OSY Consortium Materials scored an 80% or higher?

Sources of data for this evaluation report include observations by MEP staff; mobility, participation, and demographic data from the migrant database; a summary of participation and outcome data collected from reporting forms completed by MEP staff; surveys completed by MEP staff, migratory parents, and secondary-aged migratory students; the Education Data Exchange Network (EDEN) report for 2018-29, and student assessment results. The goals of the evaluation are to:

► review services to ensure that they were implemented as intended; ► document the success of services for program validation; ► analyze information to identify the strengths of services and make recommendations on

the areas in which improvement is needed; and ► report the results of the evaluation to IDOE staff to disseminate to policy makers and

decision makers within the State. This evaluation report follows the guidance found in the OME Evaluation Toolkit (August 2012) with particular emphasis on the revised checklist for written reports. The following items from the checklist are identified within this report:

► The state MEP documents the evaluation in a written report (34 CFR Section 200.84). ► The state MEP provides specific implementation results that demonstrate the level of

fidelity in the implementation of regular year and summer/intersession activities and services contained within the SDP (34 CFR Section 200.84).

o Source: the completed QSI rubric. ► The state MEP provides performance results data for students having PFS and other

migratory students compared to all other students and the state’s performance targets (34 CFR Section 200.84).

o Source: migrant database disaggregation, director reports ► The state MEP provides implications and recommendations for improvement of services,

based upon implementation results and performance results data (34 CFR Section 200.85).

► The state MEP provides a full evaluation report every two to three years and progress on MPOs annually.

► The state performs an annual performance and results evaluation in order to inform SEA decision-making.


► Upon the results of the full evaluation, the state describes specific changes to the SDP and services that were made based upon the evaluation of implementation results and performance results (34 CFR 200.85).

This evaluation report provides summary information on the accomplishments made by students, MEP staff, and migratory parents in Indiana. These accomplishments were reviewed in light of the MPOs outlined by the State MEP in its SDP and carried out through local program applications and services. An external evaluation firm, META Associates, was contracted to help ensure objectivity in evaluating the IMEP, to examine the effectiveness of services, and to make recommendations to help the State improve the quality of the services provided to Indiana migratory students. To evaluate the services, the external evaluators and/or project staff were responsible for:

► maintaining and reviewing interview records, logs, attendance sign-in sheets, meeting notes, and other anecdotal evaluation tools;

► conducting onsite visits including focus group interviews and classroom observations; ► reviewing student achievement data and other outcomes; and ► preparing an evaluation report to provide information about the extent to which program

processes such as migratory student identification and recruitment (ID&R), the comprehensive needs assessment, professional development, and the activities described in the Indiana SDP were implemented as planned. Student outcomes and achievement related to content and performance standards also are included in the annual report.


Evaluation Context Program Description

To fulfill its mission, the IMEP: 1) recruits migratory students eligible for the program; 2) provides supplemental instructional and support services designed to help students

participate fully in their education and meet state performance outcomes; 3) coordinates among LOAs, community service organizations, and businesses to assist migratory families; 4) coordinates with parents and community members on the design, development, delivery, and evaluation of migrant programs; and 5) collaborates with other states, national agencies, and organizations to ensure continuity of instruction for migratory students. Instructional and support services needs for migratory students were identified through the Comprehensive Needs Assessment (CNA) completed in July 2018. The IMEP used the CNA to design the SDP outlining how the program would meet the identified needs. Included in the SDP, is the plan for evaluation that outlines how progress toward the implementation and outcomes of migrant education services are determined. The 2018-19 reporting period is the first year in which the updated CNA and SDP served as a guide for the measurement of program implementation and performance outcomes. The MEP is a state-operated program whereby funds go directly to the SEA. The state then sub-allocates to LOAs. Indiana funds three regional Education Service Centers (see Exhibit 2). Each region is responsible for identifying and serving all migratory students in the counties for that region. Under the previous CNA and SDP, there were six funded regions, but the state chose to consolidate administration of MEP services in 2018-19 due to decreased funding. Regions were chosen based on geography and past performance. Statewide, 1,782 migratory children birth through age 21 were identified during 2018-19. Exhibit 3 provides a longitudinal snapshot of the total number of children and youth identified from birth through age 21 over the past 10 years. The 2018-19 count of migratory students is a decrease of 7% from the previous year and a decrease of 21% from the high point two years ago in 2016-17. The average count of eligible students over the previous three years is 1,988. Changes in

Exhibit 2 Indiana MEP Regional Map


agriculture and immigration concerns have impacted migratory families in Indiana as they have in other areas of the country.

Exhibit 3 Total Eligible Migratory Children 2009-10 to 2018-19

Source: Consolidated State Performance Report (CSPR) Part II, 2009-10 through 2018-19

Each Migrant Regional Center (MRC) participating in the IMEP is required to maintain a PFS form for each eligible migratory child being served by the MRC. PFS is documented on the PFS form, and MRCs use PFS status to determine which migratory children should receive services first. Definition of PFS (Title I, Part C, Section 1304 (d)): PRIORITY FOR SERVICES.—In providing services with funds received under this part, each recipient of such funds shall give priority to migratory children who have made a qualifying move within the previous 1-year period and who— (1) are failing, or most at risk of failing, to meet the challenging State academic standards; or (2) have dropped out of school. A “qualifying move within the previous 1-year period” means a child must have a QAD in the previous 1-year period, as evidenced by an Indiana certificate of eligibility (COE). IMEP cannot accept another State’s COE without making its own eligibility determination. In addition, one of the following risk factors must be true:

• Student has dropped out of school • Student is an OSY • Student is classified as an English learner as defined by the most recent WIDA

Screener, Kindergarten W-APT, or WIDA ACCESS annual assessment

15991451

1126 1087

1777 1741

21222265

19171782

0

500

1000

1500

2000

2500

2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19


• Student did not pass an Indiana State Assessment • Student was retained • Student's grade placement is not age appropriate • Student failed at least one section of a state assessment in another state • Student is behind on accruing credits towards graduation requirements • Student has a current Individual Education Plan (IEP) on file • Student is below grade level on a locally administered assessment (DIBELS, Preschool

Screener, Acuity, etc.) • Student has grades indicating below average performance in math, language arts,

sciences or social studies: • Student has missed 10 or more days of school

Exhibit 4 displays the number and percentage of migrant-eligible students identified with PFS, English learners (EL), and children with disabilities in 2018-19. Of the 1,651 eligible migratory children ages 3 through 21, 876 (53%) were identified as having PFS, 852 (52%) were classified as EL, and 92 (6%) were identified with special education needs. OSY had the largest proportion of youth identified as PFS at 91%. Other grade/age levels ranged from 35% of 6th graders to 55% of kindergarteners, not including ungraded of which there was only one student.

Exhibit 4 Indiana Migratory Student Demographics

Grade #

Students PFS EL IDEA

# % # % # % Age 3-5 198 104 53% 58 29% 7 4%

K 100 55 55% 56 56% 3 3% 1 93 47 51% 59 63% 9 10% 2 106 48 45% 65 61% 6 6% 3 107 48 45% 66 62% 14 13% 4 102 48 47% 67 66% 8 8% 5 92 38 41% 59 64% 6 7% 6 99 35 35% 52 53% 7 7% 7 87 37 43% 52 60% 12 14% 8 93 34 37% 47 51% 7 8% 9 79 38 48% 38 48% 2 3% 10 81 32 40% 30 37% 2 2% 11 90 40 44% 31 34% 4 4% 12 44 18 41% 14 32% 5 11%

Ungraded 1 0 0% 0 0% 0 0% OSY 279 254 91% 158 57% 0 0% Total 1,651 876 53% 852 52% 92 6%

Source: 2018-19 Indiana Migrant Data Check Sheet


State Education Agency Monitoring Process

Monitoring of local MEPs is the responsibility of the IDOE. This includes both the compliance monitoring process as well as the follow-up and ongoing technical assistance that supports project implementation and student achievement. The State’s monitoring plan includes a two-year cycle where each LOA is monitored onsite. Sites that are not receiving an onsite visit in a given year participate in desktop monitoring where the project application, budget, data reporting, and compliance are reviewed. Monitoring outside of the scheduled cycle may be arranged at an agency’s request or as needed if an LOA is at risk of a serious or chronic compliance problem or has unresolved issues. In the spring, LOAs scheduled to be monitored the following year are notified. All project directors receive training and technical assistance through annual application walkthroughs facilitated by the program director and/or designated staff. Monitoring staff, including IDOE staff and any monitoring contractors, identify dates and confirm the schedule with directors prior to onsite visits. An official notification letter is mailed to the director of each LOA identified for monitoring for the upcoming school year. This letter includes information on how to prepare for the visit. Monitoring teams, consisting of IDOE staff or designees, conduct the onsite reviews. The team visits the schools and sites providing services and interviews regional staff. The size of the monitoring team varies depending on the size of the LOA. In some instances, the State may determine that a follow-up monitoring visit is necessary to verify implementation of an LOA’s actions resulting from its plan for correction. The MEP On-Site Monitoring Indicators Tool will be updated during the systems alignment process to match the new strategies and MPOs. Monitoring tools are on file with IDOE.


Program Implementation and Support Services This section provides a description of the instructional and support services provided by MEPs across Indiana as well as staff and parent suggestions. The implementation of the services was examined for effectiveness using the QSI rubric.

Implementation of Strategies Summary

The summary below describes the statewide conclusions, recommendations, and implementation ratings for each of the strategies in the SDP. Ratings were based on a 5-point scale with “1” indicating a strategy that was applicable, but there was no evidence of implementation, a “2” indicates limited evidence of implementation, a “3” indicates some evidence of implementation with room for improvement, a “4” indicates sufficient evidence of implementation aligned with what was described in the SDP, and a “5” indicates evidence of implementation exceeds expectations described in the SDP. Ratings of “3” or below indicate areas where the MEP could improve strategy implementation.

MEP staff and regional coordinators/directors met to review records and document observations to arrive at ratings of strategy implementation. In summer of 2019, all regions used the QSI to rate the extent to which the strategies in the Indiana SDP were implemented (note that strategies were employed for the first time in the summer of 2019). Regional MEP staff worked in teams to discuss how each of the strategies were implemented, arrive at consensus on the level of implementation of each strategy, and identify evidence used to determine ratings.

All three funded regions rated all nine strategies. Mean ratings of the strategies implemented ranged from 3.3 to 4.7. Eight of the nine strategies (89%) had mean ratings at the “succeeding” level of 4.0 or higher. The one strategy below the “succeeding” level (Strategy 4.2) addressed the provision of services to OSY. The recommendations section contains further discussion about this strategy. The full QSI reports from each region are on file with IDOE.

Exhibit 5 displays ratings for each strategy and the number of regions implementing the strategy. Note that even though a particular strategy may have a high mean rating statewide, some regions may still have room for improvement in that area.

Exhibit 5 Mean QSI Ratings of Strategy Implementation

Strategy Number of Regions where Strategy was Implemented

Mean Rating

English Language Arts 1.1) Provide evidence-based ELA instructional services for migratory students in forms that may include tutoring, summer programming, educational resources, technology-based instruction, and self-paced materials in either site-based or home-based instructional programs. Include supports for students’ English language development as needed.

3 4.3

1.2) Provide equitable access for students to innovative technologies that support ELA instruction and develop 21st century skills.

3 4.3


Strategy Number of Regions where Strategy was Implemented

Mean Rating

Mathematics 2.1) Provide evidence-based math instructional services for migratory students in forms that may include tutoring, summer programming, educational resources, technology-based instruction, and self-paced materials through either site-based or home-based instructional programs. Include supports for students’ English language development as needed.

3 4.3

2.2) Provide equitable access for students to innovative technologies that support math instruction and develop 21st century skills. 3 4.7

School Readiness 3.1) Provide instructional and support services using evidence-based strategies within site-based or home-based early childhood education (ECE) programs. 3 4.3

3.2) Coordinate with programs fostering developmental skills aligned with the Early Childhood Foundations and State expectations to assist migratory children with participation.

3 4.0

3.3) During the regular school year or summer, provide preschool materials, training, and information to parents so they are more prepared to support their children’s growth in school readiness skills.

3 4.0

Graduation and Services for Secondary-aged Youth 4.1 Provide and/or coordinate opportunities for secondary-aged migratory students to set and meet graduation and college/career readiness goals with evidence-based supplemental instruction in a school-based, home-based, or technology-assisted model.

3 4.0

4.2 Provide OSY opportunities for vocational training, high school equivalency programming, credit accrual through evening classes, instruction with flexible scheduling, online classes, alternative programs, and English language instruction.

3 3.3

Source: QSI ratings In addition to overall rating of implementation, regions described activities and evidence of implementation on the QSI. This information provides more specific detail about how each strategy was implemented. If an MPO was not being met, the state could examine this in detail to determine which activities were effective. All MPOs were met in 2018-19, and the following pages provide more information about how that was accomplished. Strategy 1.1: All regions maintain evidence and perform activities related to the following:

• Participation of students, PFS status, EL status, and attendance logs • Results of assessments for students receiving services • Summer school calendars • Pre/post assessment results • Staff schedules • Summer school classroom daily schedules • Evidence-based instructional materials and strategies used • Staff training sign-in sheets, agendas, and notes • Parent surveys • Communication with parents/guardians (flyers, announcements, contact logs)

Other evidence and activities implemented by one or two regions included:


• How ELA instruction is targeted to needs • Progress monitoring • Individual learning plans or individual migrant plans

Strategy 1.2: All regions maintain evidence and perform activities related to the following:

• Participation of students, PFS status, and attendance logs • Staff training sign-in sheets, agendas, and notes • Staff schedules • Evidence-based instructional materials/strategies used


• Individual learning plans Strategy 2.1: All regions maintain evidence and perform activities related to the following:

• Participation of students, PFS status, EL status, and attendance logs • Results of assessments for students receiving services • Summer school calendars • Pre/post assessment results • Summer school classroom daily schedules • Evidence-based instructional materials and strategies used • Staff training sign-in sheets, agendas, and notes • Parent surveys


• Individual learning plans • How math instruction is targeted to needs • Staff schedules • Progress monitoring • Communication with parents/guardians (flyers, announcements, contact logs)


• Participation of students, PFS status, and attendance logs • Staff training sign-in sheets, agendas, and notes • Results of Digital Literacy Assessments • Evidence-based instructional materials and strategies used


• Individual learning plans • Staff schedules



• Participation records • Summer school classroom daily schedules • School readiness checklist results • Summer school calendars • Evidence base for instructional materials and strategies used


• Staff schedules • Staff training sign-in sheets, agendas, and notes


• Communication with local services providers Other evidence and activities implemented by one or two regions included:

• Sign-in sheets for meetings • Enrollment documents • Transportation documentation


• Evidence base for instructional materials and strategies used • Communication with parents/guardians (flyers, announcements, contact logs) • Parent meeting sign-in sheets, agendas, and notes


• Parent surveys • Participation records


• Individual learning plans • Participation of students, PFS status, and attendance logs • Credit earned • Evidence of interstate and intrastate coordination • Transportation documentation


• Staff schedules and assignments • Evidence of joint planning • Staff training sign-in sheets, agendas, and notes • Certificates of completion or achievement • Evidence-based instructional materials and strategies used • Progress monitoring with pre/post assessment results


• Communication (i.e., logs) with partners Strategy 4.2: All regions maintain evidence and perform activities related to the following:

• Participation of students, PFS status, and attendance logs • Evidence-based instructional materials and strategies used • Progress monitoring with pre/post assessment results • Evidence of interstate and intrastate coordination


• Communication (i.e., logs) with partners • Staff Schedules and assignments • Transportation documentation • Individual learning plans • Certificates of completion or achievement

Additional information on the aggregate implementation of instructional services, support services, professional development, and parent involvement follows. Student Services

Indiana has placed an emphasis on providing services to all migratory children and youth residing in the State. Student services include instructional services provided by teachers and paraprofessionals in various settings such as in-class tutoring, after school programs, and summer school. High school graduation services include credit accrual, identifying and working toward individual learning goals, and postsecondary preparation and planning. Support services include health and nutrition services, medical and dental services, transportation, advocacy, and other services that help migratory students participate fully in their education. Exhibit 6 shows that 1,576 migratory children and youth received services in 2018-19 (regular year and summer) representing 95% of the eligible migratory students ages 3 through 21 . The percentage served represents a substantial effort across the State to provide student services. Services were defined as either an instructional or support service provided for at least one day between September 1, 2018 and August 31, 2019. Services were provided to 90% or more of the students identified in each grade level (except ungraded, where only one student was identified).

This program is special in what it provides for migratory students and their families. The one thing that stood out to me this summer was the constant striving for improvement and looking for ways to better serve these families. I’m proud to be a part of it. I truly believe providing the students with experiences not only in the classroom but around the community is the most valuable aspect of this program and should definitely be continued. -MEP staff member


Exhibit 6

Participation in MEP Services During 2018-19

Grade

Number Eligible

# Migratory Students Served

# % Age 3-5 198 194 98%

K 100 99 99% 1 93 84 90% 2 106 101 95% 3 107 97 91% 4 102 95 93% 5 92 87 95% 6 99 92 93% 7 87 84 97% 8 93 87 94% 9 79 76 96%

10 81 76 94% 11 90 86 96% 12 44 41 93%

Ungraded 1 0 0% OSY 279 277 99% Total 1,651 1,576 95%

Source: 2018-19 Indiana Migrant Data Check Sheet

Exhibit 7 shows that substantial percentages of migratory students participated in instructional services including reading, math, and high school credit accrual. During the reporting period, 63% of eligible migratory students received an instructional service. Seventy-one percent (71%) of eligible students participated in summer services, 39% received reading instruction, and 38% received math instruction. Eleven percent (11%) of high school students received high school credit accrual services. Percentages served were similar across the grade levels, except at the high school level where reading and math services were lower. Often high school students receive instruction across multiple subjects that may not fit distinctly into a “reading” or “math” instructional service.

Exhibit 7 Migratory Students Served by Type of Service

Grade Number Eligible

Summer Participants

Any Instruction

Reading Instruction

Math Instruction

High School Credit Accrual

# % # % # % # % # % Age 3-5 198 169 85% 132 67% 70 35% 69 35%

K 100 66 66% 67 67% 56 56% 56 56% 1 93 66 71% 59 63% 47 51% 45 48% 2 106 71 67% 75 71% 64 60% 64 60% 3 107 78 73% 76 71% 66 62% 64 60% 4 102 73 72% 71 70% 53 52% 53 52% 5 92 68 74% 59 64% 48 52% 47 51% 6 99 69 70% 70 71% 53 54% 53 54% 7 87 57 66% 56 64% 42 48% 42 48% 8 93 65 70% 63 68% 47 51% 47 51% 9 79 66 84% 50 63% 25 32% 22 28% 12 15% 10 81 56 69% 47 58% 24 30% 24 30% 8 10%



Summer Participants

Any Instruction

Reading Instruction

Math Instruction

High School Credit Accrual

# % # % # % # % # % 11 90 69 77% 50 56% 21 23% 18 20% 10 11% 12 44 13 30% 18 41% 9 20% 9 20% 2 5%

Ungraded 1 0 0% 0 0% 0 0% 0 0% OSY 279 188 67% 145 52% 24 9% 17 6% Total 1,651 1,174 71% 1,038 63% 649 39% 630 38% 32 11%*

Source: 2017-18 Indiana Migrant Data Check Sheet *Percent of total number of students grades 9-12: 294. As a component of the implementation evaluation, MEP staff completing a survey rated the extent to which training prepared them to implement MEP services including technology integration, reading, math, school readiness, and secondary student instruction strategies. Staff rated training components on a 4-point scale. Ratings on the staff survey were high for all items and ranged from 3.8 to 3.9 out of 4.0. Note that the number of staff responding is a duplicated count as staff completed a survey for each training they attended. No staff member responded “not at all.” Exhibit 8 shows the ratings assigned by staff regarding key aspects that influence the MEP’s effectiveness in addressing the academic needs of students.

Exhibit 8 Staff Ratings of MEP Services

Extent to which… N Not at

all Very little Somewhat Very much Mean

The training/conference helped prepare you to effectively implement MEP services

110 0 (0%) 2 (2%) 18 (16%) 90 (82%) 3.8

Training for technology integration strategies will be useful for meeting migratory children’s needs.

107 0 (0%) 1 (1%) 19 (18%) 87 (81%) 3.8

Training for reading instruction strategies will be useful for meeting migratory children’s needs.

97 0 (0%) 1 (1%) 16 (16%) 80 (82%) 3.8

Training for math instruction strategies will be useful for meeting migratory children’s needs.

87 0 (0%) 1 (1%) 19 (22%) 67 (77%) 3.8

Training for school readiness strategies will be useful for meeting migratory children’s needs.

96 0 (0%) 0 (0%) 12 (13%) 84 (88%) 3.9

Training for secondary student instruction will be useful for meeting migratory youth’s needs.

76 0 (0%) 1 (1%) 14 (18%) 61 (80%) 3.8

Source: Staff Survey Summer 2019 In response to an open-ended question on the Staff Survey, MEP educators described the ways in which they would use the information from the training in their classrooms. Staff comments fell into the following seven categories:


• All of the information presented would be useful.

• Technology tools and lessons, especially iPads, Spheros, and robotics.

• Project-based learning strategies to improve science, technology, engineering, math (STEM) skills.

• Strategies for teaching ELs. • Reading strategies and assessment

ideas. • School readiness lessons. • Other benefits such as networking and mindfulness exercises.

In the interest of continuous improvement, staff were asked to provide suggestions for program improvement. There were some common themes as outlined below.

• No suggestions right now or continue existing programs and services. • Improve communication and collaboration among schools, regions, migratory families,

and local communities. • More STEM activities. • More strategies for EL instruction and assessment. • More strategies for providing instruction to secondary students. • Consistent training and professional development for ALL staff.

Additional comments from staff and administrators are provided in the Appendix. Parents responding to the parent survey rated the quality of MEP services. The vast majority of the parents responding (80%) rated MEP services as “excellent.” On the 4-point scale, IMEP services received a mean rating of 3.8. Exhibit 9 displays the distribution of parent responses.

Exhibit 9 Parent Ratings of MEP Services

N Poor Fair Good Excellent Mean Please rate the overall quality of the migrant program services. 120 0 (0%) 3 (3%) 21 (17%) 96 (80%) 3.8

Source: Parent Survey Summer 2019 On the survey, parents described how the MEP helped their children succeed in school. Common trends reported by parents follow. Comments representative of multiple respondents are included in the Appendix.

• They learned a lot/helped progress in school and studies. • They learned a lot, especially English. • They get to continue their education in the summer rather than just doing nothing. • They learned more in the areas where they were weak.


• Keeping up their reading and math skills. • School supplies. • Summer school. • Helped with reading. • Dental exam. • They are more prepared for kindergarten.

In the interest of continuous improvement, parents were asked to describe any suggestions that they might have for the program. Many parents had no suggestions or suggested continuing the program as it was. Parents who provided suggestions had various comments, but comments that represented more than one parent included improving food in the summer school and having the summer program last longer. Parents were asked to identify the instructional and support services they found most useful on the parent survey. As displayed in Exhibit 10, the largest percentage of parents chose summer school as the most beneficial. The next grouping included transportation, dental services, and health referrals.

Exhibit 10 Percentage of Parents Indicating a Service as Most Useful

Source: Parent survey, summer 2019 Support Services

Support services were provided to migratory students to eliminate barriers that may inhibit academic success. Focused on leveraging existing services during both the summer and regular year program, support services were aimed at ensuring migratory students were able to participate in their own education. Of the 1,651 migratory students (ages 3 through 21) identified, 95% received support services during 2018-19, and 25% received counseling services. These percentages were similar to previous years. The percentage of students receiving support services was similar across the grade levels and ranged from 90% of first graders to 99% of kindergarteners and OSY. The percentages of students receiving counseling

10%14%

18%19%

24%28%

30%30%

32%36%

43%46%

49%77%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

ReferralsMental health

High school creditsPreschool

Food bankReading program

VisionTechnology

ClothingTutors/ mentors

HealthDental

TransportationSummer School


services varied somewhat across grade levels ranging from 14% of fifth graders to 48% of ninth graders. Exhibit 11 displays the support services provided.

Exhibit 11 Support Services Provided


Any Support Service

Counseling Services

# % # % Age 3-5 198 194 98% 29 15%

K 100 99 99% 17 17% 1 93 84 90% 16 17% 2 106 101 95% 20 19% 3 107 97 91% 18 17% 4 102 95 93% 22 22% 5 92 87 95% 13 14% 6 99 92 93% 26 26% 7 87 84 97% 24 28% 8 93 87 94% 32 34% 9 79 76 96% 38 48%

10 81 76 94% 34 42% 11 90 86 96% 35 39% 12 44 41 93% 19 43%

Ungraded 1 0 0% 0 0% OSY 279 276 99% 69 25% Total 1,651 1,575 95% 412 25%

Source: 2018-19 Indiana Migrant Data Check Sheet Professional Development

MEP staff in Indiana receive training to help them better provide instructional and support services to migratory students. Professional development takes many forms including statewide conferences and institutes; local training; workshops and webinars; technical assistance; mentoring and demonstration teaching; and attending meetings and conferences. During the summer of 2019, staff that worked with migratory students were provided opportunities to learn more about students’ unique needs and culture, as well as the impact of mobility experienced by the migratory students they serve. Key State training that closely aligned with the SDP included the statewide technology institute training (STEM Summit) and summer school training. Indiana staff and administrators participated in seven local or statewide professional development and technical assistance sessions the spring and summer of 2019. Professional development was provided to a duplicated count of 140 educators. Exhibit 12 displays the dates, region, and focus of the professional development sessions.

Exhibit 12 Professional Development and Technical Assistance Provided


Date Region Topic Number of Staff

5/15-16/19 Statewide Teacher STEM Summit 29 5/29/19 2 Summer School Training All Region 2 36 5/30/19 2 Summer School Training Logansport 8 6/7/19 1 Summer School Training Huntington 10 6/10/19 3 Summer School Training 11 6/14/19 1 Summer School Training South Bend 30 6/17/19 3 Summer School Training Rock Creek 16

Total (duplicated count) 140 Source: Meeting Agendas, Sign-in Sheets, and Staff Surveys Following trainings, staff were asked to provide feedback and suggestions for future training. Many indicated that they would like to see continued training on the topics covered in 2019. The two most common suggestions were additional training on various technology tools and strategies to use with English learners. Comments regarding additional training that are representative of several staff responses follow.

• Continuing to look at tech opportunities and best practices for classroom integration. • I think training focused on EL support would be beneficial. • No specific suggestions. Anything that aids in us helping our migratory students get

caught up to their educational level required for their age groups. • Career pathways options. • Situational training, for instance if a student were to have behavioral issues or get sick in

class. What would be the best methods to take care of the situation. • Training specifically for people who are new to migrant education.

Parent Involvement

The IMEP views parents as partners with the schools in the education of their children. Through the interaction that occurs between parents and migrant advocates, there appears to be strong communication and trust, allowing a close working relationship with staff of the local MEP. The IMEP requires that local projects operating in the regular term consult with parents in the planning, review, and improvement of the local MEP in an organized, ongoing, and timely way. Each local MEP sponsors parent development, family events for sharing information and resources, and culminating activities such as end-of-year programs featuring children’s educational success in which parents are invited to participate and bring their whole family. Examples of effective topics and formats for PACs include:

• PAC meetings • Outreach • School year program information • Parenting education • College information • Summer program information


During the 2018-19 reporting period, there were 17 local parent activities, parent advisory council (PAC) meetings, and other parent meetings held across the three regions. A total of 301 migratory parents and family members (duplicated count) attended these activities. Exhibit 13 displays a list of the meetings held.

Exhibit 13 Parent Activities, PACs, and Meetings

Date Region Topic of meeting Number of Parents

9/25/18 1 Community Resources, MEP Introduction, Health and Education, Positive Parenting, Family Engagement 15

11/13/18 1 MEP Introduction, Family STEM Engagement, Parenting Partners 19

12/11/18 1 Freckle, Immigration Information, STEM Activity 25 1/15/19 1 Mental Health 19 4/17/19 1 MEP Introduction, Family STEM Kits 7 10/16/18 1 PAC Meeting, College Financial Aid 19 3/14/19 1 PAC Meeting, Parent Presentations 13 7/16/19 1 PAC Meeting, Parent Agenda Items 8 7/23/19 1 PAC Meeting, Summer School Feedback 22 7/11/19 2 Parent-lead Meeting, Classroom Visits, Community

Resources 14

6/27/19 2 Logansport PAC, Summer School Information 5 7/18/19 2 PAC Meeting 25 6/17/19 2 PAC Meeting 14 6/19/19 3 Parent Meeting 8 7/19/19 3 Parent Meeting 52 6/17/19 3 Parent Meeting 16 7/18/19 3 Parent Meeting 20

Total (duplicated count) 301 Source: Meeting Agendas and Sign-in Sheets

Following meetings, parents completed a survey about the materials and information they received. Parents rated the extent to which they found various types of information and materials useful on a 4-point scale. Exhibit 14 shows high mean ratings that ranging from 3.7 to 3.8.

Exhibit 14 Parent Ratings of Materials and Information Provided at Parent Meetings

N Not at

all Very little Somewhat Very much Mean

If you received materials or information about helping your child with school, how useful were they?

110 3 (3%) 1 (1%) 22 (20%) 84 (76%) 3.7

If you received information or materials for helping your preschool child, how useful were they?

79 1 (1%) 2 (3%) 12 (15%) 64 (81%) 3.8

If you received materials or information for your high school student, how useful were they?

54 0 (0%) 1 (2%) 16 (30%) 37 (69%) 3.7

Source: Parent Survey


Program Results State Performance Indicators

During 2018-19, the academic achievement in reading and mathematics of students attending public school in Indiana was assessed through Indiana’s Learning Evaluation

Assessment Readiness Network (ILEARN) Assessment for grades 3-8 in ELA and Mathematics, replacing the previous assessment, the ISTEP+. In the Indiana ESSA Consolidated State Plan, signed November 19, 2019, the IDOE outlines long term accountability goals and interim performance measures. ILEARN assessment results from 2018-19 serve as the baseline for all students and migratory students. Migratory students are not included as one of the identified subgroups for accountability in the State Plan, so the IMEP has adopted the interim achievement targets for all students while recognizing that these targets are set using baselines substantially higher than the performance of migratory students. The following table displays the 2018-19 baseline results for migratory students and the interim performance targets set for all students through 2026.

Exhibit 15 Proficiency Targets for Migratory Students through 2026

Interim Targets for All Students

Subject

Baseline Migratory Student Proficiency 2019 2020 2021 2022 2023 2024 2025 2026

ELA 19.6 51.6 55.4 59.1 62.8 66.6 70.3 74.0 Mathematics 22.4 51.5 55.3 59.0 62.7 66.5 70.2 73.9

Source: ESSA State Plan, November 2019 State assessment results should be interpreted with caution as the group that took ILEARN assessments represents a small subset of the overall migratory student population in Indiana. The majority of Indiana migratory students are resident in the summer and early fall, outside of the state assessment window. However, the State will continue to monitor and report outcomes for those that are resident in the State for the testing window. Because of recent increases in the number of migratory students identified during the regular school year resulting in more students assessed on state assessments, Indiana has moved from a “small state” designation to a “medium state” designation for the purposes of reporting state assessment results and progress toward the MPOs. The State will need to put in place procedures for reporting state assessment results disaggregated PFS status for the next performance period. Exhibit 16 displays the passing rate of migratory and non-migratory students by grade level on the 2019 ILEARN ELA Assessment. Migratory students are passing the ELA assessment at a rate 27% below non-migratory students. Not shown in the table is that this is a 1% reduction in the gap compared to the previous year.


Exhibit 16 Baseline Proficiency on the ILEARN ELA Assessment for

Migratory and Non-Migratory Students 2018-19

Grade

Migratory Non-migratory Gap between Migratory and Non-migratory

# Assessed

% Passing

# Assessed

% Passing

3 47 19% 75,194 45% -26% 4 50 14% 76,418 45% -31% 5 50 24% 78,636 47% -23% 6 53 25% 78,578 47% -22% 7 37 11% 77,524 48% -37% 8 43 19% 76,315 49% -30%

Total 280 20% 462,665 47% -27% Source: CSPR, Part I 2018-19

Exhibit 17 displays the passing rate of migratory and non-migratory students by grade level on the ILEARN Mathematics Assessment from the spring of 2019. Migratory students are passing the mathematics assessment at a rate 25% below non-migratory students. Not shown in the table is that this gap remained the same compared to the previous year.

Exhibit 17 Baseline Proficiency on the ILEARN Mathematics Assessment for

Migratory and Non-Migratory Students 2018-19

Grade

Migratory Non-migratory Gap between Migratory and Non-migratory

# Assessed

% Passing

# Assessed

% Passing

3 49 33% 75,480 58% -25% 4 55 27% 76,656 53% -26% 5 57 28% 78,852 47% -19% 6 60 22% 78,766 46% -24% 7 41 10% 77,752 41% -31% 8 51 12% 76,490 37% -25%

Total 313 22% 463,996 47% -25% Source: CSPR, Part I 2018-19

Twenty-two of the 25 migratory seniors enrolled in 2018-19 graduated from high school. Note that because Indiana has a small, highly mobile migratory population during the regular year (number of seniors is less than 30), only the number of graduates is reported.


Proficiency in English Language Arts

MPO 1A) By the end of the 2018-19 reporting period, migratory students in grades K-8 receiving MEP instructional services in English Language Arts for 30 sessions or three weeks of summer programming will achieve a statistically significant gain (p<.05) on a curriculum-based assessment.

The three Indiana regional MEP projects implemented the strategy related to the provision of ELA instruction beginning in the summer 2019. Regions implemented various assessments depending on availability and teacher experience with different assessment methods. Assessments used included Freckle, myON, EARS Fluency Rubric, and MobyMax. Of the 228 students assessed with both a pre-assessment and a post-assessment, 178 (78%) increased their scores on the post-assessment. Gains were significant at the p<.001 (two-tailed) level, meeting the MPO. By grade level the significance is not calculated due to low N. Exhibit 18 shows the percentage of migratory students gaining varied somewhat by grade from 62% in the fifth grade to 86% in preschool and kindergarten. The percentage of migratory students with PFS making gains was somewhat higher than the percentage of non-PFS students at 82% and 74% respectively. Both gains were statistically significant.

Exhibit 18 Results on Summer Reading Assessments by Grade

Grade #

Assessed Mean Pre

Mean Post Gain

# (%) Gaining

Sig. (two tailed)

MPO met?

PK-K 36 2.60 4.21 +1.61 31 (86%) 1 17 2.88 4.04 +1.16 12 (71%) 2 28 3.89 4.57 +0.68 21 (75%) 3 37 5.61 8.33 +2.72 31 (84%) 4 33 6.12 7.38 +1.26 28 (85%) 5 21 4.71 5.05 +0.34 13 (62%) 6 29 4.87 6.08 +1.21 21 (72%) 7-12 27 6.95 8.09 +1.14 21 (72%)

Total 228 4.77 6.14 +1.37 178 (78%) <.001 Yes PFS 112 4.97 6.27 +1.30 92 (82%) <.001 Yes

Non-PFS 116 4.58 6.01 +1.43 86 (74%) <.01 Yes Source: Reading and Math Scores Tracking Form


Exhibit 19 Percent Gaining on Summer Reading Assessments Chart

Source: Reading and Math Scores Tracking Form

Proficiency in Mathematics

MPO 2A) By the end of the 2018-19 reporting period, migratory students in grades K-8 receiving MEP instructional services in math for 30 sessions or three weeks of summer programming will achieve a statistically significant gain (p<.05) on a curriculum-based assessment.

The three Indiana regional MEP projects implemented the strategy related to the provision of math instruction beginning in the summer 2019. Regions implemented various assessments depending on availability and teacher experience with different assessment methods. Assessments used included Moving with Math, Process Standards Rubric, and MobyMax. Of the 228 students assessed with both a pre-assessment and a post-assessment, 188 (83%) increased their scores on the post-assessment. Gains were significant at the p<.001 (two-tailed) level, meeting the MPO. By grade level the significance is not calculated due to low N. Exhibit 20 shows the percentage of migratory students gaining varied somewhat by grade from 71% in the fifth grade to 89% in the second grade and grades 7-12. The percentage of migratory students with PFS making gains was somewhat higher than the percentage of non-PFS students at 88% and 78% respectively. Both gains were statistically significant.

Exhibit 20 Results on Summer Mathematics Assessments by Grade

Grade #

Assessed Mean Pre

Mean Post Gain

# (%) Gaining

Sig. (two tailed)

MPO met?

PK-K 36 2.98 4.04 +1.06 31 (86%) 1 17 1.97 3.04 +1.07 15 (88%) 2 28 2.21 3.13 +0.92 25 (89%) 3 37 3.16 4.20 +1.04 28 (76%) 4 33 3.27 4.24 +0.97 25 (76%) 5 21 2.77 3.24 +0.47 15 (71%) 6 29 2.79 3.66 +0.87 25 (86%) 7-12 27 3.02 4.12 +1.10 24 (89%)

Total 228 2.83 3.78 +0.95 188 (83%) <.001 Yes

86

71 7584 85

6272 72

8274 78

0

20

40

60

80

100

PK-K 1 2 3 4 5 6 7-12 PFS Non-PFS Total


Grade #

Assessed Mean Pre

Mean Post Gain

# (%) Gaining

Sig. (two tailed)

MPO met?

PFS 112 3.10 4.12 +1.02 98 (88%) <.001 Yes Non-PFS 116 2.58 3.45 +0.87 90 (78%) <.001 Yes


Exhibit 21 Percent Gaining on Summer Mathematics Assessments Chart


MPO 2B) By the end of the 2018-19 reporting period, 75% of K-8 migratory students participating in at least three weeks of MEP summer programming will demonstrate proficiency or grow by 5% on an IMEP Digital Literacy Assessment aligned to ISTE standards.

The IMEP has developed a science, technology, engineering, art, and math (STEAM) curriculum specifically for use in summer schools by migratory students. The curriculum is designed to fit the context in which the MEP operates and address migratory student needs (such as learning English) as identified in the CNA. The curriculum uses tools such as Ollies, Spheros, GoPro cameras, drones, 3D printers, and iPads to engage students and give them experience with the latest technologies. Units in the curriculum are built for grade bands (K-1, 2-3, 4-5, and 6-12) and are cross-curricular addressing literacy, science, and math standards. The instructional materials are available at https://sites.google.com/imep.k12.in.us/imepstem/home. A sample of titles includes:

• Build a House • Build a Student-Led Business • Tynkering Sentences • College Choice • Create a PSA • Storytelling • Electronics • Remote Ecology Sampling • Healthy Eating

86 88 8976 76 71

86 89 8878 83

0

20

40

60

80

100

https://sites.google.com/imep.k12.in.us/imepstem/home


Teachers used a rubric aligned to the International Society for Technology in Education (ISTE) standards to assess students’ progress. The rubric, called the Digital Literacy Assessment, was used after the initial unit and again after the final unit. Student projects were assessed across six domains: 1) creativity and innovation; 2) communication and collaboration; 3) research and information fluency; 4) critical thinking, problem solving, and decision making; 5) digital citizenship; and 6) technology operations and concepts. Scores across the domains were added for a composite score for the pre- and post-assessments. Overall, 180 students with both a pre- and post-assessment gained 14%, and 87% of students gained 5% or more or scored proficient, meeting the MPO. By grade level, all grades met the MPO individually except preschool students. Exhibit 22 displays mean pre- and post-assessment scores, mean gains, and the number and percent gaining by grade. Of the 20 migratory students with PFS, 60% were proficient or gained 5% on the Digital Literacy Assessment, which did not meet the MPO. Students without PFS status were proficient or made a 5% gain at a rate of 91%, which did meet the MPO. See the recommendations section for additional comments about appropriate coding of migratory PFS students on the Digital Literacy Assessment.

Exhibit 22 Migratory Student Gains on the Digital Literacy Assessment by Grade

Grade N Mean Pre

Mean Post Gain

# (%) Gaining 5%

# (%) Proficient

# (%) Proficient

or Gaining

MPO met?

PK 17 34% 48% +14% 12 (71%) 3 (18%) 12 (71%) No K 12 40% 50% +10% 11 (92%) 2 (17%) 11 (92%) Yes 1 20 33% 44% +11% 20 (100%) 0 (0%) 20 (100%) Yes 2 21 47% 60% +13% 18 (86%) 8 (38%) 18 (86%) Yes 3 26 55% 67% +12% 20 (77%) 10 (38%) 21 (81%) Yes 4 20 60% 76% +16% 19 (95%) 11 (55%) 19 (95%) Yes 5 18 59% 74% +15% 15 (83%) 7 (39%) 15 (83%) Yes 6 17 54% 71% +17% 16 (94%) 7 (41%) 16 (94%) Yes 7 10 65% 77% +12% 7 (80%) 6 (60%) 8 (80%) Yes 8 13 54% 67% +13% 12 (92%) 2 (15%) 12 (92%) Yes 9-12 6 65% 76% +11% 5 (83%) 2 (33%) 5 (83%) Yes

Total 180 50% 64% +14% 155 (86%) 58 (32%) 157 (87%) Yes PFS 20 47% 54% +7% 12 (60%) 2 (10%) 12 (60%) No

Non-PFS 160 51% 65% +14% 143 (89%) 56 (35%) 145 (91%) Yes Source: Digital Literacy Assessment Results Spreadsheet


Exhibit 23

Percentage of Migratory Students Proficient or Gaining 5% on the Digital Literacy Assessment

Source: Digital Literacy Assessment Results Spreadsheet School Readiness

MPO 3A) By the end of the 2018-19 reporting period, 75% of migratory children ages 3-5 who are not proficient on the IMEP Kindergarten School Readiness Checklist will receive site-based or home-based services in a MEP-funded regular school year or summer program for ten sessions and will make one level progress on three skills.

The IMEP used a common preschool assessment, the “IMEP Kindergarten School Readiness Checklist” which was adapted from the Pennsylvania MEP Checklist. There are 29 school readiness skills on the checklist with three levels of age-appropriate developmental progress. Sites reported the total scores for all levels and indicated whether or not students had a gain of one or more levels on three skills. Exhibit 24 shows that all 40 preschool children assessed gained one or more levels on three skills, which meets the MPO. Preschool students gained an average of 9.6 points between pre/post-testing. Gains ranged from 8.5 points for children ages 2-3 to 10.1 points for four-year-olds. The MPO was met for all age levels assessed.

Exhibit 24 Progress of Preschool Migratory Children on the IMEP Kindergarten School Readiness

Checklist, by Age

Age Number

Assessed Mean

Pretest Mean

Posttest Gain # (%) Gaining 1 or more

levels in three skills MPO met?

2-3 8 10.8 19.3 +8.5 8 (100%) Yes 4 16 14.1 24.2 +10.1 16 (100%) Yes 5 16 16.9 26.4 +9.5 16 (100%) Yes

Total 40 14.5 24.1 +9.6 40 (100%) Yes Source: Preschool Tracking Form

71

92100

8677

9583

9480

9283

60

89 86

0

20

40

60

80

100

120


Exhibit 25 shows that the MPO was met by PFS status as well. PFS students gained an average of 8.7 points and non-PFS students gained an average of 10.5 points.

Exhibit 25 Progress of Preschool Migratory Children on the IMEP Kindergarten School Readiness

Checklist, by PFS Status PFS

Status Number

Assessed Mean

Pretest Mean

Posttest Gain # (%) Gaining 1 or more

levels in three skills MPO met?

PFS 22 13.9 22.6 +8.7 22 (100%) Yes Non-PFS 18 15.3 25.8 +10.5 18 (100%) Yes

Total 40 14.5 24.1 +9.6 40 (100%) Yes Source: Preschool Tracking Form

High School Graduation and Services for Secondary Youth

MPO 4A) By the end of the 2018-19 reporting period, 80% of secondary-aged migratory students enrolled in MEP supplemental instructional services for five sessions in RSY or SSY will meet objectives set in their Individual Migrant Education Plan (IMP), which may include academic skills, life skills, English skills, college and career readiness, alternative programming, or other activities.

The IMEP completes an Individual Migrant Education Plan (IMP) for each student not assessed through another measure and reports progress toward their unique goals. IMPs were completed during the regular term 2018-19 and summer 2019. Following is a list of goals included on the plans. Many students worked toward more than one goal.

• Career Awareness and Education • Driver’s License Preparation and Testing • Elementary and Secondary Skills Building • English as a Second Language (ESL) • Life Skills • Mexico Diploma Track-Preparatoria • Postsecondary Preparation • State Assessment Preparation

Student progress was assessed regularly with a final rating provided after at least five sessions of instruction. Exhibit 26 shows that all 82 students with an IMP met or exceeded their objectives, meeting the MPO. In addition, the MPO was met at all grade levels and for OSY. Fewer than 10 students were marked as PFS, so results are not disaggregated by PFS.


Exhibit 26 Secondary Student Progress Toward Individual Learning Objectives on the IMP

Grade N Did not Meet Objectives

Met Objectives

Exceeded Objectives

Met or Exceeded

MPO met?

9 24 0 (0%) 12 (50%) 12 (50%) 24 (100%) Yes 10 26 0 (0%) 8 (31%) 18 (69%) 26 (100%) Yes 11 16 0 (0%) 4 (25%) 12 (75%) 16 (100%) Yes 12 6 0 (0%) 0 (0%) 6 (100%) 6 (100%) Yes

OSY 10 0 (0%) 10 (100%) 0 (0%) 10 (100%) Yes Total 82 0 (0%) 34 (41%) 48 (59%) 82 (100%) Yes

Source: Individual Migrant Education Plans

MPO 4B) By the end of the 2018-19 reporting period, 75% of migratory students enrolled in the MEP for one session and assessed with OSY Consortium materials will score 80% or higher.

Regional staff worked with OSY to complete lessons from the OSY Consortium life skills materials found at osymigrant.org. Lessons used included the following:

• My Basic Rights • Can I Get Sick from the Heat? • What is an STD?

• Your Budget • Creating and Using a Budget • A Visit to the Doctor

Exhibit 27 shows that 65 OSY were assessed and all (100%) scored 80% or higher on the curriculum-based assessments, which meets the MPO. OSY gained an average of 7.2% between pre/post-testing. Note that all OSY assessed were also PFS.

Exhibit 27 Results on OSY Consortium Life Skills Lessons

Number Assessed

Mean Pretest

Mean Posttest Gain

# (%) with 80% or higher

MPO met?

65 90.8% 98.0% +7.2% 65 (100%) Yes Source: GOSOSY Student Tracking Form


Implications and Recommendations Lessons learned and recommendations for action based on the evaluation of the Indiana MEP are included in this section of the report. The conclusions, commendations, and recommendations are summarized based on surveys and interviews with local MEP staff and migratory parents, results of assessments, reports provided by program directors, and other supporting information submitted to the program evaluator. Progress Made on Previous Recommendations

1. Update the SDP. The SDP update was completed in February 2019. The MPOs and strategies from that update are included in this 2018-19 evaluation.

2. Adopt statewide literacy and math assessments for migratory students. The SDP committee discussed this recommendation and reviewed relevant data. The committee ultimately decided to adopt a statewide MPO but allow the regions to select their own assessments appropriate for the length of the summer school program, resources available in the region, and previous training for teachers.

3. Focus on academic achievement in the development of strategies and MPOs. The committee reviewed existing MPOs and drafted new MPOs for the summer of 2019 that included academic performance results in each goal area.

4. Ensure MPOs align with descriptions in the OME SDP Toolkit. The committee reviewed toolkit descriptions of MPOs and strategies during SDP Committee meetings. With the assistance of the evaluator, MPOs and strategies were aligned to the descriptions. Recommendations for 2019-20

This section of the report provides recommendations for action based on the data collected for the evaluation of the IMEP. Recommendations are summarized based on evaluator observations, staff and parent surveys, student assessment results, student outcomes, records reviews, and interviews with State and local MEP staff and parents. The IMEP is commended for meeting all MPOs in 2018-19; therefore, recommendations are provided in the interest of continuous improvement.

1. Include PFS status on all data collection forms and reports for MPOs. The OME Checklist for MEP Evaluation includes a requirement that “the State MEP collects performance results data on MPOs established in the SDP for all MEP activities and services, disaggregated for PFS and non-PFS migratory students (34 CFR Section 200.83).” While the IMEP collects this data for most MPOs, some sites did not record this data element with the results. For some MPOs, the number of PFS students reported was fewer than the number expected based on numbers identified in the


previous year. Regions may need to provide technical assistance to staff regarding reporting PFS status on local assessment spreadsheets.

2. Establish data collection procedures and guidance for MPO 3b related to

preschool children participating in early childhood education. The SDP established MPO 3B: “By the end of the 2019-20 reporting period, 70% of migratory preschool children ages 3-5 will receive instructional services from existing preschool programs or MEP-funded programs.” Procedures are in place for determining the percent of preschool children enrolled in MEP-funded programs, but additional data collection is needed to identify children attending a non-MEP funded preschool program.

3. Provide training and guidance on data reporting as it relates to collecting Migrant

Student Information Exchange (MSIX) minimum data elements. Indiana changed to the MIS2000 data system during the 2018-19 reporting period. While this data system is used successfully in other states, the IMEP will need to provide training to staff and establish new procedures for monitoring the accuracy and timeliness of data input to be in compliance with Federal regulations.

4. Continue improvement of the provision of services to OSY. While the MPO related

to instructional services for OSY was met, regions reported on the QSI that the strategy related OSY services is in need of improvement. The SEA should work with regions to outline the context in which services are best provided (i.e. site, home, or technology-based instruction), and select appropriate materials to meet student needs. While the GOSOSY Consortium website (osymigrant.org) contains a wealth of materials, instructors may need assistance selecting materials that are appropriate for Indiana OSY.

5. Create an MPO to measure the impact of support services. In 2017-18, the State

reported that 96% of eligible migratory students received a support service. However, the State does not yet have an MPO to measure the impact of these services. Determining a measure to show whether or not services are effective is critical for funding and programming decisions.

Overall during 2018-19, the IMEP has established a high-quality program that focuses on student achievement and establishes methods for measuring the effectiveness of all activities provided. The program has increased learning in reading and mathematics through intentional focus on the skills migratory students need as a basis for meeting state standards. High school students are making progress toward educational goals, and preschool students are demonstrating improvement in school readiness skills. All of these achievements indicate a solid program focused on improving education for all Indiana migratory students.

Appendix: Survey Comments

Staff Surveys

What will you use from this training?

Everything • All of it • Curriculum materials, digital literacy resources, social emotional resources, breakout

session resources/ideas • I will try to use everything! I was so inspired! • I will try to incorporate as much as possible.

Technology lessons and tools

• 2-4 STEM lessons, makerbot, 4-5 iPad apps • Google training on various applications • I learned the importance of testing or reiterating previously learned knowledge,

especially related to technical fields like robotics where there are more abstract names for parts. These names can be easily seen by the students once we build our robots giving them a great visual aid to help them to remember the technical vocabulary.

• I liked using seesaw and flipgrid • I plan on using some of the technology tools that were presented during the PD potluck

with my students. • If the machine is available, I would like to use the 3D printer to make shapes, letters, and

numbers. • I will implement spheros and green screen. • I will explore Sphero.edu more for sure. • I will use the technology and curriculum resources that I learned about during this

training. • iPads & sphero • Knowing how to operate tools and share ideas efficiently. • Lesson plans • Loved the information on how to be more aware of cultural differences and the Sphero

Balls. • Sphero and iPad • Tech tools and applications to incorporate with helping students to receive engaging

instruction. • Technology for quizzes, games, and reading.

Project-based learning strategies

• I am going to use some of the lesson plans from 2.0 off the IMEP website like the bottle rocket activity and tie in the students making an imovie of their projects.

• I am looking forward to using the 3D printer, Sheros, and GoPro cameras to see what students will come up with in their classes.

• I will use the IMEP website for the PBL lessons. • PBL lessons, strategies learned in breakout sessions. • PBL/Inquiry STEM Lessons

• The hyper docs and some of the PBL lessons Other

• Charting student visits and implementing CPR curriculum • Mentally prepared for camp • Mindfulness activities, hyperdocs • My major concern moving into this training was not knowing what our numbers looked

like or having a plan for daily activities. • Staff connections • The awareness of the struggle migratory students endure.

EL Strategies

• ESL • Hyperlinks, EL strategies • Hyperdocs • I have learned how to use different techniques in communicating with the students and it

has made a huge impact in my teaching of health topics. • I hadn't thought before about the possibility that they may not be fluent in reading

Spanish either. Reading Strategies

• I learned two new ways to assess students’ reading fluency and comprehension levels. “EARS” is a simple, yet useful way to determine students’ ability and growth.

• I liked the SODA acronym and plan to use that. I also love to use storytelling, and since my students also love to hear stories, I will use stories for selective vulnerability to help engage them and help them see me as a human.

• I will be using some of the fluency strategies we learned at the training. I felt the training did a great job emphasizing the importance of all aspects of fluency not just reading speed/accuracy.

• Reading strategies and culturally appropriate resources • Story Cubes

School Readiness

• I will definitely use the lesson plans specifically created this year for preschool teachers. • Preschool Curriculum, STEM tools • Preschool skeleton • School readiness training

What suggestions do you have to improve services to migratory students in Indiana?

No suggestions • At this point, I cannot make any suggestion. • Being my first year I am not sure what we can do more just yet. • Keep up the good work. The more hands-on the better. • New to migrant services. N/A • No suggestions. I'm always impressed with this program and hope that funding

continues for the crucial services it provides to migratory families. • This is my first year so I do not have any suggestions this year.

Continue existing programs or services

• Continue providing exceptional technology programs for these students. • This program is special in what it provides for migratory students and their families. The

one thing that stood out to me this summer was the constant striving for improvement and looking for ways to better serve these families. I’m proud to be a part of it. I truly believe providing the students with experiences not only in the classroom but around the community is the most valuable aspect of this program and should definitely be continued.

Improve communication and collaboration among schools and regions

• Better communication • Encouraging teachers from the same region to collaborate during the training to help

round out each other’s lesson planning. • More communication early on about the program due to so many options in our area.

Improve communication and collaboration with migratory families

• Family liaisons to closely monitor-support migratory families (in small numbers) and work closely with the school corporation to provide support services in all areas of life for students and their families.

• Creating an ongoing, consistent dialogue throughout the year might help boost our attendance for summer sessions.

• Include a component that requires that students give back to the community to leave a positive footprint.

Other

• A curriculum would have been helpful. • A real life video of some of our actual families to make a more of an impact. • Help out more • How can we teach an appreciation of education? • Manuals delivered to teachers prior to training if possible. • Nervous about cutting to 3 regions. Travel time to transport students. • We need to help students get excited about reading. • Expectations/objectives of the program in terms of daily needs and instruction.

More strategies for EL instruction and assessment

• Finding some simple Spanish passages to assess the ELs who don't speak English would be helpful.

• I thought the picture cards were very helpful with the ELs. This year I had a student in my regular classroom whom I'd taught last summer in summer school, and we had a hard time keeping in contact with the family because cell phone numbers changed frequently. I'm not sure how to help with this problem, but it would be nice to be able to communicate with the families on a regular basis.

• Focus on learning more about families and language learning needs. • I think it would be beneficial to have more exposure to secondary instruction strategies

and EL support. Longer summer programs

• I feel like 6 weeks is not quite long enough to squeeze everything in that I would like to. I liked the 8 week course better.

• More hours in the day.... More STEM activities

• Loved the STEM activities from years past. There was not much this year. • I heard suggestions on how the planning time was much appreciated, but teachers

missed the opportunity to learn new apps/classroom implementation ideas. • Maybe a little more focus on the various apps that could be used in the classroom that

tie into the lessons. I did appreciate the work time that was given. I put a rating of n/a on the English and Math training as that was not a focus on the inservice.

Better communication and collaboration with local communities and schools

• More info in local newspapers and schools. • More information for local schools about program. • More involvement, more students.

What suggestions for professional development topics do you have?

No suggestions or don’t know • I don't know. • Nothing at this moment • No specific suggestions. Anything that aids in us helping our migratory students get

caught up to their educational level required for their age groups. • We're doing well in this area. • Whatever helps us help students more.

Training for specific technologies

• Choosing a tech and being able to attend a session specific to that piece of technology. Idea for day 2 maybe.

• Coding • Continuing to look at tech opportunities and best practices for classroom integration. • Dig deeper into the Edtech tools for hyperdocs. • Drones/3D printing • I loved the technology PD we had and the instructional PD was extremely helpful. I think

having everything ready for teachers is a huge plus. • I'd like to know a little more about SeeSaw and other technology tools to use for

language arts. • Instruction on how to use some of the technology (i.e. 3D printers).

• Introductory PDs for using specific tech tools. Maybe a pre-survey to determine a tool to focus on, and have a session on how to use the tool.

• More hands-on activities/apps that we can experience and use with kids. • More technology topics. • Refresher on google classroom, maybe flip grid. • Technology integration in the classroom • Technology resources

College and career readiness strategies

• Career pathways options Student engagement

• I am always interested in better ways to enhance student engagement within or outside my content area.

• I believe that the first connection with students is the most important one and that if we can eliminate as many unknowns as possible that would only make teachers more prepared for what this program is all about. Making them better teachers by understanding the differences and how to overcome them.

• Situational training, for instance if a student were to have behavioral issues or get sick in class. What would be the best methods to take care of the situation.

New to Migrant Education Sessions

• Consider a good PD for new schools or new teachers would be past teacher’s experiences on how to get started and build the bridge with incoming students to ease both teacher and student nerves.

Strategies for English learners

• Language barriers tend to become somewhat unfamiliar and leave a teacher lost in the beginning. The transition for students then can become lost and students lose interest fast.

• I think training focused on EL support would be beneficial. • I would have liked more focus on how to teach English learners • Interactive ideas for EL practices to build language skills • More EL strategies. • Planning for different levels of English proficiency.

Other

• Have former migratory students come speak. • Ideas to stimulate parent involvement in education. • Math • More literacy-based curriculum strategies. • Using formative assessment instead of pre/post.

Parent Surveys

How did the Indiana Migrant Education Program help your child succeed in school?

• They learned a lot/helped progress in school and studies. (14) • They learned a lot, especially English. (5) • They get to continue their education in the summer rather than just doing nothing. (4) • They learned more in the areas where they were weak. (3) • Keeping up their reading and math skills. (3) • School supplies (3) • Summer school (2) • Helped with reading. (2) • Dental exam (2) • They are more prepared for kindergarten. (2) • Tutoring helped and preschool instruction. • They offered the online Freckle Program to help with math & reading. • They have more motivation. • She got extra weeks of school to help with her education. • They learned a lot from the tutors and have interest in things about their future. • They helped my 10th grader with high school credits. • They helped my 8th grader think about studying to get into college. • Career information and field trips related to that. • Using the MP3 player they provided to learn English during free time. • Advice for parents about the schools. • Summer school helped them be more responsible, have more information about

technology, and be more organized. • Summer school provided lots of hands-on learning and increased their vocabulary. • They learned to write their name better and identify more shapes. • The tutoring and summer school were helpful. • Technology was helpful. • I believe the summer school program is so helpful to parents as they know their children

are in a safe, healthy, learning environment while they work. • I like that they come right to our door to pick them up. • Health services helped a lot.

How would you change the IN MEP to make it better?

• Everything is good (12) • I don’t know/nothing (11) • Summer school food could improve. (2) • I want the summer school to be longer. (2) • We as parents need to have more interest in the education of our children. • Monthly newsletter on upcoming events and reminders. • More information about community services for helping with homework and learning

English. • Send more information in Spanish. • Closer summer school. The bus ride was a little long. • More staff speaking English and Spanish. • More assistance for parents who want to learn to speak English so they can

communicate with teachers better. • Better communication • Maybe more school supplies. • I would like even more instruction in learning English. • More help with transportation to school.

Date post:	24-Sep-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Migrant Education Program Evaluation and Program …mathematics enrichment activities, support...

Documents