Overview and Discussion of NRS Changes

Overview and Discussion of NRS Changes

Massachusetts Department of Elementary and Secondary Education

2

NRS Changes for FY13AGENDA

Review changes on follow-up outcomes and new data requirements

Review criteria for cohorts Review changes to reporting timeline

and reporting tables Review questions and answers posted on

the ACLS website Respond to participant questions


3

Time Period Definitions Program Year (PY)– Twelve month period

from July 1 through June 30 defined by the calendar year at start of period. (federal terminology) For example, July 1, 2012 through June 30,

2013 is the time period corresponding to PY12. Fiscal Year (FY)– Twelve month period from

July 1 through June 30 defined by the calendar year at end of period. (state terminology) For example, July 1, 2012 through June 30,

2013 is the time period corresponding to FY13.


4

Changes to Follow-up Outcomes: Automatic Cohort

DesignationProgram Year (FY2012): Participants identified based on self-reported goals (program – level)

Program Year (FY2013): Participants automatically identified based on “cohort” which are based on demographic variables in the MIS system (state – level)

The definition for “cohort” varies by outcome.


5

Implications of “Automatic Cohort Designation”

Students belong to a group to be followed for NRS post-program outcomes based on pre-defined characteristics. Following students for post program outcomes willno longer be goal dependent. As has always been the case, we are only

concerned with students having at least 12 hours of attendance.


6

YES!! Local staff should continue to set goals with students Goal setting is an essential component of a good

program and good instruction but…. Students goals will no longer be used as a way to

identify students for the NRS follow-up measures

Do Student Goals Still Matter?


7


8

Secondary Credential Cohort

Obtain Secondary Credential Cohort Learners who take all GED tests, or Are enrolled in adult high school at the high

ASE level, or In the assessment phase of the EDP, and Who exit during the program year

Entry into cohort occurs only for those who take all 5 GED tests by the end of reporting period (December 31).


9

Q: Do students have to still be enrolled when they take the tests?

No, students are included in the cohort if they take all five test before the end of the reporting period for the program year in which they exit (December 31)

Q: Must a student pass all GED tests to be included as achieving the outcome?

Yes, students must pass all GED tests to be included as achieving the outcome

Question: Cohort for Exited Students and GED Tests


10

Example 1: A student who enrolls in fall of 2012, exits in March 2013, takes 3 tests then, takes 2 more in September 2013, would be included in the cohort for 2012-13

Example 2: A student who enrolls in October 2012, takes 2 GED tests in May 2013, exits in November 2013 and takes 3 GED tests would be included in the cohort for 2013-14

Example 3: A student who enrolls in December 2012, exits in March 2013, takes 3 tests then, takes 2 more in January 2014, would not be included in any cohort

Passing GED Tests: Examples


11

Postsecondary Cohort

Postsecondary CohortAll learners who

have earned a secondary credential while enrolled

hold a secondary credential or are enrolled in a class specifically

designed for transitioning to community college

and who exit during the program year


12

Questions: Foreign Credential and Transition class

Q: Are students with a secondary credential, earned in another country, in this cohort? Are ESOL students to be included?

Yes, these students are included in the cohort

Q: How is a transition class defined ? A class that has a specific purpose to prepare students

for entry into postsecondary education, training or an apprenticeship program.

The rate based class focus would be Bridge to College/Math or Bridge to College/Writing. The non rate based class focus would be Bridge to College/Math, Bridge to College/Writing, or College and Career Readiness.


13

Q: Please clarify multi-year reporting of entry into postsecondary education Students who exit during program year

achieve this outcome if they enter postsecondary any time from exit through the end of the following program year (June 30)

Follow students and report for two years, in two rows on Table 5 (unduplicated) First report (2012-13): report same as of now Future years: Update prior year—keep cohorts

separate First two year reporting on 2013 report (due December 31, 2014)

Reporting of Entry into Postsecondary--Timing


14

Entered Employment Cohort Learners who are not employed at time of

entry and in the labor force who exit during the program year

Employment Status in SMARTT: Employed Unemployed and Looking for Work Unemployed and Not Looking for Work Retired or Otherwise Not Looking for Work Homemaker Programs should exit students who are no longer in attendance by filling out the “Exit Student” screen in SMARTT.

Employment Cohorts


15

Retained Employment Cohort Learners who were not employed at time of

entry and in the labor force, who are employed by the first quarter after exit quarter

Learners employed at entry who exit during the program year

Employment Status in SMARTT: Employed Unemployed and Looking for Work Unemployed and Not Looking for Work Retired or Otherwise Not Looking for Work HomemakerPrograms should exit students who are no longer in attendance by filling out the “Exit Student” screen in SMARTT.

Employment Cohorts


16

Students, in entered employment cohort, that are employed the first quarter after exit are also in the retained employment cohort

The timeframe is the same as always–it has not been changed

States must use students’ employment status at entry to determine cohort status for employment outcomes.

Reminder on Retained Employment


17

Q: How is “unemployed, in the labor force defined?”

NRS definition (Unchanged): Learners who are not employed but are seeking employment, make specific efforts to find a job and are available for work

Employment Status in SMARTT: Unemployed and Looking for Work

Question: Definition of Unemployed in Labor Force


18

Entered EmploymentEmployed in the first quarter after exit.

Retained EmploymentEmployed in the third quarter after exit.

Timing of Employment Measure


19

Collection: One quarter after exit for entered employment, three quarters after exit for retained employment

Reporting lag (previous year) to accommodate lag in Unemployment Insurance (UI) wage records

Review: Employment Data Collection and Reporting


20

Exit Quarter MeasureCollect

Report

July 1–September 30, 2012(First Quarter)

Entered Employment October 1–December 31, 2012(Second Quarter FY 2013) FY 2013

Retained Employment April 1–June 30, 2013(Fourth Quarter FY 2013) FY 2013

October 1–December 31, 2012

(Second Quarter)

Entered Employment January 1–March 31, 2013(Third Quarter FY 2013) FY 2013

Retained Employment July 1–September 30, 2013(First Quarter FY 2014) FY 2014

January 1–March 31, 2013(Third Quarter)

Entered Employment April 1–June 30, 2013(Fourth Quarter FY 2013) FY 2013

Retained Employment October 1–December 31, 2013(Second Quarter FY 2014) FY 2013

April 1–June 30, 2013(Fourth Quarter)

Entered Employment July 1–September 30, 2013(First Quarter FY 2014) FY 2014

Retained Employment January 1–March 31, 2014(Third Quarter FY 2014) FY 2014


21

Enter Employment/Get a Job -- Those with SSN and signed release form. For all others, survey method must be used.

Retain Employment -- Those with SSNand signed release form. For all others, survey method must be used.

Survey form may be accessed at:http://www.doe.mass.edu/acls/smartt/

nrssurvey_followup.pdf

Data Matching Department of Revenue


22

Obtain GED -- All students (regardless if SSN provided and release form signed since match will be with the Oklahoma Scoring Service)

Enter Postsecondary Education -- -- All students (regardless if SSN provided and release form signed since match will be with the National Student Clearinghouse)

Data Matching Oklahoma Scoring Service, and National Student Clearinghouse


23

Follow-up Outcome Timing Secondary CredentialObtained credential before the end of the reporting period (December 31)

Postsecondary EducationEntered any time from exit until the end of the following program year (June 30)

Report for two years, in two rows on Table 5 (unduplicated)Programs should exit students who are no longer in attendance by filling out the screen found by clicking on the “Exit Student” menu option within SMARTT .


24

Reporting Changes: Table 5 Table 5 revised to include instructions for automatic cohort designation New row added for postsecondary transition: Report two years (unduplicated)

Update prior year First two year reporting on 2013 report

(Due December 31, 2014)


25

Reporting Postsecondary Example in 2013-14

In 2013-14, 125 students in cohort exit, state

matches 100 and finds 50 have entered postsecondary State runs match on 2012-13 cohort again by

June 30, 2014 and finds another 20 students have entered postsecondary in addition to the 40 from previous year


26

Example of Table 5: 2013-14

Core Follow-up Outcome Measures

Number of Participants in

Cohort

Number of Participants Included in

Survey Sample

Number of Participants Responding to Survey or

Used for Data

Matching

Response Rate or Percent

Available for Match

Number of Participants Achieving Outcome

Percent Achieving Outcome

(A) (B) (C) (D) (E) (F) (G)Entered Postsecondary Education or Training -- current program year

120 100 83%

(100/120)

40 40%

(40/100)

Entered Postsecondary Education or Training – prior program year

Leave entire row blank


27

Example of Table 5: 2014-15 Core Follow-up

Outcome Measures

Number of Participants in

Cohort

Number of Participants Included in

Survey Sample

Number of Participants Responding to Survey or

Used for Data

Matching

Response Rate or Percent

Available for Match

Number of Participants Achieving Outcome

Percent Achieving Outcome

(A) (B) (C) (D) (E) (F) (G)Entered Postsecondary Education or Training -- current program year

125 100 80% 50 50%

Entered Postsecondary Education or Training – prior program year

120 100 83% 60

(prior year was 40,

now 40+20)

60%

(60/100)


28

Implications for Implementation The number of learners to follow will

increase. Obtaining SSN’s and signed release

forms will lessen the follow-up burden.

Local staff will need to follow-up by survey method for cases where SSN’s and signed release forms were not obtained for those within any of the employment cohorts.


29

Q: How should we determine years of student education? Do we need to check? Self-reported by student, NRS does not

require verification but the state may choose to do so

Q: How do we count education in other countries that is different than our system? Use years of schooling Use in-country definition for secondary

credential

Questions: Reporting Student Education


30

Q: Can students belong to multiple cohorts? Yes, for example, an unemployed student in

the labor force, with secondary credential who exits, would be in two cohorts

Question: Multiple Cohorts


31

Q: Do we have to re-identify cohort membership for students continuing the next year? Yes, for example, employment status or

secondary credential status may change But only collect measure at exit

Question: Continuing Students


32

ADDITIONAL QUESTIONS??

Date post:	25-Feb-2016
Category:	Documents
Upload:	mead
View:	43 times
Download:	0 times

Overview and Discussion of NRS Changes

Documents