Using Multiple Data Sources to Inform Target Setting in SLOs

**The models and information shared within this presentation are for informational purposes only. NYSED does not endorse or recommend any of

the approaches described.

Kate Gerson, Senior FellowDr. Julia Rafal-Baer, Assistant Commissioner

Year 2 Implementation: A Leveraged Moment

2

Reducing the Number of Assessments

• The Department is working alongside districts to incorporate requested reductions in the number of assessments used in both the State Growth and Locally-Selected Measures subcomponents:

removing the use of pre-assessments as a baseline measurement (not a requirement in SLOs) in favor of using past performance trends, historical data, and/or prior-year assessment results to inform the baseline used to set targets within SLOs (see: SLO 103 webinar: http://www.engageny.org/resource/slo-103-for-teachers);

removing locally-developed and/or state-approved third party assessments in favor of using school-wide, group, or team measures based on State assessments, where allowable;

using the same assessment in a different way between State Growth and the Locally-selected Measures subcomponents.

3

This session will include:

1. A review of how the learning standards, assessment design, and performance indicators can help an educator determine relevant data sources that can be used to establish a baseline of student performance.

2. Discussion around the integral role data plays in SLOs, both in target setting and in driving classroom instruction throughout the year.

3. Examples of how multiple data sources, including historical data, past performance trends, individual student scores, and student work samples can be used to establish patterns of performance that can help to inform appropriate targets for student growth.

4

What are Student Learning Objectives (SLOs)?

A Student Learning Objective, or SLO, is an academic goal set for an educator’s students at the start of a course.

Remember that an SLO: -represents the most important learning (aligned to Common Core, State, or national standards, as well as any other school and district/BOCES priorities) -must be specific and measurable (based on available prior student learning data)-should be created to be ambitious but achievable (using as much available student data as possible)

Resource: Guidance on Growth Goal Setting Process for Teachers: Student Learning Objectives

5

What is the value in using SLOs?

•SLOs encourage educators to focus and align instruction with district/BOCES and school priorities, goals, and academic improvement plans.

•Many educators already set academic goals for students on a regular basis and see it as an integral part of their practice.

•Ultimately, setting SLOs can lead to more purposeful instruction, closer monitoring of student progress, and greater student growth.

6

Baseline and Trend Data

These data play two important roles in the development of SLOs.

7

First, these data will be used as the basis for target setting for SLOs.

Secondly, they frame the entire SLO and help shape teaching practice.

A Closer Look

Baseline data can take a variety of forms.

a) assessments based on the content in the class;

b) grades from previous, related classes;

c) anecdotal observations of student performance;

d) standardized test results;

e) surveys of students’ interests and experiences; or

f) any number of other measures that are available at the individual level.

Trend data is a longer term view of student performance.

Results from a variety of measures can be plotted over time to determine trends in the data.

Benefits•Reduces the need for baseline testing•Increases reliability in performance setting•Past performance is a good predictor of future performance

Trend DataBaseline Data

8

Keep in mind the goal is to collect and use as much information as possible to determine exactly what a student knows at the start of an SLO interval.

Integral Role of Data in SLOs

Teachers should use multiple sources of data when developing student growth targets.

Baseline and Trend Data Can: • Summarize student information (test scores from previous years,

performance in similar courses, etc.);• Help identify student strengths and weaknesses; and • Inform targets set, establishing the amount of growth expected.

Keep in mind, the key to effectively using data sources is recognizing the value of the data available, or in other words, understanding what it can and cannot tell you about student performance in your current course.

9

How can you help educators develop appropriate learning

targets?

10

What data sources can be used to help inform targets?

Some courses, like 4th grade ELA have antecedents that are quite clear, ranging from report card scores, to diagnostic screenings, to previous standardized

tests.

11

Kindergarten Report Card

Grade 1 Report Card

Grade 2 Regionally-Developed

Assessment

Grade 3 Report Card

Kindergarten Readiness

Grade 1 Regionally- Developed

Assessment

Grade 2 Report Card

Grade 3 ELA State Test

Grade 4 ELA

Prerequisite skills: drawing computer art geometry English

What data can be used when the course is not part of an instructional sequence?

12

CAD= Computer Aided Design/Drafting

Identify Data Sources:

The Baseline Data Source Review Sheet helps you to brainstorm the types of data available to you.

1. List the types of data available, even if it is not directly aligned with the course. Do not restrict your thinking during this stage.

2. Identify which sources of data are the most valuable source of information related to your specific course content.

3. Don’t forget to consider prerequisite skills and other student related factors.

13

Triangulation of Data

14

Performance Task

Student Survey Results

Written Test Data

More accurate picture of student

performance

•Triangulation reduces the risk of false interpretations by strengthening conclusions about finding through the analysis of multiple streams of evidence.

•Findings can be corroborated and weaknesses in the data can be compensated for by the strengths of other data, thereby increasing the confidence we can have in the results.

15

Sample Course: US History and Government

Step 1: Review the Learning Standards and the Course Summative Assessment

16

Part I: 50 multiple-choice questionsPart II: One thematic essay questionPart III: Based on several documents:•Part III A: One or more questions on each document•Part III B: One essay question based on the documents

Step 2: Review Available Student Data

18

Students

Historical Trend Data: Global History and Geography compared to US History and Government Scores

Review of Additional Data Sources

Depending on the students in the class, there may be additional sources of data that should be examined:

•Historical trends on other summative assessments and report cards•Conversations with previous grade-level/content-area teachers and service providers regarding strengths and weaknesses of students•Relevant annual IEP goals•NYSESLAT scores

19

US History and Government Sample Student Data

Student

Grade 8 ELA Achievement

Level

BOCES- developed

Assessment - English 9

BOCES- developed

Assessment English 10

Global History and Geography Scale Score

1 2 65 68 65

2 3 70 75 75

3 4 95 98 90

4 3 72 65 70

5 3 75 76 78

6 3 70 75 75

7 4 97 95 85

20

Step 3: Set Targets Based On the Data Reviewed

Student

Grade 8 ELA

Achievement Level

BOCES- developed

Assessment - English 9

BOCES- developed Assessment English

10

Global History

and Geograph

y Scale Score

Student growth target -

US History Regents

1 2 65 68 65 65

2 3 70 75 75

3 4 95 98 90

4 3 72 60 70

5 3 75 76 78

6 3 70 75 75

7 4 97 95 85

21

Step 3: Target Setting Continued…

Student

Grade 8 ELA Achievemen

t Level

BOCES developed

Assessment - English 9

BOCES developed Assessment English

10

Global History

and Geography

Scale Score

Student growth

target - US History Regents

1 2 65 68 65 65

2 3 70 75 75

3 4 95 98 90

4 3 72 60 70

5 3 75 76 78 80

6 3 70 75 75

7 4 97 95 85

22

Step 3: Target Setting Continued…

Student

Grade 8 ELA

Achievement Level

BOCES developed

Assessment - English 9

BOCES developed Assessment English

10

Global History

and Geograph

y Scale Score

Student growth target -

US History Regents

1 2 65 68 65 65

2 3 70 75 75 70

3 4 95 98 90 85

4 3 72 60 70 65

5 3 75 76 78 80

6 3 70 75 75 75

7 4 97 95 85 85

23

Guided Exercise

23

Mrs. Thompson’s 8th Grade Computing

Course

Start With What You Have

24

*Note that this list of students is abbreviated and is for illustrative purposes only

Gather Additional Data When Necessary

Alternative sources can include: • prior class information • project scores• results from interest or experience

surveys• anecdotal records• assessment scores in related

subjects• student survey results

25

Pursue Relevant Data

26

Consider the Meaning of the Data

27

Collect More Data If Needed

28

Unit 1 Assessment Score

Review of Supplemental Data Source Sheets

29

Additional Thoughts

A few general points:1. If most students performed similarly on the baseline measures, it may be reasonable to set a common growth target for all students.

2. If you review the data and determine that student performance falls into multiple groups, you may use a tiered or banded approach to targets.

For example, imagine Mr. Garcia had 76 students across his multiple sections of Math 8 and he found the following distribution of scores on the baseline measure:

Less than 20 18 students

20-40 42 students

40-60 16 students

30

3. If there is no consistency in the baseline scores you might want to consider differentiating the targets for individual students.

4. Finally, engaging students in the process of academic target setting can be a motivational tool in the classroom.

Additional Thoughts Continued…

31

For more, search “Student Learning Objectives” on engageny.org

Multi-State SLO Rubric

32