TIF-LEAP Training Fall 2009 Susan Kirkendol Senior Assessment Program Analyst Scott Reynolds Student...

transcript

TIF-LEAP Training Fall 2009

Susan KirkendolSenior Assessment Program Analyst

Scott ReynoldsStudent Learning Objective Specialist

Susan KirkendolSenior Assessment Program Analyst

Scott ReynoldsStudent Learning Objective Specialist

Review

Summary

• TIF – Teacher Incentive Fund

• LEAP – Leadership for Educators’ Advanced Performance

• 20 schools were selected

• Last year 10 schools completed the Student Learning Objectives Process

• This year all 20 schools will participate

The Design

• Centerpiece of the plan: Student Learning Objectives (SLOs)

• Teachers select a learning content from SCOS (or other state approved standards) – Selection is based on data

• Design an objective around that content with goals for student growth, teaching strategies– Provide rationale for choices made

• Incentive payouts are contingent upon goal achievement and a growth component

TIF-LEAP Goals

Devise and test an incentive plan that will:• Assist in the recruitment and retention of

highly effective and certified staff to teach every child

• Aid in recruiting teachers to high-needs schools

• Include more than EOG/EOC teachers• Focus on student learning• Improve instructional practices• Develop a systemic, comprehensive

compensation plan

Strategic Plan 2010 Goals

Goal I: High Academic AchievementGoal II: Effective EducatorsGoal III: Adequate Resources and

FacilitiesGoal IV: Safe and Orderly SchoolsGoal V: Freedom and Flexibility with

AccountabilityGoal VI: World-Class ServiceGoal VII: Strong Parent and Community

Connections

Background of SLO Process – Denver Pilot

• Piloted a similar program in Denver with 16 schools (at all 3 levels) with similar demographics over four years.

• Used objectives to link students' achievement to teachers' efforts.

• Program provided teachers with a high degree of autonomy.

• Research of the process was conducted by our grant partner, CTAC, using various statistical methodologies and arrived at several findings:

Impact of Objectives & Student Achievement

• Excellent objectives were associated with high mean achievement, as measured by their standardized tests (ITBS and CSAP).

• Meeting two objectives made a positive difference in mean student achievement.

• Objective setting is a critical initial step in the planning and delivery of instruction.

Background Effect on Pilot Results

• Achievement rose as length of time in the pilot increased.

• Ability to meet objectives increased with length of time in the pilot.

CMS Implementation

• We are in Year 3 of the 5-year grant.• Steering Committee meets monthly.• Other groups meet as needed.• Changes reflect input from this year.• With the goal of becoming the comprehensive

compensation program, TIF-LEAP has absorbed funding from – Pay for Performance– STAR Teacher & Administrator– High School Challenge– Master Teacher & Administrator

Steering CommitteeSteering Committee

Budget &

FinanceSheila Shirley

Dennis Covington

Budget &

FinanceSheila Shirley

Dennis Covington

CommunicationsLaTarzja Henry

Curriculum &

Instruction Ann Clark

Curriculum &

Instruction Ann Clark

Human Resources

Maurice Ambler

Human Resources

Maurice Ambler

Information Technology

Susan Johnson

Information Technology

Susan Johnson

Achievement Zone

Curtis Carroll

Achievement Zone

Curtis Carroll

Compensation SpecialistJan Millard

Senior AssessmentProgram AnalystSusan Kirkendol

InstructionalSupport WorkingGroup

Finance/HRWorking GroupFinance/HR

Working Group

PrincipalsWorkingGroup

SLO DesignTeam

Student Learning ObjectiveSpecialist

Scott Reynolds

Student Learning ObjectiveSpecialist

Scott Reynolds

Accountability&

AssessmentChris Cobitz

Accountability&

AssessmentChris Cobitz

Executive DirectorSusan Norwood

Sr. Admin. SecretaryCarol Turner

TIF-LEAP Organization

Overview

• Teacher identifies learner centered problems by reviewing data and student work

• Teacher begins to write Student Learning Objectives (SLOs)

• Teacher writes or selects an assessment to pre-test students knowledge/skills on this content

• Administrator approves the pre-test• Teacher administers the pre-test• Teacher submits SLOs for approval including:

– SLO PDF file – Excel SLO Student Score Sheet using pre-test results to

predict a minimum expected score– Copy of assessment(s)– Appropriate set of approved standards (available on

Riverdeep)

Teacher Eligibility

• Every certified, lateral entry, Teach for America, and VIF teacher must participate

• Must earn At- or Above-Standard ratings on each function of the TPAI-R

• Teachers must be present in the classroom for at least 85% of the interval specified for the SLO

Requirements to Participate

• Every certified teacher much complete TWO SLOs

• The first SLO will be written for an entire class• The second SLO will be one of the following:

– A Targeted Group SLO focusing on a group of students. This could be another class if appropriate. If the same class is selected for two SLOs, different objectives from the SCOS must be used.

– A Team SLO where a group of teachers work together to design common objectives and a common assessment for students.

More on Teams

• Who can be a team?– Teams could be grade-level or content-specific

teams– Teams can be inter-disciplinary– Team make-up must have the approval of your

administrator

What must be the same with team SLOs?– Learning content– Assessment

More can be in common than these two components, such as the case of inclusion teachers, etc. but these two components are the minimum that must be the same.

Payout Structure for SLOs

Payouts will be based on:

– Successful completion of the SLO(s)

– Having at- or above-standard ratings on ALL FUNCTIONS of the TPAI-R

Payout amounts:

– Will be $1400 for each successful SLO completion

– All certified teachers are required to complete at least TWO SLOs

New Growth Component

• New this year is AN ADDITIONAL incentive based on a growth component

• Actual dollar amounts are under discussion

• Will not affect SLO payout

Who Are TESTED Teachers?

THOSE WHO TEACH:• Grades 4 to 8 Reading or Math• Grades 5 and 8 Science• Algebra I• Algebra II• Biology• Civics and Economics• English I• Geometry• Physical Science• US History

Growth for TESTED Teachers

• Teachers who have State tests (as listed on the previous slide) associated with their class will be eligible for an incentive based on growth in these test scores.

• Average growth for a teacher will be calculated based on the scores of all students in tested subjects (EOC/EOG) with a minimum requirement that 5 students must be able to have growth calculated.

• If this growth score is above average, and all conditions of eligibility are met, teachers will receive an incentive.

Growth for TESTED teachers

TESTED TEACHERSREQUIRED:

• SLO 1 – Class

• SLO 2 – Targeted Group or Team

AUTOMATIC ELIGIBILITY:

• Growth measure based on tested subjects taught to all students for that area – minimum of 5 students.

Options for NON-Tested Teachers

Teachers who do not have EOG or EOC tests associated with their classes have 3 possibilities:

1. Team up with a colleague or a group of colleagues who teach tested subjects and receive an incentive based on the growth of the students taught by the non-tested teacher. It must be possible for growth to be calculated for at least 5 students. Contact tif.leap@cms.k12.nc.us if you need further information. This must have administrator approval.

2. If option #1 is not possible: Complete a 3rd SLO with a possible incentive of $1400 for this SLO.

3. Complete the 2 SLOs only and do not seek participation in a team growth measure with no additional incentive possibilities.

Options for NON-Tested Teachers

NON-TESTED TEACHERSREQUIRED:• SLO 1 – Class• SLO 2 – Targeted group or Team

OPTIONAL:• Growth measure based on team OR

SLO 3 – Class, Team, or Targeted Group (Administrator approval required for growth measure and Team SLOs.)

Deadlines to Remember

Two windows for SLO submission• FALL

– OCTOBER 2: Deadline for Fall draft SLO submission by TEACHERS

– OCTOBER 16: Deadline for Fall SLO approval completion by ADMINISTRATORS

• SPRING – FEBRUARY 12: Deadline for Spring draft SLO

submission by TEACHERS– FEBRUARY 26: Deadline for Spring SLO approval

completion by ADMINISTRATORS

Population

Which students have you identified to be involved in your SLO?

• Should begin by looking at student data to identify a learner-centered problem

• What are their strengths?• What are their areas of need?

PopulationRequirements

• CMS identification numbers and names for each student are required.

• Class SLO = students seen at one time.• Targeted group SLO = students seen at

one time targeted for a specific need.• Minimum number of students in a Class

SLO or Targeted Group SLO or Team = 5.

• Only students in attendance 85% of identified interval will count when determining the outcome of the SLO.

Population – Options to Consider

• Special cases may have population of less than 5:– Possibly EC, ESL, TD teachers, and where

pullouts are used– NOTE: Requires administrator approval

• If you want to add students who come after SLO start date – You MUST state how they would be included in your SLO before approval for implementation– NOTE: You will not be able to add them to

the Student Score Sheet until the end of the interval

– Students still must be present for 85% of the interval.

Population – Restrictions

• The same population cannot be used for both SLOs UNLESS the learning content is different.

• If no students meet the 85% attendance criteria, the teacher is not eligible for the incentive.

• Students not present for at least 85% of the interval will not count for or against the teacher for the SLO’s outcome.

• The population must be students seen during regular school hours.

Population – Options

• If you have a highly transient population:– Consider buffering the number in the group to protect

against falling below the minimum of 5 students.

• If you have flexible grouping:– Plan for your students to be in the group for at least

as long as your interval (43 days minimum) regardless of the growth achieved.

– Consider shorter intervals where you are more likely to maintain the same student roster.

Population Subcomponents

• How many students are included in your population? (Will be identified with names and ID numbers on the SLO Student Score Sheet.)

• Assigned grade level(s) of students in the population.

• How often do you see these students within the interval?

• What percentage of students much achieve their growth expectation if less than 5 students are present at least 85% of your interval?

• Describe the data reviewed for this selection and how it led to these students.

• What are these students’ strengths?• What are these students’ areas of need?

What is the SLO Student Score Sheet?

• This is an EXCEL spreadsheet• You will enter student’s names, ID numbers,

pre-test date and score, and other requested items

• Each of the columns in white must be completed prior to SLO implementation

• Student Score Sheets MUST be submitted with the SLO

• Post-test scores and growth scores will be added to this same sheet once the SLO is completed

Improving readership

Data Resources

Student Achievement, Attendance, Demographic Data

• T-sparta (by student)• I-sparta (by school/grade)• NC Wise• Cumulative Folders (report cards, IEPs)• SAS EVAAS• Goal Summary reports• Common formatives• Data inventory

Data Resources

Other sources

• School Improvement Plans – – What are the goals for your school?

• Parents/Families

• Students themselves– What was easy? What was hard?

Data Resources

Student Performance Data

• Writing samples

• Projects

• Homework

• Presentational pieces – art, music, PE, oral reports

• Student comments

Data Resources

Teacher-Generated Data

• Interest surveys

• Learning styles questionnaires

• Multiple intelligences surveys

• Pre-testing

Learning Content

The selection of objectives you will teach from one of the following:

– NCSCOS– CTE: Blueprints– ESL: Grade-specific ELA objectives – PreK: Approved Widely Held Expectations from

Foundations Standards– AVID: Approved Objectives– JROTC: Approved Objectives– Other: Contact scott.reynolds@cms.k12.nc.us. (For

grades 6-12, only courses in the 2009-2010 Planning Guides can be used. If this presents a problem, email Scott.)

Learning Content Requirements

• At least 2 objectives must be chosen from the set of standards for each SLO.

• Math: Each lettered objective counts as a single objective.

• ELA, Science, and others: Bulleted items do NOT count as a single objective.

• If an objective has multiple bullets – at least ONE bullet must be selected.

• Teacher must be certified to teach in the content area selected. – (Team SLOs may be an exception to this rule with

administrator approval.)

Learning Content Restrictions

• When 2 SLOs are written . . . – Learning content for the SLOs cannot be the same

UNLESS the population is different for the 2 SLOs

THIS IS PERMISSIBLE:SLO 1 SLO 2

Learning content X Learning content X

Population A Population B

THIS IS NOT ALLOWED:SLO 1 SLO 2

Learning content X Learning content X

Population A Population A

• Reading comprehension is too global to select for learning content.– Instead target specific aspects of reading

comprehension such as:• Character analysis• Plot development• Drawing conclusions• Interpreting figurative language

• K-12 Skill Competency Goals and Objectives for Social Studies are too global as well.– These may only be used if at least 2 grade-

specific objectives are also selected.

• ESL teachers will use English Language Arts objectives for the grade/course represented.

Learning Content – Cautions

• Learning content should align with the interval chosen.– Consult your pacing guide. – Make sure the content will require at least a

quarter to be learned.

Learning ContentSubcomponents

• Content area for the SLO• Selected objectives from the approved

set of standards• Describe the data reviewed for this

selection and how it led to these objectives. Include any needs assessments and/or baseline data sources used.

• What evidence can you provide that your content is rigorous?

Interval

Interval is the period of time during which the SLO will be completed.

• Must state a start and stop date.• Teachers must be in attendance for at

least 85% of the interval selected.

IntervalRestrictions

• The shortest interval allowed is the number of days in the shortest instructional quarter for the year.– This year this is 43 days

• The longest interval allowed is the school year.

IntervalRestrictions

• Intervals cannot change once they are approved.

• Make sure you allot enough time for students to learn the content.

• Have plans about how to handle the process if students learn the material faster than expected.– Make sure these plans are clearly stated

in your SLO.

IntervalSubcomponents

• Interval start and stop dates• Number of instructional days in

interval• Middle and high school teachers will

enter:– NCWISE title of the course– NCWISE numeric course code– During which period will you teach this

• During which quarter(s) is your interval?

IntervalSubcomponents

• Why did you select this interval?• How does your interval align with

local pacing for the content?• If your students fully learn the content

before the end of the interval, what is your plan?

• How will you ensure content depth and complexity within the interval?

Assessment(s)

• What instrument(s) will you use to measure student learning of the content?– Assessments can be

• Existing assessments• Developed with a team (i.e., common

formatives)• Teacher-designed

Assessment(s) Requirements

• Each SLO must have at least one pre-assessment and a post-assessment measure.

• Each objective of the learning content must be measured by at least one item on both the pre- and the post-assessment.

• All assessments must be approved by an administrator before they are given to students.

• Assessments must be submitted in electronic format along with your SLO.

• The pre-assessment(s) must be administered before your SLO can be approved.– Scores are needed to make predictions

about growth expectations.

• If divergent items (more than one correct answer possible) are used - a scoring guide or rubric must be developed and attached in electronic format.– A scoring guide specifies for each item how

much credit will be given for correct and partially correct responses.

– A rubric is used to measure tiers of student performance for a comprehensive task.

Assessment(s) Options

• Using more than one assessment may result in a more reliable measurement of what students have learned.

• Use both divergent (more than one correct answer possible) and convergent (a single correct answer) to measure what students know more accurately.

• You may use the same test for the pre- and the post-assessment.

Assessment(s) Options

• Rubrics can be shared with students in advance with administrator approval. – Scoring guides are similar to answer

keys and cannot be shared with students in advance.

• You can allow partial credit for answers but make sure the scoring guide clearly describes how this would work for each item.

• You may give re-tests if given administrator approval at the time of SLO approval.

Assessment(s) Restrictions

• The pre- and post-assessments cannot be used as instructional materials in the class

• Assessments, rubrics, and scoring guides cannot be changed once they have administrator approval

Assessment(s) Cautions

• Even if developed by an “expert” – every test item may not be a good item.

• Balance objectives across the assessment according to you how you wish to weight them.

• Many assessments, including “ready-made” assessments or those created from testing software may have a lack of quality alignment and/or limited reports. – Make sure you will be able to collect the

data you will need to evaluate your SLO.

• Count only those items that are related to the learning objectives chosen.

Guidelines for Assessments

Consider each of the following:

• Alignment with SCOS or other appropriate objectives

• Bias Elimination

• Reliability

• Validity

• Growth

Formative vs. Summative Assessments:

• Formative Assessment

* is a PROCESS * is assessment FOR learning * conducted to improve and promote student

success

* conducted during learning

• Summative Assessment

* is a PRODUCT

* is assessment OF learning * conducted to grade student success

* conducted at the end of learning

Assessment Basics

Convergent vs. Divergent

• Convergent– Items where there is one

correct response

– Typically use lower order thinking

– Types:• Multiple choice• True/False• Matching• Fill in the blank

• Divergent– Items where there may

be more than one correct response

– Typically use higher order thinking

– Types:• Essays/papers• Portfolios• Projects• Performance

evaluations

Assessment Basics

Assessments Recommendations

• There should be at least 2 items to measure each objective

• Use multiple forms of measure including some convergent and some divergent items

• Tests should be rigorous– Rigorous implies depth of assessment

in terms of content and thinking skills required

– Rigorous does NOT mean length

Assessments Subcomponents

• For both the pre- and the post-tests:– Total number of items– Total number of possible points– How many assessments will you use?– Are the assessment items convergent

(one correct answer), divergent (more than one correct answer), or a both?

– Names of the tests

• Are you using the same, exact assessments pre- and post?

• How many re-tests of the post-assessments are allowed?

• Is there a scoring guide and/or rubric for each divergent assessment?

• Have any/all assessment items been used before? If so, explain.

• Where and from whom did the assessment(s) originate?

• Describe how you have ensured the assessment(s) is/are aligned to the content.

• Why is this the best way to assess student learning of your selected learning content?

• How free of bias is/are the assessment(s) (such as vocabulary; scenarios, etc.?

• Describe any testing modifications you will use and whether a proctor will be present.

• If using multiple assessments, how will a single pre-test score and single post-test score be calculated?

Growth Expectations

What are your expectations for how students will grow in their learning of the selected content over the interval, as measured by the pre- and post-assessment(s)?

Growth ExpectationsRequirements

• Each student in the population must have a growth expectation that is clear.

• At least 75% of the students included in the SLO must achieve the growth expectation in order to receive the compensation. (Higher if teacher so states.)

• Expectations must be rigorous and attainable and agreed upon between the teacher and the administrator.

Growth ExpectationsRequirements

• The growth expectation for each student will be shown on the Student Score Sheet in the “minimum expected post-test score” column.

Growth ExpectationsOptions

• Growth expectations can be differentiated in many ways, including– Setting different growth expectations for the top third,

middle third, and bottom third of students or– Setting different growth expectations for each

student

• You may contact TIF-LEAP or Curriculum and Instruction for assistance in selecting appropriate growth goals.

• You may indicate that more than 75% of your students will achieve expected growth, but that is the standard to which you will be held.

Growth ExpectationsCalculation

• You may use any method for calculating growth that works for you BUT:

• Remember that on the SLO Student Score Sheet you will be asked to list an expected score for EACH student.

SLO Student Score Sheet

On the SLO SSS you will record the following items when you create your SLO:

1. RAW PRE-TEST SCORE – NUMBER OF POINTS EARNED (no percents)

2. The number of possible points on the pre-test

3. Minimum expected score on the POST-test. This is the score you want your students to reach at the end of the SLO interval

DO NOT REPORT PERCENTS, INCLUDE PERCENT SIGNS, or FRACTIONS

SLO Student Score Sheet

Once your SLO has been completed,you will return to the SLO SSS and

complete the following columns1. RAW POST-TEST SCORE – Number

of points earned --- NO PERCENTS2. The number of possible points on the

POST-TEST3. Growth made = RAW POST-TEST

score MINUS RAW PRE-TEST score4. A yes or no column indicating whether

the student met the goal set or notDO NOT REPORT PERCENTS, INCLUDE

PERCENT SIGNS, or FRACTIONS

Growth ExpectationsCalculation Options

• How many more words per minute will be read? – My students will increase their reading by at least 25 words

per minute. (Here you would enter an expected score – so a student scoring 25 on the pre-test should have a 50 recorded as minimum expected post-test score)

• How much growth will students make across one or more parts of a rubric?– I want to see all of my students move up at least 1 level on

all criteria on my rubric. So, a student who scores in the “needs revision” level should move up to “competent”, those in the “competent” level should move up to “good”, those in “good” should move up to “outstanding”. Those in “outstanding” range already should maintain that status on all of the criteria. (We would prefer to see these designated as numbers if at all possible).

Growth ExpectationsRestrictions

• Vague statements of growth cannot be accepted.

• Make sure there will be no question about whether a student made the expected growth or not. Specify a clear number in the minimum expected score column as a goal for each student.

Growth ExpectationsCautions

• Plan ahead for potential loss of students. – If you want 85% of your students to meet your growth

expectation and you begin with a class of 20, you would need 17 to meet the goal. But if you reduce to only 10 students who are present the required time, you need 9 of 10 to meet the goal – obviously a little tougher to accomplish.

• Students who score high on a pre-assessment have very little room to grow. – You may want to state that they will maintain their

score, or within a narrow range of their score.

Growth ExpectationsCautions

• Be sure you will be able to determine if this type of score can be obtained from the test chosen.– Some tests do not allow you to determine scores by

goal or objective – they just result in an overall score. These are not suitable for use with SLOs.

• When developing rubrics and/or scoring guides, be sure to consider all types of possible answers they could give.

Growth ExpectationsSubcomponents

• What percent of students must achieve their minimum expected score?

• Describe the data reviewed and how it led to these growth expectations.

• How were growth expectations calculated for each student?

• How do you know your expectations are rigorous and attainable?

StrategiesRequirements

What instructional techniques and approaches will be used to facilitate student learning?

Requirements:• List multiple strategies for each SLO.• Each SLO must clearly align with

– The School Improvement Plan– The teacher’s Individual Growth Plan

StrategiesOptions

• Using effective, research-based instructional strategies is recommended.– Innovative strategies may be used with

administrative approval.

• Work with your administrators to ensure all subjects are covered in the SIP.

StrategiesOptions

Analysis of student workConcept/Thinking MapsConstructivismCooperative LearningDirect InstructionEffective QuestioningExploring MisconceptionsEntrance/Exit TicketsField Experiences/TripsFlexible/Strategic GroupingFoldables®Formative AssessmentsGallery CrawlGraphic OrganizersGuest SpeakersGuided PracticeHands-On InstructionHomework Independent PracticeInquiry

Inside-Outside CircleIntegration of Content AreasInternational ConnectionsInterviewsJigsawJournaling/NotebookingLaboratoriesLearning CentersLearning Styles FocusLeveled Reader UseManipulativesModeling (Teacher or Student)MnemonicsMultiple Intelligence Focus Note-Taking Paideia/Socratic SeminarsProjectsQuickwritingReader’s Theatre

Reinforcing and RecognizingShared/Paired PracticeSimulation/Role-PlayStudent Peer-Assessment Student Self-AssessmentTechnology IntegrationThink-Pair-ShareTotal Physical ResponseVisual CuesWhole Language Word Wall Other:

StrategiesCautions

• Increased student engagement does not necessarily result in improved achievement. (Compliance is different than engagement.)

StrategiesSubcomponents

• You will select strategies you intend to use in this SLO from the list

• Connection to the School Improvement Plan:– Aligned SIP outcome– Aligned SIP strategy– Aligned SIP task

• What other connections does this SLO have to your SIP or other initiatives to be implemented?

• What IGP goals, strategies, etc., are addressed?

StrategiesSubcomponents

• How will you differentiate instruction?• Please provide a brief statement as to

what research and/or other data supports the selected strategies.

• Which, if any, of the selected strategies are new for you to implement?

• What support and/or professional development will you see/request to help achieve your SLO?

StrategiesRestrictions

• Instructional strategies should be age, grade, ability level, and content appropriate.

Questions?

Remember to reference the Implementation Guide for SLOs found on Riverdeep for assistance.

Contact Information

General TIF-LEAP Contact Information

980-343-1124

tif.leap@cms.k12.nc.us

Susan Kirkendol

980-343-1609

susan.kirkendol@cms.k12.nc.us

Scott Reynolds

980-343-2708

scott.reynolds@cms.k12.nc.us

TIF-LEAP Training Fall 2009 Susan Kirkendol Senior Assessment Program Analyst Scott Reynolds Student...

Documents