Post on 15-Jan-2016
description
transcript
Exploring Alternate AYP Designs for Assessment
and Accountability Systems
1
Dr. J.P. Beaudoin, CEO, Research in Action, Inc.Dr. Patricia Abeyta, Bureau of Indian Education
NIEA 2008 Annual ConferenceSeattle, Washington
Session Goals
2
• Frame the technical and resource considerations for decision-makers
• Provide clarification on the
regulatory provisions associated with NCLB
• Share assessment and accountability design features
Guiding Questions
3
urpose
How does the purpose for the change/action serve Native children?
irection What direction will the change/action
provide the educational community?
otivation To what degree will the change/action
motivate the Native peoples?
30,000ft. SYSTEM VIEW
4
School System Components
5
Academic Content Standards“What do students need to know and be able to demonstrate?”
• Instruction “How is content organized, provided, modeled, and expanded upon?”
• Assessment “To what degree is the content expressed accurately by the leaner?”
• Accountability “Which learning indicators are being reached, showing progress, or remaining unchanged?”
Integration
6
Things to Consider…
• Changes to one component will have some influence on the remaining components
• Improvements in one area often result in benefits throughout the system
• Isolated changes create bulkization, inefficiencies, and internal stressors
7
Leverage
Compliance
If you change the academic content standards used for AYP…
If you change the assessments used for AYP…
If you change how AYP is defined…
8
Then you must provide evidence that addresses Section 1 of the USDE’s Peer Review Guidance
Then you must provide evidence that addresses Sections 2-7 of the USDE’s Peer Review Guidance
Then you must create an AYP Workbook that address 10 accountability principles.
Overview: Standards and Assessments Peer Review
9
Conditioned on what is changed, evidence must be presented to the BIE and USDE Peer Reviewers…
Section 1-Content Standards
Section 2-Achievement Standards
Section 3-Coherent Assessment System
Section 4-Technical Quality
Section 5-Alignment
Section 6-Inclusion
Section 7-Reporting
Overview: AYP Workbook
10
Principle 1-Includes all schools and districts
Principle 2-Includes all students
Principle 3-Includes an allowable method for AYP determinations
Principle 4-Includes annual accountability decisions
Principle 5-Includes subgroup accountability
Principle 6-Based on academic assessments
Principle 7-Has additional indicators
Principle 8-Separate decision for reading and mathematics
Principle 9-System validity and reliability
Principle 10-Includes participation rates
Conditioned on what is changed, policy and empirical evidence must be submitted for review by the BIE and USDE…
11
Overview: Review Process
ASSESSMENT
12
Generic Assessment Life Cycle
13
Foundational Components
Academic content standards Articulate what student are expected to
know and be able to do Grade level expectations in reading/ELA
and mathematics Grade span clusters in science Contain skill and process knowledge
14
Foundational Components
Academic achievement standards Performance level descriptors No less than three achievement levels
(with applicable labels) “Cut-Score” established using a
technically valid standard-setting approach (e.g, Modified Bookmark, Body of Work, Teacher Judgment)
Established by subject matter experts (teachers) and other panelists with applicable skills
15
Foundational Components
Assessments aligned to intended purposes
Assessments in grades 3-8 and high school: Reading/ELA Mathematics Science (3 grade clusters)
Standardized administration and scoring procedures
16
Foundational Process
Typically 18-24 month development process
Documented evidence Reviewed by external (to USDE)
experts Evaluated evidence against Peer
Review Guidance Resubmissions typical
17
USDE PEER REVIEW GUIDANCE
Standards and Assessments
18
Section 1: Academic Content Standards
Students know and are able to…
Content standards are:
Adopted in mathematics; reading/ELA Adopted in science (grade spans) Coherent and rigorous Developed by stakeholders
19
Section 2: Academic Achievement Standards
The level students have mastered the content is…
Achievement standards are: Adopted in mathematics, reading/ELA, and
science Assigned achievement levels and content-
based descriptors Established using an appropriate standard-
setting procedure Aligned to the content standards
20
Section 3: Coherent Assessment System
The different assessment components within the larger assessment system contribute…
A coherent system: Is based on state and/or local assessments Provides integrated achievement data Aggregates only comparable measures Utilizes multiple measures focused on
higher order thinking skills Implements alternate assessments
21
Section 4: Technical QualityThe technical quality of the assessments is…
High quality assessments are: Valid measures based on their purpose Constructed to measure the content standards Accurate and consistent Fair and accessible Consistent across different forms and events Standardized in their administration and
scoring
22
Section 5: Alignment
The assessments reflect the content standards characteristics by…
Strong alignment is provided by using:• Using procedures to improve alignment
• Techniques to match the content standards and patterns of emphasis
• Capture the range and depth of knowledge in the standards
• Pattern of emphasis
• Approaches to measure both content and process knowledge
• Content-based performance descriptors
23
Section 6: InclusionThe assessment system includes all students
by…
Students participate in the assessments as demonstrate by:
Impact data showing high rates of student participation
Affording students with disabilities participation under accommodated conditions
Accommodating students learning to speak English
Including migrant and other highly mobile students
24
Section 7: ReportingThe assessment reports provide accurate and
timely information about student achievement by…
Student achievement data is: Reported using valid and credible procedures Based on all students from valid assessments Reported at the individual student, subgroup,
school, district, and state-levels Disseminated in a timely manner Easily understood by parents and the public
25
USDE REVIEW GUIDANCEAYP Workbook
26
Principal 1: Includes All Schools and Districts
Includes all schools and districts Holds all schools to the same
criteria Incorporates the academic
achievement standards Provides information in a timely
manner Includes report cards Includes rewards and sanctions
27
Principal 2: Includes All Students
Includes all students Consistently defines full
academic year (FAY) Includes mobile students
28
Principal 3: Method of AYP Determinations
Applies universal proficiency by 2013-2014 as the long-term goal
Uses a method to make subgroup, school, and district AYP determinations
Establishes a starting point with statewide annual measurable objectives (AMOs)
Establishes intermediate goals29
Principal 4: Includes Annual Decisions
Determines annually the progress of schools and districts
30
Principal 5: Includes Subgroup Accountability
Includes all the required student subgroups
Holds schools and LEAs accountable for the progress of student subgroups
Includes students with disabilities (SWD) and limited English proficient (LEP) students
Includes an established minimum n-count
Includes safeguards to protect student privacy
31
Principal 6: Based on Academic Assessments
Based primarily on academic assessments
32
Principal 7: Includes Additional Indicators
Includes graduate rate for high schools
Includes an additional academic indicators for elementary and middle schools
Establishes valid and reliable additional indicators
33
Principal 8: Includes Separated Decisions for Reading/ELA and Math
Holds students, schools, and districts separately accountable for reading/ELA and mathematics
34
Principal 9: System Validity and Reliability
Produces reliable decisions Produces valid decisions Addresses changes in
assessments and student populations
35
Principal 10: Includes Participation Rates
Includes a way to calculate the rate of participation on the statewide assessments
Applies the 95% participation criteria to student subgroups and small schools
36
37
Research in Action, Inc.
World-Class Educational Quality™
Voice: 1.225.571.2408 E-Mail: ria2001@eatel.netFax: 1.225.644.8472 Website: www.ria2001.org