+ All Categories
Home > Documents > Determining Validity For Oklahoma’s Educational Accountability System

Determining Validity For Oklahoma’s Educational Accountability System

Date post: 04-Jan-2016
Category:
Upload: madge
View: 28 times
Download: 0 times
Share this document with a friend
Description:
Determining Validity For Oklahoma’s Educational Accountability System. Prepared for the American Educational Research Association (AERA) Oklahoma State Department of Education April 15, 2004. Education Perspective for Oklahoma, 2002-2003. - PowerPoint PPT Presentation
25
Determining Validity For Oklahoma’s Educational Accountability System Prepared for the American Educational Research Association (AERA) Oklahoma State Department of Education April 15, 2004
Transcript
Page 1: Determining Validity For Oklahoma’s Educational Accountability System

Determining Validity For Oklahoma’s Educational Accountability System

Prepared for the American Educational Research Association (AERA)

Oklahoma State Department of Education

April 15, 2004

Page 2: Determining Validity For Oklahoma’s Educational Accountability System

Education Perspectivefor Oklahoma, 2002-2003

• All Local Education Agencies (LEAs) in the State of Oklahoma participate in Title I programs.

• There are 541 total LEAs44 LEAs with a student enrollment of 2,500 or

greater201 LEAs with a student enrollment between

499 and 2,500296 LEAs with a student enrollment of less

than 500

Page 3: Determining Validity For Oklahoma’s Educational Accountability System

Oklahoma Demographics, 2003Total school districts: 541

Independent (K-12): 430

Elementary (K-8): 111

Total School Sites: 1791

Teachers: 47,259

Page 4: Determining Validity For Oklahoma’s Educational Accountability System

Oklahoma Demographics, 2003 Average Daily Membership: 618,358

Special Education (K-12): 13.6% 83,812

Alternative Education (K-12): 2.5% 15,312

Page 5: Determining Validity For Oklahoma’s Educational Accountability System

Oklahoma Demographics, 2003American Indian/Alaskan: 17.9%

Asian/Pacific Islander: 1.5%

Black/Non-Hispanic: 10.9%

Hispanic: 7.0%

White Non-Hispanic/Other: 62.6%

Page 6: Determining Validity For Oklahoma’s Educational Accountability System

State Standards, Assessments & Accountability Timeline

1990 - Oklahoma Education Reform and Funding Act of 1990

1991 - Initial development of core curriculum with statutory requirement for

review and revision every 3 years

Page 7: Determining Validity For Oklahoma’s Educational Accountability System

1992 - Legislation mandated the development of Criterion-Referenced Tests (CRTs) in seven (7) content areas for Grades 5, 8, and 11

1993- First review and revision of core curriculum, Priority Academic Student Skills (PASS)

1995- Implementation of CRTs in Grades 5, 8, and 11

State Standards, Assessments & Accountability Timeline

Page 8: Determining Validity For Oklahoma’s Educational Accountability System

•PASS revisions – 1997, 2000, 2003• Revisions have provided more specific, detailed and clear standards• PASS includes the Oklahoma School Testing Program (OSTP) “blueprints” for the CRTs

State Standards, Assessments & Accountability Timeline

Page 9: Determining Validity For Oklahoma’s Educational Accountability System

1989 - Low-performing, high challenge schools identification process adopted in state law; includes sanctions and technical assistance

1999 - Academic Performance Index (API) adopted in state law; includes API Awards

State Standards, Assessments & Accountability Timeline

Page 10: Determining Validity For Oklahoma’s Educational Accountability System

Oklahoma’s Accountability System

To guide instruction

To measure achievement

To promote a climate of

change

Proficiency for all

Content and Performance Standards

Assessments

Additional Indicators

Decision Rules

Sanctions and

Rewards

Page 11: Determining Validity For Oklahoma’s Educational Accountability System

Validity Questions for Oklahoma’s Accountability System

Do our standards help guide instruction?

Do our assessments and performance indicators measure performance standards (i.e. achievement)?

Do our decisions rules accurately identify schools?

Do our sanctions and rewards promote change towards intended outcomes?

Page 12: Determining Validity For Oklahoma’s Educational Accountability System

Key Elements in Determining Validity and Reliability

1. Ensure that the system has provided the intended outcomes.

2. Conduct research on additional information to corroborate findings.

3. Analyze design and implementation of each component of the system.

4. Conduct Analysis on several levels.

Page 13: Determining Validity For Oklahoma’s Educational Accountability System

Sources of External Evidence

• Studies and research using additional indicators to corroborate findings or results of the system.

• Surveys on attitudes and opinions.

• Outside reviews for components of system.

Page 14: Determining Validity For Oklahoma’s Educational Accountability System

Sources of Internal Evidence

• Ongoing review of policies and procedures.

• Data analysis for various levels such as State level, district level, and for particular types of schools.

• Evidence of Quality Control measures and Data Audits.

• Trend analysis over time.

Page 15: Determining Validity For Oklahoma’s Educational Accountability System

PASS - external reviews have included:• 2001-02: Achieve, Inc.• 2001-02: Southwest Educational

Development Laboratory (SEDL)• 2001-02: Technical Issues in Large Scale

Assessment (TILSA)• 1997-2003: “Quality Counts” report cards by

Education Week

Oklahoma’s Steps Towards ValidationStudies and Reviews

Page 16: Determining Validity For Oklahoma’s Educational Accountability System

Studies and Reviews

TILSA Review: Curriculum and Assessment Alignment Study (2001-02) with

Dr. Norman Webb, University of Wisconsin• Results of the study enabled Oklahoma to conduct

more in-depth alignment of standards and assessments

• Provided common language to facilitate communication in building curriculum and designing aligned assessments

Page 17: Determining Validity For Oklahoma’s Educational Accountability System

Research Study (2002-03) by

Dr. John Poggio, University of Kansas

• Study to determine levels of cognitive complexity of multiple choice test items

Studies and Reviews

Page 18: Determining Validity For Oklahoma’s Educational Accountability System

Implementation Survey (January 2002)

• After initial implementation of the API an attitude and opinion survey was collected.

• Additional surveys specifically on Adequate Yearly Progress (AYP) will be collected in October 2004.

Studies and Reviews

Page 19: Determining Validity For Oklahoma’s Educational Accountability System

• Study on School Improvement Schools– Collecting additional local LEA assessments

from additional grades.

• Purpose: – To corroborate the results of the decision rules.– To look at trends of schools being identified as

school improvement .

Studies and ReviewsOngoing Study

Page 20: Determining Validity For Oklahoma’s Educational Accountability System

Grants and ProjectsNo Child Left Behind (NCLB)

Enhanced Assessment Grant

• Oklahoma selected as lead state in collaborative curriculum/assessment alignment project with Council of Chief State School Officers (2003)

• Continues the alignment work done in 2001

• Includes vertical alignment of curriculum and assessments (Grades 3-8)

• Includes alignment of special education alternate assessments

Page 21: Determining Validity For Oklahoma’s Educational Accountability System

Grants and Projects

Project with Council of Chief State School Officers (SCASS) - 2003

• Develop survey tools to determine the level of implementation of a state’s standards-based language arts curriculum

• To provide support for teachers in meeting NCLB requirement to increase student achievement

Page 22: Determining Validity For Oklahoma’s Educational Accountability System

PoliciesA few examples of changes or development

as a result of reviewing the system.

• Developed policy for appeals of Adequate yearly progress.

• Developed a Technical Advisory Committee to review and offer expertise in regards to technical issues of the State testing program.

• Implemented new rules and regulations regarding data audits and security.

Page 23: Determining Validity For Oklahoma’s Educational Accountability System

AYP Statistics• 367 (20.5%) of Oklahoma Schools did not

make AYP.• 198 (36.5%) of Oklahoma School Districts

did not make AYP.• 1.7 is the average number of subgroups that

did not make AYP by school.• 1.5 is the average number of subgroups that

did not make AYP by district.

Page 24: Determining Validity For Oklahoma’s Educational Accountability System

AYP Statistics

• Total number of subgroups not meeting specific performance targets, ordered from greatest to least.– Reading target– Math target – Additional Indicator (attendance or graduation)– Participation target

Page 25: Determining Validity For Oklahoma’s Educational Accountability System

AYP Statistics

• Total number of subgroups not meeting specific performance targets, ordered from greatest to least.– Reading target– Math target– Additional Indicator (attendance or graduation)– Participation target


Recommended