Date post: | 04-Jan-2016 |
Category: |
Documents |
Upload: | barnaby-ferguson |
View: | 213 times |
Download: | 1 times |
®
Copyright © 2009 Educational Testing Service.
1
Observations from a Lifetime in Accountability Testing
Wendy M. YenETS
CERANovember 19, 2009
®
1. Everyone wants to help children learn.
Copyright © 2009 Educational Testing Service.
2
• Opinions differ about how to help children.• There is conflict in educational policy.
• Opinions are strongly held about how to help children.
®
2. Everyone wishes that every student would learn a lot.
Copyright © 2009 Educational Testing Service.
3
• All educators believe that their students’ test scores should be higher than they are.
• Many people work very hard every day to help students learn.
®
2. Everyone wishes that every student would learn a lot (cont.)
Copyright © 2009 Educational Testing Service.
4
• There must be problems with the existing education system.• Every system can be improved.• If the existing system is changed, it will be improved.
• The amount that students are learning is never enough.
There seem to be general beliefs that:
®
3. Educational systems are very complex.
Copyright © 2009 Educational Testing Service.
5
• Their social, economic, & political surroundings are constantly changing and being changed by education.• All the implications of any action or change cannot be foreseen.
• Educational systems are reactive and dynamic.
®
4. Within this complex system what can accountability testing contribute?
Copyright © 2009 Educational Testing Service.
6
• AT can spur thought into the meaning and alignment of content standards, curricula, instruction, tests, and performance standards.
• AT can help define what we want our students to learn and increase focus on it.
®
4. What can accountability testing contribute? (cont.)
Copyright © 2009 Educational Testing Service.
7
• AT can help us see what change is taking place & what change is not:
• From 2003 to 2009 the CST average increase per year in percent proficient was 2.5% in ELA and 1.8% in Math.
• Gaps in performance persist.
®
4. What can accountability testing contribute? (cont.)
Copyright © 2009 Educational Testing Service.
8
• Historical AT information provides setting in which we can evaluate the difficulty of goals and the resources needed to reach them.
• Stretch goals that are attainable can be developed.
®
5. There can be unintended negative consequences of accountability testing:
Copyright © 2009 Educational Testing Service.
9
• Everyone is stressed by AT.
• High standards are good but unrealistically high standards are counter-productive.
• AT can receive too much emphasis in evaluating educational effectiveness.
®
6. Within this ever-changing, imperfect system, what can a psychometrician contribute?
Copyright © 2009 Educational Testing Service.
10
• Offer honest advice about what works and does not work, what is accurate and fair and what is not
• Ask lots of questions
• Attempt to communicate meaningfully; explain “why”
®
6. What can a psychometrician contribute? (cont.)
Copyright © 2009 Educational Testing Service.
11
• Bias analyses
• A scientific basis for data analysis procedures, such as
• Scaling• Equating
• Documentation of those procedures
®
6. What can a psychometrician contribute? (cont.)
Copyright © 2009 Educational Testing Service.
12
• Equating is what allows score comparisons from year to year.
• It’s the equating, Stupid.
• If scores “bounce around,” they are not useful.• Changes in test structure undermine good equating.
®
6. What can a psychometrician contribute? (cont.)
Copyright © 2009 Educational Testing Service.
13
• Dedication to 100% accuracy
• Detailed focus on procedures, data, and results• Broad awareness of the possibility of the unexpected• Constant query: Do the results make sense?
®
6. What can a psychometrician contribute? (cont.)
Copyright © 2009 Educational Testing Service.
14
• Humility, given that
• Respect for the unique knowledge, good intentions, and hard work of others
• many people are affected by AT
• others, not AT, are the direct cause of student learning