Paul [email protected] State CoordinatorMichigan Department of Education
National Conference on Student Assessment, Minneapolis, MinnesotaJune 2012
Paul [email protected] State CoordinatorMichigan Department of Education
National Conference on Student Assessment, Minneapolis, MinnesotaJune 2012
Describing NAEP Background Question Data: A Cautionary Tale
Describing NAEP Background Question Data: A Cautionary Tale
NAEP Background Question Data Analysis
NAEP Grade 4 Reading Trends ComparisonNAEP Grade 4 Reading Trends Comparison
2
NAEP Background Question Data Analysis
NAEP Grade 4 Reading Trends Comparison (Enlarged)NAEP Grade 4 Reading Trends Comparison (Enlarged)
3
NAEP Background Question Data Analysis
What is different about these comparisons?What is different about these comparisons?
4
NAEP Background Question Data Analysis
What is different about these comparisons? (enlarged)What is different about these comparisons? (enlarged)
5
Avg SS 202 214 219 222 * 229 237 238 * 217 219 221 * 215 225 224 Nat. Pub. MA MI OHAvg SS 220 237 219 224
NAEP Background Question Data Analysis
Life is Good!Life is Good!
6
NAEP Background Question Data Analysis
NAEP Trends in Grade 4 Reading Average Scaled ScoresNAEP Trends in Grade 4 Reading Average Scaled Scores
NAEP Background Question Data Analysis
Wait a Minute, What Happened?Wait a Minute, What Happened?
8
NAEP Background Question Data Analysis
Wait a Minute, What Happened? (enlarged)Wait a Minute, What Happened? (enlarged)
9
Avg. SS * 217 219 222 * * 191 190
NAEP Background Question Data Analysis
What explains this? What about other TUDAs?What explains this? What about other TUDAs?
10
NAEP Background Question Data Analysis
What about OHIO and Cleveland?What about OHIO and Cleveland?
11
NAEP Background Question Data Analysis
What about MA and Boston?What about MA and Boston?
12
NAEP Background Question Data Analysis
Why the discrepancies?Why the discrepancies?
Two Possibilities
A. If the data is in error? Differences in respondent question interpretation –
definitions? Need to improve the question
Social Desirability
B. If data is “true,” perhaps more interesting? Internal School Variables External School Factors
13
NAEP Background Question Data Analysis
Checking Social Desirability 09 to 11 ComparisonChecking Social Desirability 09 to 11 Comparison
14
NAEP Background Question Data Analysis
Why the discrepancies?Why the discrepancies?
If data is “true,” perhaps more interesting?What are the possible intervening factors?
Internal School Variables Professional development Rigor Instructional Methods/Impact Professional knowhow vs. execution Administration, management and follow through Resources and resource utilization
External School Factors Chronic Absenteeism Concentration of Poverty Health Factors Cultural differences
15
NAEP Background Question Data Analysis
Conclusions?Conclusions?
Descriptive data can be helpful and informative
Have to keep reminding ourselves it is not causative/predictive
Are we jumping to solutions without fully understanding how best to solve these problems of large urban school districts?
Do we infer too much from snapshot assessment results about the quality of teaching and administration?
Does arguing otherwise sound like excuses?
Aren’t we all guilty of trying to connect the dots? Trying to simplify the story Making inferences beyond what the data is telling us.
16
NAEP Background Question Data Analysis
FinallyFinally
Prediction is very hard, especially about the future. - Yogi Berra
17
NAEP Background Question Data Analysis
Contact:
Paul Stemmer, Ph.D., NAEP State Coordinator
Michigan Department of EducationOffice of Educational Assessment and
AccountabilityPO Box 30008
Lansing, MI 48909
(517)[email protected]
For More Information:
![Technology-Forum-EN-Stemmer-Imaging-Hyperspectral ...€¦ · sTEMMER@ sTEMMER@ sTEMMER@ sTEMMER@ 1900 nn-, 1470 r,rn 5260 6800 avl+ * Raman peaks are given in [805]. 401 nm 24940](https://static.documents.pub/doc/80x56/60508727ac4f6661f64aa54f/technology-forum-en-stemmer-imaging-hyperspectral-stemmer-stemmer-stemmer.jpg)
