ICCS Main Survey Overview

National Research Coordinators Meeting Madrid, February 2010

NRCMeetingMadrid

February 2010

Content of presentation

• Overview of participating countries and international options

• Data collection and quality control• Data processing and analysis• Sampling outcomes• Reporting of ICCS data

NRCMeetingMadrid

February 2010

Reporting of ICCS data

• International Report and European Regional Report last week of June 2010

• Latin American and Asian Regional Reports to be released some time afterwards (not yet decided)

• Technical Report and International Database to be released in late November 2010

NRCMeetingMadrid

February 2010

Participation

• 38 countries or educational systems participated in ICCS

• 30 different languages were used in survey

• 35 countries participated in regional modules:– European: 24 participants– Latin American: 6 participants– Asian: 5 participants

NRCMeetingMadrid

February 2010

International options

• Student Questionnaire: – Ethnicity: 19 participants– Household members: 37 participants– Religion: 29 participants

• Teacher Questionnaire: – CCE teachers: 37 participants

• Some countries used incorrect filter question!

• Trend study– 17 participating countries– Some with additional grade data

NRCMeetingMadrid

February 2010

Data collection

• Southern Hemisphere school calendar– October 2008 -December 2008

• Northern Hemisphere school calendar– February - May 2009

• Three countries on SH school calendar tested at beginning of new school year– Results to be annotated (not yet in draft

report tables!)

NRCMeetingMadrid

February 2010

Quality Control - 1

• Adaptation review– Discussion of National Adaptations with

International Study Centre (ISC)

• Translation verification– Review of translation by language experts

(organised by IEA Secretariat)

• Layout verification– Final optical check by ISC

NRCMeetingMadrid

February 2010

Quality Control - 2

• International quality monitoring– Organised by IEA Secretariat– ~15 schools per country visited

• National quality monitoring– Organised by national centres– Recommended to visit 10% of tested

schools

• Re-verification of specific items– Translation checks for items that behaved

differently (after scaling analysis)

NRCMeetingMadrid

February 2010

Data Processing

• IEA DPC received data files and undertook cleaning, weighting and computation of item statistics

• ISC at ACER:– Analysis of item dimensionality– Scaling analysis– Item adjudication– Final scaling– Analysis for reporting tables

NRCMeetingMadrid

February 2010

Sampling outcomes - 1

• Sample participation categories1. Reached before inclusion of

replacement schools > no annotation

2. Reached after inclusion of replacement schools > annotation

3. Not reached even after inclusion of replacement schools > reporting in separate section of tables

NRCMeetingMadrid

February 2010

Sampling outcomes - 2

• Sampling adjudication at meeting in Hamburg 7-8 December 2009

• Some countries could not be adjudicated before mid-January 2010

• National centres were informed about outcomes via email by the ISC

NRCMeetingMadrid

February 2010

Sampling outcomes - 3• Separate adjudication and reporting rules for

student and teacher survey• Student survey adjudication

– 8 countries reached requirement only after replacement

– 2 countries

• Student survey adjudication– 5 countries reached requirement only after

replacement– 9 countries failed to meet sampling participation

requirements– Two countries will not be included in report

NRCMeetingMadrid

February 2010

Questions or comments?