Home >Documents >The CSES Questionnaire Design Process

The CSES Questionnaire Design Process

Date post:22-Feb-2016
Category:
View:67 times
Download:0 times
Share this document with a friend
Description:
The CSES Questionnaire Design Process. David Howell University of Michigan [email protected] Jessica Fortin GESIS – Leibniz Institute for the Social Sciences [email protected] European Survey Research Association (ESRA) Lausanne, Switzerland July, 2011. CSES Questionnaire. - PowerPoint PPT Presentation
Transcript:

Slide 1

The CSES Questionnaire Design ProcessDavid HowellUniversity of [email protected]

Jessica FortinGESIS Leibniz Institute for the Social [email protected]

European Survey Research Association (ESRA) Lausanne, SwitzerlandJuly, 2011

CSES QuestionnaireOver 60 countries participate

Common module of survey questions new theme every five years some questions the same some different according to new theme to be run as is

Administrative, demographic and voting variables to national standards

2Questionnaire Design Process

Questionnaire Design ProcessA lengthy development process

Over a three year period, 2008-2011

Iterative

As transparent as possible

Collaborative, with broad input

Planning Committee FormationA Planning Committee, comprised of, selected by, and informed by collaborators, designs and oversees each Module.

Public call for nominationsNominating Committee recommends membersPlenary Session confirms list and suggests additional membersResult = 20 members from 15 countries

Theme DevelopmentEvery five years a new module is developed to specifically address a big question in science.

Multiple public calls for theme proposals Task Force collects proposalsTask Force presents and evaluates proposals for Plenary Session and Planning CommitteeTheme(s) tentatively decided upon before questionnaire design begins

Questionnaire DevelopmentAfter a theme is arrived at, attention turns to developing the questionnaire for it.

Subcommittees formed to pursue the new theme(s) and improvements to past questionsSubcommittees return to proposers for revisions to their original proposalsDraft questionnaire producedPlanning Committee discussion, revisionRepeat as necessary

PretestingThe draft questionnaire is pretested cross-nationally in different contexts.

Partial pretests in Brazil, Germany, UKFull pretests in Canada, IrelandReports received from pretest countries

Questionnaire FinalizationAfter having received extensive feedback, the questionnaire can now be finalized.

Subcommittees formed to address sections outside of the new theme(s)A Plenary Session provides final feedback Review of pretest results, subcommitteesPlanning Committee makes final changesFinal questionnaire disseminated

Challenges

Design Challenges10-15 minutes is not much questionnaire time but, pressure to reduce collaborator burden competing with other content on surveys competing with other comparative studies

Existing, validated questions preferred

Priority to cross-national questions must be applicable in most all countries must work well across most all countries must be feasible in a variety of study designsInformation ChallengesA lot of people are involved Managing and tracking lots of input Reconciling many opinions and approaches Consensus not always possible

Making decisions with imperfect information More testing in more contexts would always be better, but relies on volunteers who...Have an election coming up (optimally)Have a funded survey with available spaceCan produce reports in a timely fashion Eventually decisions must be finalized

Collective ChallengesA lot of countries are involved Cross-national and cross-cultural applicability and equivalence Accommodating the variety of institutional arrangements in the many countries Difficult to predict or account for all situations, even with broad cross-national participation in the process

Where to go from here

Where to go from hereA formal post-process evaluation Weve done this a number of times now What worked well, what didnt Consider improvements Formalize and document process for next time

Some possibilities:

More extensive pretesting Allow more time for pretesting Test in more cultures and contexts Possible use of focus groupsWhere to go from hereMore focus on cross-national/cultural issues... concept equivalence... collaborator/interview instructions... translation issues

More methodological working groups

Learning from the approaches of other cross-national studies

Thank you for your time!

To learn more about CSES, or to download data:www.cses.org

...or email your questions to:[email protected]

Popular Tags:

Click here to load reader

Reader Image
Embed Size (px)
Recommended