Post on 22-Mar-2021
transcript
CEDEFOP –Employer Survey on Skill Needs in Europe
Project overview
Arnold Riedmann, TNS Infratest SozialforschungMiriam Gensicke, TNS Infratest SozialforschungBernd Dworschak, Fraunhofer IAOFrancis Green, Institute of Education, University of London
Cedefop Expert workshopPrague, 29/30 April 2010
Chart 2Cedefop expert workshop, Prague, 29/30 April 2010
Agenda
Aims and challenges of the project
The research group, its tasks and work organisation
Work packages of Phase 1
Methodological outline of the project
Chart 3Cedefop expert workshop, Prague, 29/30 April 2010
1. Aims and challenges
Chart 4Cedefop expert workshop, Prague, 29/30 April 2010
The challenges
The aim of the project
is to develop and pilot a survey instrument to reliably identify future needs of occupations, skills, competences and qualifications in enterprises in Europe, covering the whole economy.
As a result,
a detailed, robust and validated methodology should be made available for future regular use at EU-27 level that ensures data according to agreed international quality standards.
The tasks
range from agreeing the topics to be investigated, via developing adequate questionnaires and sampling frames to data collection and reporting.
Chart 5Cedefop expert workshop, Prague, 29/30 April 2010
Major aspects to be considered
Which type of questions related to current and future skill needs can be answered by employers?
…of different size-classes (firms with a specialised Human Resources department vs. firms without such a department) ?
…of different sectors (highly innovative sectors vs. “normal” sectors) ?
Which type and depth of information is needed in order to get an “added value” from an European employer survey?
Additional information must be complementary to other types of information (incl. existing Cedefop forecast tools), filling a real “gap”
Additional information must be useful for the policy making process
(How) Is it possible to bring the practical possibilities/limitations and the information needs in line with each other?
Chart 6Cedefop expert workshop, Prague, 29/30 April 2010
Different approaches
Critical view on approaches discussed in the feasibility study:
Vacancies (“normal” and “hard to fill” vacancies)
Occupational structure and its evolution over time (stock and flow)
Vocational training
These approaches have their merits, but have limitations especially regarding their usefulness for forecasting
Alternative proposal: “task-oriented” approach
Chart 7Cedefop expert workshop, Prague, 29/30 April 2010
Future of the project
Prospects for the future of the survey:
New survey coordinated by Eurostat?
New survey managed by Cedefop (similar to Eurofound or EU-OSHA surveys)?
(“Only”) toolkit for national skill need surveys?
Survey with compulsory or voluntary participation?
The future of the survey is still open
The current project thus has to cover different options/scenarios (e.g.: national/international, small/large samples, ad hoc/regular, compulsory/voluntary…)
Chart 8Cedefop expert workshop, Prague, 29/30 April 2010
2. The research group, its tasksand work organisation
Chart 9Cedefop expert workshop, Prague, 29/30 April 2010
The research group: institutes and persons
“Core research group”:
TNS Infratest Sozialforschung (Arnold Riedmann, Miriam Gensicke)
Fraunhofer IAO, Stuttgart (Bernd Dworschak, Hartmut Buck, Alexander Karapidis, Helmut Zaiser)
Prof. Francis Green, LLAKES, University of London
“Extended expert group”:
Jos Sanders/Karolus Kraan (TNO Work and Employment, NL)
Lina Vaitkute/Beleckiene Giedre (Qualifications and VET Development Centre, LT)
Olympia Kaminioti (Employment Observatory Research Informatics S.A., EL)
Vera Czesana/Hana Zackova (National Training Fund NVF, CZ)
Magdolna Benke (National Institute for Vocational and Adult Education NIVE, HU)
John McGrath (Irish Employment and Training Authority FAS, IE)
Cristina Mocanu (National Research Institute for Labour and Social Protection INCSMPS, RO)
Ferran Mané (URV Foundation, ES)
Chart 10Cedefop expert workshop, Prague, 29/30 April 2010
Tasks and time schedule – All PhasesYearMonth 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12
PHASE 1: Development of the survey instrument
WP 1 Specification of required survey outcomesDef. of information needs and required survey outcomes
Analysis of different approaches for operationalisation
WP 2 Statistical methodology and survey guidelinesChoice of approach(es);development of survey concept
WP3 Development of the master questionnaireDevelopment of draft master questionnaire versions
PHASE 2: Improvement of the questionnaire and pilot survey preparation
WP 4:Development of national quest.; pre-test
Preparation of national language versions, pre-testing and analysis of pre-test results
Final questionnaire revisions (master & national)
WP 5: Preparation of the pilot survey
Writing of pilot survey manual
PHASE 3: Implementation of the pilot survey
WP 6 Fieldwork
Preparation of fieldwork and fieldwork phase
WP 7: Data processing and quality assessment
Weighting, data checks & data validation
WP 8 Data analysis, summary pilot survey report and manual for final survey instrument
Data analysis & report
Writing of a handbook for employer surveys on skill needs
20112010
Chart 11Cedefop expert workshop, Prague, 29/30 April 2010
Task distribution I: TNS Infratest Sozialforschung
Background/ relevant experiences:Survey institute with a European-wide network of fieldwork institutes (TNS group)
Broad experiences in the conceptualisation and implementation of surveys among employers: IAB Establishment Panel, Eurofound establishment surveys (ESWT 2004/05 and ECS 2009), Establishment survey of the European Agency for Safety and Health at the Workplace (EU-OSHA) etc.
Experiences in the field of skill research: ADeBar studies (in cooperation with Fraunhofer IAO), PIAAC Germany, BiBB/BAUA Employee survey (question bloques on qualifications and training) etc.
Tasks in the project:Formal lead
Main responsibility for statistical concept
Involvement in questionnaire development
Main responsibility for phases 2 and 3 (translations, pre-testing, fieldwork, data-cleaning, reporting)
Chart 12Cedefop expert workshop, Prague, 29/30 April 2010
Task distribution II: Fraunhofer IAO
Background/ relevant experiences:Founding member and coordinator of the German FreQueNZ network on early identification of skill needs (started in 1999)
Member of CEDEFOP’s Skillsnet
Experience in the field of early identification of skill needs for many years: Various sectoralstudies (e.g. ADeBar, in cooperation with TNS Infratest, on renewable energies, pharma, facility management and many others)
Tasks in the project:Lead in Phase 1 (conceptual work, questionnaire development)
Participation in phases 2 and 3 (pre-testing, fieldwork, reporting)
Chart 13Cedefop expert workshop, Prague, 29/30 April 2010
Task distribution III: Prof. Francis Green
Background/ relevant experiences:UK skills survey
Major role in the development of the JRA module to be used in the PIAAC surveys
Participation in the MEADOWS project (development and testing of employer survey questions related to organisational and technological changes)
Tasks in the project:Participation in all work packages of Phase 1
Leading role in the questionnaire development
Participation in phases 2 and 3 (pre-testing, fieldwork, reporting)
Chart 14Cedefop expert workshop, Prague, 29/30 April 2010
Task distribution IV: Extended expert group
Background/ relevant experiences:Direct involvement in the conceptualisation of various skill need surveys, both among employers and employees (e.g. in Spain, Romania, Ireland)
Experiences with European wide skill surveys as users ( usefulness for the political process at the national level)
Participation in Cedefop’s Skillsnet
Tasks in the project:Support in the definition of information needs
Support in the choice of approaches and design of the survey concept
Advice on the practicability of the preferred approach in the own country
Support in the design and revision of questionnaires
Chart 15Cedefop expert workshop, Prague, 29/30 April 2010
3. Work packages of Phase 1
Chart 16Cedefop expert workshop, Prague, 29/30 April 2010
Work Package 1: Specification of required survey outcomes
Critical review of feasibility study
Identification of key political questions with respect to skill need forecasting
Assessment of (national or cross-national) employer surveys dedicated to future skill needs (approaches, outcome, data-collection methodology)
Definition of information gaps to be filled with the survey and of expectations on part of Cedefop and its stakeholders
Which information gaps do exist in the field of skill need forecasting?
Which of these could be filled with an employer survey?
Which approach(es) would be most suitable to this end?
Results will be documented in the interim report (June 2010)
Chart 17Cedefop expert workshop, Prague, 29/30 April 2010
Work Package 2: Statistical methodology and survey guidelines
Development of a concept for a detailed and complete employer sample survey:
Choice of an approach and description of the approach
Choice of data collection methodology and description of methodological implementation:
Data-collection method
Sampling strategy
Translation procedure
Etc….
Selection of countries for the pilot interviews
Results will be documented in the final report (November 2010)
Chart 18Cedefop expert workshop, Prague, 29/30 April 2010
Work Package 3: Development of the master questionnaire
Procedure:
Joint development of 1st draft questionnaire within the core group
Discussion of the draft within the research group and with Cedefop
Revised 2nd and 3rd draft (modified on base of further comments)
Important criteria for all draft versions:
Easily understandable questions
Questions that can reasonably be answered in an interview situation (level of knowledge on part of respondent)
Cross-national comparability! (differences in definitions/concepts to be taken into account)
End product: English master questionnaire in a version (almost) ready for being pre-tested
Chart 19Cedefop expert workshop, Prague, 29/30 April 2010
4. Methodological outline of theproject
Chart 20Cedefop expert workshop, Prague, 29/30 April 2010
Data collection methodology – general options I4 basic options:
Face-to-face (PAPI or CAPI)
+ High response rates, possibility to apply visual means (e.g. lists of occupations), complex filtering possible
- Most expensive method (in most countries); in PAPI version no ad hoc data plausibility control
Telephone (CATI)
+ Reasonably high response rates and low response bias (interviewer-based), complex filtering possible
- No visual means applicable, limited interview duration, no possibility to look up statistical information in company files
+ Application of visual means, respondent can take time to reflect on questions, to discuss issues with colleagues or to look up information in the company files
- In some countries extremely low response rates (if voluntary survey); enhanced danger of a response bias; no ad hoc data quality control (call-backs)
Online (CAWI)
+ similar advantages as mail, complex filtering and ad hoc data plausibility control possible
- Similarly low or even lower response rates than mail; danger of response bias (incomplete questionnaires), no full internet coverage of businesses
Chart 21Cedefop expert workshop, Prague, 29/30 April 2010
Data collection methodology – general options II
There is no generally best method; method has to suit the chosen approach towards the mapping of skill needs
+ CATI/CAPI: higher response rates & low self-selection bias due to interviewer intervention; complex filtering possible
+ Mail/(Online): More time to reflect & time to discuss issues with colleagues; particularly useful for the questions on FUTURE skill needs; but: response rates for a voluntary survey unacceptably low in some countries, in mail no complex filtering possible
Method mix as a possible further option
Unit of enquiry: Company or establishment
Chart 22Cedefop expert workshop, Prague, 29/30 April 2010
Data collection methodology I – Proposed pilot design
Preliminary recommendations for the pilot survey:
CATI survey or maybe 2-3 countries by CATI and 2-3 countries by mail
1.000 interviews per country
20-25 minute interview
Unit of enquiry: Preference for establishment level for reasons related to contents (but: feasibility & later usage!)
Universe: Establishments/companies from 1 dependent employee onwards, all relevant sectors
But: Changes of the pilot survey design are still possible
Main criterion: Which method renders more/higher quality information? The method has to suit best the defined information needs and the approach finally chosen
Chart 23Cedefop expert workshop, Prague, 29/30 April 2010
Thank you for your attention!