QCM system in the PSOs

Good practices from the EU Countries

Søren B Schmidt

17.01-18.01.2013Shenyang

2

European Union – Qualitywise

• 27 Member states• 27+ Different Languages• 27+ Different cultures• This leads to a variety of approaches to quality management• There is no common EU legislation on quality management • Most common approaches to Quality Management include

• EFQM Excellence Model• The ISO 9000 family. • Lean

3

European Union – Public Sector

• The EFQM Excellence Model open for the public sector• EFQM Excellence Awards and Prizes

– is to recognize Europe’s best performing organizations, whether private, public or non-profit.

• 1998 / 1999: Development of the Common Assessment Framework (CAF)

• 2000: Launch of the first version of the CAF model– Developed by the public sector for the public sector– Inspired by the EFQM Excellence model

• 2001: Creation of the European CAF Resource Centre at EIPA (European Institute of Public Administration, http://www.eipa.nl)

• 2010: Special version: CAF and Education

4

European UnionVocational Education and Training Sector

• Education and hence quality management in education is not subject to harmonisation across the EU.

• The Copenhagen Process: Joint Declaration in Copenhagen, October 2002 provides a political framework for cooperation at the European level:– Promoting cooperation in quality assurance with particular focus on

exchange of models and methods, as well as common criteria and principles for quality in vocational education and training

• This agenda was implemented by a technical working group (TWG) in which Member States, candidate countries, EFTA-EEA countries, European social partners and the European Commission were represented.

• The Result was CQAF, Common Quality Assurance Framework,

CQAF (Common Quality Assurance Framework)

• CQAF Comprises– a model to simplify planning,

implementation, evaluation and review of systems at appropriate levels in Member States;

– a methodology for assessing and reviewing systems: with emphasis on self-assessment, combined with external evaluation;

– a monitoring system: to be identified as appropriate at national or regional levels, and possibly combined with voluntary peer review at European level;

– a measurement tool: a set of reference indicators aimed at helping Member States to monitor and evaluate their own systems at national or regional levels.

5

For each of these elements core quality criteria have been identified. …. the core criteria are presented as possible answers to specific questions which are transversal to any VET system or provider when reviewing policies

Source: Cedefop panorama series: Fundamentals of a common quality assurance framework (CQAF) for VET in Europe

6

Planning

7

Implemen-tation

8

Evaluation andAssessment

9

Review

10

Methodology

11

CQAF

Common Quality Assurance Framework

• By 2011, 19 countries reported having devised a national quality assurance approach in line with EQAVET

– (Germany, Estonia, Ireland, Greece, France, Lithuania, Luxembourg, Latvia, Malta, the Netherlands, Austria, Poland, Portugal, Romania, Finland, Sweden and the UK (England, Wales, Northern Ireland).

• Ten further countries were preparing one – (Belgium (FL), Belgium (FR), Bulgaria, Cyprus, the Czech Republic, Denmark,

Hungary, Italy, Norway, Slovakia, Slovenia, Spain and the UK (Scotland)).

• In Germany, Estonia and Sweden, a national quality assurance approach is included in legislation.

• Most countries have quality standards for VET providers which are often part of legislation and are also used as a condition to accredit/approve and/or allocate funding. – Which types of standards this refers to is, however, not clear from the

responses.Source: Cedefop: Trends in VET policy in Europe 2010-12

12

European UnionThe national and provider level

• The Danish approach to Quality in Vocational Education and Training (Booklet available at http://pub.uvm.dk/2008/vetquality2)

• Quality strategy for the VET Sector – “[…] it is not possible to say anything definitive and universal about

quality in an education system. It is neither possible nor desirable to authorise one specific concept – be it in regard to methods or objectives and values.

– This is a basic democratic principle, which takes into consideration the fact that it is possible to achieve the same goals by different routes and with different means and methods”

13

Quality policy measures

• No single, nation-wide approach to quality• But common principles and measures at system level and different

approaches at both system and provider level. • The Danish Ministry of Education has defined nine common

principles/measures concerning the policy on quality issues. • The measures apply to the entire Danish education system, but are

given different weight, and take different forms, within the education system.

• A tenth measure should also be included for the VET system: the funding of innovation and development projects.

14

15

16

17

Besides the model, the CQAF also comprises a set of indicators aiming at facilitating the member states to monitor and evaluate there own quality systems.

These indicators include the following three areas:• Employability• Access• Matching

The table to the right provide a qualitative description of how the Danish VET System matches these indicators

18

Executive order on quality development and assessment

• All VET providers must document that they have, and use, a system for quality assurance and development

• Planning: the providers must draw up an annual plan for evaluation.

• Implementation: the providers must draw up procedures for how to evaluate at specific levels, and within specific VET programmes. These procedures must specify how users/trainees/enterprises will be involved in the evaluations.

• Evaluation: the providers must write a report on the evaluation results within specific areas (i.e. the six indicators) and publish them on their website.

• Review: the providers must assess the results, and draw up a follow-up plan taking resources and time into consideration. This follow-up plan should be part of the next year’s action plan.

19

20

The main barrier to deploy quality assurance

• Seen from the system level the major problem in a highly decentralised environment has been lack of motivation to work with quality assurance in some of the institutions

• The heads of the institutions face the same problem with some internal departments and/or managers

• The Danish ministry of Education have been using a number of methods to overcome these problems:

– Increased competition on quality, openness and transparency leads to “user controlled inspection”

– Extended demands on internal quality work of the institutions– More grants should be given according to the principle of “something for

something” (“Quality for money”)– Quality policy, action plans and results etc. must be presented on the internet– If an institution fails to deliver, grants can be reduced or withdrawn. But normally

“sharpened attention” for some time will solve the problem.– Once a year the instituting must present an action plan to the ministry of education,

signed by the chairman of the board, thus securing that the board has quality on the agenda

– These methods have proven to be effective at the national level as well asd at the institutional level

21

The provider levelAARHUS TECH as an example

• All material presented here are available on the internet (http://aarhustech.dk)

• Methodology and tools– Annual Self evaluation in all teaching teams

• Facilitated discussions on processes in and around the teaching• Based on the discussions the teams decides upon improvement measures

and development of common practice. The resulting action plan is published on the Web

– Monthly or quarterly ‘board meetings’ in the teaching teams• Board meetings are short meetings around a whiteboard comprising follow-

up on the teams results for the previous period. Improvements plans for the upcoming period are decided upon – by the teaching team

– Satisfaction surveys• Learner satisfaction surveys• Employee satisfaction surveys are also carried through, but not as part of

the quality policy for the teaching department.

22

The Quality System for AARHUS TECH

QUALITY OBJECTIVES

No drop-outs% Completion ≥ National

average

*

Satisfied learners Result ≥ Satisfied

High Professionalism Average grade ≥7

High Participation % Absence ≤ 5

TEACHINGFOLLOW-UP ON QUALITY – EDUCATIONS PRGRAMMES & COURSES

The educational programs and courses are evaluated on a continuous basis through a variety of measurements, documentation control and internal audits. The measurements are, for example, satisfaction surveys, measurement of dropouts, absence and grades etc. The result of the teams are discussed at lean board meetings and self evaluation and are dealt with by the educational manager.

DEVELOP AND IMPROVE QUALITY OF EDUCATIONAL PROGRAMMES & COURSES

Teaching teams and education managers collaborate on achieving our quality objectives through continuous improvement.This is inter alia done at lean board meetings and the annual self evaluation, where improvement is in the core of the discussions

Internal and external

requirements for quality

assurance and -developmentin education programmes

and short courses

23

Student satisfaction survey, overall resultsincluding external benchmarking

AARHUS TECH

Best in region

National average

Best institution

Results presentation

24

Student satisfaction winning team

• Based on the results of the annual student satisfaction survey a winning team is appointed and awarded.

• The winning team should– achieve an outstanding result of the overall satisfaction level, i.e. an

index above 70 on a 0-100 scale.– have a satisfaction level above national average for the particular

programme– have a relatively high response rate– have improved in relation to the previous survey

Results presentation

Results by department for internal benchmarking

25

(Not all departments are shown here)

Award winner 2011

Results presentation

Priority mapinstitutional level

26

Satisfaction level

Impact of improvementResults presentation

27

Grades

• The Ministry of Education’s grade database is published on the internet

• Institution by institution, programme by programme making external and internal benchmarking possible

• The access is too complicated to show in this presentation

Results presentation

28

Educational environment

• As part of the annual student satisfaction surveys, the educational environment is investigated.

• The investigation comprises– Social environment– Physical environment– Aesthetic environment– Results prepared for

internal and external benchmarkingas shownbefore

Results presentation

29

Study time and completion rates

• The Ministry of Education’s Study time and completion rates on the internet

• Institution by institution, programme by programme making external and internal benchmarking possible

• Below is only part of the results for AARHUS TECH shown

Programme

Completion time (months% completion

Results presentation

30

Employment Rates

• The Ministry of Education’s Employment rates database is available on the internet

• Institution by institution, programme by programme making external and internal benchmarking possible

• Below is only part of the results for AARHUS TECH shown

Results presentation

Programme

31

Evidence of self evaluation and action plan for team:

Hairdresser, AARHUS TECH (1)

Programme HairdresserSelf evaluation date 30. august 2011

Participants dk, ekr, khso, lri, nla, sbuc, tb

Programmer manager MKW

Methodology used Student focused self evaluationFacilitator MIRA

Improvement issues to be dealt with until next self evaluation (only samples are shownAction Revision of basic courseWhy? Local education plan to be revised due to new

ministerial order on the programmedHow? Technical Workgroup To be finalized no later than

August 2012

The evidences of self evaluation is available on the Internet

Results presentation

Evidence of self evaluation and action plan for team:

Hairdresser, AARHUS TECH (2)

32

Action Establish competiton teamsWhy? To support the professional pride among the

hairdressing students at AARHUS TECHHow? Participate in Danish Championships in skills contest To be finalized no later than

August 2012

Action Establish continuing education and TrainingWhy? To offer hairdressers in employment possibility to

Follow recent trends within the tradeHow? Offer Look and Learn Events, Creative evenings

etc.To be finalized no later than

August 2012

Results presentation

33

Action Plan – Institution level

• Once a year the instituting must present an action plan to the ministry of education, signed by the chairman of the board, thus securing that the board has quality on the agenda

• The issue to act upon is decided by the Ministry of Education: Improved completion (reduced no. drop-outs)

• The 2011 action plan of AARHUS TECH comprised the following actions– Action 1: Development of student competences based on appreciative

pedagogy – Action 2: Freshmen’s trip. All new students to the Basic course (appr. 800)

and 51 teachers on a one day trip to enhance affiliation to the College– Action 3: Upgrading of competencies of teachers to work as business

consultants having contact with (potential) “apprenticeship-companies”– Action 4: Apprenticeship dating (potential) apprenticeship-companies meets

potential apprentices