European Experience on Quality Assurance
in Higher Education Nora Skaburskienė
ENQA Board member,
European Association for the Quality Assurance of Higher Education (ENQA)
Director,
Centre for Quality Assessment in Higher Educationof (SKVC), Lithuania
23.3.2017 1
International Forum “Evaluation and Accreditation of Higher Education in Mexico”, 17 March, Pachuca
What is ENQA? European Association for Quality Assurance in
Higher Education
• ENQA is a membership organisation of external Quality Assurance Agencies and represents its members at the European level and internationally. ENQA members are quality assurance agencies from the European Higher Education Area (EHEA) that operate in the field of higher education
• ENQA was established as the European Network for Quality Assurance in Higher Education in 2000 to promote European cooperation in the field of quality assurance (QA) in higher education
• In 2004 it became the European Association for Quality Assurance in Higher Education but retained the ENQA acronym
2
ENQA
23.3.2017 3
• The criteria for membership of ENQA is that an agency undergoes a
successful external review against the European Standards and
Guidelines for Quality Assurance in the European Higher
Education Area (ESG) every five years
• Organizations that do not wish to or are unable to apply to become
members of ENQA may request affiliate status within ENQA. Affiliates
are bona fide organisations or agencies with a demonstrable interest in
the quality assurance of higher education
• As of February 2017, ENQA has 46 full members, 3 members under
review and 51 affiliates and is representative of 41 of the 48 members
states of the EHEA
ENQA at a glance
• Based in Brussels
• Permanent Secretariat of 5 led by Director
• Elected Board of 9 members
– President
– 2 Vice-Presidents
– Treasurer
– 5 ordinary members
• Term of office of Board members is 3 years (renewable
once)
4
ENQA at a glance
• Consultative member in the Bologna Follow-up Group since
2005 (QA as one pillar of EHEA since the beginning in 1999)
• Founding member of the European Quality Assurance Register
(EQAR)
• Co-operation within E4 (European University Association,
European Students Union, EURASHE)
• Two formal types of affiliation with ENQA: Membership and
Affiliate status
23.3.2017 5
Bologna Declaration (1999)
Signed by 29 Countries
23.3.2017 6
Bologna Process and Quality Assurance
• One of the purposes of the Bologna Declaration (1999) -
to encourage European cooperation in quality assurance
of higher education with a view to developing comparable
criteria and methodologies
• The European Ministers of Education adopted in 2005
the "Standards and Guidelines for Quality Assurance in
the European Higher Education Area (ESG)"
• A new version was adopted in 2015 at Yerevan
23.3.2017 7
About Bologna Process
• The ESG - part of the Bologna Process, which is seen as an
intergovernmental process of policy making and policy
coordination. This means that agreements among the 47
countries must be translated, by those countries and by actors
within countries, into national policies and regulations
• The Bologna Process depends on countries binding
themselves to implement declarations, communiqués and
reports - such as the ESG.
• It is not able to bring sanctions to bear on countries for non-
implementation, other than the ‘naming and shaming’
associated with the open method of coordination
23.3.2017 8
European Higher Education Area (2017)
48 countries
23.3.2017 9
European QA short story
1999 2000 2003
Berlin
communiqué
ESG
2005 2006 2008 2009 2010 2015
Development of the European Standards
and Guidelines (Version 1 – 2005)
• Recommendation of the Berlin Communiqué of Ministers (September 2003)
• Ministers invited ENQA (in cooperation with EUA, EURASHE and ESIB) to develop:
o „an agreed set of standards, procedures and guidelines on quality assurance‟ and
o „to explore ways of ensuring an adequate peer review system for quality assurance and/or accreditation agencies‟
and to report back to the Ministers in 2005
23.3.2017 11
Development of the European Standards
and Guidelines (Version 1 – 2005) In 2005 (in Bergen, Norway)
ENQA reported back with a recommendation for the
Ministers to adopt:
• European standards for internal and external quality
assurance and for external quality assurance agencies
• European QA agencies will be expected to submit
themselves to a cyclical review within 5 years
• A European register of quality assurance agencies
will be produced
• A European Consultative Forum for Quality
Assurance in Higher Education will be established
(EQAF)
23.3.2017 12
European Standards and Guidelines (ESG)
• Membership criteria “Standards and Guidelines for Quality
Assurance in the European Higher Education Area” (ESG)
• ESG the core document for all operations
• Cyclical reviews of quality assurance agencies in the EHEA
• ENQA coordinated review can also be used for application to
be listed in the European Quality Assurance Register (EQAR)
23.3.2017 13
European Standards and Guidelines (ESG)
A key element of ESG 2005 was the 3 Part Structure of QA:
Part 1. ESG for Internal QA within Higher Education Institutions (7
standards) - the corner stone of QA in HE
Part 2. ESG for External QA of Higher Education (8 standards) - a
condition of the credibility of the results of the internal evaluation
Part 3. ESG for External QA Agencies (8 standards) - External
evaluators (QA agencies) are accountable for the quality of their
activities
23.3.2017 14
Main principles for QA in EHEA
• Independence of QA agencies (not parts of the ministry)
• Supports trust - also central role in internationalisation (mobility
and recognition)
• Provides transparent and independent information on the
quality of HEIs and programmes - but is it accessible and easy
to use?
• External and internal QA build on each other (one is not
complete without the other)
23.3.2017 15
European Standards and Guidelines (ESG)
23.3.2017 16
ESG 2.1: External QA procedures
should take into account the
effectiveness of the internal QA
processes described in Part 1 of
the ESG
ESG 3.1: The external QA of
agencies should take into account
the presence and effectiveness of
the external QA processes
described in Part 2 of the ESG
Scope of the current and revised ESG
• Standards and guidelines for quality assurance, not quality as
such
• Apply to all higher education offered in the EHEA regardless
of the mode of study or place of delivery (TNE, e-learning,
short courses…)
• Apply to all types of QA activities and agencies (quality audits,
programme accreditation, institutional review...)
• Quality assurance can serve a variety of purposes:
enhancement – accountability
• Generic, not specific: provide the framework and common
basis for national and institutional activities
23.3.2017 17
ESG 2015 and ESG 2005
• Mandate, Bucharest, 2012
• ENQA, ESU, EUA, EURASHE in
cooperation with EI,
BUSINESSEUROPE, EQAR
• Steering committee and drafting
group, 11 meetings and a lot of work
in-between
• Stakeholder driven: based on
outcomes of the MAP-ESG and other
experience with the ESG 2005
• Open consultation and dialogue with
the BFUG (= ministry representatives)
• Approval by the BFUG, adoption by
the Yerevan Ministerial Conference in
May 2015
18
Guiding principles for the revision
• Keep the strengths: integrated concept and understanding
of QA, broad applicability, broad ownership
• Overcome the weaknesses: vagueness, redundancies,
inconsistencies, confusion between “standard” and
“guideline”…
• Update: ESG as part of the ‘Bologna-infrastructure’ - taking
into account recent developments in QA and HE in general
(NQFs and QF-EHEA; ECTS; LOs based approach…)
• Enable adaptability to future developments
ESG 2015 and ESG 2005
• Comparative analysis of the ESG 2015 and ESG 2005
• Changes in context, scope, purposes and principles
• Changes in standards’ wording
• Changes in guidelines’ content
• ESG are a document that needs to be read as a whole: introductory part is of central importance! Also: parts 1, 2 and 3 build on each other and are parts of one whole.
• All parts have undergone significant changes, and requirements have become more stringent
• Biggest changes in part 1
www.equip-project.eu/activities/comparative-analysis
20
ESG 2015 purposes
• They set a common framework for quality assurance
systems for learning and teaching at European, national and
institutional level
• They enable the assurance and improvement of quality of
higher education in the European Higher Education Area
• They support mutual trust, thus facilitating recognition and
mobility within and across national borders
• They provide information on quality assurance in the EHEA
23.3.2017 21
…22…
ESG 2015
principles
• HEIs have primary responsibility for the quality of their
provision and its assurance
• QA responds to the diversity of higher education systems,
institutions, programmes and students
• QA supports the development of a quality culture
• QA takes into account the needs and expectations of
students, all other stakeholders and society
Some important changes overall
• Slighlty modified scope: focus on learning and teaching in higher education, but now including – Learning environment
– Links to research and innovation
• The ESG 2015 explicitly apply to all programmes whichever mode or place of delivery in the EHEA (e-learning, cross-border HE…) many questions to HEIs and agencies:
– How to carry out QA of programmes delivered abroad? Responsibilities, costs, cooperation with local agencies….
– How to assess e-learning? Can the same criteria be used as for traditional forms of education (drop-out rate, m2/student, teacher-student ratio…)?
– EUA study in 2014: only 29% of HEIs had IQA for e-learning, other 35% were discussing
• Apply to agencies wherever they carry out EQA and weather the activities are compulsory or voluntary Cross-border QA needs to be ESG compliant, too
23
Some important changes overall - 2
• Underlines the importance of enhancement for all QA
processes, and the support EQA needs to give to the
development of a quality culture at HEIs
• Strong focus on the shift to student-centered learning
• Remain generic principles that allow for diversity of practical
implementation
• Standard is a “should”, based on agreed and accepted practice;
guidelines describe why the standard is important and how it
might be implemented, and set out good practice
23.3.2017 replace txt View menu > Header and footer 24
Key features of QA in Europe
• Diversity in developing solutions and strategies (according to national
regulations)
• Diversity is compatible with enhancement oriented approaches:
enhancement a priority to almost al QA agencies
• Increased focus on institutional QA and the creation of quality culture
• Agencies are more and more open to involve stakeholders (by now a
“must” – students!)
• International dimension is increasingly expected (experts, cooperative
QA, cross-border QA)
• Improvement of information and communication
• Agencies’ scope of activity has broadened (consultancy - enhancement)
• Agencies’ area of operation is broadening (cross-border activities)
• National regulations still driving the majority of QA agencies’ activities
23.3.2017 25
Remaining challenges in EHEA
• Understanding when local adaptations are related to quality
and when to traditions
• Recognition of QA decisions and foreign qualifications
• QA’s role in raising quality standards/levels - Impact of QA?
• Meaningful involvement of all parties (challenge: employers)
• Information value and usability of QA reports
• QA of joint programmes
• QA of cross-border higher education
23.3.2017 26
What do we mean with quality assurance?
Internal and external process and criteria to:
– Ensure minimum standards (accountability)
– Support quality enhancement
– Provide reliable and transparent information to users and
stakeholders (consumer protection)
– Create trust in the HE system and its components
– Ensure, fundamentally, that students (can) reach the
intended learning outcomes
23.3.2017 27
Implementation and Translation
Central Government makes formal HE policy and establishes funding mechanisms
Rectors and their management teams interpret and respond to policy in different ways
Heads of departments balance competing pressures, employ, reject or ignore demands for compliance, encode policy in internal processes
Academic staff in different departments and HEIs apply, ignore or adapt policy as they think appropriate, only some of which reaches them and which they receive and interpret in different – sometimes unpredictable – ways
Students respond in unpredicted ways, changing relationships and practices in teaching and learning situations
23.3.2017 28
NATIONAL
INSTITUTIONAL
DEPARTMENT
CLASSROOMS,
OFFICES
Implementation staircase. (adopted from Trowler (2002)
Implementation and Translation - 2
• Much may depend on, for instance, the country or the
academic discipline
• For instance, Britain has different legal, policy and economic
environments than Lithuania
• Or, in terms of disciplines, in engineering connections with the
non-academic world of work play a much larger role than in
theoretical physics
23.3.2017 29
Recent developments
23.3.2017 30
• Declining trust in the ability of higher education institutions to
ensure quality
• Movement towards accreditation (the Netherlands, Portugal,
Denmark)
• Multi-dimensional ranking system for HEIs
Trends in QA agency structures
Mergers of agencies across HE sectors:
• AUSTRIA (public universities, fachhoschule, private
universities)
• BELGIUM-FLANDERS (Universities, university colleges)
• IRELAND (universities, institutes of technology)
23.3.2017 31
Trends in agency scope
Wider responsibility for ENQA member agencies:
• FINEEC, Finland (from early childhood to higher
education)
• QQI, Ireland (higher education, further education (VET),
NQF, NARIC)
• NOKUT, Norway (higher education, VET, NARIC)
• EKKA, Estonia (Higher education, VET)
• SKVC, Lithuania (Higher education, ENIC/NARIC)
23.3.2017 32
Trends in QA methodologies
• Movement from programme accreditation (only) to a
mixture of institutional accreditation and some
programme audits/accreditations
• Movement to more risk-based QA
23.3.2017 33
Trends at Agency level
• More countries establishing agencies and more agencies
seeking to become full members of ENQA or listed on
EQAR (lately – two new agencies from Kazakhstan)
• 24 of the 48 EHEA countries have agencies that are
ENQA members
• A further 16 countries have agencies that are ENQA
affiliates
23.3.2017 34
Trends in QA
• Development of a market in German-speaking Europe –
Germany – Austria – Switzerland
• Proliferation of Regional QA agencies in Spain – 7 ENQA
members – 3 ENQA affiliates
23.3.2017 35
Trends in complexity of decision-making
• More separation of responsibility for organising QA
evaluations and decision-making
• Denmark (Accreditation Institute(AI) and Danish
Accreditation Council)
• Switzerland (AAQ and Swiss accreditation council)
• Netherlands - Flanders (NVAO and agencies organising
evaluations)
23.3.2017 36
Trends in QA
Increasing of Internationalisation of QA :
• Cross-border quality assurance (mobile agencies)
• Quality assurance of cross-border higher education
/transnational education (mobile institutions)
• Quality assurance of joint programmes (mobile students)
23.3.2017 37
Trends in QA
QA for enhancement
• Greater emphasis on Internal QA and the encouragement
of an institutional “Quality Culture”
Much greater use of Explicit Statements of Requirements
• Qualification Frameworks
• Subject Benchmarks
23.3.2017 38
Trends in QA
Greater Focus on Outputs
• Direct – What a student has learnt
• Indirect – Where a graduate has gone
In some countries increasing government involvement
Main QA approaches in the EU include evaluation, accreditation and audit
23.3.2017 39
Strengths of European QA
• There is strong consensus regarding the principles and/or the
functioning of QA in HE in Europe
• Internal QA is primarily focused on continuous quality enhancement
• External QA extends internal QA and provides the
institution/programme with a mirror and suggested improvements;
• External QA is the key element in accountability
• External QA is independent and involves all stakeholders, conducts
a site visit and publishes its outcomes, as well as follow up its
recommendations in a systematic and cyclical way
• the external QAAs themselves are subject to cyclical review
• The European QA that is systematic, professional, fit-for-purpose,
targeted towards enhancement and public accountability, and based
on trust
23.3.2017 40
KEY FINDINGS
• The most common national QA system in the EHEA is one characterised by a single agency. In some countries, there are several agencies
• Most European QAAs comply with the ESG. At EU-level, this is considered a precondition for operating abroad
• In most countries it is mandatory to accredit study programmes. Institutional assessments have recently become more prominent. Whether these will substitute programme assessments is not yet clear
• Most quality criteria are similar for programme and institutional assessment. Programme assessments apply more content-related indicators; HEI reviews have a systemic approach and focus on internal management structures
• Most systems apply an ordinal measurement scale to rate the different quality criteria as well as for the overall rating. In some scales the best value refers to compliance with a standard
• Stakeholder participation is common in EHEA countries and a specific feature of QA in the region
• Most common assessment outcomes are the permission to further operate a programme or an institution and, in some countries, impacts on public funding. In other countries, HEI and programmes simply receive recommendations for improvement (enhancement led evaluation)
23.3.2017 41
KEY FINDINGS
• Quality assurance must respond to the changing higher education
landscape in order not to be become an obstacle for innovation and
modernisation
• Common trends include a stronger focus on internal QA, more stakeholder
engagement and further internationalisation of QA
• Agencies explore more flexible and lighter procedures leaving primary
responsibility for QA within the HEIs and focussing on quality enhancement.
• External QA may measure quality more and more beyond minimum
standards through distinct features or excellence labels
• QA agencies broaden their scope of activities, by taking up advisory
activities and providing more enhancement and support activities
23.3.2017 42
Scope of Activities of QAAs
Different countries use different tools of QA:
• Belgium – evaluation of programmes
• Bulgaria – accreditations of HEIs and programmes
• Croatia – accreditations of HEIs, programmes and research
institutions
23.3.2017 43
Scope of Activities of QAAs
• Greece – accreditation of IQA systems of HEIs and
programmes
• Holy See – evaluation of HEIs and programmes
• Hungary – accreditation of HEIs and programmes
• Kosovo – accreditation of HEIs and programmes
23.3.2017 44
Scope of Activities of QAAs
• The Netherlands – audits of IQA systems, accreditation of
programmes
• s
• Romania – accreditation of HEIs and programmes
• Serbia – accreditation of HEIs and programmes
• Slovenia – accreditation of HEIs and programmes
• Switzerland – accreditation of HEIs
23.3.2017 45
46
Finland
Audits of IQAS
Sweden
Evaluation of HEIs
and programmes
Norway
Audits of IQAS
Accreditation of HEIs
and programmes
UK
Review of
Universities and
other degree-
awarding bodies
Ireland
Approval of QA
procedures
Evaluation of HEIs
Poland
Accreditation of HEIs
and programmes
Latvia
Accreditation of HEIs
and programmes
Estonia
Evaluation of
programmes groups
Accreditation of HEIs
Spain – 8 ENQA QAAs
Accreditation of HEIs
and programmes
France
HCERES –
evaluation of HEIs and
programmes;
CTI – accreditation of
engineering progr.
German
7 QAAs-ENQA m.
Evaluation of
HEIs……
Austria
Accreditation of HEIs
and programmes
Audits of IQAS Hungary
Accreditation of HEIs
and programmes
Lithuania
Accreditation of
HEIs and
programmes
Portugal
Accreditation of HEIs
and programmes
Romania
Accreditation of HEIs
and programmes
• Effectiveness of IQA
• ESG – Part 1
Consideration of IQA
• Clear aims
• Stakeholder involvement
Methodologies fit for purpose
• Self-assessment · Visit by peers
• Report · Follow-up Process of EQA
• Student member · Employers
• Selection procedure · Training Experts
• Pre-defined
• Published
Criteria for outcomes
• Full text
• All decisions Reporting
• Clearly defined
• Part of EQA process
Complaints and Appeals
23.3.2017 47
23.3.2017 48
Qquality Assurance ≠ Evaluation of Quality
Consultations
Capacity
building
Follow-up
Publicity
Analytical
activities
Development
of guidelines
CONCLUSIONS
23.3.2017 49
When introducing a quality assurance framework to enable mutual
national and international understanding, it could be useful to
reflect upon:
• The key concepts:
– What is higher education?
– What is common understanding of the quality?
– What is quality in higher education?
– How does quality relate to a strategy, leadership and
information structures?
– What should be the prevailing purpose of quality assurance
– accountability or enhancement?
– What organizational structure will be used for national quality
assurance? Who will develop QA tools?
