1
1
An integral quality assurance approach
external quality assurance by VLIR
PRIUM
‘Promoting a model of Integrated University in Macedonia’
Wednesday, 10th September 2008
dr. ir. E. VAN ZELE
Cell Quality Assurance, Flemish Interuniversity Council, VLIR
2
Outline
1. quality assurance (QA) and
quality assurance systems
2. QA model applied in Flanders
3. Questions
2
3
quality assurance
and
quality assurance systems
4
1. The Concept of Quality in Higher Education
Quality is a complex and multidimensional concept
The interpretation of the concept
- depends on the target group and the level of education - changes in time
- is determined by preconditions with respect to quality
- is determined by the views on education
3
5
1. The Concept of Quality in Higher Education
Parties involved in the quality discussion :
• institutions of higher education (HEI)• students and their parents
• employees
• trade organisations
• society as a whole
• the government• …
6
2. Quality Assurance System and Functions
Most often 2 kinds of Quality Assurance Systems :
� internal quality assurance :
the study programme or institution is responsible for the evaluation
� external quality systems:
evaluation done by experts (peers)
who are not associated with the study programme or institution
Remark :
the separation may not always be clear
depends also on the functions of evaluation in QA system
4
7
2. Quality Assurance System and Functions
Some functions of quality assurance
� Improvement and guarantee : results of QA are used to improve the quality of the study programme
� Accountability:
results of QA are used to inform governments and consumers--) to broader society :
transparency about the use of public funds
--) to the students/parents and employers :
public reports of the assessment
(Q: How do study programmes fulfil quality standards)
8
2. Quality Assurance System and Functions
Some functions of quality assurance
� Regulation of the higher education system:
results of QA are input for concrete consequences for
steering and regulation
Quality indicators are applied as parameters for decision-
making
on the teaching qualifications of institutions,
the acceptance of new study programmes,
the distribution of resources between education and research
etc.
5
9
First-generation quality assurance systems (1991 – 2004)
Established around 1990s, in Europe, in Flanders
set up by the HEI’s in exchange for greater autonomy
concept : peer review assessment
of the study programmes at the institutions
main focus :
suggestions and recommendations for improvement
ownership : HEI
coordination : VLIR and VLHORA
3. Generations of external quality assurance
10
First generation educational evaluations
HEI(universities)
Flemish Government
VLIRVLHORA
6
11
First generation educational evaluations
Internal quality assurance
external quality assurance
12
Second-generation quality assurance systems (from 2004 onwards)
� Internal quality assurance: all activities oriented to maintaining and improving quality at the institution
� the internal character of QA is especially expressed in SER
� strengths and weaknesses analysis: critical analysis of education
� SER is link between IQA and EQA
� The external character of the quality assurance:
shaped by the assessment visits at the institutions � the basis for EQA is SER
� a panel of independent external experts conducts a quality study and an assessment
� Assessment Panel(AP) publishes a public report of its findings
� the study programme/institution organises the response to the recommendations of the AP
3. Generations of external quality assurance
7
13
� meta-assessment :
process is supervised by the government
� evaluates the quality control at the HEI
� checks the degree to which the institutions process the results of the quality
assessment in their policy
3. Generations of external quality assurance
14
Simultaneous introduction of
second generation QA systems
and accreditation in Europe
Stimuli
- challenges from the first generation of QA systems
- new evolvements:
e.g. the development of the knowledge society
globalisation
liberalisation of higher education policy
3. Generations of external quality assurance
8
15
Second-generation quality assurance systems
Def of accreditation is:
‘Accreditation is a formal public pronouncement,
made by an independent institution
and based on a quality assessment,
which implies that specific standards, agreed on in advance are being realised’
In Flanders the 4 April 2003 Structure Decree introduced
� an accreditation system
� and the bachelor’s-master’s structure
Accreditation is a condition:
� for the funding of the study programme by the government
� for the right to award recognised degrees
� for awarding study grants to students
3. Generations of external quality assurance
16
subject of assessment : is a study programme
assessment panel:
peer review of domain experts, methodologist and student
accreditation :
• decision
• oriented towards recognition of the presence of generic quality
• uses the assessment visit report as a source of information
• must be applied for through the NVAO by the HEI
• decision is binary (positive or negative decision : accredited or not)
results of educational assessment and accreditation are public
3. Generations of external quality assurance
9
17
HEI
(universities)
Flemish Government
VLIR
NVAO
Recognition Committee for
Higher Education
VLHORA
2004
Second generation educational evaluations
18
2004
Second generation educational evaluations
Internal quality assurance
external quality assurance
accreditation
10
19
Second-generation quality assurance systems
Expectations of accreditation
� study programmes fulfil agreed generic quality standards
� (leaving sufficient space for internal reference frames and profiling)
� international quality assurance standards
� criteria and procedures to promote international mobility
� international recognition of diplomas
� tightens the accountability function of QA
� sustains the improvement function of QA
� makes possible the link between QA and regulation mechanisms (such as recognition of diplomas and financing)
3. Generations of external quality assurance
20
Third generation quality assurance systems
Currently being developed (towards 2012)
expectations
� Evaluation of QA and structure of the HEI as a whole
+ : Full responsibility of HEI for QA and for its study programmes
- : check of some study programmes and window of time for improvement
Strongly increases the accountability function of QA
Need for internal system of reporting of quality of the study programmes
3. Generations of external quality assurance
11
21
2012 Prospect (?)
Third generation educational evaluations
internal quality assurance
accreditation
external quality assurance
22
Outline
QA model applied in Flanders
12
23
5. VLIR QA model (1)
preparationSER at HEI
APcomposed
at VLIR
assessment
by AP
on site visitat institution
report
by AP (written by project
manager – Cell QA
VLIR)
follow up
and
accreditation
external quality assurance
24
5. VLIR – procedure (1)
1. Preparation phase
- Workshop (~ 1 year before), support/advice for writing SER
- Writing the SER at the HEI / study programme
- Composition of AP (proposal by the study programmes, academic coordinator, check independence by Higher Education Recognition committee)
- Preparatory meeting AP (preparation AP, task description, ref frame)
- Preparation of assessment audits, practical organisation, time schedule, parties involved to be interviewed
project manager (staff VLIR cell QA)
communication and support of the AP, the study programme,
contact person, drafting reports
13
25
5. VLIR – procedure (2)
2. Assessment on site visits
(educational audits)
- interviews with all parties involved
(students, admin, assistant, teaching staff, professional field, alumni, general HEI
services : st. counselling – guidance, IQA)
- on site visit to infrastructure (library, lecture halls, ICT…)
- counselling hour (private time with AP)
- private debate of members of AP --) first findings : checklist
( with scores and argumentation, recommendations for improvement)
- presentation of the preliminary findings of the AP at the institution
26
5. VLIR – procedure (3)
3. Assessment report
Input :
- checklist with scores for each theme and standard
filled out during the assessment audit
- notes from project manager
- notes from discussions of the AP
- composition of draft report in 1st editorial meeting AP
- study programme’s comments on the draft report
- composition of final version of the report in 2nd editorial meeting AP
- copy of final report to the institutions (poss. for remarks)
14
27
5. VLIR – procedure (4)
3’. Contents of the assessment report :
I. General section
- introduction
- domain specific reference frame
- general observations of the AP
- comparison between comparable study programmes
- table with scores per theme and per standard
II. Reports of the individual study programmes
- description per theme
- motivation and argumentation for the judgement per theme/standard
- recommendations for further improvement
- general judgement about the quality of the study programme
- distinctive quality features
III. Appendices (CV, time schedules)
28
5. VLIR – procedure (5)
4. Publication report and follow up
- Publication of the final report (booklet – website)
- Official handing over of the final report of the AP to VLIR
- Follow up
- HEI file for accreditation at NVAO (Ass report is input)
- Questions from NVAO about the judgement in the report
(to the AP and the VLIR as QAA)
- additional information of recent evolvements
(in between ass report and filing for accreditation)
- procedures and well-founded argumentation checked by the NVAO
15
29
Questions
30
Questions