This project has been funded with support from the European Commission. This publication (communication) reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein

Peer Learning Event

on Assessment and

Demonstration of

Achieved Learning Outcomes

The Hague 29-30 October 2015 26 October 2015

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 2

Table of Contents

1 Preface 3

2 Programma (v.1.0) 4

3 Abstracts 8

4 Biodata of the presenters 14

5 List of participants 16

The organizing team wishes you a pleasant stay in The Hague and a fruitful conference.

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 3

1 PREFACE

The Accreditation Organisation of the Netherlands and Flanders (NVAO), together with the

Dutch Ministry of Education, Culture and Science (OCW), is organizing the Peer Learning

Event on the Assessment and Demonstration of the Achievement of Learning Outcomes on

Thursday 29 - Friday 30 October 2015 from at 9.30 am in The Hague, The Netherlands.

The conference is part of the EU-financed project Facilitating Bologna Tools (FaBoTo) of the

Dutch Ministry of Education (OCW), the National Agency Erasmus+ and NVAO.

What is it about?

Using achieved learning outcomes in quality assurance and accreditation and assessing

and demonstrating these has been identified as a development that needs further

enhancement and training. The peer learning event provides a platform for sharing and

discussing good practices and eventually formulate a set of practical guidelines, which will

later on be made available for a wide audience.

Themes and questions

– Do you know of any good practices or case studies from HEI when it comes to describing

intended learning outcomes in a clear and measurable way, both at course unit and at

programme level so that learning outcomes are understandable and verifiable for all

target groups concerned?

– Do you know of any HEI serving as examples of good practice when it comes to

demonstrating convincingly the achievement of the intended learning outcomes to the

stakeholders?

– Do you know of any good practices or case studies from QA when it comes to evaluating

intended learning outcomes and the demonstration of their achievement?

– Are there interesting case studies on the involvement of internal/external stakeholders in

the formulation of intended learning outcomes or on how HEI organize this involvement?

– Do you know of any training programmes HEI have access to in order to facilitate the

proper formulation and assessment of intended learning outcomes?

– Do you know of any convincing case studies of methods programme committees apply to

monitor the appropriate link between learning outcomes, curriculum design and learning

activities, and assessment methods and criteria?

– Are there interesting case studies of properly defined expectations that QA have with

respect to the use of intended learning outcomes and their assessment and

demonstration of achievement?

– Do you know of any convincing case studies of methods programme committees apply to

monitor the appropriate link between learning outcomes, curriculum design and learning

activities, and assessment methods and criteria?

– Are there interesting case studies of properly defined expectations that QA have with

respect to the use of intended learning outcomes and their assessment and

demonstration of achievement?

Participants

The Peer Learning Event has participants mainly from Higher Education Institutions (HEI)

and Quality Assurance Agencies (QA). The event is set up in response to interest for this

theme from a number of countries, who have set up delegations. Besides these, a large

number of independent participants have registered.

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 4

2 PROGRAMME (v.1.0)

Times and titles of presentations are provisional and can change.

Thursday 29 October 2015

9.00-9.30 Reception of guests at the venue: the Ministry of Education,

Culture and Science (OCW)

9.30-9.45 1. Welcome address by Lucien Bollaert, member of the board of

the NVAO

9.45-10.30 2. General introduction to the topic of the event

Fiona Crozier, Head of International, QAA, UK

Dr. Kath Hodgson MBE, Director of Learning and Teaching Support,

Leeds University, UK

3. Two Case Studies from HEI’s related to the achievement of intended learning

outcomes

2x20 min presentation and joint discussion of 20 minutes chaired by Fiona Crozier.

10.30-10.50 3.A Dr. Daan Andriessen (HU University of Applied Sciences

Utrecht)

Title: Developing a Protocol for the Assessment of Achieved

Learning Outcomes for the Netherlands Association of

Universities of Applied Sciences [Vereniging Hogescholen]

10.50-11.10 3.B Andy Gibbs BSC (Hons), MSC, PGCE, RN

Senior Lecturer International, School of Nursing and Community

Health, School of Health and Life Sciences, Glasgow Caledonian

University

Title: Differing Approaches to Programme Learning Outcomes

11.10-11.30 BREAK

11.30-11.50 3.C Questions and discussion on the two case studies

Do these produce a set of general principles or good practices?

Chair: Fiona Crozier

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 5

4. Two Case Studies from QA’s; The use of achieved learning outcomes in external

quality assurance

2x20 min presentation and joint discussion of 20 minutes

Chaired by Kath Hodgson

11.50-12.10 4.A Karin Järplid Linde, Head of Department of Quality Assurance

– Swedish Higher Education Authority (UKÄ)

Title: External quality assurance of higher education in Sweden

12.10-12.30 4.B Dietlinde Kastelliz - Head of Department Audit & Consulting -

Agency for Quality Assurance and Accreditation Austria (AQ

Austria)

Title: Learning Outcomes as Subject Matter of External Quality

Assurance

12.30-12.50 4.C Questions and discussion on the two case studies

Do these produce a set of general principles or good practices?

Chair: Kath Hodgson

12.50-14.00 LUNCH

14.00-16.00

5. Breakouts in two rounds of three workshops: Presentations of

good practices and discussion (follow the color codes)

- Each workshop starts with short presentations of good practices by

participants (10-15 minutes) followed by discussion or assignments

Desired outcomes:

- Could the presented practices be used by participants in their own

context?

- Are there useful alternatives not discussed before?

- Which elements constitute a good practice approach?

14.00-15.00 5.A Round 1 Workshops

Workshop 1 (Room 15.003, 15th

Floor)

Daan Andriessen (HU University of Applied Sciences Utrecht)

- Pilot Guidelines from Dutch Universities of Applied Sciences

Workshop 2 (Room Barcelona, 16th

Floor)

Johan Alvfors (Swedish National Union of Students)

- TBA

Oliver Vettori (Vienna University of Economics and Business)

- Meaningful measurements? - Between the poles of rigor and

relevance

Workshop 3 (Room Lissabon, 16th

Floor)

Petra Bulthuis & Anke Thijsen (Saxion University of Applied Sciences)

- The AuCom Model - Guidelines for researching, defining and

demonstrating achieved learning outcomes

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 6

15.00-15.20 BREAK

15.20-16.20

5.B Round 2 workshops

Workshop 1 (Room Barcelona, 16th

Floor)

Paul Garré (Odisee)

- EQF-levels as a corner stone for institutional quality assurance

approaches

Workshop 2 (Room Lissabon, 16th

Floor)

Luc Van de Poele (Ghent University)

- Systematic Data Gathering of Intended and Experienced Learning

Goals

Workshop 3 (Room 15.003, 15th

Floor)

Robert Coelen (Stenden University of Applied Sciences)

- Workplace Acceptance of Program Learning Outcomes

Kees Kouwenaar (VU Free University Amsterdam)

- Required learning achievements at the entrance of the master’s

programme: Mastermind Europe (www.mastermindeurope.eu )

16.20-17.00 6. Synthesis from the six groups, discussion chaired by Axel

Aerden, NVAO

17.00-18.00

Free time

18.00-22.00 DINNER IN THE HAGUE, at Madurodam

Busses will be waiting at Central Station (instructions will be

announced)

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 7

Friday 30th October

9.00-9.30 1. Opening by Mark Frederiks, NVAO International Affairs

Introduction to a possible outline of guidelines

9.30-10.00 2. Plenary discussion on an outline of guidelines

10.00-11.00 3. Breakout in two workgroups with a mix of HEI and QA

representatives, chaired by Els van der Werf (Bologna expert, the

Netherlands) and Oliver Vettori (Vienna University of Economics

and Business)

- how can the presented and identified good practices be included in the

guidelines?

- what elements can be used to formulate guidelines?

- how should they be described as guidelines?

11.00-11.20 BREAK

11.20-12.00

4. Plenary discussion and synthesis from workgroups on main

conclusions regarding guidelines, chaired by Axel Aerden, NVAO

12.00- 12.30 5. Concluding talk by Lucien Bollaert, NVAO Board member

Summing up the results

12.30- 13.30 End of the programme and LUNCH

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 8

3 ABSTRACTS

1. General introduction to Achieved Learning Outcomes by Fiona Crozier and Kath

Hodgson

Our presentation aims to set out two sides of the „learning outcomes coin‟; i.e. transparency

or straitjacket (to put it very simply). We ask whether quality assurance agencies and

academics represent those two sides of the coin in their approach to learning outcomes and

whether, in fact, both sides of the coin impact positively on all stakeholders. What about the

impact on the views of students, their parents, employers and professional, regulatory

bodies? Do learning outcomes and their assessment as currently used improve the

students‟ overall experience?

We will look at the importance and impact of learning outcomes from these perspectives –

we have a lot of questions…and not many answers! We hope that our questions will

encourage debate and discussion and we look forward to hearing what the audience has to

say.

2. Case studies from Higher Education Institutions

2.1 Daan Andriessen, HU University of Applied Sciences Utrecht

Developing a Protocol for the Assessment of Achieved Learning Outcomes

In response to a crisis of public confidence in Dutch Professional Education a group of

experts developed a Protocol for systematic quality assurance and external validation of

final examination in professional education. This protocol is based on a model for the

assessment of achieved learning outcomes. The Netherlands Association of Universities of

Applied Sciences conducts a pilot with the protocol to gather information about the

effectiveness and the conditions for successful implementation.

Prof. dr. Daan Andriessen is professor in the methodology of applied research at the HU

University of Applied Sciences Utrecht and chairman of the Expertgroup Protocol.

2.2. Andy Gibbs, Glasgow Caledonian University

Differing approaches to Programme Learning Outcomes

This session outlines the background to the role of Learning Outcomes in European Higher

Education Reform and discusses module learning outcomes, programme learning outcomes

and specifications. Ways in which intended learning outcomes of a Higher Education

programme and the means by which the outcomes are achieved and demonstrated are

defined. The concept of constructive alignment is discussed and exemplar material is

provided to assist colleagues in higher education to implement constructive alignment in

their programmes. Differing approaches to formulating degree programme profiles are

highlighted and potential inconsistencies in demonstrating achievement of intended

programme learning outcomes are flagged for discussion. The implications of a harmonised

rather than a standardised approach are considered in the context of employers and HEI

providers.

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 9

3. Case studies from Quality Assurance Agencies

3.1 Karin Järplid Linde, Swedish Higher Education Authority

External quality assurance of higher education in Sweden

The first generation of quality assurance in higher education in Sweden was developed in

the beginning of the 1990‟s. However, the Swedish National Agency for Higher Education

(HSV), later Swedish Higher Education Authority (UKÄ), was not established until 1995. The

evaluation model at the time was based on the concept of quality audit and focused on the

enhancement of internal quality assurance processes at the higher education institutions.

From 2001 and onwards, programme and subject evaluation became the predominant form

of external quality assurance. In addition, thematic evaluations were conducted. In 2010 the

Government commissioned UKÄ to develop a new model for the evaluation of first and

second-cycle programmes, focusing on student attainment of intended learning outcomes

specified in the national qualification descriptors.

In contrast to previous evaluation models, this cycle almost exclusively focused on output,

and very little on pre-requisites and processes. A peer review based model was used where

self-evaluations submitted by the institutions, interviews with representatives from the

institutions and students were taken into account. However, the student‟s independent

projects became the most important factor in the experts‟ assessment of the programmes.

The assessment of independent projects was seen as a way of confirming that the students

had achieved the intended learning outcomes laid down in the qualification descriptors and

as a measure of assuring the quality of a study programme. The approach was somewhat

controversial.

Between 2011 – 2014 about 2100 programmes were evaluated according to this output

oriented evaluation model. For programmes where shortcomings were identified, a follow –

up is done after one year. UKÄ has the power to revoke the entitlement to award

qualifications, if shortcomings persist after that year. In these follow – ups, many examples

of quality enhancement activities have been shown, such as: review of courses and

programmes, curricula reviews, strengthened link between education and research, review

of internal quality assurance procedures. According to a Danish evaluation of this quality

assurance model , the evaluation system has had many effects on the institutions‟ internal

quality assurance systems. There has been an increased awareness of the national

qualification descriptions and due to stakeholders involved in developing action plans,

cooperation within universities has also increased. You could argue that the model has been

sharp as institutions were compelled to take immediate actions, for the benefit of the

students. However, the evaluation model has been criticized for having too much focus on

the students‟ output rather than on the quality of programmes.

In 2016 a new model for external quality assurance will be launched in line with the

European Standards and Guidelines (ESG). Apart from the appraisal of entitlement to award

qualifications, it is foreseen to include both institutional audits and programme evaluations

as well as thematic evaluations. The focus on output will be kept, taking the internal quality

assurance procedures at the higher education institutions into account. Thus the model will

include parts from all former quality assurance approaches, balancing the national legal

requirements as wells as international agreements. We foresee a model where the role of

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 10

controlling that quality is being maintained is balanced with being a driver for quality

enhancement of higher education.

3.2. Dietlinde Kastelliz, AQ Austria

Learning outcomes as subject matter of external quality assurance

Learning outcomes have gained considerable significance as one of the action lines in the

Bologna Process. The ECTS users‟ guide determines learning outcomes as follows:

“Statements of what a learner knows, understands and is able to do on completion of a

learning process. The achievement of learning outcomes has to be assessed through

procedures based on clear and transparent criteria. […]”

The Austrian Agency for Quality Assurance and Accreditation (AQ Austria) is a national

body with competences in all sectors of higher education. The agency‟s responsibilities

include external quality assurance procedures, such as institutional and study programme

accreditation or audit of internal quality management systems at higher education

institutions, as well as information and consulting services, and system-wide analyses. With

this broad field of responsibilities in mind, the agency takes a holistic view on external

quality assurance.

In the Austrian higher education system, two types of external quality assurance are

predominant: one is the ex ante accreditation of study programmes, the other is the audit of

the internal quality management system of an institution.

Learning outcomes in the ex ante accreditation of study programmes are considered in at

least four criteria which refer et.al. to their definition and the scientific and professional

requirements of the respective qualification objectives, the content of the programme, the

didactics, students‟ workload, and assessment. The actual check whether the intended

learning outcomes have turned into achieved learning outcomes by the individual student is

not part of a review process by AQ Austria.

In the audit, a higher education institution has to demonstrate that its internal quality

management system is functional and efficient. This includes mechanisms and processes

for the development and further development of curricula. Therefore, the object of the

review are not the defined learning outcomes of each study programme, but the way the

institution ensures their explication and deals with the evaluation of a full study programme

and single units within it.

AQ Austria adheres to the principle that autonomous higher education institutions are in

charge of quality assurance and quality development with respect to degree programmes,

teaching, research and development, and organisation. Taking this idea of autonomy

together with the legal stipulations in the country, learning outcomes as subject matter of

external quality assurance in Austria are considered on the level of study programmes (as

opposed to the individual student) and on the process level, whereas the responsibility for

the control and assessment of their achievement stays with the individual higher education

institution.

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 11

5. Presentations of good practices

5A.1 Daan Andriessen: Pilot Guidelines from Dutch Universities of Applied Sciences

5A.2 Oliver Vettori: - Meaningful measurements? – Between the poles of rigor and

relevance

The assessment of achieved learning outcomes remains a theoretical and practical

challenge in several aspects: It requires a carefully balanced assessment design that is

methodologically sound, yet also feasible; It needs to provide detailed information on the

performance of individual learners, but on the same time is expected to feed into institutional

processes of curriculum development and stakeholder information; and it should cover a

broad variety of competences and skills on various levels. In simple words: The

measurements should be accurate but also create meaning - demanding a tightrope walk

between rigor and relevance. This presentation will not only delve into this issue, but also

relate it to the practical case study of implementing a quality-assurance driven learning

outcome assessment process at an Austrian public university. The decisions taken in the

context of this example will provide a starting point for actively discussing the question of

priorities in assessing achieved learning outcomes.

5A.3 Johan Alvfors

5A.4 Petra Bulthuis & Anke Thijsen: The AuCom Model - Guidelines for researching,

defining and demonstrating achieved learning outcomes

The responsibility of our institutions is to ensure that our students are prepared for and can

succeed in their future work environment. The ways that we teach and the types of

assignments, projects and internships must be relevant to todays‟ world: high quality and be

developed and assessed at a level that is appropriate to the programme.

Although we have national and international qualification frameworks for determining the

different levels, they are rather abstract. It is not immediately clear what the level of a

student who graduates from the University of Applied Sciences is.

That‟s the reason why Saxion University of Applied Sciences has developed a model, the

AuCom Model, which can help formulate the level of programmes and of assignments and

projects.

In the presentation we will explain why and how the AuCom Model was developed and how

the model can be used at your own institution. To experience this model we do a practical

assignment to allow everyone to put the model into practice. By seeing how the model

works, it may generate a more practical dialogue.

5B.1: Paul Garré: EQF-levels as a corner stone for institutional quality assurance

approaches

Examples from a specific professional bachelor programme will be given to illustrate some

crucial elements of the institutional quality assurance approach. All programmes of the

university college use the same competence framework, defining in rather general terms

four categories of competence achievement (basic, growth, integration, expert). These

categories relate to developing knowledge and skills, degrees of autonomy and

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 12

responsibility, growing complexity and novelty of contextual elements. The third level

(integration) corresponds with the attainment of EQF6. Each programme has formulated

domainspecific key learning objectives, adjusted well to this third integrative level, and thus

corresponding with EQF6. Furthermore, each course unit of the degree programme is

related to one or more of these learning objectives (specifying the expected level of

achievement). Course units from the first year of the degree programme assess their

students at level 1 or 2 of the competence framework. Exams and papers in the third (final)

year will have to be assessed at the third level (EQF6). Students are informed about these

expectations via online available ECTS information guides. Once in a six yearly quality

cycle, assessment procedures of all degree programmes and course units are thoroughly

screened by a panel of external peers and representatives of the labour market. Teachers

as well as programme responsibles receive feedback reports about the quality of their

assessment practices (in terms of transparancy, relevance, level …). Moreover, these

reports contain suggestions for improvement and appraise good practices. Again, the

attainment of EQF6 is an explicit criterium for reflection and feedback during this peer

review. Additionally, after one and a half year of work experience, alumni receive a

questionnaire containing ten crucial competences (selected from the intended learning

outcomes). They are asked if these competences (in terms of intended learning outcomes)

are still considered as relevant taking into account their first steps in the labour market and if

so, whether they had achieved these competences at the expected level when entering their

first job. This precious feedback is used to adapt the degree programmes and learning

objectives where necessary.

Points for discussion

Collateral damage of extreme emphasis on learning outcomes driven evaluation.

Does quality assurance based on learning outcomes leaves room for innovation?

What could be our role as a teacher, as a dean, as a quality agency, as a legislator to find

the right balance?

5B.2: Luc van de Poele: Systematic Data Gathering of Intended and Experienced

Learning Goals

Ghent University Quality Assurance has always put great emphasis on learning outcomes

as the basis for curriculum development and monitoring. In a research-intensive university,

the level of specification and orientation on particular professions is permanently under

discussion. Explicit learning outcomes, or competences, as the term is systematically used

at Ghent University, are described at the level of each program, and at the level of each

course. Their coherence is monitored by the Ghent University Competence Model, which

has been in use/in place since 2005. Staff is permanently being trained in formulating

good/coherent? competences. And this, in turn, is systematically monitored, university-wide

as well as ad hoc. Quality assurance at Ghent University partially relies on student surveys,

in which the students are asked to give feedback on a set of general outcomes. The results

of these two-yearly questionnaires are openly reported in a comparable format, and are

what drives the quality assurance. The surveys have become even more important now

that the external review panels on the level of the study programs have been suspended by

decree, giving the Flemish higher education institutions an opportunity to experiment with an

internal quality assurance system. At Ghent University, our programs complement the

university-wide monitoring instruments with more specific data gathering among students

and alumni about the program competences. Since 2015, both program and course

competences are part of a new competence management system, where teaching staff

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 13

indicate to which program competences their courses contribute, with which course

competences they work in order to achieve the program goals, and which teaching

techniques they use. Ultimately, the system also captures which program competences are

being evaluated and what evaluation techniques are being used. The system is still in pilot

phase and some issues have yet to be resolved.

Points for discussion:

Collateral damage of extreme emphasis on learning outcomes-driven evaluation.

Does quality assurance based on learning outcomes leave room for innovation?

5B.3: Kees Kouwenaar: Required learning achievements at the entrance of the

master’s programme: Mastermind Europe

5B.4: Robert Coelen: Workplace Acceptance of Program Learning Outcomes

Between conceiving the program learning outcomes (PLOs) and making them achieved

outcomes through appropriate testing, there is a verification step in professional education

that involves the work field. This presentation discusses an innovative way of aligning the

PLOs with the work place needs. At the same time this approach creates input for testing

that further enhances the proximity of work place reality with class room practice and

preparation. The approach is at variance with suggested practice. The benefits and pitfalls

will be discussed.

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 14

4 BIODATA OF THE PRESENTERS

Fiona Crozier is currently Head of International at UK Quality Assurance Agency (QAA).

Before that she was Director of the Quality Promotion Unit at (since April 2013) at the

University College Cork, Ireland and responsible for the University's internal quality

assurance processes and system. Prior to that she was Assistant Director at the UK Quality

Assurance Agency for 13 years (responsible for policy development and for implementation

of external QA processes through institutional review) and was a board member and Vice-

President of European Association for Quality Assurance (ENQA) for five years of that time.

She has been involved in many European and international projects including chairing the

MAP-ESG project and as member of the drafting group that produced the latest version of

the ESG. She has spoken at numerous conferences, workshops and seminars. She also

worked at a UK university and for the Higher Education Funding Council for England and

Northern Ireland. She is currently a member of the Advisory Board of ACSUCYL and of the

Quality Assessment Committee of AQU Catalunya.

Dr Kath Hodgson MBE Following 20 years as a school teacher and completion of her PhD

in education management Dr Kath Hodgson began working at the University of Leeds as a

Planning Officer before moving into a learning and teaching role. She became head of the

Quality Assurance Unit and subsequently the University's Director of Learning and

Teaching, responsible for curriculum enhancement and support. She also undertook over 15

institutional reviews on behalf of the UK Quality Assurance Agency. In 2012 she was

awarded an MBE for services to Higher Education.

Dr Daan Andriessen is professor of the Methodology of Applied Research at HU University

of Applied Sciences Utrecht from 2013. His research deals with the continuing

professionalization of applied research in the higher professional education in the

Netherlands. The research group conducts research into best practices and techniques for

applied research. Another focus of their work is how students and teachers learn to carry

out research. Before this, Daan Andriessen was Professor of Intellectual Capital at

Hogeschool Inholland University of Applied Sciences. He was also a member of the Forum

for Applied Research and assumed responsibility for establishing a Code of Conduct for

Applied Research in higher professional education (HBO) in the Netherlands. In addition to

his work as a professor, Daan Andriessen is also a research coach at Onderzoekcoach.nl.

Through this website, he supervises Bachelor‟s, Master‟s and PhD students who are

designing and conducting research. Daan Andriessen gained his PhD in 2003 from

Nyenrode Business University with a thesis on the value of knowledge. Before he was

appointed as professor, he spent twelve years as an organizational consultant at

consultancy firm KPMG.

Andy Gibbs is Senior Lecturer at Glasgow Caledonian University responsible for

internationalisation of the curriculum. He is a UK European Higher Education Expert with a

particular focus on quality, mobility and curriculum development. He has completed a

number of related projects in Europe and established trans national education curricula in

SE Asia and China.

Karin Järplid Linde is Head of Department of Quality Assurance at the Swedish Higher

Education Authority. She has 13 years of experience in external quality assurance of higher

education, with different positions on a management level for the past five years. She has

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 15

had several international engagements in external quality assurance in Europe and

represents the Swedish Higher Education Authority in various networks, such as ENQA.

She was also a member of the National Team of Bologna Experts 2007-2014.

Dietlinde Kastelliz is currently head of the department “Audit and Consulting” at the

Austrian Agency for Quality Assurance and Accreditation (AQ Austria) and was a project

coordinator at one of the agency‟s predecessors. Prior to that, she worked in the Austrian

Federal Ministry of Foreign Affairs. After graduation from the University of Graz, she acted in

different educational contexts as lecturer and office manager in Kyrgyzstan, the Ukraine and

Croatia for seven years, before coming back to work at the University of Graz. Dietlinde has

been involved in several projects related to quality management in higher education.

Oliver Vettori is Dean for Accreditations and Quality Management as well as Director of the

Department for Programme Management and Teaching & Learning Affairs at WU, the

Vienna University of Economics and Business, Austria. He obtained his PhD from the

University of Vienna in the field of Sociology and Organisation Theory and is still is also

liaised as a research associate with the Institute for Public Management and the Institute for

Organization Studies (both at WU). Oliver Vettori is the initiator and coordinator of the

Network for Quality Assurance and Quality Development of the Austrian Universities, a team

coordinator for the Institutional Evaluation Programme of the European University

Association (EUA) and has been frequently working as a reviewer , consultant and

evaluator, i.a. for ENQA, evalag, ZeVa, AQ-Austria and ASEAN-QA as well as for numerous

universities across Europe.

Johan Alvfors

Petra Bulthuis MSc is senior educational advisor at the School of Marketing & International

Management of Saxion University of Applied Sciences. Her job is to advise management

and teachers on improving the educational quality. She has broad experience as a teacher

and educational advisor at Institutes for MBO (vocational education) and Higher Education.

She develops tools and guides on assessment, study career guidance, curriculum design

and other subjects.

Anke Thijssen MSc is senior policy advisor at the department of Quality Assurance of

Saxion University of Applied Sciences. Her particular focus now is on quality assurance

issues, such as development and implementation of quality assurance protocols,

coordination of accreditation for Saxion and coaching schools in the preparation for internal

and external audits. Since this year she is a member of the Dutch Bologna Expert Group.

Prior to this job she worked as a policy officer at the economic unit of one of the precursors

of the MBO Raad, the Dutch association of VET Colleges, and at the University of Twente

as a researcher on curriculum development.

Paul Garré

Luc van de Poele

Kees Kouwenaar

Robert Coelen

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 16

5 LIST OF PARTICIPANTS

Gabriele Abermann

Salzburg University of Applied Sciences

Austria

Axel Aerden

NVAO

Belgium

Johan Alvfors

Swedish National Union of Students (SFS)

Sweden

Christine Anderton

Nyenrode New Business School

The Netherlands

Dr. Daan Andriessen

HU University of Applied Sciences Utrecht

The Netherlands

Dr. Katrien Bardoel

EP-Nuffic

The Netherlands

Hélène Bartelds MA

Erasmus Universiteit Rotterdam

The Netherlands

Bartlomiej Banaszak

Ministry of Education

Poland

Rianne Bechet-Tjoonk

The Hague University

The Netherlands

Isabel Belmonte

Agency for Quality Assurance in the

Galician University System (ACSUG)

Spain

Ilze Birzniece

Riga Technical University

Latvia

Audbjorg Bjornsdottir

University of Akureyri

Iceland

Professor Emita Blagdan

Agency for Science and Higher Education

Croatia

Rihards Blese

Higher Education Quality Agency (AIKA)

Latvia

Lucien Bollaert

NVAO

Belgium

Hege Brodahl

Norwegian Agency for Quality Assurance in

Education

Norway

Elisabeth Brunner-Sobanski

University of Applied Sciences bfi Vienna

Austria

Petra Bulthuis MSc

Saxion University of Applied Sciences

The Netherlands

Bente Ringlund Bunæs

The Norwegian Association of Higher

Education Institutions Address

Norway

Dr. Robert Coelen

Stenden University of Applied Sciences

Leeuwarden

The Netherlands

Anna Collard

NOKUT

Norway

Perry Coppiëns

NUFFIC/OCW

The Netherlands

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 17

Fiona Crozier

The Quality Assurance Agency for Higher

Education, UK

UK

Steven David

NVAO

The Netherlands

Dr. Thomas de Bruijn

NVAO

The Netherlands

Erick de Jong

UNESCO-IHE

The Netherlands

Nicole de Leeuw

NHTV Breda University of Applied Sciences

The Netherlands

Yvonne de Vries

Tilburg University

The Netherlands

Lieselotte de Zutter

NVAO

Belgium

Gisli Fannberg

University of Iceland

Iceland

Berta Fernandez Alvarez

UNESCO-IHE

The Netherlands

Dr. Mark Frederiks

NVAO/ECA

The Netherlands

Ingrid Furfjord

The Norwegian Police University College

Norway

Paul Garré

Departement Onderwijs Vlaanderen

Belgium

Drs. Guðrún Geirsdóttir

University of Iceland

Iceland

Andy Gibbs BSC (Hons), MSC, PGCE, RN

School of Health and Life Sciences,

Glasgow Caledonian University

UK

Anna Gover

European University Association

EU

Inge Greving

Hanzehogeschool Groningen

The Netherlands

Vidar Gynnild

Norwegian University of Science and

Technology

Norway

Varduhi Gyulazyan

ANQA

Armenia

Dr. Jon Haakstad

Norwegian Agency for Quality Assurance in

Education (NOKUT)

Norway

Professor Hans Hellsten

Lund University, Faculty of Performing Arts

in Malmö

Sweden

Dr. Kath Hodgson MBE

Leeds University

UK

Ellen Jagtman

NUFFIC/OCW

The Netherlands

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 18

Karin Järplid Linde

Swedish Higher Education Authority

Sweden

Hallur Jonasson

The Icelandic Centre for Research

Iceland

Mag. Dietlinde Kastelliz MA

Agentur für Qualitätssicherung und

Akkreditierung Austria

Austria

Maria-Regina Kecht Mag.

Austrian Private University Conference

Austria

Åsa Kettis

Uppsala University

Sweden

Monique Knoester

NVAO

The Netherlands

Eveline Kok

VU Free University Amsterdam

The Netherlands

Jonna Korhonen

European University Association

EU

Kees Kouwenaar

VU Free University Amsterdam

The Netherlands

Professor Berta Leeb

Private Pädagogische Hochschule der

Diözese Linz

Austria

Drs. Patrick Leushuis

Ministry of Education, Culture and Science

The Netherlands

Dr. Eric Lindesjöö

Linköping University

Sweden

Sandra Louwersheimer

NVAO

The Netherlands

Mariusz Maciejczak

Warsaw University of Life Sciences

Poland

Marta Mallo Rey

Agency for Quality Assurance in the

Galician University System (ACSUG)

Spain

Maciej Markowski

Polish Accreditation Committee

Poland

Professor Vladimir Mrša

University of Zagreb, Faculty of Food and

Biotechnology

Croatia

Dr. Andreas Neuhold

Ministry of Science, Research and

Economy

Austria

Professor Anna Elisabet Olafsdottir

Bifrost University

Iceland

Joy Plokker

NUFFIC/OCW

The Netherlands

Henri Ponds

NVAO

The Netherlands

Dr. Alba Prieto-Gonzalez

European Union (EACEA)

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 19

Maria Próchnicka

Jagellonian University, Krakow

Poland

Ana Proroković

replaced by: Leonardo Marušić

University of Zadar, Department of

Psychology

Croatia

Dr. Dagmar Provijn

NVAO

Belgium

Christina Raab

University of Innsbruck

Austria

Rebekka Silvía Ragnarsdóttir

Icelandic Erasmus+ National Agency -

Icelandic Centre for Research

Iceland

Dr. Andrejs Rauhvargers

Quality Assurance Agency

Latvia

Gudrun Salmhofer

University of Graz

Austria

Harald Scheuthle

Evaluationsagentur Baden-Württemberg

Germany

Drs. Anne Klaas Schilder

NUFFIC/OCW

The Netherlands

Tineke Sessink

Platfom Life Long Learning

The Netherlands

Sigurdur Sigurdsson

The Icelandic Centre for Research

Iceland

Gerard Sijben MSc

Fontys University of Applied Science

The Netherlands

Professor Paweł Stępień

University of Warsaw

Poland

Britta Stoelinga

Rotterdam Business School

The Netherlands

Una Strand Vidarsdottir

Ministry of Education, Science and Culture

Iceland

Susann Stroemsvaag

The National Union of Students in Norway

Norway

Prof.dr. Ana Tecilazić Goršić

Ministry of Science, Education and Sports

Croatia

Anke Thijsen MSc

Saxion University of Applied Sciences

The Netherlands

Professor Ruben Topchyan

ANQA

Armenia

Jeff van As

NUFFIC/OCW

The Netherlands

Lineke van Bruggen

NVAO

The Netherlands

Luc Van de Poele

Ghent University

Belgium

Tim van den Brink

NUFFIC/OCW

IJda van den Hout

NVAO

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 20

The Netherlands The Netherlands

Leonard van der Hout

NUFFIC/OCW

The Netherlands

Jolien van der Vegt

NUFFIC/OCW

The Netherlands

Dr. Els van der Werf

NUFFIC/OCW

The Netherlands

Annemiek Verkuijl

International Business School/ Amsterdam

University of Applied Science

The Netherlands

Oliver Vettori

WU -Vienna University of Economics and

Business

Austria

Professor Annick Vlaminckx

Karel de Grote-Hogeschool

Belgium

Dr. Martina Vukasovic

European Union (EACEA)

Belgium

Michèle Wera

NVAO

The Netherlands

Agata Wroczyńska

University of Warsaw

Poland

Drs. Sarmite Zeire

Riga Stradins University

Latvia

NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 21

Accreditation Organisation of the Netherlands and Flanders (NVAO)

Parkstraat 28

P.O Box 85498 | 2508 CD DEN HAAG

T 31 70 312 23 00

E info@nvao.net

www.nvao.net