Quality assurance at HUB

23-11-2011, Brussels

Paul Garré

director quality and education

Quality Assurance at HUB

• Internal policy– Total quality management (education,

supporting organisation and services, research, services to society)

– Strong stakeholder participation (students, staff, alumni, relevant external stakeholders)

– Documented quality system (internal audits)

Quality Assurance at HUB• External regulations

– Peer review by independent expert panels (national quality agency: VLUHR)

– Accreditation (international Dutch-Flemish Accreditation Council: NVAO)

– NEW: starts up in 2015: institutional reviews by international expert panels (international Dutch-Flemish Accreditation Council: NVAO)

• Check• Act

• Do• Plan

Mission statement

Strategic goals

Operational plans

Quality Manual

Surveys

Data analysis

Self evaluation

Internal audit

Peer review

Project teams

Changed procedures

New structures

A mission to accomplish …a long journey

Quality policy: 10 basicsTeam work

Teach as you preach

Facts and figures

Personal commitment

Shared focus

Stakeholder orientation

Exchange of expertise

Open communication

Simplicity

Empowerment

Educational policy: 3 premises

Competence

•Integrated knowledge, skills and attitudes

•Authentic contexts•Professionally relevant learning outcomes

•Evidence/research based contents

Learning

•Adequate student counseling•Increasing student autonomy•Active knowledge construction

Powerful learning environment

•Interaction and collaboration•Flexibility•Diversity, interculturality•Mobility•Blended learning•Studiability•Transparancy

A quality system is just a way to coordinate some basic building blocks

Commitment

Authority and responsability

Budget

Time

Capacity

Expertise

Information

Communication

Vision

Adequate methods and instruments

Organisational Structure

Education and Quality Department

1. Educational development and innovation unit

2. Quality assurance unit

3. Statistical Analysis and Measurements unit

1 Educational development unit

• Curriculum development• Educational training programmes• Flexible learning paths en competence

assessment• New trends and innovations

Educational Training at HUB

1. Policy model 2. Educational Training: main characteristics3. Administration

Policy modelThe main goal of the training model at HUB is to develop the organisation and to achieve its policy goals (so the main goal is not individual development).

• Individual training is the responsability of the individual employee. If relevant and related to their job, employees can follow external training.

• In order to achieve strategic and operational goals, HUB provides internal training, i.e. training of the competences of all employees and teaching staff or specific groups:

• Educational Training: Training needs in Faculties and Fields of Study are discussed and listed by deans/directors, central training needs by the Management Committee. The Education and Quality Department is responsable for realisation. Some examples

• Functional Training: Training needs are discussed and listed by Staff and/or Management Committee. The HR Department is responsable for realisation. Some examples

Educational Training: main characteristics Central projects- HUB Education-day:

Organised by Faculties and Fields of Study / Preferably 1 central theme / Networking and knowing each other

- Strategic projects – ICTO: Individual consultancy or made-to-measure training by the ICTO Department. – Diversity and Corporate Sustainability: made-to-measure training

• Preferably made-to-measure training tracks with more than one training session

for specific groups or curriculum teams in order to achieve work related implementation • Realisation in collaboration with the coordinator Diversity or Corporate Sustainability

Training for Faculties and Fields of Study– Preferably made-to-measure training tracks with more than one training session

for specific groups or curriculum teams in order to achieve work related implementation – Realisation in collaboration with the coordinators educational development in every Faculty / Field of Study– Communication and follow-up of implementation by deans and directors

Other Trainings– Initiatives of a specific group or curriculum team within a Faculty or Field of Study– No direct link with the strategic or operational goals of the Faculty or Field of Study

Examples: Doing filosophy with children (BALO), Statistic programme Maple (CT Maths & Statistics), ….

AdministrationCommunication

– External Training: Intranet, monthly overview by mail, meetings of the team

– Internal Training: Intranet for central projects, made-to-measure trainings by mail

Approval of participation and registration– External Training: Digital procedure for approval and registration

• For teaching staff: advice by directors and approval by faculty president/ approval by deans• For administrative employees: advice by head of department and approval by the director HR Department• Automatic registration in a databank

– Internal Training: • For teaching staff: no approval needed (teaching hours can be replaced but cannot be cancelled)• For administrative employees: approval by the head of department• Registration: attendance list with signature

– All registrations (also for trainers/instructors) are transferred to an individual training portfolio• This portfolio is on the agenda in performance interviews / can be used in the context of promotion

One central budget divided in two: – External: Administration by faculty president/deans for teaching staff / director HR Department for all administrative employees

– Internal: Administration by the Education and Quality Department

Evaluation and follow-up• Training sessions: Internal evaluation of every training • External (and most internal) trainings are on the agenda of curriculum teams and departments• Global policy: Follow-up in the annual Scorecard / the annual Strategy and Effectiveness plan

2 Quality assurance unit

• Quality manual• Complaints handling• Internal audits• External audits and peer reviews• Accreditation procedures• Self assessment excercises

3 Statistical analysis and measurement unit

two main tasks:

Drawing up statistics using the information in existing databases

Organizing and analyzing surveys

Statistics and survey results are posted on the intranet

Information is used on various levels:

Micro: teacher wants to know how his students performed

Meso: a degree programme has to write a self evaluation report, statistical data are vital

Macro: general management gets insight in the top layer of statistical information (through strategy book and strategy and effectiveness report)

Distinction between:

Routine information: • Statistics that are drawn up every year (based

on existing internal/external frameworks)• Surveys organized using a measurement

planning covering multiple years

BUT! Take into account changes in routines, frameworks, survey methods, the educational landscape…

Ad hoc: statistics drawn up on request

Strategy Book/Strategy and effectiveness report

statistics (existing databases)

surveys

Routine • Number of students• Influx of students• Characteristics of (influx of) students

• Student performance• Academic level of degree programmes

• Internationalization

• Evaluation of Didactic Practice

• Alumni• Study time• Student quality barometer

• Personnel survey+ some smaller surveys

ad hoc    

Short presentation of the different surveys

Evaluation Of Didactic Practice

with a standardized questionnaire students evaluate a combination of a professor and a course (used to be on paper, now digitally)

Planning (3-year cycle)Year x: courses BA1, MA 1, MA2Year x+1: courses BA3Year x+2: courses BA2

Evaluation Of Didactic Practice (cont.)

Goals:

Provide students with an opportunity to give feedback on their professors

Inform professors about their didactic practiceHR tool: corrective measures for professors with low

scores, element in promotionsOverview of Didactic Quality of a certain degree programme

Alumni survey

alumni receive an extensive (digital) questionnaire asking them about (i) their professional career up until that moment; (ii) how they look back on their studies at HUB

Every alumnus gets a questionnaire one and half year after graduation (=march of graduation year +2),

Alumni survey (cont.)

Goals:

Get an overview of where alumni work (or additional courses), job characteristics and job satisfaction

Get to know view alumni now have on their degree programme (competences acquired, impact of international experience, general satisfaction…)

Study time measurements

students are asked to estimate the study time they needed for their courses, two methods (designed on association level):

• Estimating after the course has finished• Keeping a study time diary (every week)

Every degree programme stage has to have at least one study time measurement in a period of eight years

Study time measurements (cont.)

Goals:

Legislation states that 1 credit should entail 25-30 hours of study time – measurement is necessary

Getting an idea of the balance of the study load within a degree programme (stage) and within the academic year

Measures if necessary (if study time is either too low or too high)

Student Quality Barometer

using a standardized questionnaire students give their opinion on a high number of topics concerning their studies at HUB

Every four years, all students are asked to fill out the student quality barometer

Student Quality Barometer (cont.)

Goals:

Get an overview of how student’s score different aspects of studying at HUB (also important for external justification)

Getting an idea of the problem areas, element to prioritize actions

Assessing if former problem areas have been dealt with

Other (smaller) surveys include

Quality of internshipQuality of course materialExperience of exchange students

(incoming/outgoing)Student’s motivation behind a study choice

Statistics + Survey results = Loads of information

Useful for individuals/degree programmes(they only use what they need)

Less useful for general management wanting to assess the state of things at HUB (bird’s eye view)

Management tool that summarizes the information is needed = strategy book

Short presentation of the strategy book

genesis:

From the Mission statement goals are derived

To assess if these goals are met, indicators for each goal need to be determined and neccessary data collected.

Each goal has its own scorecard with indicators

Number of studentsMarket position

Practice orientednessScientifically foundedSocial formation

Didactic excellenceQuality oriented (improve, consolidate, justify)Services and facilitiesStudent and study guidance

Students’ performanceStudy time

EmploymentOn the job training

What problems is STAM confronted with?

Pitfalls:

central data provider for more than 20 degree programmesVisibility?Local commitment?

Ad Hoc questions: don’t let them get in the way of routine information

Pitfalls (cont.):

Surveys: response = continuous source of frustration

Actions that can/need to be taken:• Group survey sessions in pc rooms• For some digital questionnaires: ask student

number (use only to target non-respondents, guarantee anonimity)

• make (previous) survey results available• Communicate!• Plan!

Pitfalls (cont.):

Strategy book = still too extensive to really get a bird’s eye view

Plan to develop an all-encompassing management tool (max. 20 à 25 indicators)

Thanks for your attention!

Any questions?