NVAO’s external quality NVAO’s external quality assurance proceduresassurance procedures
Ann Van Neygen
La Rochelle11th of May 2012
2
‘Higher education’
‘Government’ ‘Society’
Enhancement
Accountability Information
Recognition
FundingSupport
Appreciatio
n
Value for €
Proof of useQuality
Access
(Consumer) protection
Support
• Stimulate quality culture institutional audit
• Commit professionals / Increase academic ownership
programme assessment
• Reward earned trustlimited programme assessment
• Stimulate HE to aim above thresholdaccredit as satisfactory, good, excellent
3
• Institutional audits • Initial accreditation
• (New) programmes that want to offer a recognised degree
• All bachelor and master’s programmes, including associate degrees
• Accreditation• programmes that already offer a recognised degree
• Publication of decisions (and officially recognised degrees)
4
Publication
Report
Official register
(Initial) accreditation• Concentrate on content
(& focus on performance) learning outcomes
Audit• Focus on policy & practice
re. internal QA system teaching and learning
5
Institution
Programme Report
6
1. What is the vision of the institution regarding the quality of the education it provides & of the development of a quality culture?
2. Does the institution have an appropriate policy to realise this vision?
3. How does the institution measure the degree to which this vision is realised?
4. How can the institution demonstrate that it systematically improves the quality of its programmes?
5. Does the institution have an effective organisation and decision-making structure regarding the quality of its programmes?
Positive / Negative / ConditionalPositive / Negative / Conditional
• Installed by NVAO
• At least 4 members, including 1 student
• The panel commands administrative (Board), educational and audit expertise, is acquainted with developments in the higher education sector at home and abroad, and is authoritative
• One of the members with Board expertise will act as chair
• The panel is independent (its members have had no ties with the institution over at least the past 5 years)
• Not part of panel: NVAO process co-ordinator and secretary (also independent of the institution)
7
• Conversation on management level
• Institutional profile: results of previous accreditation
• Critical reflection by HEI (max. 50 pages + appendices)
• Site visit• 1st visit (1-2 days): exploration of issues, first interviews and
feedback, and panel decision on topics for audit trails
• 2nd visit (2-3 days; 2 to 4 weeks after 1st visit): issues emerging from the meetings & documents studied during 1st visit; audit trails; at the end feedback to HEI on overall judgement and considerations
• Audit trails: panel follows the trail from the institutional level to the implementation of policy and/or the management of problems or vice versa
8
• Judgement on standards:• Meets, does not meet or partially meets the standard
• Overall judgement: • Based on its vision of the quality of the education
provided, the board of the institution has implemented an effective quality assurance system, which enables it to guarantee the quality of the programmes offered.
• Positive, negative or conditionally positive
9
• If positive judgement then NVAO approval valid for 6 years
• If conditionally positive then explicit statement of relevant conditions and positive judgement within 1 year:
• If conditions are met then NVAO approval for 6 years• If conditions are not met then approval expires
• If negative judgement then approval is withheld for at least 3 years:
• comprehensive programme assessments• additional assessments for those programmes which
have already had limited assessment
10
• For each standard:• Findings
• Considerations
• Judgment
• Improvement suggestions
• Summary for wider audience
• Report and decision is published on the site of NVAO
11
12
Extensive programme assessment
Limited programme assessment
Yes No
Is this institution recognised?
Recognition procedure
Has this institution undergone an institutional audit?
Yes No
++ decision -- decision
13
Is this programme on the register/ recognised?
(Lim./Compr.) initial accreditation
procedure
(Lim./Compr.) accreditation
procedure
-- decision ++ decision ++ decision -- decision
Extensive programme assessment
Limited programme assessment
Satisfactory - Good - Excellent
Deleted from register Included on register
Impr
ovem
ent
perio
d
14
Limited programme assessment
Extensive programme assessment
Assessment of•content of the programme•focus on performance
Detailed assessment of•content,•policy and•procedures
3 standards 16 standards
Themes
1. Intended learning outcomes 2. Programme and staff quality
3. Achieved learning outcomes & Learning assessment
1. Intended learning outcomes 2. Programme3. Staff4. Services5. Quality assurance system6. Achieved learning outcomes &
Learning assessment
Conclusion: unsatisfactory, satisfactory, good, excellent quality
• At least four members, one of whom is a student;• at least two authoritative subject-matter experts
• At least one with teaching experience at relevant level;
• aware of latest international developments in the discipline
• expertise in the professional field (where applicable)
• educational expertise
• assessment or audit expertise;
15
• The panel is independent• no ties with the institution offering the programme for
at least five years;
• an independent, external secretary trained and certified by NVAO;
• signed declarations of independence and codes of conduct;
• panel composition and declarations of independence will be published and made public;
• the parties involved in the assessment can report any matters that could affect the independence to NVAO.
16
17
Programme Module x Module y Module z ...
Intended LOs
Intended LOs
Intended LOs
Intended LOs
...
(C. &) LOs (C. &) LOs (C. &) LOs
Assessment Assessment Assessment ...
(ECTS) Course catalogue
Achieved?
Achieved?
Achieved?
Achieved?
…
• Who ?• New programmes: NVAO• Recognised programmes: evaluation organisation
• Information dossier (max. 50 pages + appendices)
• Site visit• Conversations with management, teachers, students, work
field • Overall judgment and considerations• Short feedback to HEI
18
• Judgement on standards:• New programme: Satisfactory, unsatisfactory
• Recognised programme: Satisfactory, good, excellent, unsatisfactory
• Overall judgement: • New programme: Satisfactory, unsatisfactory
• Recognised programma: Satisfactory, good, excellent, unsatisfactory
19
• New programmes:• If positive judgement then NVAO approval valid for 6
years
• If negative judgement: no approval
• Recognised programmes:• If positive judgement then NVAO approval valid for 6
years
• If negative judgement:• Limited approval• Conditions for improvement• If conditions are not met approval expires
20
• Generic (basic) qualityThe quality that in all reasonableness could be expected of a bachelor’s or master’s programme within higher education, and this from an international perspective.
• UnsatisfactoryThe programme does not provide generic quality.
• SatisfactoryThe programme provides generic quality.
• GoodThe programme is of notably higher quality than generic quality.
• ExcellentThe programme is of a quality very much above generic quality and fulfils an exemplary role for other relevant programmes.
21
• For each standard:• Findings
• Considerations
• Judgment
• Improvement suggestions
• Summary for wider audience
• Report and decision is published on the site of NVAO
22
23
Application
Allocation
Admissibility
Analysis
Proposed decision
Final decision
Publication
Letter Institutional details Programme details Expert panel report By managing director
First handler=policy advisor
Second handler Portfolio handler
= executive board member
Screening application Panel report
All standards judged Proof of payment Completeness
Template for Analysis Substantiation judgements
(Objective) findings (Subjective) considerations
Quality panel report ≠ quality programme
Decision NVAO Procedure followed Panel composition Essential programme details Substantiation decision Status and period of validity
NVAO website Netherlands: Centraal Register
Opleidingen Hoger Onderwijs (CROHO) Flanders: HE Register
24
25
• Open and inclusive approach• One framework for all assessments in a given type of
procedures• E.g. one assessment framework for all types of initial
accreditation
• Fit for purpose (~appropriate burden)
• Respect for institutional autonomy • Internal quality assurance system
• Assessment in line with institutional choices on the basis of the relevant framework
• Programme level• Institutional level
26?
• Generic, descriptive standards≠ Narrow, prescriptive and highly formulated
Broad acceptance
• Focus on what should be done (~fitness of purpose)
≠ How they should be achievedDo not dictate practice
• Focus on teaching & learning (~education)
• Incl. academic standards ≠ Research
• Incl. professional orientation ≠ Service to society
27
• Central documentation for External QA• Self-contained document – relatively brief
• Basic information• Institution / programme details
• Presents perspective shared by (internal) stakeholders• Addresses each standard in relevant framework
• rationale, practice, ambition
• Provide insight into strengths & weaknesses• Maximising strengths?
• Converting weaknesses into strengths?
• Mandatory annexes
28
• Direct window on what programmes aims to do & what a programme actually does• Less focus on policy, process & procedures
• But process & procedures are easier to assess
• Facilitates international transparency & comparability• E.g. International benchmarking
• Open to all types of teaching & learning• Technology Enhanced Learning / Distance learning
• Work-based learning / Assessment of prior learning
• Provides a common language for HE & stakeholders• Teaching & Learning
29