European Quality AssuranceRegister for Higher Education

Agency reviews against the ESG 2015:Expectations of the EQAR Register Committee

Expert training workshop8 January 2016, Berlin

Colin Tück

EQAR Policy on Use andInterpretation of the ESG

Aims

Create transparency ex ante Consistency and understanding

Replaces “Practices and Interpretations” (2013)

Adopted by Register Committee, after consultation ofEuropean stakeholder organisations and governments

New element: “Reports should at least demonstrate”

Read in conjunction with the (full) ESG

Example

Shorter and fewer than inPractices & Interpretations 2013,only where needed

New: ensure that external reviewreports provide evidence needed

Changes in EQAR'sapplication process

New Old

Application Before review After review

Check of eligibility First step of process Prior check optional

External review Follows eligibility check Before application

Flags No longer used In approval decision

Publication policy: Full documentation Decision only

Coverage of reviews

All external QA activities within the scope of the ESGThe focus of the ESG is on quality assurance related to learning and teaching in highereducation, including the learning environment and relevant links to research andinnovation. […] The ESG apply to all higher education offered in the EHEA regardless of themode of study or place of delivery. Thus, the ESG are also applicable to all highereducation including transnational and cross-border provision. In this document the term“programme” refers to higher education in its broadest sense, including that which is notpart of a programme leading to a formal degree. […] The term ‘quality assurance’ is usedin this document to describe all activities within the continuous improvement cycle (i.e.assurance and enhancement activities).

Obligatory and voluntary Within the EHEA and outside the EHEA Clear procedures & criteria for activities abroad Clear separation of other activities (e.g. consultancy) See also letters to agencies (confirmation of eligibility)

2.1 Consideration ofinternal quality assurance

External quality assurance should address the effectiveness of theinternal quality assurance described in Part 1 of the ESG.

Reports should at least demonstrate:• How the agency addresses the effectiveness of internal QA processesin its evaluations, audits and accreditations.• How standards 1.1 – 1.10 (see Annex 4) are addressed in the agency'scriteria and processes for institutions/programmes.

2.2 Designing methodologiesfit for purpose

External quality assurance should be defined and designed specificallyto ensure its fitness to achieve the aims and objectives set for it, whiletaking into account relevant regulations. Stakeholders should beinvolved in its design and continuous improvement.

Reports should at least demonstrate:• How the agency develops, reviews and updates the processes andcriteria used in its different activities.• How the agency ensures that its methodologies are fit for purpose.• How stakeholders are involved in the design and continuousimprovement of the agency's processes.

2.3 Implementingprocesses

External quality assurance processes should be reliable, useful, pre-defined, implemented consistently and published. They include- a self-assessment or equivalent;- an external assessment normally including a site visit;- a report resulting from the external assessment;- a consistent follow-up.

Reports should at least demonstrate:• How the agency ensures that its processes are reliable, useful andconsistent.• How the key features in the standard are implemented by the qualityassurance agency in each of its activities.• If no site visits are used, how evidence provided by institutions isvalidated through other mechanisms.

2.4 Peer-review experts

External quality assurance should be carried out by groups of externalexperts that include (a) student member(s).

Reports should at least demonstrate:• How the agency's groups of experts are composed.• How the agency ensures in the selection process that experts haveappropriate skills and competences, and no conflict of interest.• How the agency organises training or briefing of experts.

2.5 Criteria foroutcomes

Any outcomes or judgements made as the result of external qualityassurance should be based on explicit and published criteria that areapplied consistently, irrespective of whether the process leads to aformal decision.

Reports should at least demonstrate:• How the agency published the criteria used in each of its activities.• How the agency ensures consistency in its application of criteria forall types of reviews.

2.6 Reporting

Full reports by the experts should be published, clear andaccessible to the academic community, external partners andother interested individuals. If the agency takes any formaldecision based on the reports, the decision should be publishedtogether with the report.

Reports should at least demonstrate:• How reports are made accessible to the public for all types of reviews.• How the agency ensures that its reports are clear and understandablein their structure, content and style.

2.7 Complaints andappeals

Complaints and appeals processes should be clearly defined as part ofthe design of external quality assurance processes and communicatedto the institutions.

Reports should at least demonstrate:• Which appeals processes are in place for each of the agency'sactivities.• How the agency handles complaints.

3.1 Activities, policy andprocesses for qualityassurance

Agencies should undertake external quality assurance activities asdefined in Part 2 of the ESG on a regular basis. They should have clearand explicit goals and objectives that are part of their publiclyavailable mission statement. These should translate into the dailywork of the agency. Agencies should ensure the involvement ofstakeholders in their governance and work.

Reports should at least demonstrate:• How the agency complies with standards 2.1 – 2.7 (in a distinctchapter, each standard should be addressed separately for eachdifferent type of review).• How the agency's mission translates into its daily activities.• How stakeholders are involved in the agency.• How the agency ensures a clear distinction between external qualityassurance and its other fields of work, if applicable.

3.2 Official status

Agencies should have an established legal basis and should beformally recognised as quality assurance agencies by competentpublic authorities.

Reports should at least demonstrate:• What is the legal status of the agency.• In which higher education system(s) the agency is formallyrecognised as a quality assurance agency.

3.3 Independence

Agencies should be independent and act autonomously. They shouldhave full responsibility for their operations and the outcomes of thoseoperations without third party influence.

Reports should at least demonstrate:• How the agency's organisational independence is demonstrated byofficial documentation.• How the agency operates independently de facto, especially in termsof defining procedures and methods as well as nomination andappointment of experts.• How the agency ensures that the outcomes of its quality assuranceprocesses are its independent responsibility.

3.4 Thematic analysis

Agencies should regularly publish reports that describe and analysethe general findings of their external quality assurance activities.

Reports should at least demonstrate:• How the agency conducts and publishes analyses of the generalfindings from its activities.• How it uses the outcomes of these analyses.

3.5 Resources

Agencies should have adequate and appropriate resources, bothhuman and financial, to carry out their work.

Reports should at least demonstrate:• How the agency's financial arrangements ensure the sustainability ofits activities within the scope and in line with the ESG.

3.6 Internal qualityassurance andprofessional conduct

Agencies should have in place processes for internal quality assurancerelated to defining, assuring and enhancing the quality and integrity oftheir activities.

Reports should at least demonstrate:• How the agency's internal QA system guarantees the quality andintegrity of its activities.• How the internal QA system fosters continuous improvement withinthe agency.

3.7 Cyclical externalreview of agencies

Agencies should undergo an external review at least once every fiveyears in order to demonstrate their compliance with the ESG.

Reports should at least demonstrate: N/A

The cyclical review of an agency is a prerequisite for (continued) EQARregistration and inherently fulfilled by the agency undergoing a review.

Decision-making

Conclusions for each standard Overall judgement

Review panel

Substantially compliant

Not substantially compliant

either ...

… or

Register Committee decision

Full compliance

Substantial compliance

Partial compliance

Non-compliance

Compliance (full or substantial)

Partial compliance

Non-compliance

All standards

One or more

One or more → holistic judgement

Clarification requests to panel chair if needed If conclusion differs from panel, explained in public decision

Publication