ENQA procedures for external ENQA procedures for external review – the Bulgarian experiencereview – the Bulgarian experience

Todor.Shopov@yahoo.com

Vilnius,30 May 2009, 11.15-11.45 Hrs.

Objectives

• The Agency (3 – 16)• The External Review (17 – 30)• The External Review Report (31 – 35) • The EQAR Application (36)• The Current State of Affairs (37 – 40)

2

The Agency (est. 1996)

3

Official Status

a statutory body for evaluation, accreditation and monitoring of the quality in higher education institutions and scientific organizations, aiming at the enhancement of their teaching and research as well as of their development as scientific and cultural organizations

4

Activities

• reviews the ability of institutions to provide good quality of education and scientific research

• performs project evaluation, institutional and programme accreditation, post-accreditation monitoring of quality of higher education

5

Mission

to encourage higher education institutions in assuring and enhancing the quality of education they offer by sustaining high academic standards and good education traditions in Bulgaria

6

Structure

Decision-making bodies• Accreditation Council • Standing Committees on areas of higher

education

AdministrationExpert teams (ad hoc)

7

Standing Committees

• formed by the Accreditation Council which appoints their members through selection by applications

• comprised of 3 to 7 members, one of whom is Chairman of the Committee

• the Chairman of the Accreditation Council appoints the members of the Committees for a three-year term

8

Standing Committees

(in alphabetical order)1. AGRARIAN SCIENCES AND VETERINARY MEDICINE 2. ECONOMIC SCIENCES AND MANAGEMENT 3. EDUCATIONAL SCIENCES AND MUSICAL AND DANCE ART 4. HEALTHCARE AND SPORTS 5. HUMANITIES AND ARTS6. NATURAL SCIENCES, MATHEMATICS AND COMPUTING7. SOCIAL SCIENCE, LAW AND NATIONAL SECURITY STUDIES8. TECHNICHAL AND MILITARY SCIENCES

9

Standing Committee on POST-ACCREDITATION MONITORING

AND CONTROL

supervise HEI’s implementation of the recommendations received in the evaluation

and accreditation process

10

Internal Quality Assurance of NEAA

• IQA system• Corrector-NEAA

11

IQA system

Four subsystems:1.legal base2.standards, policies, procedures and

documents3.management subsystem4.material resources and financing

12

Corrector-NEAA

Mechanisms for collecting feedback and recommendations on improving the activity:

• Internal relations for feedback from administration, Standing Committees and Accreditation Council

• External relations from individuals (academic experts, students, professionals) and higher-education institutions

13

External Quality Management (Control, Assurance, Improvement)

quality terms: Quality Improvement can be distinguished from Quality Control

in that Quality Improvement is the purposeful change of a process to improve the reliability of achieving an outcome.

Quality Control is the ongoing effort to maintain the integrity of a process to maintain the reliability of achieving an outcome.

Quality Assurance is the planned or systematic action necessary to provide enough confidence that a product or service will satisfy the given requirements for quality.

14

The national system of quality management in higher education

PRINCIPLES• quality is not in the hands of TPTB ("the

powers that be")• the "chorus of stakeholders" rules

15

The national system of quality management in higher education

16

The External Review

• the application: 7 November 2007• the language: "it's not an English exam“• the patterns: to design the preparation

process effectively

17

The Patterns

• The patterns (A and B) help us plan and execute the external evaluation procedure.

• That provides the Agency with positive experiences.

18

Pattern A, identified in the preparation for the external review

• Pattern A is a complicated scheme. It is known as an ill-formed system.

• We could not use it effectively.

19

20

Pattern B, identified in the preparation for the external review

• We needed a well-organised, lean structure.

• So, in June 2007, we developed Pattern B, based on classic model of quality

management.

21

Pattern B, identified in the preparation for the external review

INPUT → PREPARATIONPROCESS → OUTPUT → SITEVISIT → EFFORTS TO IMPLEMENTRECOMMENDATIONS

22

INPUT

the state of affairs in June 2007

23

PREPARATION PROCESS

• translation of “Guidelines ”; “Marinov” task force appointed, issues identified: leadership, transparency of processes, accountability of participants, timelines, etc.

• other issues (we are working on them even now): Participation in NEAA’s expert teams (ESG 2.4, 3.7); Reporting (ESG 2.5); Independence of recruiting experts (ESG 3.6); Composition of decision-making bodies (ESG 3.6, 3.8)

24

OUTPUT

self-evaluation report (May 2008)

25

The site visit of the Review Panel• On June 17-18, 2008 an external expert group under the aegis of ENQA,

coordinated by the Spanish Quality Assurance Agency ANECA, visited the NEAA to perform an external review of its activities. The on-site visit is part of the procedures for ENQA membership established in the ESG.

• Within the two-day visit the expert group had working meetings with representatives of the NEAA leadership and its staff as well as with representatives of the State Administration, Local Communities, Trade-Unions, Rector’s Council, Higher Education Institutions and Students.

26

The TOR

The TOR were approved by the NEAA(14 Feb 2008) and ENQA (3 March2008).

27

describe the purpose and the structure of the external review, i. e. what has to be achieved, who will take part in it, how it will be achieved, when it will be achieved. The TOR are enclosed in the evaluation report. They contain 8 sections explaining the review process, the review panel (6 members), the self-evaluation report, the site visit, the evaluation report, the final decision by the board of ENQA, the publication of the evaluation report, and the budget of the review.

28

Official correspondence with ANECA

29

The self-evaluation report

The task force, appointed by the Chairmanof NEAA (June 2007), met regularly in theperiod June 2007 – June 2008.

30

THE EXTERNAL REVIEW REPORT

It was produced, as planned, in July 2008.Its conclusions are presented in Chapter 5,entitled "NEAA's compliance with ESG". Thefirst section, "Summary of strengths and areasof improvement" describes the positive andnegative aspects of the activities of NEAA(strengths and weaknesses).

31

The second section of Chapter 5, "Final statement", includes the following positive evaluation

"In the light of the documentation and oral evidenceprovided during the site visit, the Review Panel is satisfiedthat in the performance of its functions NEAA is insubstantial compliance with the ENQA Membership Criteriaand with the Standards and Guidelines for QualityAssurance in the EHEA. The Panel therefore recommendsto the Board of ENQA that NEAA should be awarded FullMembership for a period of five years.”

32

July 2008

comments of NEAA on the factual accuracy of the external review report

33

34

ISSUES

Problems solved at present:• Participation in NEAA’s expert teams (ESG 2.4,

3.7)• Reporting (ESG 2.5)• Independence of recruiting experts (ESG 3.6)• Composition of decision-making bodies (ESG

3.6, 3.8)

35

THE EQAR APPLICATION

The application was submitted inFebruary 2009. Two letters withadditional explanations were sent to theEQAR Chair in March and May 2009.

36

THE CURRENT STATE OF AFFAIRS

NEAA has a new goal: to make the process of evaluation andaccreditation leaner, more user-friendly andmore effective. For example, the twoprocedures of institutional and programmeaccreditation may be merged into oneappraisal procedure at institutional level.

37

THE CURRENT STATE OF AFFAIRS

NEAA has formed a working group toexamine all issues related to its work and topropose fundamental changes in theaccreditation of higher education institutionsin line with current developments in qualityassurance in education.

38

THE CURRENT STATE OF AFFAIRS

In September 2009, the term of office of allStanding Committees will come to an end.New members will be elected and in thisprocess students and non-academicprofessionals will be included as membersof the committees. This decision was takenby the Accreditation Council in March 2009.

39

THE CURRENT STATE OF AFFAIRS

In 2010, the NEAA will submit an interimreport on the progress in implementing thechanges recommended by ENQA.

40

THANK YOU!