Is ‘Quality’ a Beleaguered Concept? The Place of

Quality Audits in a PBRF Environment

Gareth Jones

Introduction

Striking contrastsStriking contrasts

Quality auditQuality audit

on teaching and teaching programson teaching and teaching programs

no repercussions for individual staffno repercussions for individual staff

emphasis on processes and claims of the institutionemphasis on processes and claims of the institution

PBRFPBRF

evidence of research status of institutionevidence of research status of institution

significant financial implications for institutionsignificant financial implications for institution

ranking of individual academic staffranking of individual academic staff

emphasis on emphasis on outcomesoutcomes

Introduction

Uneven battleUneven battle

Why bother about quality in teaching?Why bother about quality in teaching?

Doesn’t research bring status and Doesn’t research bring status and recognition?recognition?

Introduction

Effects of RAE in UK - chasm between Effects of RAE in UK - chasm between research and teachingresearch and teaching

Asymmetry between teaching and Asymmetry between teaching and research institutionalised by external research institutionalised by external reward systemsreward systems

If we downgrade teaching, are we If we downgrade teaching, are we negating a basic tenet of research-led negating a basic tenet of research-led universities?universities?

The centrality of quality

Used ‘quality’ in very narrow sense - to Used ‘quality’ in very narrow sense - to apply to teaching alone and teaching-apply to teaching alone and teaching-related activitiesrelated activities

Concept of quality applies to everything Concept of quality applies to everything we do: research as much as teachingwe do: research as much as teaching

Speak another language in research areaSpeak another language in research area

The centrality of quality

Link between all teaching and research Link between all teaching and research should be provided by should be provided by qualityquality

Should never be content with first-rate Should never be content with first-rate research and mediocre teaching, or with research and mediocre teaching, or with first-rate teaching and mediocre researchfirst-rate teaching and mediocre research

Hallmark of illustrious universities is Hallmark of illustrious universities is that they are outstanding in teaching and that they are outstanding in teaching and researchresearch

The centrality of quality

2006 2006 GuardianGuardian table assessing teaching quality, table assessing teaching quality, staff-student ratios and graduate job prospectsstaff-student ratios and graduate job prospects

Top 10 places:Top 10 places:

• OxfordOxford• CambridgeCambridge• LSELSE• UCLUCL• Imperial CollegeImperial College

• SOASSOAS• KCLKCL• WarwickWarwick• BathBath• BristolBristol

Living with asymmetry

Status and financial rewards are far Status and financial rewards are far greater in case of research than teachinggreater in case of research than teaching

How do we take quality seriously in the How do we take quality seriously in the relatively poorly rewarded area of relatively poorly rewarded area of teaching?teaching?

Major distinction between RAE and Major distinction between RAE and PBRFPBRF

Living with asymmetry

In PBRF research rewards are In PBRF research rewards are outcome outcome drivendriven

In academic audits teaching rewards are In academic audits teaching rewards are process drivenprocess driven

Living with asymmetry

What one wants to see are What one wants to see are good good processes that lead to good outcomesprocesses that lead to good outcomes

On the surface, neither quality audits nor On the surface, neither quality audits nor PBRF encourage both partsPBRF encourage both parts

A A dual emphasisdual emphasis upon teaching and upon teaching and research will not be driven by externally research will not be driven by externally imposed reward mechanisms: institution imposed reward mechanisms: institution has to establish its own ethoshas to establish its own ethos

• Confirmation (individual; T+R)• Promotion (individual; T+R)• Departmental reviews (academic unit; T+R)• Professional accreditation (course; T+R)• Student Opinion Surveys and Graduate Opinion Surveys

(academic unit; T)• Postgraduate supervision (individual; T+R)• Whole institution academic audit (institution; T+R)

Role of processes

• PBRF (individual and unit: R)

Role of processes

• Confirmation (individual enhancement)

• Promotion (individual enhancement and status)

• Departmental reviews (feedback and self-improvement)

• Professional accreditation (continued existence of professional course)

• Student Opinion Surveys and Graduate Opinion Surveys (feedback and self-improvement)

• Postgraduate supervision (feedback and self-improvement)

• PBRF (status; financial rewards)

Role of processes

What rewards accompany these processes?What rewards accompany these processes?

Role of processes

Imagine an institution that decided to Imagine an institution that decided to place little store by internal processes place little store by internal processes and placed all its emphasis on research and placed all its emphasis on research and PBRFand PBRF

Would the institution’s reputation be Would the institution’s reputation be enhanced by its research emphases to enhanced by its research emphases to such an extent that it could get away such an extent that it could get away with what might be very poor quality with what might be very poor quality teaching?teaching?

1 an institution’s standing in the eyes of quality agencies is far from insignificant

2 accreditation of professional courses is crucial to those courses and the students in them

3 students wield considerable influence; student dissatisfaction with teaching and courses will result in long-term loss of EFTS and a drift from those areas

Role of processes

Cannot get away with poor quality teaching:Cannot get away with poor quality teaching:

Why have processes that link teaching and research?

Our processes take account of both teaching Our processes take account of both teaching and researchand research

Teaching alone, in the absence of any research Teaching alone, in the absence of any research component, including research into teaching, is component, including research into teaching, is untenable at a university that aspires to untenable at a university that aspires to research-led teaching. Conversely, research research-led teaching. Conversely, research alone, in the absence of any teaching, has alone, in the absence of any teaching, has major limitations for staff employed to interact major limitations for staff employed to interact with studentswith students

Why have processes that link teaching and research?

OutcomesOutcomes are critical in all cases are critical in all cases

Congruence between high quality Congruence between high quality research and high quality teaching: the research and high quality teaching: the research-teaching nexus is crucialresearch-teaching nexus is crucial

Quality is not a beleaguered concept, Quality is not a beleaguered concept, even in a PBRF environment, because it even in a PBRF environment, because it is common to both teaching and researchis common to both teaching and research

Focus is entirely on teaching-related matters; teaching quality, program delivery, and the achievement of learning outcomes

2006 academic audit

1 Induction and support of HODs (RES)2 Development of a best practice data base (RES)3 More effective use of data from student and graduate

opinion surveys (RES)4 Foster formative methods of evaluating courses5 Celebrating success in supporting Mäori students6 Ensuring primary supervisors are research active (RES)7 Development of generic workload models (RES)

2006 Quality Audit

Improvement initiatives (central to teaching)Improvement initiatives (central to teaching)

8 Integrate Graduating Year Reviews into the University’s review processes

9 Undertake strategic benchmarking activities (RES)

10 Integrate external accreditation and internal review processes more closely

11 Formalisation of mentoring for junior supervisors (RES)

12 Review the PhD progress reporting system and confirmation processes (RES)

13 Support more students to be awarded top external scholarships (RES)

14 Rationalisation of the graduate attributes

2006 Quality Audit

Improvement initiatives (central to teaching)Improvement initiatives (central to teaching)

“By rewarding research excellence … PBRF reinforces collegiality within an institution, and strengthens teaching. PBRF encourages research productivity, as well as quality, which in turn allows frequent peer review and public presentation and further promotes collegiality.”

2006 academic audit

Geoff WhiteGeoff White

2006 Academic Audit

CollegialityCollegiality is central is central

This is a problem with the academic unit This is a problem with the academic unit and also with PBRF. A teaching-driven and also with PBRF. A teaching-driven academic audit tends to drive a wedge academic audit tends to drive a wedge between teaching and research, while between teaching and research, while PBRF can become far too individualistic.PBRF can become far too individualistic.