Sheffield Hallam University - APRIL 2006 · Sheffield Hallam University APRIL 2006. Preface The...

Sheffield Hallam University

APRIL 2006

Preface The Quality Assurance Agency for Higher Education (QAA) exists to safeguard the public interest insound standards of higher education (HE) qualifications and to encourage continuous improvementin the management of the quality of HE.

To do this QAA carries out reviews of individual HE institutions (universities and colleges of HE). InEngland and Northern Ireland this process is known as institutional audit. QAA operates similar butseparate processes in Scotland and Wales. For institutions that have large and complex provisionoffered through partnerships, QAA conducts collaborative provision audits in addition toinstitutional audits.

The purpose of collaborative provision audit

Collaborative provision audit shares the aims of institutional audit: to meet the public interest inknowing that universities and colleges are:

providing HE, awards and qualifications of an acceptable quality and an appropriate academicstandard, and

exercising their legal powers to award degrees in a proper manner.

Judgements

Collaborative provision audit results in judgements about the institutions being reviewed.Judgements are made about:

the confidence that can reasonably be placed in the soundness of the institution's present andlikely future management of the quality of the academic standards of its awards made throughcollaborative arrangements

the confidence that can reasonably be placed in the present and likely future capacity of theawarding institution to satisfy itself that the learning opportunities offered to students throughits collaborative arrangements are managed effectively and meet its requirements; and

the reliance that can reasonably be placed on the accuracy, integrity, completeness andfrankness of the information that the institution publishes, (or authorises to be published)about the quality of its programmes offered through collaborative provision that lead to itsawards and the standards of those awards.

These judgements are expressed as either broad confidence, limited confidence or no confidenceand are accompanied by examples of good practice and recommendations for improvement.

Nationally agreed standards

Collaborative provision audit uses a set of nationally agreed reference points, known as the'Academic Infrastructure', to consider an institution's standards and quality. These are published byQAA and consist of:

The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ),which includes descriptions of different HE qualifications

The Code of practice for the assurance of academic quality and standards in higher education

subject benchmark statements, which describe the characteristics of degrees in different subjects

guidelines for preparing programme specifications, which are descriptions of the what is onoffer to students in individual programmes of study. They outline the intended knowledge,skills, understanding and attributes of a student completing that programme. They also givedetails of teaching and assessment methods and link the programme to the FHEQ.

The audit process

Collaborative provision audits are carried out by teams of academics who review the way in whichinstitutions oversee their academic quality and standards. Because they are evaluating their equals,the process is called 'peer review'.

The main elements of collaborative provision audit are:

a preliminary visit by QAA to the institution nine months before the audit visit

a self-evaluation document submitted by the institution four months before the audit visit

a written submission by the student representative body, if they have chosen to do so, fourmonths before the audit visit

a detailed briefing visit to the institution by the audit team six weeks before the audit visit

visits to up to six partner institutions by members of the audit team

the audit visit, which lasts five days

the publication of a report on the audit team's judgements and findings 22 weeks after theaudit visit.

The evidence for the audit

In order to obtain the evidence for its judgement, the audit team carries out a number of activities,including:

reviewing the institution's own internal procedures and documents, such as regulations, policystatements, codes of practice, recruitment publications and minutes of relevant meetings, aswell as the self-evaluation document itself

reviewing the written submission from students

asking questions of relevant staff from the institution and from partners

talking to students from partner institutions about their experiences

exploring how the institution uses the Academic Infrastructure.

The audit team also gathers evidence by focusing on examples of the institution's internal qualityassurance processes at work through visits to partners. In addition, the audit team may focus on aparticular theme that runs throughout the institution's management of its standards and quality.This is known as a 'thematic enquiry'.

From 2004, institutions will be required to publish information about the quality and standards oftheir programmes and awards in a format recommended in document 03/51, Information on qualityand standards in higher education: Final guidance, published by the Higher Education FundingCouncil for England. The audit team reviews how institutions are working towards this requirement.

© The Quality Assurance Agency for Higher Education 2006

ISBN 1 84482 583 3

All QAA's publications are available on our website www.qaa.ac.uk

Printed copies are available from:Linney DirectAdamswayMansfieldNG18 4FN

Tel 01623 450788Fax 01623 450629Email [email protected]

Registered charity number 1062746

Summary 1

Introduction 1

Outcome of the collaborative provision audit 1

Features of good practice 1

Recommendations for action 2

National reference points 2

Main report 4

Section 1: Introduction: the institutionand its mission as it relates tocollaborative provision 4

Background information 5

The collaborative provision audit process 5

Developments since the institutional audit of the awarding institution 6

Section 2: The collaborative provision audit investigations: theawarding institution's processes for quality management incollaborative provision 7

The awarding institution's strategicapproach to collaborative provision 7

The awarding institution's framework for managing the quality of the students'experience and academic standards in collaborative provision 8

The awarding institution's intentions for enhancing the management of itscollaborative provision 9

The awarding institution's internal approval, monitoring and reviewarrangements for collaborative provision leading to its awards 11

External participation in internal reviewprocesses for collaborative provision 14

External examiners and their reports in collaborative provision 15

The use made of external reference points in collaborative provision 16

Review and accreditation by externalagencies of programmes leading to theawarding institution's awards offered through collaborative provision 17

Student representation in collaborativeprovision 17

Feedback from students, graduates andemployers 18

Student admission, progression, completion and assessment information for collaborative provision 19

Assurance of the quality of teaching staff in collaborative provision; appointment, appraisal, support anddevelopment 20

Assurance of quality of distributed anddistance methods delivered through anarrangement with a partner 21

Learning support resources for students in collaborative provision 22

Academic guidance and personal support for students in collaborativeprovision 23

Section 3: The collaborative provision audit investigations: published information 24

The experience of students in collaborative provision of the publishedinformation available to them 24

Reliability, accuracy and completeness ofpublished information on collaborativeprovision leading to the awardinginstitution's awards 25

Findings 27

The effectiveness of the implementation of the awarding institution's approach to managing its collaborative provision 27

The effectiveness of the awarding institution's procedures for assuring the quality of educational provision in its collaborative provision 28

The effectiveness of the awarding institution's procedures for safeguarding the standards of its awards gained through collaborative provision 29

Contents

The awarding institution's use of the Academic Infrastructure in the context of its collaborative provision 30

The utility of the collaborative provision self-evaluation document as an illustration of the awarding institution's capacity to reflect upon its own strengths and limitations incollaborative provision, and to act on these to enhance quality and safeguard academic standards 31

Commentary on the institution's intentions for the enhancement of its management of quality and academic standards in its collaborative provision 31

Reliability of information provided by the awarding institution on its collaborative provision 32

Features of good practice 33

Recommendations for action 33

Appendix 34

Sheffield Hallam University's response to the collaborative provision 34audit report

Summary

Introduction

A team of auditors from the Quality AssuranceAgency for Higher Education (QAA) visitedSheffield Hallam University (the University) from24 to 28 April 2006 to carry out an audit of thecollaborative provision offered by the University.The purpose of the audit was to provide publicinformation on the quality of the programmesof study offered by the University througharrangements with collaborative partners, andon the discharge of the University's responsibilityas an awarding body in assuring the academicstandard of its awards made throughcollaborative arrangements.

To arrive at its conclusions the audit team spoketo members of staff of the University, and reada wide range of documents relating to the waythe University manages the academic aspects of its collaborative provision. As part of theaudit process, the team met with four of theUniversity's collaborative partners, where itspoke to students on the University's collaborativeprogrammes and to members of staff of thepartner institution.

The words 'academic standards' are used todescribe the level of achievement that a studenthas to reach to gain an award (for example, a degree). It should be at a similar level acrossthe UK.

Academic quality is a way of describing howwell the learning opportunities available tostudents help them to achieve their award. It is about making sure that appropriateteaching, support, assessment and learningopportunities are provided for them.

The term 'collaborative provision' is taken tomean 'educational provision leading to anaward, or to specific credit toward an award, of an awarding institution delivered and/orsupported and/or assessed through anarrangement with a partner organisation' (Code of practice for the assurance of academicquality and standards in higher education, Section2: Collaborative provision and flexible and

distributed learning (including e-learning), 2004,paragraph 13, published by QAA).

In an audit of collaborative provision bothacademic standards and academic quality are reviewed.

Outcome of the collaborative provisionaudit

As a result of its investigations the audit team'sview of the University is that:

broad confidence can reasonably beplaced in the soundness of the University'spresent and likely future management ofthe academic standards of its awardsmade through collaborative arrangements

broad confidence can reasonably beplaced in the present and likely futurecapacity of the University to satisfy itselfthat the learning opportunities offered to students through its collaborativearrangements are managed effectively and meet its requirements.

Features of good practice

The audit team identified the following areas as being good practice:

the contribution that collaborativeprovision makes to the University'sstrategy particularly in respect of wideningparticipation and continuing professionaldevelopment

the way in which the University is movingits AQR processes to support theenhancement of collaborative provision.

the support provided for staff in partnerorganisations through formal and informalcommunication channels and processesincluding the collaborative conference

the use of its virtual learning environmentboth in its delivery of programmes and asa way of effective communication withstudents and partners

Collaborative provision audit: summary

page 1

Recommendations for action

The audit team also recommends that theUniversity should consider further action in anumber of areas to ensure that the academicquality of programmes and standards of theawards it offers through collaborativearrangements are maintained. The teamconsiders it advisable that the University:

makes better use of its statisticalinformation to monitor and compare the performance of particular groups of students

ensures the process of revalidation forcollaborative provision is as rigorous as the on-site periodic review and includesinvolvement of students

implements an effective process forproviding timely feedback to students on assessed work

develops a process for ensuring thechecking of transcripts produced bycollaborative partners

and considers it desirable that the University:

defines the criteria by which 'Chair'saction' is appropriate in respect ofinstitutional approval and programmeapproval and modification

continues to monitor the efficacy of thechanges to the structure and operation of the assessment process.

National reference points

To provide further evidence to support itsfindings, the audit team also investigated theuse made by the University of the AcademicInfrastructure which QAA has developed onbehalf of the whole of UK higher education. The Academic Infrastructure is a set of nationallyagreed reference points that help to define bothgood practice and academic standards. Theaudit found that the University was makingeffective use of the Academic Infrastructure inthe context of its collaborative provision.

In due course, the audit process will include acheck on the reliability of the Teaching Quality

Information (TQI) published by institutions inthe format recommended in the HigherEducation Funding Council for England'sdocument 03/51, Information on quality andstandards in higher education: Final guidance.The audit team was satisfied that theinformation the University and its partners ispublishing currently about the quality of itscollaborative programmes and the standards ofits awards is reliable, and that the University ismaking adequate progress to providing TQIdata for its collaborative provision.


page 2

Main report

Main report1 An audit of the collaborative provision(CP) offered by Sheffield Hallam University (the University or SHU) was undertaken duringthe period 24 to 28 April 2006. The purpose ofthe audit was to provide public information onthe quality of the programmes of study offeredby the University through arrangements withcollaborative partners, and on the discharge of the University's responsibility as an awardingbody in assuring the academic standard of its awards made through collaborativearrangements.

2 CP audit supplements the institutional auditof the University's own provision. The process ofCP audit has been developed by the QualityAssurance Agency for Higher Education (QAA) in partnership with higher education institutions(HEIs) in England. It provides a means forscrutinising the CP of an HEI with degreeawarding powers (awarding institution) wherethe CP was too large or complex to have beenincluded in the institutional audit of theawarding institution. The term 'collaborativeprovision' is taken to mean 'educational provisionleading to an award, or to specific credit towardsan award, of an awarding institution deliveredand/or supported and/or assessed through anarrangement with a partner organisation' (Codeof practice for the assurance of academic qualityand standards in higher education (Code ofpractice), Section 2: Collaborative provision andflexible and distributed learning (including e-learning) 2004).

3 The CP audit checked the effectiveness ofthe University's procedures for establishing andmaintaining the standards of academic awardsthrough collaborative arrangements; forreviewing and enhancing the quality of the programmes of study offered throughcollaborative arrangements that lead to thoseawards; for publishing reliable information aboutits CP; and for the discharge of its responsibilityas an awarding body. As part of the collaborativeaudit process, the audit team visited four of theUniversity's collaborative partners.

Section 1: Introduction: theinstitution and its mission as itrelates to collaborative provision4 The recent history of the University can be traced to the merger of three colleges:Technology, Commerce, and Art and Design,into Sheffield Polytechnic in 1969. A change ofname to Sheffield City Polytechnic came withfurther mergers with three teacher trainingcolleges during the 1970s. Sheffield CityPolytechnic was incorporated in 1989 and wenton to become Sheffield Hallam University withthe authority to award its own degrees in 1992.

5 The University has a well-establishedportfolio of CP. It currently has partnershipswith 86 organisations of which 14 are overseas.At the time of audit the University had over28,000 students of which more than 3,000 are studying on collaborative programmes.Approximately 2,000 of these fall within thescope of this audit.

6 The University has recently completed theprocess of a major academic restructuring from10 schools to four faculties: Arts, Computing,Engineering and Sciences; Development andSociety; Health and Wellbeing; and Organisationand Management. The move to facultiesprovided an opportunity to introduce somegreater standardisation in the operation of CP.Faculties instigate collaborative partnerships and provision which best suit their needs. TheUniversity has not sought to impose particularmodels for collaboration or draw boundaries asto the type of organisation considered suitable.

7 The faculties' development of CP growsdirectly from the University's institutionalmission as expressed in its Vision and ValuesStatement. Thus the University seeks to:promote access for a diversity of students, offer flexible course delivery; enable students to meet the challenges and opportunities of a rapidly changing world; and increasestudents' employability by encouraginginnovation, creativity and enterprise. Therealisation of these objectives is being metpartly through the development of CP with arange of partners.


page 4

8 The implementation of the University'sCorporate Plan 2003-2008, Positioning forGrowth, has provided an opportunity to betteralign CP to the aims of the University. Thecorporate plan specified a number of areas ofgrowth: multiprofessional development; publicsector education and training; internationalprovision; and continuing professionaldevelopment (CPD). The collaborative portfoliois integral to these developments, and it alsoenables the University to maintain its long-standing commitment to widening participationespecially through the further development ofFoundation Degrees (FDs). A good example ofCP which is at once multiprofessional, based in the public sector, and incorporating wideningparticipation and CPD is the FD in Health CareInformatics. The audit team recognised as goodpractice the manner in which CP contributes to broader University strategies particularlywidening participation and professionaldevelopment.

Background information

9 The published information available forthe audit included the following recent reports:

Institutional Audit Report (April 2005)

the outcomes of developmentalengagement reports for Architecture,Architectural Technology and LandscapeArchitecture (April 2003), Accountancy(May 2003), Law (December 2003) and Geography (October 2004)

FD review reports for Food ManufacturingManagement (May 2002), AppliedComputing - Defence GeographicInformation (July 2005), AppliedComputing (October 2005) and BusinessInformation Technology (October 2005)

subject review report of English, Sociology and Anthropology, Social Policy and Administration and Social Work (July 2002)

major review report of healthcareprogrammes (December 2003).

10 The University also provided QAA with aseries of documents and information including:

an institutional CP self-evaluationdocument (CPSED)

undergraduate and postgraduateprospectuses

access to the University intranet

documentation relating to the partnerinstitutions visited by the audit team.

11 During the briefing and audit visits, theaudit team was given ready access to a rangeof the University's internal documents inhardcopy and through intranet access. Theteam identified a number of partnershiparrangements that illustrated further aspects of the University's provision, and additionaldocumentation was provided for the teamduring the audit visit. The team was grateful for the prompt and helpful responses to itsrequests for information.

The collaborative provision auditprocess

12 A preliminary meeting was held at theUniversity in July 2005 between a QAA officerand representatives of the University andstudents. QAA confirmed in September 2005that four partner visits would be included in the audit. The University provided its CPSED inDecember 2005 and briefing documentationfor the selected partner institutions in February2006. Students' Union (SU) officers were invitedto reflect views of students studying for SHUawards through collaborative partners and ashort written statement was provided inFebruary 2006. The audit team is grateful tothe SU officers at the University for theirassistance during the audit.

13 The audit team visited the University from27 February to 1 March 2006 to discuss withsenior members of staff of the University, seniorrepresentatives from partner institutions, andstudent representatives from the University andpartner institutions, matters relating to themanagement of quality and academic standardsin CP raised by the University's CPSED and other

Collaborative provision audit: main report

page 5

documentation. At the close of the briefing visit,a programme of meetings for the audit visit wasagreed with the University. Additionally, it wasalso agreed that selected document audit trailswould be followed to gain a clear understandingof the range of collaborative arrangements andprocedures.

14 Visits to partners, which included the useof video-links, took place between the briefingand audit visits and members of the audit teammet senior staff, teaching staff and studentrepresentatives. The team is grateful to the staff of the partner institutions for their help inadvancing its understanding of the University'sarrangements for managing its collaborativearrangements.

15 The audit visit took place from 24 to 28April 2006, and included further meetings withstaff of the University. The audit team is gratefulto staff who participated in meetings. Theauditors were Dr P Campbell, Dr T Joscelyne,Professor P Periton, and Professor N Whiteleywith Ms M Sheehan as audit secretary. Theaudit was coordinated for QAA by Professor HColley, Assistant Director, Reviews Group.

Developments since the institutionalaudit of the awarding institution

16 The findings of the institutional auditreport (April 2005) highlighted a number ofpoints which were relevant to the audit of theUniversity's CP. In the CPSED the Universityprovided the audit team with a summary of its response to the audit report.

17 In the audit report the University wasadvised to reassess how the staff appraisal andpeer-supported review of learning, teachingand assessment might be more effectively usedfor the assurance of teaching quality in additionto the enhancement of teaching standards. Inresponse the University stated that Link Tutortraining along with CP conferences wereaddressing the development needs of its staff.The University was also actively consideringhow it could be more proactive in its approachto the development of partner staff.

18 The audit report indicated it would bedesirable for the University to review the internalprocesses for responding to the reports of the external examiners to avoid potentialduplication and ensure timely responses. Arevised procedure is now in place under whichfaculties respond to issues raised by externalexaminers at module and course level, ensuringthat both the partner and the external examinerare advised of actions taken. The new procedurerequires Registry to track the faculty responsesand also produce an overview response toinstitutional issues raised which is shared withexternal examiners and partner organisations.

19 As a consequence of the institutional audit, although not in direct response to arecommendation, the University stated in theCPSED that it continues to work on its studentsupport framework with the intention ofclarifying a threshold standard of studentsupport available to students including those atpartner organisations. The University has madea virtual learning environment (VLE) an integralpart of the enhancing of the student learningexperience. Full access to the VLE has untilrecently been available only to enrolled and not registered CP students. Technical andcontractural barriers for access to the VLE forregistered students have now been overcomeand it can be made available to all students. The audit team found that CP partners andregistered students feel that access to electronicdatabases would benefit the student learningexperience. The University's Executive Group has supported a proposal for a set of technicaldevelopments that will make targeted access,specific to particular programmes for registeredstudents, on the basis of prior agreement onrequired resources. Licensing and financial issueswill be resolved as part of the prior agreement.

20 The Associated College Network hasplayed an important part in the development of CP at the University. Recently its role haschanged to focus more on facilitatingrecruitment to the University and to reflect this change it has been re-titled the AssociateCollege Partnership. A paper submitted to theUniversity Executive Group in January 2006


page 6

recognised that major partners of the Universitywould benefit from an institutional strategic linkto coordinate planning of collaborativeactivities. Consequent upon this, discussions are under way at a senior level to explore thepossibilities of setting up institutional-level linksand streamlined link-tutor arrangements withpartners with whom the University has anumber of programmes.

21 The University is currently engaged throughthe Assessment Working Group in consideringhow best to implement the recommendation of the institutional audit report to provide clearguidelines for the timely feedback on assessedwork to students. In addition this audit teamwould stress the desirability of the Universitycontinuing to monitor the efficacy of changes to the assessment process arising from the workof the Assessment Working Group.

22 FDs have formed a significant part of UK-based CP. The University explained that this provision provides an invaluable way ofsupporting the University's wideningparticipation agenda and a awards are to bevalidated in 2006. Work is now progressing on updating University FD curriculum designprinciples in response to the latest FoundationDegree Qualification Benchmark, published byQAA and to lessons learnt from recent reviews of FDs.

23 The audit team considered that theUniversity had engaged with therecommendations made in the institutionalaudit report and had a well-plannedmechanism for effecting operational change.The intended impact of the measures taken was appropriate but it was too early to judgeon eventual effectiveness.

Section 2: The collaborativeprovision audit investigations: theawarding institution's processesfor quality management incollaborative provision

The awarding institution's strategicapproach to collaborative provision

24 In its Collaborative Provision Policy,published in 2005, the University describedhow, over the years, the University hasdeveloped a range of types of CP andpartnerships. The document went on to explainthat it has not sought to impose particularmodels for collaboration, or draw boundaries asto the types of organisation considered suitable.The range of CP arrangements is expressed inthe University's 'collaborative typology'. Thetypology indicates the types of relationship:agent, articulation, enrolled courses (shareddelivery single award or joint award), registeredcourses (licence or validated/credit-rated) andthe responsibilities of the parties. In 2001-02 the total number of collaborative partners wasreduced in response to the precepts of the Code of practice, Section 2 and to an assessmentof the risks of the portfolio. Since then, revisedprocedures have operated that have controlledthe type and number of collaborative coursesand, in particular, overseas provision.

25 The University's approach andcommitment to CP, expressed in its CP policystatement, is directed by its strategic objectivesas reflected in the Vision and Values Statement.Thus, it:

seeks to promote access to advanced levelskills and knowledge for a diversity ofstudents within a culture of lifelong learning

puts students at the heart of teaching and learning by offering flexible coursedelivery with time, pace and, increasingly,place chosen by students

enables students to meet the challengesand opportunities of a rapidly changingworld through educational excellence and enterprise


page 7

is committed to increasing students'employability by encouraging innovation,creativity and enterprise.

26 It is University policy to integrate CP intoon-site provision as far as possible, whether bymeans of progression from FDs to on-sitehonours degrees, or the development ofpartnerships encompassing in the sameinstitution both CP and the placement ofstudents taking on-site courses. The Universitystated in its CPSED that it aims for 'richpartnerships' with 'substantial, mature andexperienced organisations' in order to facilitatesustainable relationships. All CP courseproposals are validated by the University'sCollaborative Standing Panel (CSP) whichconsists of a core team of highly experiencedstaff well equipped to gauge the risks andrequirements of differing collaborativearrangements.

27 In its International Strategy 2005-2010 the University describes how it seeks to become'an important contributor to internationaldevelopments in the UK higher educationsector'. International developments are seen asmainstream rather than an additional part ofUniversity business. The approach involves theselection of a small number of partners ofcomparable mission and status and theminimisation of risk. There is a UniversityInternational Group (IUG) chaired by the ProVice Chancellor (PVC) - Academic Development(PVCAD), which meets regularly to prioritiseopportunities, coordinate developments andshare best practice.

28 The Corporate Plan specified a number of areas of growth: multiprofessionaldevelopment; public sector education andtraining; international provision; CPD; andresearch and business development. Thegrowth is focused into the following academictheme areas: creative industries, computing and communication technologies; health andwellbeing; management; and social development.Corporate Plan implementation provided anopportunity to undertake an analysis of theUniversity's CP register, to reflect on experience

of the management of CP, and to share bestpractice. This took the form of an extensivereview of CP through faculty mini-audits(FMAs). The review resulted in reports sharedwith faculties and in action plans forenhancement of the provision.

29 Each faculty implements CP, in the contextof the University policy statement, as anelement in the portfolio of activities throughwhich its strategic objectives and those of theUniversity are achieved. From faculty minutes,and meetings with staff, the audit team wassatisfied that faculties were paying due regardto wider University strategies when developingcollaborative, including international, provision.

The awarding institution's frameworkfor managing the quality of thestudents' experience and academicstandards in collaborative provision

30 The Quality and Standards Managementand Enhancement (QSME) framework is themain vehicle through which the Universitymonitors and assures the establishment andmaintenance of academic standards and themanagement and enhancement of quality. TheQSME framework was established in 2001 andembraces the University's CP to ensure that thequality and standards of CP are managed asrigorously as for on-site provision. TheUniversity seeks to ensure that the academicstandards of CP satisfy the nationalexpectations embedded in the AcademicInfrastructure, and that the very diversepopulation of students studying by means ofCP is enabled to achieve the standards througha learning experience which is fit for purpose.

31 The Academic Frameworks, Policies andRegulations (AFPR) inform and regulate allQSME activities and take account ofappropriate external reference points. Theelements of the AFPR which are particularlyrelevant to CP include the academic awardsframework; standard assessment regulations;admissions policy and minimum entryrequirements; and criteria for the appointmentof external examiners and assessors for


page 8

University awards. Staff at partner institutionshave access to the AFPR through the provisionof the collaborative partners' web pages. Theaudit team's meetings with staff confirmed the importance, easy accessibility andcomprehensibility of the AFPR information.

32 The Academic Board (AB) is the senioracademic body in the University. It endorses the recommendations of validation panelswhich consider new collaborative courseproposals, and approves the Annual QualityReview (AQR) of CP. One of the AB's two subcommittees, the Academic DevelopmentCommittee (ADC), is responsible forrecommending to the AB 'policy frameworks for collaborative partnerships involvingacademic provision' and for monitoring theimplementation of relevant strategies throughapproved frameworks. The ADC is therefore thekey body within the University's deliberativestructure providing central oversight of themanagement of the quality and standards ofCP. The ADC is assisted in its monitoring role bythe Monitoring Sub-Committee (MSC) whichmonitors the effectiveness of the managementof quality and standards of CP, principallythrough scrutiny of the reports whichcontribute to the AQR of CP provision.

33 The responsibilities of the AB are mirroredat faculty level by the Faculty Academic Board(FAB), which is the senior academic body with a focus on academic policy and the effectivenessof academic delivery. The FAB is assisted in thisrole by a faculty QSME Committee whichreports to the FAB. Within each faculty, theAssistant Dean - Academic Development(ADAD) has senior management responsibility,under the Executive Dean, for the strategicdevelopment of the CP to meet the faculty'sstrategic and business plans and stakeholderneeds. The ADAD is assisted in this role by thefaculty Head of Quality and Enhancement, who has oversight of the academic health ofthe faculty's CP.

34 A Negotiator from the faculty works with partners in the very early stages of thedevelopment of CP proposals. For 'enrolledstudent' courses (students with a direct

enrolment contract with the University for theprovision of their education), the negotiationwill normally be undertaken by a Universitycourse leader. For courses involving 'registeredstudents' (students registered with SHU for anaward but having an enrolment contract withthe partner) the Negotiator often becomes thelink tutor who is appointed to liaise, monitorand verify the effectiveness of the partner'sengagement with the University's QSMErequirements.

35 The University stated that it seeks abalance of tight corporate control of academicstandards, freedom for faculties to grow andinnovate, and the need for differentialtreatment of partners in a way which isappropriate to their needs and proportionate tothe risks involved. Faculties have some flexibilityin their local management arrangements andstructures although there is no relaxation of keyrequirements. The Overall University Statementregarding the implementation of the facultyQSME systems in 2004-05 clearly outlinesstandards, procedures and the allowablevariations and the framework, therefore, has avital role in ensuring consistency and rigour inprocesses and standards. The audit team foundthe framework to be comprehensive in scopeand robust for use and implementation.

The awarding institution's intentionsfor enhancing the management of itscollaborative provision

36 As outlined in the International Strategy2005-2010 the University is seeking to take amore strategic approach to the development of its CP in general and to its internationalprovision in particular. The PVCAD and seniorcolleagues in the faculties are meeting on aregular basis to identify cross-facultyopportunities and ways to enrich partnershipswhich promote the growth of trans-nationaleducation. The audit team was told that theUniversity's Executive Group is also developinga coherent framework for the proposal,planning, delivery and management ofUniversity partnerships.


page 9

37 The establishment of four large faculties,each of which has its own strategic approach to CP, albeit within the context of Universitypolicy, has the potential, in the opinion of theaudit team, to meet the key objectives of theUniversity set out in its International Strategyand the CP policy. A potential risk is a set ofdisparate faculty approaches insufficientlyfocused on the wider University initiatives. Theaudit team considered that the University hadin place sound management structures andapproaches to deal with such tensions. Forexample, there has been an extensive review of CP through FMAs. However, the team alsoconsidered that the overarching Universitypolicy for CP, while recognising thecontributions which can be made by thefaculties, will need to retain a strong centralsteer to keep it on course for successful deliveryof objectives.

38 The University is developing its studentinformation systems and processes to provide asingle data source for module information. It isenvisaged that this will cover CP and will formpart of the wider student information andmanagement system. A key aim is theproduction of transcripts by the University, andthe holding of full assessment and progressiondata, for all CP students. Currently such data isavailable for enrolled students. A significantamount of system development, includinginitial training for partner staff and an ongoingcommitment to staff development, has beennoted by the University for the successfulintroduction of the extended system. The auditteam noted the advisability of the Universitydeveloping a process for ensuring the checkingof transcripts produced by all CP partners (see also paragraph 72).

39 The AQRs Reports currently incorporatedata on the progression and achievement ofstudents on CP programmes but data isrelatively perfunctory. Any analysis of registeredstudents for ethnicity, gender or disabilitywould occur at the collaborative partnerinstitution and therefore would not be reportedin the University AQR Reports. Similarly,although evidence for enrolled students was

available for analysis this was not consistentlyundertaken in AQR Reports. Consequently,because the University is not systematicallymonitoring the progression of particular groupsof CP students, it is not in a position to considerthe comparative performance of such students.The student information systems and processesproject might usefully provide the necessaryinformation to overcome this lack ofmonitoring. However, currently the audit teamnoted that monitoring was incomplete andadvised the University to make better use of itsstatistical information to monitor and comparethe performance of particular groups ofstudents (see also paragraph 90).

40 The VC Executive Group hasrecommended that the approach to quality andstandards should become more enhancement-led. This is to be discussed by the University'sStanding Panels Chairs Forum and will bereflected in the agendas of future CPconferences and in changes to the AssessmentBoard structure to include considerations ofquality assurance and enhancement.

41 The University has established proceduresfor undertaking risk assessment and negotiatingwith new partners. Central to this is the role ofthe Faculty Negotiators. Detailed guidelines forthe role of Negotiator are available and theRegistry has convened a Negotiator workshop.In its CPSED the University recognised the needto improve the sharing of good practice andexpertise and is planning to provide forumswithin which this can take place. Similarly theIUG has noted the need for explicit limits onthe authority of staff negotiating internationalcollaborations. The changes agreed by IUG willbe incorporated into the guidelines forNegotiators.

42 The audit team concluded that theUniversity's intentions for enhancing themanagement of its CP are timely andappropriate within the context of its evolvingstrategy. The team also supported theUniversity's intention for quality and standardsprocedures to become 'more enhancement-ledand student-focussed'.


page 10

The awarding institution's internalapproval, monitoring and reviewarrangements for collaborativeprovision leading to its awards

43 Procedures for the approval andmonitoring of CP programmes werecomprehensively reviewed and updated during2004-05, partly to ensure that the University'sapproach is consistent with the expectations ofthe revised Code of practice, and to re-presentthe procedures to staff in new roles in thefaculties. The procedures now provide forapproval and monitoring arrangementsappropriate to the six broad types ofarrangement identified in the University'stypology of CP (see also paragraph 24), and to the different levels of inherent risk.

Planning approval44 The University requires that all new CPcourse proposals are approved through eachfaculty's Business Planning and OperationalReview process. Central departments such asRegistry and the Learning InformationTechnology Services (LITS) are able to provideadvice and support. Each faculty has anAcademic Portfolio Development Group orequivalent, chaired by the relevant assistantdean, which is responsible for advising thefaculty on the strategic development of theeducational portfolio. For each new CP courseproposal, an Outline Proposal (CPA1) form isrequired by Registry to ensure that the proposalis consistent with the AFPR and to agree anappropriate process and timescale for validation.

45 Where the new CP course proposalinvolves collaboration with a new partnerorganisation, the University guidance is for therelevant faculty to supplement the CPA1 formwith an Application for Approval of a NewCollaborating Organisation (CPA2) form, priorto validation of the course, to consider whetherthe organisation is an appropriate partner forthe University and, where necessary is suitablefor developing HE programmes. Forinternational proposals, the risk assessmentmust include an assessment of any risks posedby the particular overseas location. All CPA2

forms are submitted to the PVCAD for approvalin principle, subject to validation. Where it isproposed that a new international collaboratingorganisation will be delivering or assessingsubstantial components of a University award, a University-level institutional approval visit isrequired as part of the approval process. Inother cases which are judged to be of lower risk(as in UK proposals) the institutional approval is informed by faculty visits. For all new CPproposals, a financial risk assessment is neededas part of the process of seeking approval inprinciple to proceed to validation (CPA3). Allinternational proposals, and any UK proposalsoutside the financial limits for faculty sign-off,are also referred to the Director of Finance.

46 The audit team noted that the planningprocess has comprehensive documentation andis robust. The team heard that modifications tothe process are continuing in order to make the process more effective and efficient, but the team regards the current system as fit forpurpose.

Validation47 The primary aim of the University'svalidation process is to test that proposalsincorporate academic standards which are inaccordance with the AFPR and that programmesare of appropriate quality and are fit forpurpose to deliver a high quality studentexperience. CP proposals, which are scrutinisedby the University's CSP, are subject to the samevalidation process requirements as for on-siteprovision, adapted as appropriate toaccommodate the involvement of acollaborating organisation, and, if appropriate,representatives of its industrial or commercialpartners. The sequence involves a preliminarymeeting, normally involving a representative ofthe partner organisation, to identify the issuesthat will be explored at validation and to agreethe form of the validation event. Validation willthen involve the completion of requireddocumentation, a validation event involvingexternal expertise and a representative of thepartner, production of a validation reportmaking approval recommendations to theUniversity's AB which also sets out any


page 11

conditions of approval required by thevalidation panel, the submission of a definitivecourse document, and the signing of a formalagreement by the University and the partner.

48 Not all validations include an institutionalvisit and the interpretation of the guidance bystaff of the necessity or not of a visit relates tothe degree of risk associated with the validation.However, in the view of the audit team it is notimpossible that what may be perceived to be alow risk partner may have inadequate learningresources which would not be identified if a visitwere not to be completed. The team concludedthat the University might address the issue ofclarifying the need for a visit.

49 The CSP considers validation of CP courseproposals. It is not clear, however, where theboundary is between a 'light touch' but stillcommittee-based approach (including use ofexternal advisers), and the use of 'Chair's action'form of approval (that does not make use ofexternal advisers). Although Chair's action wasused on only three CP validations in 2004-05, the audit team underlines the importance of avalidation process that, however light in its touch,is robust and systematic. Part of the robustness isthe use of external advisers, and the team wouldadvise the University to consider the desirabilityof defining clear and unambiguous criteria for theuse of Chair's action.

50 Validation activity for each academic year isreported in a Validation Review (VR), producedby Registry, which includes a section devoted toCP. The audit team agreed with the assertion inthe CPSED that reviews from recent yearsconfirmed the continuing effectiveness of the CSPin securing appropriate quality and standards inCP proposals. The team acknowledges theimportance of the VRs in effectively raising anddealing with matters of general CP relevance.

51 A review of the validation process wasundertaken during 2004-05 to ensure that theprocess remains fit for purpose in the newcontext of the responsibilities devolved tofaculties. A revised methodology for validationwas agreed in January 2005. One of theoutcomes of the review was to develop the

guidance which is available for faculties andpartners on CP documentation to be submittedfor validation. This guidance is intended to raisethe quality of documentation in submissionsand to secure in advance of validation moreinformation about how the QSME andadministrative aspects of CP are expected tooperate. This accords with the University'sintention, which is supported by the audit team,that the approach to quality and standardsshould become 'more enhancement-led'.

Periodic review52 Six years is the maximum approval periodpermitted for validated programmes, whetheron-site or CP, before a review exercise must beundertaken to ensure that the quality andstandards of provision remain sound. TheUniversity's annual validation schedule, whichidentifies CP separately, includes existingprogrammes subject to review or revalidation,and any new ones to be approved.

53 A new and comprehensive periodic reviewis currently being piloted with a view toreplacing a number of former approaches:progress review (programme level), internalacademic review (subject group level) andrevalidation. In order to maintain control ofquality and standards in the case of CP,revalidation will continue to be operative andwill not be replaced with the introduction ofperiodic review. The key components of thenew process are: critical review; use of existingdocumentation; external input; student and Students' Union involvement; and programme,subject and enhancement focal points. In theprevious processes, there was sometimes aninconsistency in the engagement with thestudent learning experience. For example, theinternal academic review included a studentpanel member and a meeting with studentsfrom relevant programmes, whereas,revalidation did not always address directly thestudent learning experience because it could bea paper-based exercise. The audit teamrecognised the advisability of the Universityseeking to ensure the process of periodic reviewis rigorous and consistent across all aspects ofprovision and recognised that the piloting of


page 12

the new periodic review process offered theopportunity to address the problem ofinconsistent engagement with the studentlearning experience.

Annual Quality Review54 All University CP is subject to AQR. Alongwith external examiner reports, the AQR is animportant source of ongoing quality andstandards information for the University andtherefore the completion of AQR is a significantrequirement of agreements with partners. Thisprocess operates on the same principles asthose which underpin the AQR of on-siteprovision with annual monitoring operating ata variety of levels from module up to faculty,drawing on a wide evidence base and makinguse of action planning to ensure that strengthsare consolidated and areas for improvement areaddressed. The guidelines for AQR of CP havebeen reviewed and, in certain places,strengthened in light of the revised Code ofpractice, Section 2. They are more prescriptivethan those for on-site provision, reflecting thechallenges and risks of CP, and provide for thecompletion of a prescribed course reporttemplate designed to address key quality andstandards issues, together with submission of amore discursive report on the operation of theCP and an action plan. For 'registered student'courses there is also a Link Tutor report, whichcomplements the course report and evaluatesthe effectiveness of the annual monitoringprocess. Consideration of course and Link Tutorreports informs the drafting of a faculty AQR ofCP with a faculty action plan to follow up locallevel actions.

55 The AQR and Link Tutor reports are, in turn, considered by the Registry and theMSC in order to identify generic themes, assessthe effectiveness of the course-level monitoring,and to inform the drafting of the UniversityOverview Report of the AQR of CP. TheOverview Report is also informed by Registryreview of all reports from external examinersappointed to CP and is a summary of theoutcome of the AQR process. It is discussed bythe ADC and AB and is intended to assist AB inits responsibility for approving academic

standards and the validation and review ofcourses. The MSC considers the University AQROverview Report and the faculty AQRs to agreekey points relating to CP to be included in theannual refreshing of the University's Quality andStandards Profile. The Profile offers an up-to-date evaluation of the academic health of theUniversity in terms of academic standards andits systems and processes for QSME. It containsa section devoted to CP and is considered byfaculties, ADC, AB, and the Board of Governors.It was commended as good practice in theUniversity's Institutional Audit Report of 2005.The audit team confirmed this judgement andnoted also as good practice the way in whichthe University is moving its AQR processes tosupport the enhancement of its CP. The teamalso noted a limitation of the AQR Reports inthat the statistics they provide are relativelyperfunctory. A more sophisticated range ofstatistics could contribute further toenhancement of the AQR process through thebetter use of its statistical information tomonitor and compare the performance ofparticular groups of students (see alsoparagraph 90).

56 The University seeks to ensure that partnersinvolved in delivery, assessment or studentsupport in CP, have a shared understanding ofresponsibilities in respect of QSME. It does this in a number of ways but one of the mostimportant is through formal and informalcontact with the Link Tutor, a role that the auditteam found to be valued highly by partners. Alsovalued highly by partner staff was attendance atUniversity-organised staff development events,including the annual CP Conference, whichprovide the potential to share knowledge,understanding and good practice on internalapproval, monitoring and review.

Institutional monitoring and review57 At present the University does not have in place an institutional monitoring mechanism.Currently the University has only onemultiprogramme relationship with a majoroverseas partner and this is underpinned byinstitutional QSME arrangements which reflectthe size and significance of this relationship.


page 13

Detailed audits of this partner's participation inthe QSME systems were carried out in 2002and 2004-05, the latter in the light of therevised Code of practice. The University'sexpectations regarding quality and standardsmanagement were updated and made morerobust in light of the audits. Some issues areongoing, including diversity of practicebetween the various partner schools, both indelivery of courses and the monitoring ofacademic quality; and delays within the partnerin making consistent use of SHU's externalexaminer report form. The audit team wasinformed that these issues are being worked on within the University and will be discussedfurther with partner staff.

58 The University is 'committed to piloting anew institutional monitoring mechanism'. It isintended that this will draw upon existingprocesses, principally the AQR, and in discussionwith partners identify common themes and use a risk-based approach to areas for improvement.University staff acknowledge the need for aninstitutional monitoring mechanism and the auditteam welcomes this development and believes itwill contribute to the shift toward enhancement.

External participation in internalreview processes for collaborativeprovision

59 CP validation proposals are subject to the same requirements as on-site provision,adapted as appropriate to accommodate theinvolvement of a collaborating organisation,and, if appropriate, representatives of itsindustrial or commercial partners. However, inthe case of new programmes, revalidations andmajor modifications to existing programmes,the format of the validation process may varydepending on the degree of scrutiny deemedappropriate for a given proposal, and this mayaffect the level of external scrutiny required fora particular proposal.

60 The University declares that, as part of thevalidation of new programmes, 'External peerreview is…vital in ensuring University provisionis of at least a comparable standard to that

offered elsewhere'. The CSP will normallyinclude at least one external panel member(either an academic and/or practitioner) withrelevant subject expertise. More than one canbe co-opted onto a panel if it is necessary toreflect both practitioner and academic aspects.The audit team found that the University wasupholding this principle, although not all panelsinclude an external member with UK HEexperience.

61 A preliminary phase in the validationprocess is designed, inter alia, to agree the natureand extent of the involvement of externalmembers and receive faculty nominations.Nominations are approved by the Chair of theCSP, through the Registry. The Chair of theValidation Panel considers whether external inputshould come from an academic or practisingprofessional or both, where appropriate. This maybe determined by whether the provision is to beaccredited by a professional, statutory andregulatory body (PSRB). Employer representativesmay also be nominated to join the CSP. Inputfrom all external members during validationprocesses is sought either by the submission ofinitial comments followed by attendance at thevalidation meeting, or by correspondence alone.The external member is also asked specifically tocomment on the curriculum's content andconsistency with relevant national subjectbenchmark statements. The revalidation processfor CP has, to date, also included an externalpanel member, and this will continue.

62 During 2004-05, 43 external panelmembers contributed to the 53 validationoutcomes (some External Panel Membersserved more than once), 17 of which involvedCP. In addition, five approvals (all validations)were made by Chair's action, with threerelating to CP. Chair's Action is a process thatdoes not normally make use of externality, andthe audit team consider it desirable that thecircumstances in which Chair's Action isappropriate be made fully explicit, so as not to compromise the function of externality ininternal review processes.


page 14

External examiners and their reportsin collaborative provision

63 The CPSED set out procedures onappointment, induction, briefing andcommunication regarding external examiners,and the practice regarding response to, andevaluation of, external examiner reports. Itclaimed that its external examining proceduresfor CP are the same as those for on-siteprovision and that they adhere to the Code of practice, Sections 4 and 2 on externalexamining and CP respectively. The audit teamwas able to examine a wide range of materialrelating to the external examining and inparticular the University's External Examiners'Handbook. In addition, the team examinedevidence about the working of the externalexamining system and the processing of theirreports at four partners. The team was able to confirm that the appointment of externalexaminers follows clear and appropriate criteria for proposal and approval, with formalconsideration given at faculty level. TheAcademic Registry gives close scrutiny to and signs off the appointment on behalf ofAcademic Board. Registry maintains aninformation base on all examiners, includingthose involved with CP, and holds a pivotalcommunication role between externalexaminers and the University.

64 In the case of CP the University seeks to appoint external examiners with someexperience in the role, although the UniversityAQR of CP 2006 reported to Academic Boardthat this was becoming increasingly difficult.The link tutor liaises between the partnernominating an examiner and the relevantfaculty QSME Committee and has theobligation to 'assist and advise' on CPappointments. In cases where a local overseasCP examiner is required arrangements forappointment are monitored by the link tutor.An experienced nominee in the subject area isidentified by the overseas partner but isappointed by SHU and taken through thenormal external examiner inductionprocedures. Additional briefing is provided byan experienced SHU external examiner along

with mentoring through the examinationprocess. In those instances where it is notpossible to appoint an external examiner withexperience in the role to a UK-based oroverseas CP course, a well qualified candidatewith full expertise in the discipline is appointed and mentored by an experiencedUK-based external examiner; about 7 per cent,of external examiners fall into this category. Inthe examples investigated by the audit teamthe mentoring and induction arrangements for local overseas and inexperienced externalexaminers were found to be in place. At themoment all courses taught in a language otherthan English are examined in English and theaudit team confirmed this was the case withone overseas partner. Arrangements are in placefor instances where a course is taught orassessed in a language other than English, andin such cases the relevant external examiner will require fluency in both languages. Theteam was able to confirm that appointmentwas robust and conformed to the precepts ofthe Code of practice and the team was of theopinion that effective measures were beingtaken by the University to bridge the differentacademic cultures relating to external scrutinyfor CP.

65 The University states that all externalexaminers receive a briefing pack from Registryand have access to the appropriate pages of thewebsite with information on University policy,procedures and regulations. Training ofexaminers, which was introduced in 2004-05, is now mandatory and now provides a morestructured briefing for all new examiners. Theaudit team was assured during meetings withstaff that, before engaging in external examining,examiners are offered training session places. If no regular slot is suitable for a particularindividual, a one-to-one session is arranged. The team concluded that training of externalexaminers was now mandatory and robust.

66 Reports from external examiners go bothto the relevant faculty for response to module-specific issues, and to Registry where thereports are scrutinised in order to identifyoverarching issues. External examiners receive


page 15

written responses from both faculty anduniversity levels. The institutional audit teamhad found that duplication and delay inresponse was almost unavoidable in thissystem, and considered a review of the internalprocesses to be desirable. The CP audit teamhad the opportunity to examine a range ofreports and concluded that the University nowdoes have in place, through the AQR process, a reliable system of receiving, noting and actingon external examiner reports. A template isprovided for all reports, and although somereports are very short and some are veryrepetitive, the system of using a single templatedoes ensure that information is collected andacted upon.

67 The audit team noted that the AQR is the vehicle for registering and transmitting theviews of the external examiners to faculty andultimately to Academic Board. Link tutorsmonitor the implementation of any changesnecessary to CP through attendance at coursecommittees and in other ways. Link tutorsreports include references to the implementationof changes. Full discussion of the views ofexternal examiners is conducted annually atfaculty level. An external examiners annualoverview report, based on Registry reading ofthe external examiner reports, is producedtogether with an action plan and this isconsidered by the Monitoring Sub-Committee.The team found that the process of reviewingexternal examiner reports and deciding onappropriate action on issues raised appears to be carefully implemented and monitored.

68 Overall, the audit team considered thatthe external examiners' reports made aconsistent and positive contribution to themaintenance of standards and that the reportswere carefully considered at all levels within theUniversity. Appropriate action was taken inresponse to the reports. The team did havesome concern where in the same module oneexternal examiner may deal with the CPstudents and another external examiner withhome-based students. The team accepts theUniversity's justification for this on grounds ofpracticality but considered that the University

needs to do more to ensure that there is anopportunity for comparison and calibration ofmarks for both sets of students.

The use made of external referencepoints in collaborative provision

69 The CPSED claimed that external referencepoints had been embedded in the University'sQSME Framework. It stated that the Universitysystematically identified the elements of theAcademic Infrastructure, and the associatedmonitoring and review mechanisms that are in place at University level to ensure their fullintegration. These mechanisms are also used forquality enhancement purposes. The audit teamwas provided with access to a wide range ofdocumentation relating to alignment of itsprovision with external reference points. TheAQR of CP for 2004-05 identified areas wherework was continuing to align provision with the Code of practice relating to flexible anddistributed learning and the AcademicInfrastructure.

70 Validation processes of CP take account of the Academic Infrastructure. For example,generic learning outcomes and University credit requirements now explicitly refer toqualification descriptors in The framework forhigher education qualifications in England, Wales and Northern Ireland (FHEQ), subjectbenchmarks and the Code of practice. Thisapproach assists partners in focusing andaligning module outcomes and assessmentcriteria. Programme specifications are draftedby the course planning team, advised by theNegotiator where appropriate. At validation,the external member of the validation panelwill scrutinise the appropriateness ofprogramme specifications. It was claimed thatfollowing validation all CP programmespecifications are available on the HigherEducation and Research Opportunities (HERO)or the University web pages, although the teamfound that availability was affected by problemsassociated with the HERO website (see alsoparagraph 117).

71 It was made clear to the audit team, atpartner visits, that staff in partner institutions


page 16

depended heavily on University staff, inparticular faculty negotiators and link tutors, to handle the issues of alignment with externalreference points such as the FHEQ, subjectbenchmarks, programme specifications and theCode of practice. Overseas partners were awareof the use of standards in the UK but were notoften clear what they were.

72 The audit team found that the University'sapproach to the Academic Infrastructureincluding the Code of practice is, largely,thorough and appropriate but that alignmentwith regard to CP is continuing and noted twoinstances where the University would wish toprogress action. Firstly, in the case of registeredstudents at two overseas partner institutionstranscripts are produced by the partners andthe University does not appear to haveappointed a person to check the security andaccuracy of these transcripts. This view wasconfirmed in meetings with staff and at partnervisits. The team considered that the Universityshould set in place mechanisms to check thesecurity and accuracy of transcripts producedby its CP partners. Secondly, the teamreiterated the view, first stated at institutionalaudit, that the University may find it desirableto review its practices with regard to the Codeof practice, Section 6: Assessment of students andconsider the firm implementation of Universityguidelines for the timely feedback on assessedwork to students (see also paragraph 105).

Review and accreditation by externalagencies of programmes leading tothe awarding institution's awardsoffered through collaborativeprovision

73 The CPSED noted that the University'sapproach to receiving and consideringprogramme level reviews by PSRBs was an areaof good practice identified in the institutionalaudit report. Where the involvement of a PSRBis proposed in a CP programme this is noted atvalidation and in the programme specification.With these programmes informal meetings and industrial liaison panels enable regularinterchange of views between University staff

and professional colleagues. Through meetingswith staff and reading of minutes ofcommittees, the audit team was provided withconfirmatory evidence that external agenciesare closely involved at all stages of many CPprogramme validations and development.

74 At the time of the audit, the Universitywas responsible for 27 CP awards which wereaccredited by 23 PSRBs. As far as is possible,monitoring and quality assurance are in linewith the institution's normal practice. FacultyQSME committees identify issues for action,and form action plans in response to PSRBreports and the MSC is then responsible for theoversight of both report and action plan. PSRBreports are included as a regular item on theagenda of the MSC and it produces an annualreport on PSRB provision and this forms part of the University's Quality and Standards Profile.Generic areas for improvement are takenforward by the MSC and examples of goodpractice are noted.

75 Through the study of documentationprovided for partners running programmeswith PSRB recognition the audit team was ableto confirm that University procedures andapproaches to PSRB related provision werecarefully and conscientiously implemented.

Student representation incollaborative provision

76 The CPSED stated that through theirpartnership agreement with the Universitycollaborative partners are required to operatestudent representation and feedbackmechanisms, either by course committees or byother means. The audit team was informed thatthe form of representation to be operated wasdetermined through validation and establishedby the agreement document. It was confirmedby the team's reading of documentationrelating to the partner visits that agreementsnormally specified the model of studentrepresentation to be adopted. The team found,however, that while partners were generallyoperating some form of staff-studentconsultation, not all CP programmes hadestablished course committees and that there


page 17

was variety, and in some cases informality, inthe mechanisms adopted. Student representativeswere selected by a variety of methods and insome cases students were not aware of therepresentative system. The team was thus notable fully to confirm the CPSED statement that'where provision is delivered to students whocome together regularly in cohorts the standardUniversity model of Course Committees withstudent representation is followed', although it did recognise that where the CP was partlydelivered at the University this was normally the case. The team also recognised that insome overseas contexts the operation of formal student representation was alien to localpractice. On the other hand, the team foundevidence overseas and in relation to distancelearning courses of good use being made ofelectronic media to assist student representation.

77 Where formal course committees did notexist the audit team found it more difficult todetermine how the link tutor was in a positionto confirm that student representative processeswere feeding effectively into AQRs and actionplans. According to the CPSED 'Universityguidance for preparing AQRs requires CourseCommittees to draw upon a wide range ofevidence' which includes 'the views of theCourse Committee, including studentrepresentatives'. The tick-box dimension of theAQR pro forma, which limits presentation ofevidence, can further impede the transparencyof this process, although the University isconscious of the need to keep under review the balance between the tick-box anddiscursive elements of the AQR format.

78 During the Briefing Visit, the audit teammet with a Manager and an Officer of the SUand with students, including studentrepresentatives from partner colleges andreceived a written submission from the SU. The meeting established that enrolled studentsin CP, but not registered students, have accessto the SU of the University, including itsrepresentative functions on behalf of students.It was evident that the SU was willing to extendits services to CP students within the constraintsimposed by resourcing and the way that the

University had formally determined its relationswith the two categories of CP students. Theaudit team would encourage the SU andUniversity managers to continue to considerways of increasing support to all CP students.

Feedback from students, graduatesand employers

79 The CPSED stated that feedback atmodule level is obtained for all CP and that this is conducted by a questionnaire or theequivalent and informs the AQR process. Fromits discussions with staff and with students theaudit team ascertained that individual studentmodule feedback by questionnaire was notnecessarily the norm and that module feedbackwas often gathered collectively and by staff.Nonetheless the team considered that feedbackat module level was satisfactory.

80 The University's Student Experience Survey(SES) has not been extended to students in CPwith the exception of one group of studentswho attend the University for a summersemester. The National Student Survey (NSS)included the enrolled students within CP but it is not possible to disaggregate these for thepurposes of analysis. Given the constraintsimposed by the structure of the NSS, the auditteam would encourage the University to extendits SES to a larger cohort of CP students.

81 Students and graduates are not regularlyand systematically engaged in the review andrevalidation of programmes, indeed thisappears only rarely to happen. There was no evidence available to the audit team of anygraduate feedback on CP. The team wouldencourage the University to reflect on how itmight achieve more systematic engagementwith CP students in review and revalidation.

82 As indicated in the CPSED, a considerableproportion of the University's CP, such as FD andCPD programmes, involves employers in thedesign, development and delivery ofprogrammes. Employers may also be directlyinvolved in the assessment of some elements of programmes, notably work-based learning.Professionals and employers frequently act as


page 18

external assessors for the validation andrevalidation of such programmes. Discussion withstaff by the audit team in the audit and partnervisits provided ample evidence of effectiveinteraction with employers and practisingprofessionals, including PSRB professionals.

Student admission, progression,completion and assessmentinformation for collaborative provision

83 All CP course entry requirements mustcomply with the University's Admissions Policyand minimum entry requirements. Course entryrequirements for CP are discussed with thepartner, defined in the course agreement andapproved at validation. Any exceptions made aremonitored by the link tutor, who is involved inall applications for accreditation of prior learning(certificated or experiential) (APL or APEL) andparticularly with respect to registered students.

84 The audit team found that APL and APELissues arose in a fair percentage of cases in CPand that the MSC had noted 'a high number of approval conditions attached to collaborativeapprovals related to admissions criteria'. TheMSC recommended that this issue be raised withlink tutors and University staff with responsibilityfor the negotiation of CP, and for guidingpartner organisations through the planning andvalidation process. The team noted that facultieshad taken responsibility in this area with, forinstance, the Faculty of Arts Computing,Engineering and Science reporting twosuccessful applications for APEL and a ratificationprocedure of approval through the appropriateassessment board, chaired by a member ofUniversity staff with the link tutor in attendance.

85 It was clear to the audit team that CP waslikely to involve more cases of applicants withnon-standard qualifications than for on-siteUniversity courses, especially in the case of FDsand 'top-up' awards for mature students. Theteam saw the need for continuing careful andoverarching scrutiny by the University to ensureparity across its CP provision and maintenance ofadmission standards.

86 Data for enrolled students has beenmaintained using the University's studentmanagement system, which tracks each studentfrom application to qualification and producesdata for assessment boards. Work is underwayto upgrade the reporting tool to ensure that itcan provide continued effective support toassessment boards, particularly with theintroduction of the revised two-tier boardstructure. The CPSED reported that onlyabridged student records are maintained by the Registry and the partner maintains fullerrecords for registered students. Work isunderway to obtain fuller details of thesestudents in order to track their progression and achievement and to explore how thefunctionality of the University's studentmanagement system might be extended to hold fuller data on all categories of CPprovision. Data on registered students isprovided also to faculties at assessment andAQR events. Generally this has operatedsmoothly although there have been some local instances of delays in provision of data to boards in one faculty. The AQR of CP for2004-05 reported to Academic Board thatsystems for recording programme and studentdata, and transferring data between partnerand University are generally secure andunderpin the quality and standards of CP.

87 The audit team noted that partners as well as faculties and the Registry consideredstatistical data relating to student achievementand progression through the AQR process. The overview of this data is presented annuallyto Academic Board by ADC, where trends onstudent performance and attainment and issues raised by external examiners are noted.The team found limited analysis of the dataalthough some use of data had been made, forexample, for one CP course resulting in limitingthe number of times it had run owing to smallenrolments of students. In another instance,low standards of achievement in a postgraduatecertificate programme led to an increased levelof monitoring of admission standards. However,overall there appeared to be very littlecomparison of admission, progression,retention or completion data across thedifferent CP courses.


page 19

88 The audit team was informed that allboards now operate within a newly implementedtwo-tier structure, at subject level and at awardlevel unless formally exempted. There isnormally one Postgraduate Awards AssessmentBoard per faculty and one UndergraduateAward Assessment Board (AAB) per divisionwithin a faculty for each assessment period;these boards have full delegated authority fromAcademic Board and have both ratification andquality enhancement functions. SubjectAssessment Boards (SAB) report to AABs and themembership reflects the CP arrangement byincluding the Faculty Collaborative Coordinatoror nominee (not the link tutor) as the Chair,one internal examiner for each module, subjectleaders, the subject external examiners, asecretary represents the Academic Registrar, ex-officio members and the Faculty Head ofQuality and Enhancement. The role of the SABis to verify students' marks and moderate markssets; and receive Academic Conduct andExtenuation Circumstances decisions. It has the responsibility for confirming overall moduleresults and these cannot be changed by anAAB. It also undertakes a quality assurance andenhancement role at module level, with reportsand action plans for each module beingconsidered. The AABs in ratification mode aredesignated as Ratifying Assessment Boards(RAB) and these boards meet after SABs. TheChair of the RAB then forwards the data to theannual quality meeting of the AAB.

89 The AABs are the decision-making bodiesagainst whose decision student appeals can bemade. They have an explicit role in qualityenhancement of the overall course provisionand ratify progression and awards. For CP theChair could be the Chair of the SAB providedthe person meets the criteria to conduct theAAB function. Membership includes awardexternal examiners and University level staff. An annual AAB meeting is conducted wherecourse statistics (for example, good honoursand progression details), and cross-Universityissues are considered but given the recentintroduction of the two-tier system the auditteam was not able to assess the effectiveness

of AAB consideration of cross-University issues.It was clear to the team that at the present timethe new process is not fully understood by allUniversity staff, partner staff or externalexaminers.

90 On the whole, the audit team consideredthat the University was making good progresswith its management of student data onadmissions, progression, completion and levelof achievement. It considered, however, thatthe University would find it extremely useful togather fuller data on the admission, retentionand progression of all its CP student body (for example by gender, race, age, class) and toput in place a mechanism for comparing theseelements. This would provide the Universitywith an invaluable tool in managing thedevelopment of CP at SHU. The team wouldalso recommend the advisability of theUniversity adopting a more thoroughinterrogation of the available statistical data onstudents. Such an approach as outlined abovewould, for example, provide a better analysis ofhow CP programmes are contributing to theUniversity's widening participation objectives.

Assurance of the quality of teachingstaff in collaborative provision;appointment, appraisal, support and development

91 The capacity of teaching staff in partnerorganisations to deliver programmes leading to awards of the University to the appropriatestandards and quality undergoes initial scrutinythrough the validation process and isconducted by the CSP on the basis ofcirriculum vitaes. Responsibility for thenotification of subsequent changes of staffinglies with the partner organisation and ismonitored through the dialogue between thelink tutor and the partner and in particularthrough the link tutor's report on the AQR pro forma. Through discussion with Universityand partner staff, the audit team perceived thisto be approval through notification with anymore formal approval only being by exception.


page 20

92 While staff in partner organisations are approved to deliver programmes of theUniversity they remain subject to the staffmanagement arrangements of the partner.Thus, staff in partner organisations are subjectto the appointment and appraisal systems of the partner organisation. For example, in thecase of local further education college (FEC)partners, college staff participate in the localappraisal and peer-observation systems. TheUniversity seeks to maintain a balance in itsrelationship with partner staff, betweenassurance of the standards of programmes and quality of delivery and support, anddevelopment for partner staff. The latter isdesigned to promote capacity building for thepartner organisation. The audit team consideredthat a reasonable balance was achieved.

93 The CPSED recognised the diversity ofpartners within the University's CP portfolio and hence their diversity of needs. There is no policy in CP for providing a standardentitlement of staff development, butdiscussions with both partner and Universitystaff provided ample evidence both of makingsome University staff development available topartner staff, and of providing customised staffdevelopment to partners, for example inassessment. While the prime mover for theidentification and communication of suchdevelopment was normally the link tutor orcourse leader, it was evident that a wide rangeof agencies or services within the Universitywere involved including faculty and subjectacademic staff, Registry staff and LITS staff. A recent and positive innovation has been theestablishment of a Collaborative Conference tobring together key players from partners andthe University with a view to sharing issues andgood practice. The Centre of Excellence forEmbedding, Developing and IntegratingEmployability also indicated its intention toengage partner institutions centrally in thedevelopment of its initiatives. Overall, thesupport and development provided to partnerinstitutions and partner staff by the Universitythrough both formal and informal mechanismswas found by the audit team to be a feature ofgood practice.

Assurance of quality of distributedand distance methods deliveredthrough an arrangement with apartner

94 In its CPSED the University explained thatit has a significant (circa 300) body of distance-learning (DL) students which falls under thedefinition of CP since the learning is facilitatedor supported by local partner organisationsacting as agents for the University. The mostnotable example of this is a network of nineFECs in the UK acting as agents for the deliveryof the e-top-up (H-level) of the BSc AppliedComputing.

95 In the case of DL proposals, validationdocumentation is required to includespecification of the student support arrangementswhich subsequently must be expressed ininformation available to students. An addition tothe standard validation procedures also considersthe DL materials.

96 The use of an agent to support delivery of a DL programme is based upon a formalagreement setting out the respectiveobligations of the University and the agent.Such an agreement covers the arrangementsfor: communication and information exchangebetween the two parties; production andapproval of publicity materials; management of assessment; provision of tutors; student feecollection; staff development; and maintenanceof records. Agreement with an agent is positedupon an agent approval process whichnormally involves a visit and report conductedby a LITS professional. There are alsoprocedures for the monitoring of agents whichinvolve at least one annual visit by Universitystaff, although in many programmes there aremore frequent visits by staff, to attend studyschools, when the efficacy and efficiency of thelocal provision can be checked. Students areprovided with a named University contact,contactable by email in the event of a problemproving insoluble locally.

97 The CPSED used the terminology ofagents to refer to local support providedoverseas for DL. Discussion with senior Registry


page 21

staff indicated that agents are not considered'partners' in the same way as othercollaborative partners in that their academicengagement in the delivery of the Universityprogramme is minimal. In the development ofits collaborative strategy through the fosteringof rich or complex partnerships, the Universitydoes not perceive DL and agents as fulfilling amajor role, although a partner might be theagent for delivery of a specific DL programmeas one of its multiple roles and functions. Thevalidation report for the multisite UK-based FDin Applied Computing refers to the collegesinvolved as study centres and states thatapproval should be sought through 'standardSHU partner approval procedures'. Thesecolleges do provide support tutoring andmentoring.

98 Overall, the audit team found that theUniversity was meeting the precepts of theCode of practice, Section 2: Collaborativeprovision and felxible and distributed learning(including e-learning), even though there mightbe some further clarification of terminology andprocedure. As it develops its practice in thisarea, the University may wish to considerproviding further clarification as to itsdefinitions of 'partner' and 'agent' to avoid anypossible confusion or ambiguity about roles.The team also encountered one exemplarymodel of the operation of e-learning in anoverseas college.

Learning support resources forstudents in collaborative provision

99 CP proposals are required to presentevidence that there will be adequate andappropriate learning resources in place to meetthe needs of students. Where the responsibilityfor the provision lies with the partnerinstitution, evidence is required of the partner'scapacity to provide and maintain suchresources. Such evidence is normally confirmedby a visit from University staff, often a LITSprofessional, although this may not be the case if the risk analysis indicates low risk in thisdomain, for example, programme revalidationor validation with an established partner in a

subject where other programmes are alreadyrunning. Where visits occur or where LITS staffform part of a panel undertaking institutionalapproval of a new partner, use is made ofstandard information service benchmarks.

100 The AQR reporting mechanism is thefundamental means of assuring themaintenance and, where necessary,development of learning resources. Partners are required to report on learning resourcesthrough the AQR pro-forma with confirmationprovided by the link tutor's report. Reporting ofCP AQR reports through faculties to AcademicBoard enables the University to identify anycommon or recurring issues in respect oflearning resource provision for CP students.

101 Developments, both within the Universityand in an increasing number of CP programmes,of a VLE delivery system has raised one of theissues reported through the AQR process.Currently, enrolled CP students have full accessto the VLE and thereby to all the electronicresources for learning within the University'sLearning Centre, subject to licence constraints.Registered students, as students of the partnerinstitution, have not benefited from such accessand this distinction has also previously informedthe operation of the student informationmanagement system. Contractual constraintsrelated to the site licences for electronic materialshave also hitherto impeded access for registeredstudents posing problems reflected in the AQRreports of 2003-04.

102 Members of the audit team found veryeffective use of the VLE in some CP and thepotential for its development in others. Theteam also encountered the differentiation ofentitlement between enrolled and registeredstudents within a single partner organisation.The team consequently welcomes the workbeing undertaken by the University, throughLITS and the Registry, to extend to registeredstudents targeted access in agreed electroniclearning resources and the commitment ofsignificant University funding to bring this toeffect. The team recognises that this is on thebasis of revised financial arrangements with thepartners and that therefore further negotiation


page 22

is required. Nonetheless the team recognises asgood practice the use of the VLE in the deliveryof programmes and as a way of effectivecommunication with students and partners.

Academic guidance and personalsupport for students in collaborativeprovision

103 Access to academic guidance and pastoralsupport for students in CP is determined by thedistinction between enrolled and registeredstudents. Enrolled students are treated thesame as internal students of the University andhave full access to the range of academicguidance and pastoral support provided by theUniversity, although qualified by the constraintsof physical accessibility.

104 For all CP proposals it is a Universityexpectation, established through theinstitutional approval and programmevalidation processes, that provision will bemade for academic guidance and personalsupport that would meet University norms. Inthe case of programmes for registered students,the responsibility for such provision will lie withthe partner organisation. It was also indicatedto the audit team that the AQR process couldbe used to communicate to partner institutionschanges in the level of support required as aconsequence of legislative change, for examplein respect of the Disability Discrimination Act.

105 Students that the audit team met throughthe briefing and partner visits were generallyvery satisfied with the levels of academicguidance and personal support being providedon and around their courses. Needs appearedto vary significantly according to the natureand location of the programme and thecomposition of the student body. In virtually allcases the most immediate source of guidanceand support was the staff and resources of the partner institution at the location forprogramme delivery. However, students weregenerally aware of the potential for recourse to the University, most frequently through thelink tutor or a designated contact, or emailcontact in the case of DL programmes. The oneissue that had given rise to some dissatisfaction

was the lack of an established policy or definedparameters for the timely return of feedback on student assessment, an issue which had also arisen in the institutional audit. It wasascertained in discussion with University staffthat this issue was under consideration in thecontext of a broader review of the structure andoperation of the assessment process. The teamwas assured that, when an institution-widetimeframe for feedback on assessed work had been established, this would also beimplemented across CP. It appeared to the auditteam that the University was close to establishingsuch a time norm and it would emphasise theadvisability of early implementation an effectiveprocess for providing timely feedback to studentson assessed work.

106 The University is in the process of movingon its notion of student entitlement from thatas expressed in the Partnership in Learningstatement and related documents to that of the Student Support Framework developed aspart of the Corporate Plan implementationprogramme and being led by the newlyappointed Director of Student Affairs. It isenvisaged that the framework 'will provide a sharper context for the identification andresolution of student support issues incollaborative partnerships, encouragingattention to aspects of the student experiencethat are likely to be particular to specific CPprogrammes'. As established in discussion withsenior staff, it is not yet clear how this will berealised and implemented.


page 23

Section 3: The collaborativeprovision audit investigations:published information

The experience of students incollaborative provision of thepublished information available to them

107 The University stated in its CPSED that itdoes not dictate how partners should conveyinformation but it does provide guidance onthe content required and arrangements for theprovision of relevant information are checked at validation or through course agreements.Information published by the University and its partners comes in a plethora of forms whichreflects the diverse nature of the CP. Keyinformation may be contained in module aswell as programme handbooks. Increasinglyinformation is provided in electronic format for students by means of the student portal orat local Associate Colleges through studentintranet sites. Similarly a wide range of publicitymaterial may be used to promote CP topotential students. The audit team was able to view information across this range and itsaccuracy, reliability and helpfulness wasdiscussed in meetings with students.

108 The University process for monitoringinformation produced by its collaborativepartners begins at an early stage withdiscussions regarding the format and content of any publication taking place during theplanning stage. Subsequently this is agreedthrough validation and documented as part ofthe programme agreement. Monitoring of thearrangements is undertaken by the link tutorwho normally will also be the personauthorising changes to any promotionalmaterials bearing the University's name. Linktutors have a crucial role in ensuring theaccuracy of the information provided bycollaborative partners and they report on this as part of the AQR of CP. The University AQR of CP in 2006 considered that this workedsatisfactorily but not all programme AQRReports seen by the audit team demonstrated

that the checking of information provided forstudents had taken place. The Universitysupports the link tutors with induction sessionsand workshops which stress the importance ofchecking the provision of information madeavailable to students. Link tutors met by theaudit team indicated that they were aware oftheir responsibilities in respect of publishedmaterials used by collaborative partners and thestudents confirmed the usefulness of suchmaterials.

109 Information regarding assessmentregulations, and supporting regulations such as those governing complaints, appeals,plagiarism and extenuating circumstances aremade available to students in a variety of ways.Of increasing importance is the University'sstudent portal from which students can readilygain information.

110 Students met by the audit team spokehighly of the on-line regulatory informationavailable to them and indicated that they wereaware of how to source such information if theneed arose. The University has also made keyinformation on regulations and proceduresavailable to collaborative partner staff on thecollaborative partners' webpages.

111 The awarding of certificates for bothenrolled and registered students is undertakenby the University. Transcripts for registeredstudents are normally prepared and issued bythe CP partner. The University has no formalmechanism in place to assure itself of theaccuracy of such transcripts. The audit teamformed the conclusion that the Universityneeded to establish a procedure so as to beable to exercise the ultimate responsibility forthe security and accuracy of such transcripts.

112 The University has revised its programmespecifications to make them more readilyunderstandable to students. Where appropriate,collaborative partners have been involved in thedrafting. The University is in a transition periodwhere new programme specifications approvedthrough validation will eventually replacespecifications on old templates. It is theintention that the new style programme


page 24

specifications will provide a sounder basis forinforming students about their programmes.Students met by the audit team generally didnot indicate that they had used programmespecifications in this way but gatheredinformation regarding their programmes fromother sources such as handbooks.

113 Overall, the audit team concluded that,other than in the oversight of transcriptsproduced for registered students, the Universityhad sound procedures for ensuring theaccuracy and reliability of published material on its CP and students were satisfied with theusefulness and access they had to suchinformation.

Reliability, accuracy and completenessof published information oncollaborative provision leading to the awarding institution's awards

114 The CPSED gave an account of theUniversity's progress in relation to the TeachingQuality Information (TQI) requirements. It hadaddressed the initial requirements relating toCP contained in HEFCE's document 02/15,Information on quality and standards in highereducation, and developed in document 03/51,the Final guidance.

115 The audit team was able to confirm thatthe University meets the TQI requirements inrespect of statistical information. The Universityprovides information on all enrolled students to the Higher Education Statistics Agency whichis used by HERO for Teaching QualityInformation (TQI) purposes. All the University'ssummaries of external examiners' reports for2004-05 for standard undergraduate and non-standard provision, including those relatingto CP, were available on the TQI website.

116 At the time of the audit visit, theUniversity confirmed that it had uploaded all of the periodic review reports to date. TheUniversity has used a process of revalidation toreview the academic standards of its CP. Theaudit team was able to view summaries of tworevalidation reports relating to CP programmeson the TQI website.

117 Programme specifications were introducedinto the University in 2001 and have been usedfor all award-bearing CP since. A total of 17 CP proposals were considered for approval in2004-05. This total consisted of both newprovision and existing CP undergoingrevalidation. As part of these processes,programme specifications were approved orreviewed. The audit team was able to view suchprogramme specifications and confirm theywere fit for purpose. The University has madeavailable all of its post 2001 programmespecifications, including those for CP, on itswebsite. However at the time of the audit visitthese were not linked to the TQI site due to thenon-availability of the HERO portal fordownloading programme specifications.

118 The audit team was able to conclude, onthe basis of the available evidence, that theUniversity's currently published information onits CP was both accurate and reliable. The auditteam was satisfied that the University had inplace processes to meet the requirements ofHEFCE's document 03/51 and that theUniversity was making good progress inproviding TQI data for its CP.


page 25

Findings

Findings 119 An audit of the collaborative provision(CP) offered by Sheffield Hallam University (the University) was undertaken during theperiod 24 to 28 April 2006. The purpose of theaudit was to provide public information on thequality of the programmes of study offered by the University through arrangements withcollaborative partners, and on the discharge of the University's responsibility as an awardingbody in assuring the academic standard of its awards made through collaborativearrangements. As part of the collaborative audit process, the audit team visited four of the University's collaborative partners. Thissection of the report summarises the findings of the audit. It concludes by identifying featuresof good practice that emerged during theaudit, and making recommendations to theUniversity for action to enhance currentpractice in its collaborative arrangements.

The effectiveness of theimplementation of the awardinginstitution's approach to managingits collaborative provision

120 The University's approach andcommitment to CP, expressed in its CP policystatement, seeks to promote access toadvanced level skills and knowledge for adiversity of students by offering flexible coursedelivery with time, pace and, increasingly, placechosen by students. It is University policy tointegrate CP into on-site provision as far aspossible, and aim for 'rich partnerships'. Forinstance, this approach involves the selection of a small number of international partners of comparable mission and status and at aregional level the establishment of the AssociateCollege Partnership (ACP) with shared goals(for example, for recruitment) for the Universityand partners. The University's CP, includinginternational developments, is not, in itself, astrategy but a way of enabling achievement ofother University strategies. An example of goodpractice recognised by the audit team is the use of CP, through further development ofFoundation Degrees with the ACP, in enabling

the University to maintain its commitment to widening participation.

121 The Quality and Standards Managementand Enhancement (QSME) framework, whichembraces CP, is the main vehicle through which the University monitors and assures theestablishment and maintenance of academicstandards and the management andenhancement of quality. The AcademicFrameworks, Policies and Regulations (AFPR)inform and regulate all QSME activities and takeaccount of relevant external reference points soallowing the University to match the academicstandards of CP against national expectationsembedded in the Academic Infrastructure.

122 The Academic Board endorses therecommendations of validation panels whichconsider new collaborative course proposals,and approves the Annual Quality Review (AQR)of CP. One of the Academic Board's twosubcommittees, the Academic DevelopmentCommittee (ADC), is the key body within theUniversity's deliberative structure providingcentral oversight of the management of thequality and standards of CP. The ADC is assistedin its monitoring role by the Monitoring Sub-Committee (MSC) which monitors theeffectiveness of the management of quality andstandards of CP. In addition, all CP proposalsare validated by the University's CollaborativeStanding Panel (CSP) which consists of a coreteam of highly experienced staff well equippedto gauge the risks and requirements of differingcollaborative arrangements.

123 At faculty level CP quality and standardsare monitored by Faculty Academic Boardassisted in this role by a faculty QSMEcommittee. The University has developed arange of types of CP and partnerships and does not seek to impose particular models for collaboration and seeks a balance of tightcorporate control of academic standards andfreedom for faculties to grow and innovate.Faculties have some flexibility in their localmanagement arrangements and structuresthough there is no relaxation of key Universityrequirements. The range of CP arrangements is expressed in the University's 'collaborative

Collaborative provision audit: findings

page 27

typology'. The typology indicates the types ofrelationship: agent, articulation, enrolledcourses (shared delivery single award or jointaward), registered courses (licence orvalidated/credit-rated) and the responsibilitiesof the parties.

124 The QSME framework, which has a vitalrole in ensuring quality and standards throughreference to the AFPR, was found by the auditteam to be comprehensive in its scope androbust in its use and implementation. Theteam's meetings with staff confirmed theimportance, easy accessibility andcomprehensibility of the AFPR information.Each faculty implements CP, in the context ofthe University policy statement, and throughfaculty minutes and meetings with staff, theteam was satisfied that faculties were payingdue regard to wider University strategies whendeveloping collaborative, includinginternational, provision. The faculty mini-audits(FMAs) were particularly effective in producingaction plans for enhancement of the provision.

The effectiveness of the awardinginstitution's procedures for assuringthe quality of educational provisionin its collaborative provision

125 All new CP course proposals are approvedthrough each faculty's Business Planning andOperational Review process. Centraldepartments such as Registry and the Learningand Information Technology Services are ableto provide advice and support. For each newCP course proposal, an Outline Proposal (CPA1)form is required by Registry to ensure that theproposal is consistent with the AFPR and toagree an appropriate process and timescale forvalidation. A faculty negotiator works withpartners in the very early stages of thedevelopment of CP proposals. Where it isproposed that a new international collaboratingorganisation will be delivering or assessingsubstantial components of a University award,an institutional approval visit is required as partof the approval process.

126 CP proposals, which are scrutinised by theCSP, are subject to the same validation processrequirements as for on-site provision, adaptedas appropriate to accommodate theinvolvement of a collaborating organisation,and, if required, representatives of its industrialor commercial partners. Validation involves thecompletion of obligatory documentation, avalidation event involving external expertiseand a representative of the partner, andproduction of a validation report makingapproval recommendations to the University'sAcademic Board. The University has alsoadopted a new fast-track, 'light-touch'validation process for proposals based largelyupon existing provision and this, along withsome other circumstances, may result in a'Chair's action' form of approval. Validationactivity for each academic year is reported in a Validation Review (VR), produced by Registry,which includes a section devoted to CP. Areview of the validation process, undertakenduring 2004-05, has contributed to theapproach to quality and standards becomingmore enhancement-led in line with UniversityExecutive Group recommendation thatUniversity approach to quality and standardsshould become 'more enhancement-led andstudent-focussed'.

127 Six years is the maximum approval periodpermitted for validated programmes, whetheron-site or CP, before periodic review is required.Periodic Review is currently replacing ProgressReview (programme level), Internal AcademicReview (subject group level) and Revalidation(for CP programmes). In these processes therewas variation with internal academic reviewincluding a student panel member and ameeting with students from relevantprogrammes whereas revalidation could be apaper-based exercise. In addition, all UniversityCP is subject to AQR operating at a variety oflevels from module up to faculty. The AQRreports currently incorporate data on theprogression and achievement of students on CPprogrammes but data is relatively perfunctory.The guidelines for AQR of CP have beenreviewed in light of the revised Code of practice,Section 2 and are more prescriptive than those


page 28

for on-site provision with submission of a morediscursive report on the operation of the CPand an action plan. For 'registered student'courses (delivery of course provided by thepartner) there is also a link tutor report, whichcomplements the course report and evaluatesthe effectiveness of the annual monitoringprocess. AQR reports are considered by theRegistry and the MSC the drafting of theUniversity Overview of the AQR of CP. The MSCconsiders the University AQR Overview Reportand the faculty AQRs agree key points relatingto CP to be included in the annual refreshing ofthe University's Quality and Standards Profile.The Profile offers an up-to-date evaluation ofthe academic health of the University in termsof academic standards and its systems andprocesses for QSME. It was commended asgood practice in the University's institutionalaudit report of 2005. The audit team confirmedthis judgement and noted also as good practicethe way in which the University is moving itsAQR processes to support the enhancement ofits CP. Student feedback at module level maybe obtained through individual questionnairesor the equivalent and is often gatheredcollectively by staff. Students and graduates arenot regularly and systematically engaged in thereview and revalidation of programmes.

128 The audit team found the planningprocess for new CP proposals to be fit forpurpose. With regard to validations the teamrecognised as sound practice the use of VRs in addressing matters of relevance to CP andnoted the general soundness of validationprocedures. Some validations do not include a visit and the team considered that theUniversity could provide clearer guidance onthe need for visits to ensure that there can beconfidence that where a visit does not takeplace all aspects of the validation will still beconsidered. Chair's Action is an option in thenew fast-track validation process; the auditteam maintains the importance of a validationprocess that, however light in its touch, isrobust and systematic. Part of the robustness is the use of external advisers, and the teamwould advise the University to consider the

desirability of defining criteria for the use ofChair's action. Given previous variation ininternal review the team welcomed the newperiodic review process, and advised theUniversity to seek ways of ensuring the newprocess is rigorous and consistent across allaspects of provision. The team also noted alimitation of the AQR reports in that thestatistics they provide are relatively perfunctory.A more sophisticated range of statistics couldcontribute further to enhancement of the AQRprocess through the better use of its statisticalinformation to monitor and compare theperformance of particular groups of students.The audit team also welcomed the UniversityExecutive Group recommendation that theUniversity approach to quality and standardsshould become 'more enhancement-led andstudent-focussed' and noted that FMAs wereparticularly concerned with producing actionplans for enhancement of the provision.

129 Overall, the audit team notes that broadconfidence can reasonably be placed in thepresent and likely future capacity of theUniversity to satisfy itself that the learningopportunities offered to students through itscollaborative arrangements are managedeffectively and meet its requirements.

The effectiveness of the awardinginstitution's procedures forsafeguarding the standards of itsawards gained through collaborativeprovision

130 Procedures on appointment, induction,briefing and communication regarding externalexaminers for CP, and University practiceregarding the response to and evaluation ofexternal examiner reports, are the same asthose for on-site provision. In the case of CPthe University seeks to appoint externalexaminers with some experience in the role.Where it is impossible to appoint an externalexaminer with experience in the role, a wellqualified candidate with full expertise in thediscipline is appointed and mentored by anexperienced UK-based external examiner.


page 29

The link tutor liaises between the partnernominating an examiner and the relevantfaculty QSME Committee and has theobligation to 'assist and advise' on CPappointments. Reports from external examinersgo both to the relevant faculty for response tomodule specific issues, and to Registry wherethe reports are scrutinised in order to identifyoverarching issues. External examiners receivewritten responses from both local andUniversity levels. An external examiner annualoverview report and action plan is consideredby the MSC.

131 Overall, the audit team considered thatthe external examiners' reports made aconsistent and positive contribution to themaintenance of standards and that the reportswere carefully considered at all levels within theUniversity. Appropriate action was taken inresponse to the reports. The team did identifythat, on occasion, one external examiner maydeal with the CP students and another withhome-based students for the same module. The team formed the view that the Universityneeded to do more to ensure that there wasopportunity for comparison and calibration ofmarks for both sets of students.

132 All CP course entry requirements mustcomply with the University's Admissions Policyand minimum entry requirements. Course entryrequirements for CP are discussed with thepartner, defined in the course agreement andapproved at validation. Any exceptions madeare monitored by the link tutor, who is involvedin all applications for accreditation of priorlearning. Data for enrolled students (with fullentitlement to support from the Unversity) havebeen maintained using the University's studentmanagement system, which tracks each studentfrom application to qualification and producesdata for assessment boards. Abridged studentrecords are maintained by the Registry forregistered students (with entitlement to supportprovided by the partner) with the partnermaintaining fuller records. Data on registeredstudents is provided to faculties at assessmentand AQR events. The AQR of CP for 2004-05reported to Academic Board that systems for

recording programme and student data, and transferring data between partner andUniversity are generally secure and underpinthe quality and standards of CP.

133 Generally the audit team considered thatthe University was making good progress with its management of student data on admissions,progression, completion and level ofachievement. It considered, however, that theUniversity would find it very useful to gatherfuller data on the admission, retention andprogression of all its CP student body (by forexample, gender, race, age, class) and to put in place a mechanism for comparing theseelements. This would provide an invaluable toolin managing the development of CP at SHU. The team would also recommend the advisabilityof the University adopting a more thoroughinterrogation of the available statistical data onstudents.

134 Overall, the audit team notes that broadconfidence can reasonably be placed in thesoundness of the University's present and likelyfuture management of the academic standardsof its awards made through collaborativearrangements.

The awarding institution's use of the Academic Infrastructure in thecontext of its collaborative provision

135 The University stated that externalreference points have been embedded in theUniversity's QSME framework. It systematicallyidentified the elements of the AcademicInfrastructure, and the associated monitoringand review mechanisms that are in place atUniversity level to ensure their full integration.The AQR of CP for 2004-05 identified areaswhere work was continuing to align provisionwith the Code of practice for the assurance ofacademic quality and standards in highereducation (Code of practice) relating to flexibleand distributed learning and the AcademicInfrastructure. It was made clear to the auditteam, at partner visits, that staff in partnerinstitutions depended heavily on Universitystaff, in particular faculty negotiators and linkTutors, to handle the issues of alignment with


page 30

external reference points such as The frameworkfor higher education qualifications in England,Wales and Northern Ireland, subjectbenchmarks, programme specifications and the Code of practice.

136 The audit team found that University'sapproach to the Academic Infrastructure includingthe Code of practice is, largely, thorough andappropriate but that alignment with regard to CPis continuing and noted two instances where theUniversity would wish to progress action. Firstly, in the case of registered students at two overseaspartner institutions transcripts are produced bythe partners and the University does not appearto have appointed a person to check the securityand accuracy of these transcripts. The teamconsidered that the University should set in placemechanisms to check the security and accuracy oftranscripts produced by its CP partners. Secondly,the team reiterated the view stated at institutionalaudit that the University may find it desirable toreview its practices with regard to Code of practice,Section 6: Assessment of students and consider thefirm implementation of University guidelines forthe timely feedback on assessed work to students.In addition, the team would stress the desirabilityof the University continuing to monitor theefficacy of changes to the assessment processarising from the recommendations of theAssessment Working Group.

The utility of the collaborativeprovision self-evaluation document as an illustration of the awardinginstitution's capacity to reflect uponits own strengths and limitations incollaborative provision, and to act on these to enhance quality andsafeguard academic standards

137 The audit team found the CP self-evaluation document (CPSED) to be wellstructured although rather descriptive inproviding detail on CP procedures. On occasionit did not reflect fully the allowable variability inprocedures. It was particularly informative withregard to developments since the institutionalaudit conducted in 2004 and contained anumber of useful and carefully constructed

appendices. In some areas, for example, onexternality in review procedures, professionalinput into courses, and statistical data, theCPSED was more limited. As a generalobservation the team considered that theCPSED did not fully reflect the careful analysisand evaluation of quality and standards mattersthat became apparent during discussions withstaff and students from the University and itspartners. In particular during these meetings,and in reading other documentation, the teamnoted the strong commitment within theUniversity and its partners to self-evaluation and quality enhancement.

Commentary on the institution'sintentions for the enhancement of itsmanagement of quality and academicstandards in its collaborativeprovision

138 The University Executive Group hasrecommended that the approach to quality andstandards should become more enhancement-led and is developing a coherent framework for the proposal, planning, delivery andmanagement of University partnerships. This isto be discussed by the University's StandingPanels Chairs Forum and will be reflected in the agendas of future CP conferences and inchanges to the assessment boards structure toinclude considerations of quality assurance andenhancement. The Pro Vice-Chancellor AcademicDevelopment and senior colleagues in thefaculties are meeting on a regular basis toidentify cross-faculty opportunities and ways toenrich partnerships which promote the growthof trans-national education. Faculties are alsodeveloping CPD frameworks within which CPplays a significant part in meeting theirambitions.

139 The University continues to work on itsstudent support framework with the intentionof clarifying a threshold standard of studentsupport available to students including those atpartner organisations. The University has madea virtual learning environment (VLE) an integralpart of the enhancing of the student learningexperience and audit team found very effective


page 31

use of the VLE in some CP and the potential forits development in others. Full access to the VLEhas only been available to enrolled and notregistered CP students. CP partners andregistered students feel that access to electronicdatabases would benefit the student learningexperience. The University's Executive Grouphas supported a proposal to make such accessavailable to all CP students subject to anagreement on the recovery of costs involvedfrom CP partners.

140 The CPSED recognised the diversity ofpartners within the University's CP portfolio and hence their diversity of needs. There is no policy in CP for providing a standardentitlement of staff development, butdiscussions with both partner and Universitystaff provided ample evidence both of makingsome University staff development available topartner staff and of providing customised staffdevelopment to partners, for example inassessment. A recent and positive innovationhas been the establishment of a CollaborativeConference to bring together key players frompartners and the University with a view tosharing issues and good practice. The Centre of Excellence for Embedding, Developing andIntegrating Employability also indicated itsintention to engage partner institutionscentrally in the development of its initiatives.

141 The audit team considered that theUniversity had well-planned mechanisms foreffecting operational change and concludedthat the University's intentions for enhancingthe management of its CP are timely andappropriate within the context of its evolvingstrategy. The team recognised good practice in the use of the VLE for the delivery ofprogrammes and as a way of effectivecommunication with students and partners. In addition, the support provided to partnerinstitutions and partner staff by the Universitythrough both formal and informal staffdevelopment mechanisms was found by theteam to be another example of good practice.

Reliability of information provided by the awarding institution on itscollaborative provision

142 The University does not dictate howpartners should convey information but it does provide guidance on the content of the information which is checked throughvalidation or course agreements. Keyinformation may be contained in module aswell as programme handbooks. Increasinglyinformation is provided in electronic format for students through the student portal or atlocal Associate Colleges by student intranetsites. Students met by the audit team spokehighly of the on-line regulatory informationavailable to them and indicated that they wereaware of how to source such information if theneed arose. Similarly a wide range of publicitymaterials may be used to promote CP topotential students. Link tutors have a crucialrole in ensuring the accuracy of the informationprovided by collaborative partners and theyreport on this as part of the AQR of CP.

143 The CPSED gave an account of theUniversity's progress in relation to the TeachingQuality Information (TQI) requirements. It hadaddressed the initial requirements relating to CP contained in the Higher Education FundingCouncil for England (HEFCE's) document 02/15,Information on quality and standards in highereducation, and developed in 03/51, the Finalguidance. At the time of the audit visit, theUniversity confirmed that it had uploaded allexternal examiner summaries for 2004-05 andperiodic review reports to the Higher Educationand Research Opportunities (HERO) website. TheUniversity has made available all of its currentprogramme specifications, including those for CP,on its website, however, at the time of the auditvisit these were not linked to the TQI site due tothe non-availability of the HERO portal fordownloading programme specifications.Currently, the University is making good progresswith the introduction of a new template forprogramme specifications.

144 The audit team was able to conclude, onthe basis of the available evidence, that theUniversity's currently published information on its


page 32

CP was both accurate and reliable. Students weresatisfied with the usefulness and access they hadto such information. The audit team was satisfiedthat the University had in place processes to meetthe requirements of HEFCE's document 03/51and that the University was making goodprogress in providing TQI data for its CP.

Features of good practice

145 The following features of good practicewere identified during the audit:

i the contribution that collaborativeprovision makes to the University'sstrategy particularly in respect of wideningparticipation and continuing professionaldevelopment (paragraph 8, 120)

ii the way in which the University is moving its AQR processes to support theenhancement of its collaborative provision(paragraph 55, 127).

iii the support provided for staff in partnerorganisations through formal and informalcommunication channels and processesincluding the collaborative conference(paragraph 93, 141)

iv the use of its virtual learning environmentboth in its delivery of programmes and as a way of effective communication with students and partners (paragraph 102, 141)

Recommendations for action

146 The audit team also recommends that theUniversity should consider further action in anumber of areas to ensure that the academicquality of programmes and standards of theawards it offers through collaborativearrangements are maintained. The teamconsiders it advisable that the University:

i develops a process for ensuring thechecking of transcripts produced bycollaborative provision partners.(paragraphs 38, 72, 111, 136)

ii makes better use of its statisticalinformation to monitor and compare theperformance of particular groups ofstudents. (paragraphs 39, 55, 90, 133)

iii ensures the process of revalidation forcollaborative provision is as rigorous as the on-site periodic review and includesinvolvement of students (paragraphs 53,128)

iv implements an effective process forproviding timely feedback to students onassessed work (paragraphs 72, 105, 136)

and considers it desirable that the University:

v continues to monitor the efficacy of thechanges to the structure and operation of the assessment process. (paragraphs 21, 136).

vi defines the criteria by which 'Chair'sAction' is appropriate in respect ofinstitutional approval and programmeapproval and modification.(paragraphs 49, 62, 128)


page 33

Appendix

Sheffield Hallam University's response to the collaborative provision audit report

The University welcomes the audit team's judgement of broad confidence in the soundness of itsawards and the effectiveness of its management of collaborative arrangements. The University ispleased to note the many strengths considered in the audit report, as well as those specificallyidentified as features of good practice.

As the report notes, the University has a well-established portfolio of collaborative provision,involving 86 organisations and giving a diverse population of more than 3,000 students access toadvanced level skills and knowledge within a culture of lifelong learning. We have a particularcommitment to widening participation regionally, nationally and internationally, and to continuingprofessional development. An important route to the achievement of these objectives is viacollaborative provision with other organisations in the UK and overseas, and so it is particularlypleasing to note the audit team's identification of the support provided for staff in partnerorganisations as a feature of good practice.

We are also especially pleased to note that the audit report echoes the QAA's Institutional Auditreport of the University in 2005, in commending the use of the virtual learning environment bothin the delivery of programmes and as a way of effective communication with students and partners.

The University notes the advisable recommendations for change.

We will make better use of statistical information to monitor and compare the performance ofparticular groups of students, including via revised arrangements for assessment boards whichare being implemented in the 2006-07 academic year.

As part of implementation of the University's recent review of its validation process, we willensure that the process of revalidation for collaborative provision is as rigorous as onsiteperiodic review and always includes involvement of students.

As part of the University's current Assessment for Learning initiative we will implement aneffective process for providing timely feedback to students on assessed work, although werecognise that this is a particularly challenging task for all universities.

We will build on the current close and excellent working relationships with our partners todevelop a process for ensuring the checking of student transcripts produced by collaborativepartners.

Actions addressing all the recommendations in the report will be monitored during 2006-07through a University action plan.


page 34

RG

274 08/06

Date post:	14-Jun-2020
Category:	Documents
Upload:	others
View:	7 times
Download:	0 times

Sheffield Hallam University - APRIL 2006 · Sheffield Hallam University APRIL 2006. Preface The...

Documents