Reviews Group: Access to Higher Education
Assessment and Qualifications AllianceM A R C H 2 0 0 7
brought to you by COREView metadata, citation and similar papers at core.ac.uk
provided by Digital Education Resource Archive
© The Quality Assurance Agency for Higher Education 2007
ISBN 978 1 84482 733 6
All QAA's publications are available on our website www.qaa.ac.uk
Printed copies of current publications are available from:Linney DirectAdamswayMansfieldNG18 4FN
Tel 01623 450788Fax 01623 450481Email [email protected]
Registered charity numbers 1062746 and SC037786
Foreword 1The review process 1
The AVA context 2AVA statistics 2005-06 2
Principle 1 2The organisation has governance structures which enable it to meet its legal and public obligations, to render it appropriately accountable, and to allow it to discharge its AVA responsibilities securely
Principle 2 5The organisation is able to manage its AVA responsibilitieseffectively, and to maintain an appropriate structure to support them
Principle 3 9The organisation is able to assure the quality and fitness for purpose of Access to HE programmes at the point at which they are granted formal approval, and to have effective means to develop, evaluate and review the Access to HE provision for which it has responsibility
Principle 4 14The organisation is able to secure the standards of achievement of students awarded the Access to HE qualification
Conclusions 19
The AVA licence 20Review outcome 20Conditions 20Recommendations to the AVA 22
Appendix 23Aims and objectives of AVA review 23
Contents
Foreword1 The Quality Assurance Agency for HigherEducation (QAA) is responsible to the Departmentfor Education and Skills (now the Departmentfor Innovation, Universities and Skills) for therecognition of Access to Higher Educationcourses. QAA exercises this responsibilitythrough a national network of authorisedvalidating agencies (AVAs), which are licensedby QAA to recognise individual Access to HigherEducation (HE) courses, and to award Access toHE qualifications. The AVAs are responsible forimplementing quality assurance arrangementsin relation to the quality of Access to HEprovision and the standards of studentachievement. QAA has developed a scheme forthe licensing and review of AVAs, the principlesand processes of which are described in theQAA Recognition Scheme for Access to HigherEducation in England, Wales and Northern Ireland.The Recognition Scheme is regulated andadministered by the Access Recognition andLicensing Committee (ARLC), a committee ofthe QAA Board of Directors.
2 The ARLC is responsible for overseeing theprocesses of AVA licensing and periodic reviewand relicensing. The criteria applied by theARLC and by review teams operating on theCommittee's behalf, in reaching judgementsabout whether and under what terms an AVAlicence should be confirmed or renewed, areprovided within the Recognition Schemedocumentation. These criteria are groupedunder the four principles that provide the main section headings of this report.
3 Following an AVA review, a member of theteam presents the team's report to the ARLC. TheCommittee then makes one of four decisions:
i unconditional confirmation of renewal oflicence for a specified period
ii conditional confirmation of licence withconditions to be met by a specified date
iii provisional confirmation of licence withconditions to be met and a further reviewvisit by a specified date
iv withdrawal of licence for operation as an AVA.
4 This is a report on the review for theAssessment and Qualifications Alliance (AQA).QAA is grateful to AQA and to those whoparticipated in the review for the willingcooperation provided to the team.
The review process
5 The review was conducted in accordancewith the process detailed in the QAA RecognitionScheme for Access to Higher Education in England,Wales and Northern Ireland. The preparation forthe event included an initial meeting between aQAA officer and AQA representatives to discussthe requirements for the Overview Document(the Overview) and the review process; thepreparation and submission by AQA of itsOverview, together with a selection ofsupporting documentation; a meeting of the review team to discuss the Overview andsupporting documentation and to establish the main themes and confirm the programmefor the review; and negotiations between QAA and AQA to finalise other arrangements for the review.
6 The review visit took place on 12 to 14March 2007. The visit consisted principally ofmeetings with representatives of AQA, includingAVA officers; members of the AQA Council, theQuality Assurance Committee and AccessManagement Group; moderators for Access to HE programmes; Access to HE coordinators;and representatives from HE.
7 The review team consisted of ProfessorBrian Anderton, Associate Dean (AcademicPlanning and Development), University ofCentral England in Birmingham; Mike Farmer,education consultant, Farmer Associates,member of the ARLC and lead reviewer;Professor James Hughes, formerly Professor ofIndustrial Relations, University of Kent. Thereview was coordinated for QAA by KathDentith, Head of Access.
Assessment and Qualifications Alliance
page 1
The AVA context
Background and developments since thelast review
8 AQA is a company limited by guarantee anda registered charity. The company in its presentform was created in April 2000, following amerger between the Associated ExaminingBoard and the Northern Examinations and Assessment Board (NEAB). AQA isaccredited by the Qualifications and CurriculumAuthority to award a broad range ofqualifications, including GCSE and GCE A-levels.The original AVA licence was granted to the JointMatriculation Board (one of four constituentbodies which were amalgamated to form NEAB)in 1991, transferred to the NEAB when it wasestablished in 1992 and to AQA in 2000.
9 The organisation is the largest of the threeEnglish examination boards. Its annual turnoveris in excess of £130 million and it employsaround 900 full-time staff and a substantialnumber of part-time staff. Income directlyattributable to Access to HE is less than 0.1 per cent of the overall total.
10 AQA has offices in Guildford, Manchester andHarrogate. The AVA functions are coordinatedand managed from the Harrogate office, havingbeen relocated from the Manchester office sincethe previous review in 2001.
11 The last review resulted in the provisionalrenewal of the AVA licence, subject to nineconditions. The conditions were approved byQAA as having been met in four stages over theperiod July 2002 to July 2004. The review teamsaw evidence to indicate that action to meetthese conditions had led to a number ofimprovements. In particular, positivedevelopments have taken place with regard toclarification and strengthening of governancestructures for the AVA; the development ofregular strategic planning processes relating to Access to HE; the collection and analysis ofstatistical information from providers; andclarification of moderators' roles.
12 The AVA has provided regular annual reportson its operations to QAA. One was not approved
because of the amount of material that wasmissing or not presented on the requiredformat; one was approved after furtherinformation had been submitted; and threereports were approved by QAA with no furtheraction required.
AVA statistics 2005-061
Providers and programmes
Total number of providers 11
Total number of Access to HE programmes available 30
Total number of Access to HE programmes running 30
Total Access to HE learner registrations 1,985
Total Access to HE certifications 880
13 Over the period since the last review, until2005-06, the number of providers remainedfairly static at ten or 11, while the number ofprogrammes rose from 16 to 30 and thenumber of registered learners rose significantlyfrom 590 to 1,985. At the time of the review,three new providers had joined the AVA,bringing the total to 14 and the number ofprogrammes available to 37.
Principle 1
The organisation hasgovernance structures whichenable it to meet its legal andpublic obligations, to render itappropriately accountable, andto allow it to discharge its AVAresponsibilities securely
Legal and constitutional status
14 AQA is a company limited by guarantee andhas charitable status. The governing body is theCouncil, consisting of 29 trustees, includingrepresentatives of education, local governmentand business. The Articles of Association andterms of reference of its committees identify theconstitutional arrangements and governance
Access to HE review report
page 2
1 As provided by the AVA in its annual report to QAA in December 2006
structures, the appointment of Councilmembers, and the appointment of the DirectorGeneral and Deputy Director General.
Council
15 Of the 29 Council members, eight aredrawn from schools and colleges, and sevenfrom higher education institutions (HEIs). The Director General, his Deputy and othermembers of the Executive Board (see paragraph29, below) also attend Council meetings,thereby ensuring that, whatever subject isunder discussion, there will always be someonewith direct line management responsibilitypresent to answer any queries that might ariseduring the Council's deliberations.
16 The Council is the locus of all decision-making in AQA, including all matters relating to Access to HE. Thus it has ultimate authorityfor the AVA licence. The composition of itsmembership and the relationship between theCouncil and the committee and managementstructures which are responsible for Access toHE matters are, therefore, as noted in the 2001 report, of some importance. Of the eight Council places reserved for schools and colleges, none is currently filled by arepresentative of a further education (FE)member of the AVA 'partnership' (seeparagraph 24, below), while only one of theuniversity places is filled by a representative of one of the AVA's 'partner' institutions. AVApartnership representation on Council istherefore limited. Clearly, from an AVA point of view, having a Council member drawn fromone of its FE partners with direct and currentunderstanding of Access to HE, would be seenas a positive development.
17 The Articles of Association distinguishbetween governance committees, which takedecisions and determine policy because Councilhas delegated specific powers to them, andadvisory committees, which have no decision-making powers. Council has recently reviewedthe committee structure and terms of referenceof all committees and, as a consequence, bothtypes of committee are now chaired by atrustee and sometimes include additional
trustee members. The main governancecommittees which have a bearing on the workof the AVA are the Council Access ReviewGroup (CARG) and the Access ManagementGroup (AMG). Although the AVA does not have any advisory committees, there is aQuality Assurance Committee (QAC), which is a subcommittee of AMG, and its Chair, who is elected by the membership, becomes an ex officio member of the AMG.
Council Access Review Group
18 CARG is not listed as part of the formalcommittee structure but is a small subgroup ofCouncil, consisting of five trustees, includingthe Chair of Council. It meets only once a yearand its sole function is to consider the AVA'sannual report to QAA on behalf of Council, andadvise the Chair of Council on its approval. Theminutes of CARG confirm that it discharges itsscrutiny and amending role very thoroughlybefore approving the draft report on behalf ofCouncil. While the report does not, therefore,go to a full Council meeting for approval, thereview team considered that this developmentwas a positive one in ensuring that at least fivemembers of Council, including its Chair, gavethe report detailed scrutiny and therebyreceived and considered detailed informationabout the AVA's activities.
Access Management Group
19 The AMG is responsible for managing allAVA activities and advising Council on thestrategic direction for AQA's Access to HE work.In addition to the trustee Chair appointed byCouncil, there are ten members, dividedequally between representatives of HE and FE.Nine of these members are currently drawnfrom, and nominated by, partnershipinstitutions and this ensures that the partnershave a prominent role in both the managementof Access to HE and the development ofstrategy within the AVA. The authority of AMGis reinforced by the fact that it is chaired by atrustee and reports directly to Council.
20 In its management role, AMG is involvedwith all aspects of Access to HE, including
Assessment and Qualifications Alliance
page 3
approving the validation of programmes;reviewing programme reports from collegecoordinators; approving the draft AVA annualreport; approving and monitoring the actionplan; and managing external relations withQAA and members of the partnership. It isassisted in its work by the QAC, which reports to it.
Quality Assurance Committee
21 The QAC has eight members, all of whomare appointed by AMG from nominationssubmitted by the partners. As noted above,these members elect a Chair from amongthemselves. Its other seven members areappointed from nominations submitted by the partners. As QAC is formally neither agovernance nor an advisory committee of AQA (see paragraph 17, above), its terms of reference are not included in AQA's maindocument, which details the terms of referencefor those committees in a consistent formatwith common categories of information. Rather,the QAC's terms of reference are provided in aseparate document with a different format.QAC's terms of reference specify that 'Memberswill be appointed for a period of three years'.However, unlike committees for which terms ofreference appear in the main document, nolimit to the number of times a member mightbe reappointed is specified. Presumably thisomission is an oversight, and the teamrecommends that this be addressed when thecommittee's terms of reference are nextreviewed. In making appointments to QAC,AMG ensures there is 'an appropriate spread ofexpertise and experience', not only between HEand FE, but also between those with expertisein curriculum matters, quality issues andexperience of running Access to HE courses.That the majority of its members are frompartnership institutions and collectively have awide spread of expertise ensures that QAC isappropriately constituted to keep AMGinformed about developments affecting Accessto HE within the partnership.
22 The Committee, which meets three times a year, is an important advisory committee to
AMG. Its key functions are to monitor theoperation of all Access to HE courses within theAVA, to oversee quality assurance processesand, where necessary, make recommendationsto AMG. QAC considers in detail all moderatorreports, validation and revalidation reports, aswell as any other issues affecting the quality ofcourses. As well as providing oversight of thequality of the AVA's provision in these ways,through the work of the Committee, individualQAC members are directly involved - and aretherefore a key resource - in the validationprocess itself, since all validation panels includetwo members of QAC (see paragraph 57, below).
The AVA partnership
23 Membership of the AVA partnershipconsists of all those providing institutions thatdeliver Access to HE courses validated by theAVA and some of the HE institutions (HEIs) thatadmit students from these courses. Currentlythere are 14 partners in each group, although itis part of the AVA's strategy to increase both(see paragraph 36 and 40, below).
24 The partnership acts in a purely advisorycapacity to AMG, although its members play a central role in electing AMG as well asproviding the majority of members for bothAMG and QAC. The AVA partnership wasoriginally established as a mechanism to identifythe AVA's stakeholders and allow them to beinvolved in the strategy and development of theAVA. The partnership is now defined by aMemorandum of Association which providescategories of membership, rights, and duties andobligations of members. The introduction of theMemorandum clarifies a number of mattersabout the AVA's expectations of those it workswith. However, the constitutional significance ofthe Memorandum is unclear. Indeed, with thestrengthening of the governance structures andprocedures through which AQA meets its formalresponsibilities as an AVA, the purpose of thepartnership itself - which stands outside thosestructures - has become less certain.
25 The annual partnership meeting provides a useful forum for discussion of both local andnational developments in Access to HE, and
Access to HE review report
page 4
staff believe that communication and feedbackfrom the meeting inform the day-to-day thinkingof AVA officers and influence longer-termstrategy. Attendance at these meetings hasbeen low in recent years, particularly among HErepresentatives. An explanation for this poorattendance might be that offered by the AVAitself in its self-evaluation document, namelythat the 'precise remit (of the partnership) stillneeds further clarification'. A furthercontributory reason for poor attendance by HErepresentatives is that the AVA may not betargeting those in HE with most direct interestin the development of Access to HE.
26 Members of AMG expressed the view thatmore could be done to make the partnershipmeetings more relevant and proposed that theyshould be used as a forum for consultation onstrategic issues. While, in the past, the AVA hasnot consciously used partnership meetings toeither consult on or promote its strategicobjectives, the minutes of the March 2007meeting suggest that, in future, strategic issuesmight figure more prominently as agenda items.
27 The partnership meeting has the potentialto provide an important forum for facilitatingthe flow of information and the exchange ofideas, but it is not currently fulfilling thatpotential. The review team considers that, in order to do so, there would need to be achange of focus, with agendas reflectingstrategic issues, and an increase in theparticipation of the HE members.
28 Although there are still matters relating topartnership and the development of strategy tobe resolved (see also paragraphs 40 to 46,below), the review team considers that therehave been several positive developments ingovernance since the last review. In particular, theappointment of a trustee to chair AMG and theclarification of the Group's role in the governancestructure; the establishment of CARG; and theincreased representation of partnershipinstitutions on QAC and AMG are all positivedevelopments which lead the team to believethat the AQA Council is a responsible location for the AVA licence. The team also noted thestatement in the 2005-06 annual report, that theAVA 'remains an institutional priority for AQA'.
Principle 2
The organisation is able tomanage its AVA responsibilitieseffectively, and to maintain anappropriate structure to supportthem
Management
29 The Director General and Deputy DirectorGeneral of AQA, together with five divisionaldirectors, constitute the Executive Board. TheExecutive Board is responsible for themanagement of the organisation and theimplementation of the Council's strategy. Eachdirector has line management responsibility forthree or four assistant directors. Together, themembers of the Executive Board and theassistant directors form the Senior ManagementTeam (SMT). Within the SMT, the AssistantDirector, Business Management (Harrogate),has some responsibility for Access to HEincluding line management of the Access to HEofficer. The Assistant Director reports to theDirector of Finance, although the review teamheard that Access to HE matters are alsoreported to the Director of the QualificationsDevelopment and Support Division, whenappropriate. The job description states that theAssistant Director has 'overall responsibility forall aspects of the work of the Harrogate officeand, in particular, for the overall managementof staff and the development of new businessopportunities in the Harrogate office', as well asa wide range of 'specific accountabilities',including 'encouraging best practice and mosteffective use of resources in the overallmanagement' of a number of specified areas,including Access to HE. The post holder is alsorequired to 'provide continuing leadership' forthe teams which manage the variousqualifications administered from the Harrogateoffice. Inevitably, the time which the AssistantDirector is able to dedicate to managing andproviding leadership for Access to HE activitieswill be limited.
Assessment and Qualifications Alliance
page 5
Access to HE section
30 The Access to HE section consists of fourindividuals, three of whom are on fractional,fixed-term contracts. There are two sectionheads and a clerical assistant, all working half-time (0.5) and, at a more senior grade, an Access to HE Officer on a 0.6 contract. Thus,ignoring grade differences, the Access to HEteam is equivalent to 2.1 full-time staff. Thisamounts to almost a doubling of dedicatedstaff on the previous year although, at thatearlier point, administrative support was beingprovided to Access to HE by staff outside thesection. According to the job descriptions forthese posts, the team is based in the Division ofQualifications Development and Support, andthe Access to HE Officer reports to a PrincipalSubject Manager. In discussion with the Accessto HE Officer, the review team heard thatalthough the Access to HE Officer oncereported to a Principal Subject Manager, shenow reports directly to the Assistant Director,Business Management. The job descriptions forstaff in the Access to HE section would appearto be in need of updating. The review teamrecommends that job descriptions are reviewedto ensure that they reflect reality and, inparticular, that reporting and line managementresponsibilities are clear.
31 The two half-time section heads, who arethe first points of contact for partners and areresponsible for the day-to-day running of theAccess to HE section, report to the Access to HE Officer whose overarching responsibility isto 'administer and develop' Access to HE withinthe AVA. This responsibility embraces a range of duties, including servicing committees andvalidation panels; coordinating moderationarrangements; collecting and analysingcolleges' data; and preparing the annual reportto QAA.
32 The review team noted that the AVA hadsuffered from a high turnover of Access to HEOfficers during the last two years and, forsubstantial periods of time, this post hasremained unfilled. The post remained unfilledfor several months in late 2005 and the failureto fill the post quickly prompted the Chair of
AMG to express his concern to Council aboutthe level of Access to HE staffing and the needfor Council's support. While the post was thenfilled, it again fell vacant again in August 2006.
33 A lack of dedicated Access to HE staff and,in particular, the availability of appropriateexpertise for the future, was identified as anissue in the 2001 review. Now, although thenumber of staff with dedicated Access to HEposts has increased, the team remains small.When members of Council met the reviewteam they expressed some 'concern' aboutwhether, with current staffing levels, the AVAwould have the capacity to take on additionalprovision, as planned, as well as taking forwardthe work to implement the Access to HEDiploma. The review team was informed thatthe staffing situation would be reviewed later in the year and, if the expansion in AVA activitythat is planned was realised, then a case foradditional staff would be made. The teamconsidered this a positive move.
34 The review team concurred with theCouncil representatives' view that, given the demands associated with the AVA'sresponsibilities, including demands relating tothe introduction of Access to HE Diploma, theremust be some concern about whether thecurrent level of staffing could provide both themanagement and support necessary to ensurethat the AVA is able to discharge itsresponsibilities fully and effectively. The team'sconcerns were reinforced by recent instances of lapses in quality assurance procedures (asdescribed later in this report), which suggestedthat the Access to HE team was working undersignificant pressure. Given these circumstances,the team concluded that the Access to HE staffwould be unlikely to have any spare capacityfor engaging in additional development activity.
35 The AVA is therefore required, as acondition of licence, to carry out a review andanalysis of its staffing arrangements for Accessto HE, and report on the outcomes of thereview to demonstrate how it will ensure thatits responsibilities as an AVA licence holder willbe met, including responsibilities for qualityassurance and development, actions identified
Access to HE review report
page 6
in its action plan (see paragraph 36, below),actions relating to the implementation of theAccess to HE Diploma, and actions relating toimplementing the conditions of this report.
Strategic planning
36 AQA operates a regular strategic planningprocess for its Access to HE activity, and anannual action plan is considered and approvedby the AMG in November each year. The 2006-07 plan sets out specific actions and targetsunder five broad headings: strategicdevelopment; quality assurance, enhancementand development; financial; statistical data; andpreparation for QAA review. The specific targetsinclude actions relating to developing thenecessary structures for, and implementation of,the Access to HE Diploma, as well as attractingnew business during the period of introductionof the Diploma and establishing links with moreHEIs. The plan also includes reducingexpenditure and reviewing charges. Within theplan, responsibilities for undertaking actionsand achieving targets are assigned and reviewdates for each are set. The plan does notidentify resources for meeting the targets eitherin terms of finance or in staff time, and therelationship of this plan to budget settingwithin the organisation was not explored by thereview team, although the Access to HE sectionhas a separate budget, for which the Access toHE officer has monitoring responsibility.However, discussion of the plan at the AMGmeeting which approved it did not appear tohave addressed the question of resources. Itwas consequently not clear to the review teamwhether the 2006-07 targets were realistic andachievable, given the current staffing base andthe plan to reduce expenditure.
37 Achievement of the plan is monitored byAMG and an assessment of the achievement oftargets is included in the AVA's annual report.The plan is therefore considered by themembers of AQA's Council through the CARGat its annual meeting to approve the annualreport, and in this way Council is involved inthe planning process. At the most recentmeeting of CARG, it expressed concern aboutthe level of resources, particularly staffing,
necessary to meet the AVA's plans, although itdid not amend the plan itself.
38 In parallel with this regular planningprocess, AQA Council has been conducting areview over the last three years of a number ofits awards which were not central to its mainbusiness, including Access to HE, in particularto ensure that these areas did not make a loss.In December 2006, Council considered areport which assessed the financial position ofAccess to HE, the potential for growth and theanticipated financial and staffing implications,and the resource implications for thedevelopment of the Access to HE Diploma. The review team sought information on theoutcome of this review. Members of Councilaccepted that Access to HE is a very small partof AQA's business, but confirmed theircontinued commitment to Access to HE.
39 The review team concluded that the AVA'sstrategic planning processes were generallyappropriate for their purposes, involving AQA'sCouncil and therefore the potential to ensurethat resources can be matched to the plannedactions. An example of this is the decision toinvest in data resources to support thedevelopment of the Access to HE Diploma.However, there is a requirement for the AVA togenerate sufficient income not only to pay for itsuse of AQA's general infrastructure, but also tomake a financial surplus. In practice, therefore,there is not a sufficiently clear articulationbetween the AVA's plans and the resources toenable these plans to be implemented. Thereview team recommends that the AVA reviewsits approach to strategic planning and makesexplicit the articulation between plans andresources to support these plans.
Regional strategy
40 The 14 providers at the time of the review(see paragraph 13, above) were located in fiveEnglish regions, with the largest single groupin Yorkshire and the Humber. The 14 HEImembers were located in five regions, in fourof which there were also providers of Access toHE programmes validated by AQA. The AVA isseeking additional HEI members and two newmembers have recently joined.
Assessment and Qualifications Alliance
page 7
41 Council members highlighted thedifferentiation that they were trying to createfrom other players in Access to HE and theimportance of the AQA brand as an alliancebetween schools, FE and HE. As an AVAoperating in a national context, AQA is open toany provider, irrespective of the region in whichit is located. The AVA stresses its national scopeand the view was expressed by officers andCouncil members that, since AQA is a nationalbrand, its mission is to facilitate Access to HEwherever there is a demand. They stated thatthey had decided not to prioritise any particularregions, and that the AVA would respond toapproaches from any provider in any region.
42 At the same time, the AVA accepts the need to have a regional focus and the plan for2006-07 (see paragraph 36, above) includesthe development of regional links, and commitsthe AVA to identifying and publicising relevantregional priorities and agendas to Access to HEproviders. The AVA states that the 'regional skills agenda should be at the centre ofstrategic development', and it has committeditself to developing regional subgroups of theAVA, using a bulletin board to aidcommunication, to working with Aimhigherpartnerships and Lifelong Learning Networks,and to exploring ways of using AQA's networkof regional forums and regional officers torespond to this agenda. Plans in these areas are still at an early stage, however.
43 While AQA has stated its intention todevelop regional networks in these ways, theAVA acknowledged in its Overview that, as anational organisation, it has problems inpromoting a regional agenda and stated thatalthough regional skills agendas are keyelements in policy and planning, in practice,the labour market and progression to HE ismore localised. Its main approach in respondingto regional agendas is, therefore, through itsapproach to course development. Thisapproach focuses on giving providers flexibilityto develop Access to HE provision tailored tomeet local priorities and local HE progressionopportunities in the travel-to-work area.
44 The review team noted that the AVA wasconsidering expansion to include a number of particular providers in a region in which the AVA was not currently active. The teamtherefore sought to clarify the AVA's position,including how it proposed to undertake theregional agenda it had set itself, which regionor regions it proposed to operate, how it wouldcollaborate with the regional organisations inthese regions and, in particular, how inresponse to the licensing criteria it wouldanalyse the needs of each region in which itproposed to operate, identify those groups in these regions which have most need ofopportunities to progress to HE, and how itwould communicate the results of this analysisto its providers to assist them in their planning,and what resources it planned to devote tothese developments. AMG members indicatedthat strategies in each region would bedeveloped from the bottom up, and wouldneed to reflect the interests of providers in eachregion. They reported that a number of ideashad been discussed about how to develop theregional dimension, including greater use of the annual partnership meeting.
45 The review team came to the view that theAVA licensing criteria which relate to regionalplanning pose a particular challenge for AQA as a national organisation. Moreover, the AVA'sapproach to regional strategy, in particular itsdecision not to prioritise any particular regionor regions, and to accept applications formembership from any provider without anyregionally-based criteria, add to this challenge.Its plans for expansion are to attract providers,principally by relying on its reputation forflexibility and responsiveness, rather than toprovide strategic leadership and direction in the planning, development and promotion ofAccess to HE at a regional level. It was not clearto the team that the AVA had committedsufficient resources to sustain this competitiveapproach, or to the necessary joint work withsuch agencies as Aimhigher and LifelongLearning Networks, or to the work required to analyse the needs for Access to HEdevelopment in all the five regions in which it currently operates, and support thedevelopment of new provision. Indeed, should
Access to HE review report
page 8
its plans for expansion be successful, theparticular difficulties in engaging with, andresponding to, regional strategies, andproviding robust quality assurance of provisionover a wider geographical area, are likely to beexacerbated if the new providers attracted aredrawn from a still wider range of regions.
46 It is therefore a condition of licence that theAVA should review its approach to developing a regional strategy to ensure that it can meet its licence obligations in those regions in which it operates, and that it should report, in particular, on:
the regions in which it intends to operate,including any phasing of expansion
the means whereby it will engage and workcollaboratively with other organisations andinstitutions working to promote wideningparticipation in each region
its approach to the analysis of regionalpriorities and needs for Access to HEdevelopment
its approach to the promotion of Access to HE in each region
the role of its regional officers, regionalforums and bulletin boards in this strategy
the resources necessary to support theimplementation of this strategy.
Principle 3
The organisation is able toassure the quality and fitness for purpose of Access to HEprogrammes at the point atwhich they are granted formalapproval, and to have effectivemeans to develop, evaluate andreview the Access to HE provisionfor which it has responsibility
Access to HE Handbook
47 The AVA publishes an Access to HEHandbook, which is updated on an annual basisand constitutes the main means through which
it advises its members about its processes forassuring quality and standards. The latestversion, at the time of the review, was datedSeptember 2007. During the course of thereview, the review team was also provided witha copy of the AVA's Annual Office Procedureswhich includes some of the procedural aspectsof validation and approval of courses, but whichhad not, at that time, been published.
Fees and charges for validation
48 The Handbook describes the AVA's chargingpolicy, stating that there are separate fees forinitial validation and for subsequentamendments to validated courses. The reviewteam was provided with a more detailedcharging schedule. This showed that thevalidation fee comprises two elements - a basicvalidation fee and variable fees according to thenumber of units to be validated. It also showeda standard basic fee for validation at theprovider's own premises and for an 'office-based'validation at AQA's offices if the validation usedalready validated units, in which case the feewould be reduced by just over 70 per cent.
Course development
49 The Handbook provides detailed guidancefor providers on course development. This setsout the AVA's requirements and providescomprehensive guidance on the developmentwork necessary prior to submission forvalidation, including preliminary research andidentification of target groups, course structureand content, organisation and management,assessment, and consultation with representativesof HE. Access to HE coordinators whom thereview team met expressed satisfaction with thelevel of support that they had received fromAVA officers in the developmental phase of thecourses for which they were responsible. Theteam noted that the advice on course structurehad been updated to address the requirementsfor the Access to HE Diploma, includingdetailed guidance on rules of combination.
Validation and revalidation procedures
50 The Handbook also provides guidance onvalidation and revalidation procedures, and for
Assessment and Qualifications Alliance
page 9
course approval by the AVA. The Handbookprovides a standard timetable starting with avisit by an AVA officer in October andculminating with a validation panel meeting in May/June. The procedures for revalidationare the same as for validation.
51 Before a provider can proceed with asubmission, the centre first needs to beapproved as an AQA Examination Centre, usingthe standard procedures that AQA uses for allits centres, and linked to AQA's GeneralRegulations. The review team considered thatthe requirements for centre approval, whilst avaluable initial check on a centre's standing, are mainly concerned with matters relating toexamination centres for AQA's mainqualifications, rather than requirements foroffering Access to HE courses, and that this also applied to a range of other standardpublications. The general requirements forAccess to HE providers are covered in theMemorandum of Association for the AVApartnership (see paragraph 24, above), whichplaces certain obligations on providers in termsof resources, student support, reporting to theAVA, and payment of annual subscription fees.AVA officers, in responding to questions fromthe team about procedures for approval andwithdrawal of approval for Access to HEcourses, referred to the terms of theMemorandum of Association.
52 The review team noted that neither therequirement to be approved as an AQAExamination Centre, nor the requirement toadhere to the terms of the Memorandum ofAssociation, are specifically referred to in theHandbook as necessary prerequisites forvalidation or for continuing approval as acentre. In view of the importance of some ofthe expectations articulated in these documentsfor providing the context for securing qualityand standards, the team considered that theAVA should clarify these requirements (in theHandbook, or other formal document), andthat they should be verified and confirmed aspart of the validation process.
53 The AVA's procedures for validation providefor scrutiny of units by subject specialists andfeedback to the centre, before a validation
meeting. Members of the QAC stated that byundertaking this prevalidation scrutiny of units,it was unusual for detailed issues on the units tobe raised during the validation event itself.However, the review team noted that there isno procedure for new or revised units to beformally approved prior to the meeting of thevalidation panel, that there appeared to besome overlap between the issues on whichsubject specialists were asked to comment andthose that the validation panels were asked toaddress, and it saw at least one example of avalidation report where detailed issues relatingto the content of the units were addressed bythe validation panel.
Membership of validation panels
54 The AVA's procedures provide for avalidation panel to be appointed by QAC foreach Access to HE course for which a provider isseeking validation, comprising three members:two members of the QAC and one from an HEI.Although the AVA's own requirements are for avalidation panel for each course, the reviewteam noted examples of validation events wherethe same panel had considered a number ofcourses and, in the most recent case, the samepanel had considered seven courses at threedifferent colleges. As noted in paragraph 58, theAVA was unable to provide written reports ofthese three validation events. Membership ofthese panels is also supposed to include arepresentative from an HEI outside thepartnership, although finding such volunteershas recently proved difficult and members ofQAC have suggested that this requirementmight be abandoned in future.
55 The Handbook states that, in addition tothe normal complement of panel members,one or more external subject specialists mayalso be co-opted. The panel concluded that theprecise relationship between the role of subjectspecialists and that of the validation panel inrelation to the subject content and approval ofunits was not clear.
56 Given that, under the new Access to HEDiploma, each programme might include anumber of different pathways leading todifferent named awards, the review team
Access to HE review report
page 10
sought clarification about whether the AVAwould regard each pathway as a separatecourse, for which a separate validation panelwould be appointed, or whether it would addadditional members to a validation panel totake account of the range of pathways andawards under consideration. QAC membersstated that the number of pathways underconsideration would not necessarily have anybearing on the size of the validation panel.Since a range of subjects could be covered bydifferent pathways at an individual provider,and the AVA's own procedures allow foradditional subject specialists to be added tovalidation panels, the team considered that theAVA should review this practice, particularly ifunits continue to be formally validated byvalidation panels.
Validation meetings
57 The role of the validation panel is to make a recommendation via the QAC to the AMGabout the suitability of the course aspreparation for entry to HE. The Handbookprovides guidance on the issues whichvalidation panels should address in order tomake that judgement. However, the reviewteam noted that although there is guidance toproviders on course development in relation tothe requirements of the Access to HE Diploma,the guidance to validation panels does notappear to have been updated to ensure thatthe validation process confirms that a coursemeets the standard requirements for theDiploma, including the specifications for thecommon credit framework, the rules ofcombination, or the methods of recordingindividual student achievement. The teamconcluded that guidance to validation panelsneeds to be revised to ensure, at the point ofvalidation, that all courses will meet therequirements of the Diploma.
58 The Handbook, in describing the validationprocedures, states that a validation panel willvisit the provider, or that there will be an'office-based panel'. However, although theHandbook provides an outline agenda for avalidation panel meeting on the provider's
premises, none is provided for an office-basedmeeting, nor does the Handbook specify underwhat circumstances there would be an office-based panel. QAC members explained that thisprocedure would be adopted with an existingprovider in circumstances where most of theunits had already been formally approved bythe AVA, and had been developed partly for the Access to HE Diploma, although not allrevalidations would necessarily be office-based.The review team noted, however, that office-based validation had been used in summer2006 for three centres which were new to theAVA. Members of the QAC described thisprocedure as 'experimental' and explained thatnormal procedures had not been followed inthat instance. AMG members referred to themas 'atypical'. The AVA provided a paper setting out the rationale for these validations whichreferred to an intention to 'test run andstreamline procedures' and to address'excessive' 'costs to AQA and institutions'.Unfortunately, there is no formal written recordof these three validation events, so the teamwas unable to confirm that the AVA'sprocedures had been followed fully. AVA officersstated that the decision on which type ofvalidation would take place would be taken byan AVA officer, although it would probably alsogo to the Chair of QAC. The AVA's Overviewreferred to plans to move towards a risk-based approach to validation andrevalidation and AMG members confirmed that this was planned, although no details were provided to the team. The team formedthe view that, in the absence of clear publishedcriteria for determining when a validation couldbe office-based, or what that procedure shouldinvolve, and a lack of involvement by theappropriate committee in decisions about what kind of validation procedure should befollowed, there was a real possibility that anoffice-based validation which includedinsufficiently robust quality assuranceprocedures might be adopted.
59 After the validation, the AVA officerattending the event writes a report. TheHandbook describes the possiblerecommendations of a validation panel,
Assessment and Qualifications Alliance
page 11
including approval with or without conditionsor recommendations. An officer writes to theprovider informing them of the outcome and ofthe deadlines which the panel has set by whichconditions are required to be met. TheHandbook does not describe this process butthe Office Procedures document states thatconditions have to be met by 20 July forcourses starting in September. AVA officersconfirmed that the decision on whetherconditions are met is taken by the AVA officer.Once officers are satisfied that conditions havebeen met, the AVA officer writes to the providerto confirm this and takes the report throughthe AVA's committee structures for approval.
Course approval
60 The report of the validation meeting,including the panel's recommendations, ispresented to the QAC and then, throughreceipt of the QAC's minutes, the AMG grantsformal approval. The review team noted thatthe meetings of AMG to approve courses hadroutinely taken place around November eachyear, and that courses therefore had not beenapproved until after they had started. AMGmembers acknowledged that this was late. TheOffice Procedures document provides a revisedtimetable, with AMG meeting in June/July toconsider validations. The Chair of AMG statedthat, in order to ensure that all courses wereapproved prior to their commencement, Chair'saction might need to be considered. However,this process would depend on a formal writtenrecord of the validation event being available.
61 The audit trails also raised concerns aboutthe process for withdrawal of course approval(see paragraph 88, below). The AVA will wish toconsider how the particular events came about,and the process by which the different decisionstaken were communicated to the provider, andaddress any weaknesses in procedures to ensurethat such events cannot recur.
62 The AVA is required, as a condition oflicence, to reconsider its procedures forvalidation and revalidation and revise the AVA'sformal documentation to clarify:
the role of standard AQA centre approvaland of the AVA's Memorandum ofAssociation, and requirements relating to these
the distinction between approval of acentre and the validation and approval ofparticular courses and pathways offered bya centre
composition and formal approval of thevalidation panel membership, and theprocess for ensuring that the panelmembership is appropriate for the range of courses and/or pathways which it is toconsider
the point at which units are formallyapproved including the distinctive roles ofsubject specialists prior to the validationevent and the validation panel
matters to be addressed by a validationpanel, including requirements relating tothe Access to HE Diploma specification
expectations for the content and standardformat of the panel report, andresponsibility for its production
process for the confirmation of course andpathway validation
process and responsibility for monitoringconditions and recording that conditionshave been met and formally approved
processes and timing for formal approval,and for the formal withdrawal of approval,including notification to the provider
the rationale for, and distinction between,different styles of validation event and thecriteria and process used to decide on thestyle for a particular event
how its proposed 'risk-based' approach tovalidation will be implemented.
Progress with plan for implementing theAccess to HE Diploma
63 The AVA produced a plan for introducingthe Access to HE Diploma and provided anupdate in its annual report for 2005-06. Thisincluded a schedule which provided for the
Access to HE review report
page 12
revalidation of courses at nine providers during2006-07; five between November 2006 andFebruary 2007, and four between April andJune 2007, with a further five during 2007-08.The review team sought clarification onprogress with the plan. It was reported thatnone of the scheduled revalidations had takenplace by the time of the review and the teamwas provided with a revised schedule, with fiverevalidation events to take place duringFebruary to June 2007, and nine during 2007-08. However, during the review, some membersof QAC suggested that only two of the fivevalidations scheduled for 2006-07 might takeplace before June 2007. The team concludedthat there had been significant slippage fromthe original schedule, and there wereindications of potential further slippage fromthe revised schedule. Some members of AMGconsidered that these revalidations would notbe substantial events, as they would onlyinvolve a few units, but some providersindicated that the amount of work could besubstantial. No analysis of the volume of workenvisaged for each event was provided by theAVA. The possibility of new pathways had notbeen included in the schedule since it onlylisted providers. The team also noted that oneof the AVA's strategic objectives in its 2006-07Action Plan was to attract new business duringthe period of the introduction of the Diploma.
64 The review team noted that the AVA'sstandard method of planning validations andrevalidations, and recording them in AVAcommittee minutes, in correspondence withproviders, and on validation schedules, was torefer to a particular provider rather than thetitles of the individual courses or pathways,although in the most recent AMG minutes fullcourse titles had been recorded. The teamnoted that this was also true for the revalidationschedule for the introduction of the Access toHE Diploma (see paragraph 56, above). Theteam came to the view that the practice ofrecording only the name of the provider makesit difficult to track decisions about individualpathways, and to judge the appropriateness of membership of individual validation panels,and concluded that the AVA should review this
practice, with particular reference to therequirements of the Access to HE Diploma.
65 The review team concluded that, inaddition to the pressures it will place on AVAstaff, the revised revalidation schedule will posea significant logistical challenge to QAC, giventhat current validation procedures require twomembers of each validation panel to bemembers of QAC, from a total membership ofnine at the time of the review. The team cameto the view that the delays in revalidations, andthe consequent 'bunching' of validation events,together with the possibility of new orsubstantially-revised pathways to be validated,and new providers, would add to thischallenge, given the AVA's current practice ofconsidering each provider separately.
66 The AVA is therefore required, as acondition of licence, to review and develop itsplan for the introduction of the Access to HEDiploma, to ensure that all its providers are ableto offer the Diploma from 2008 onwards, andto provide a revised schedule with confirmeddates for all stages of validation, with particularreference to:
the revalidation schedule for existingproviders and for pathways
the incorporation of new providers into theschedule
the style of validation (provider or office-based) in each case
AVA officer support for providers
scrutiny of new and revised units
membership of, and officer support for,validation panels
the AVA's database of units
the training of moderators and subjectspecialists.
Annual review
67 Providers are required to submit an annualreview report which is considered by AMG.Immediately before the review, AMG hadreviewed the content of these reports and hadagreed that more detailed guidance to
Assessment and Qualifications Alliance
page 13
providers should be developed, a view withwhich the review team concurred. The reviewteam also noted that the process for theconsideration of moderators' reportsundertaken by QAC, was considerably moredetailed than the process for the considerationof providers' annual reports undertaken byAMG. While the revised guidance to providersis an improvement, the team recommends thatthe AVA should review its processes to ensurethat that it is explicit about how matters ofconcern are addressed, how data on Access to HE courses and students are formallyconsidered, the level of scrutiny by AMG, and how the AVA disseminates good practiceidentified though annual review. It furtherrecommends that these procedures arepublished to providers in the Handbook or in some other appropriate way.
Access to HE promotion and development
68 The AVA provides opportunities to sharegood practice through its moderator andcoordinator training events, development days,and its annual conference. These cover a rangeof matters and the most recent have addressed,in particular, matters relating to the introductionof the Access to HE Diploma. The events appearto be generally well-attended, and Access to HEcoordinators reported that they welcomed thesupport provided by the AVA and itsresponsiveness to queries.
69 The review team identified a tensionbetween the AVA's desire to be flexible andresponsive to the needs of its providers on theone hand (see paragraph 43, above), and theneed to provide leadership and direction in thepromotion of Access to HE at regional level on the other. This tension is heightened by thefact that the AVA operates in a number ofgovernment regions, but has not developeddistinctive promotion and development policiesfor each (see also paragraphs 44 and 45, above).
Principle 4
The organisation is able to securethe standards of achievement ofstudents awarded the Access toHE qualification 70 At the time of the last review in 2001, the AVA had a system of both moderators and external examiners which led to someuncertainty about roles. Subsequent to the2001 review, the AVA modified its approach to external programme monitoring andassessment by phasing out the role of externalexaminer and by redefining the role ofmoderator. Under these new arrangements,moderators fulfil the roles which werepreviously undertaken by external examinersand, where several moderators are appointedto one provider, they operate as a teamcoordinating their visits to the provider andtheir reporting to the AVA. A group ofmoderators whom the review team metindicated that they found these new team-based arrangements a positive development. A report presented to the AVA with the collectedviews of two or three moderators tended tocarry more weight, while the team-basedapproach was also helpful in mentoring newmoderators. The team also met with a group ofcoordinators who were similarly positive aboutthe changes in moderation. They consideredthem a significant improvement on theprevious system, being rigorous yet supportiveand generating constructively critical commentson the provision.
71 The role and function of moderators aredetailed in the AVA's Handbook. These includeensuring conformity of the course as delivered,with the course as approved by the AVA;oversight of the conduct of assessmentincluding approval of assessment materials,moderation of assessment outcomes andattendance at examination boards; andreporting twice yearly to the AVA through QAC.The review team was of the view that the AVA'sHandbook provided moderators with a clearspecification of their duties in relation to theAccess to HE provider. In particular, there is an
Access to HE review report
page 14
explicit requirement that moderators must visitcolleges for three full days each year and thatthey must attend all examination boards. TheHandbook provides moderators with a detailedchecklist of activities which must be undertakenon each of their three visits, geared to theannual academic cycle, and there are templatesfor reporting to the AVA twice yearly. There isalso clear guidance on the moderation ofassessment outcomes, including sampling ofstudent work. Although they operate within ateam, there is no hierarchy of moderators, andeach moderator takes responsibility forparticular subject areas, with all the moderatorscontributing to the report to the AVA.
Moderator appointment and training
72 Moderators are appointed by the AVA andare independent of the provider institution. Thereview team examined the profile of currentmoderators, and it was able to confirm thisindependence. However, the team did note aninstance of a 'back-to-back' moderationarrangement, whereby the moderator of adiscipline area in College A came from College B,and the moderator in the same discipline area inCollege B came from College A. Asked whetherthere was an explicit policy relating to sucharrangements, the AVA confirmed there was not.The team was of the view that the AVA shouldavoid situations of 'back-to-back' moderationarising, and recommends that it should developan explicit policy statement to this effect.
73 While there is a general statement of thequalities which the AVA seeks when appointingmoderators, there is no explicit statement ofthe criteria against which the suitability of apotential moderator might be evaluated. QAChas identified the need for transparent criteriafor the appointment of moderators, and hasindicated it would undertake a review andapprove new criteria and procedures. However,the review team saw no evidence that this workhad so far been undertaken. The QAC ischarged with reviewing the role and practice ofmoderators but it is not part of its terms ofreference to make appointments of moderators.The team was told that appointment of
moderators is the responsibility of the Access to HE Officer. While there was no suggestionthat current practices were leading to theappointment of inappropriate moderators, theteam was of the view that this process placedtoo great a reliance on a single officer.
74 The review team was told that, throughinformation provided on application andthrough taking up references, the AVA couldconsider a combination of factors includingacademic qualifications and experience, and toseek to match appointments to the courses andsubject areas in the provider for which a newmoderator was required. The range of subjectareas covered by individual moderators is, insome cases, quite wide ranging. The teamasked the group of moderators with which itmet for their views on the breadth of subjectcoverage within their areas of responsibility.Generally, the moderators felt comfortable thatthey had the necessary expertise to cover therange of subjects which had been allocated tothem. However, one moderator indicated thathe felt one subject for which he hadresponsibility was outside his subjectcompetence, and another moderator indicatedshe had refused a request from the AVA tocover a subject area for the same reason. To ensure that provision is moderated byappropriately experienced moderators, the AVAis required, as a condition of licence, to developclear criteria, including curriculum expertise,and process for the selection of moderators.including recommendation to the QAC, orother appropriate committee for appointment.
75 Moderators are appointed initially for oneyear. Their appointments are, thereafter,renewed annually until they have completed a maximum period of four years. It is possiblefor the AVA not to renew a contract if it hasconcerns about the performance of amoderator. Moderators receive a letter ofappointment and schedule of required duties,but there is not a formal contract document.The team recommends that this letter ofappointment should incorporate a prohibitionon subcontracting of duties by the moderator.
Assessment and Qualifications Alliance
page 15
76 The AVA is able to make judgements aboutthe performance of its moderators through thereports which they make during the year toQAC. In the past, QAC has expressed concernsabout variability in the quality of moderatorreports, but introduction of standard reportingtemplates was said to have improved thequality and consistency of reports. It was alsopossible for college coordinators to raiseconcerns about moderation in their annualreports to the AMG.
77 The AVA provides an annual ModeratorTraining Programme and recent events havecovered updating, good practice andprocedural issues. Moderators with whom theteam met confirmed they found the annualtraining events helpful and that they provided a sufficient level of support. While the level ofattendance by moderators was reasonable, the AVA has recognised that it is an area ofweakness that attendance is not compulsory.The team was told that the AVA intends tointroduce compulsory attendance at the annualtraining programme for all moderators from2007-08 and, recognising that unforeseencircumstances or late appointment may preventsome moderators from attending, it alsointends to introduce online training materials.The AVA is required, as a condition of licence,to implement its plans to make moderatortraining mandatory.
Standardisation
78 The review team was of the view that,through the move to working on a team-basedapproach, moderators within one collegewould be able to compare standards andjudgements in the different subject areas whichmade up the Access to HE provision in thecollege. The team members sought tounderstand how moderators would be able to make comparisons of standards andjudgements between different colleges. Theywere advised that moderator training hadincluded standardisation exercises using actualstudent assessments, and this would be carriedover into the mandatory training programmescheduled to commence in 2007-08. However,
the AVA confirmed that there was no individualor body within the AVA explicitly charged withoversight of comparability of standardsbetween different college providers, and theteam recommends that the AVA reviews theterms of reference of the QAC (or otherappropriate body), in order to ensure thatresponsibility for comparability of standardsbetween providers is explicitly located withinthe AVA.
Moderator reports
79 Moderators are required to complete tworeports each year. The first focuses on thecollege's response to the previous year'smoderator reports, while the main, summerreport, made after the examination board hastaken place, comments on assessmentperformance during the year, and makesrecommendations to the college for futureaction. The reports are received by the QAC forconsideration. The AMG is made aware of theoutcomes of moderation through receiving theminutes of QAC which reports to it.
80 The summer moderator reports areanalysed by members of the QAC working inpairs in order to produce summaries of the keypoints. A draft copy of the report is sent to thecollege coordinator for checking for factualaccuracy before the reports and summaries ofkey points are presented to the September QACmeeting for consideration. The summaries andcomments of QAC are then sent to theprincipal of the relevant college, and they formthe basis for the first moderation visit of thefollowing academic year. The review teamheard from moderators that this system ofreporting had proved very effective, and theywere able to cite examples of instances todemonstrate that the quality of provision hadbeen enhanced as a result. Responses torecommendations for action are reflected in thecoordinator's annual report. The moderator alsomonitors and reports back on the response ofthe college to the recommendations made inthe previous moderator report. Clearly, it couldtake up to another year before it became clearthat a college was not responding
Access to HE review report
page 16
appropriately. In this case, the Access to HEOfficer would raise the issues directly with thecollege and, should a college not be receptive,the AVA has the ultimate sanction ofwithdrawing approval, although the team wastold this had not been necessary to date.
81 The AVA's Handbook provides clearguidance on the membership of collegeexamination boards. There is also guidance onthe agenda for the examination board. The AVArequires colleges to have their own internalverification procedures to ensure internalmoderation and standardisation of assessmentsprior to their external moderation. Access to HEawards are only confirmed after the exam boardhas been concluded. The chair of the examboard and the moderator are required to signthe schedule of results to signify agreement tothe awards, and this schedule is sent to the AVAin order to trigger the issue of certificates.
Certification
82 The review team was able to inspect copiesof the certification issued by the AVA tostudents and to confirm that these were in thestandard format and used the wordingspecified by QAA for Access to HE certificates.The team asked officers about the officeprocedures for the issue of certificates tostudents. Certification is prepared against theschedule of results sent by the colleges, afterthe results have been checked for accuracy. It was clear that, while the approach used tomanage the issuance of certificates wassatisfactory, there were no written proceduresto which reference might be made, and theprocedure relied on the tacit knowledge of themember of staff responsible for this activity. Theteam recommends that the process be codifiedin the form of a set of written procedures.
83 The team was told that blank certificateswere issued to the responsible officer in batchesfrom AQA's secure central store, for preparationin the Access to HE section and onwardtransmission to the colleges who issued themto the individual students. The certificates carrya unique serial number, and the numbers ofeach batch are recorded. The team asked
whether there was a system whereby detailswere recorded of to which student eachcertificate was issued. The answer was that therewas no such system, and the AVA had no recordof to whom each certificate was issued. Theteam considered that this was a serious omission.There was no system whereby the issue of blankcertificates from the central store and the issue ofcompleted certificates to students was audited orreconciled, and the AVA had no records of theserial numbers of the certificates which it issuedto individual students. The team was of the viewthat this weakness in the AVA's administrativesystems was sufficiently serious as to require theAVA to introduce a robust and effective systemto manage and record the issue of its Access to HE certificates, as a condition for renewal ofthe licence.
Audit trails
84 In the course of the visit, the review teamconducted audit trails on eight of the AVA'sprovider colleges. The purpose of these trailswas to enable the team to consider theconsistency and effectiveness of the AVA'sprocesses at centre level.
85 The providers selected were seven publicsector colleges, and one private sector college.Three of the public sector colleges had anextensive portfolio of Access to HE programmes,covering a diverse range of discipline areas. Theprivate sector provider had one Access to HEcourse in business. Geographically, while four of the providers including the private sectorcollege were located in Yorkshire, the other fourproviders were in the Midlands, the South andthe South-West of England.
86 The review team was presented with audittrail files which included, where available,moderation, monitoring and evaluationdocumentation for the years 2004-05 and2005-06. The files included validation andrevalidation reports as appropriate, moderators'and coordinators' reports, letters to collegeprincipals regarding the outcomes ofmoderation, and course handbooks.
Assessment and Qualifications Alliance
page 17
87 In the examples of the validation andrevalidation processes presented, the reviewteam saw evidence of variability in practice.There were examples of rigorous and effectiveprocesses, with scrutiny of modules by subjectspecialists, and course approval by panels whichincluded external academic expertise. However,the team also saw examples of validations wherethe AVA did not follow its own procedures, forexample, in relation to panel membership andthe production of written reports. It was also thecase that, from the communications the teamsaw in some of the audit trail files, there was anabsence of standardised letters which wouldprovide clarity of communication, for example,formal letters of confirmation of approvalfollowing a course validation. It was not alwayspossible for the team to confirm, from thedocumentation presented, that validationconditions had been followed through. Also,there was minimal communication from thepartner colleges presented in the audit traildocumentation, suggesting an incompleteness of record-keeping.
88 The review team saw one example withinthe audit trails which gave rise to significantconcerns about the robustness of the AVA'sprocedures and the consistency with which itfollowed these. The validation panel had notbeen constituted in accordance with the AVA'sown procedures as laid down in its Access toHE Handbook with, in particular, no externalrepresentation on the panel. The team sawdocumentation confirming approval of theprovision in the AMG's minutes but, followingthe examination board for the first studentcohort in September 2005, the moderatormade recommendations that amendmentsshould be made to the programme. Themoderator's report was sent to the principal ofthe college with a covering letter in November2005. The team noted that, while themoderator report pro forma referred to 'futureaction required by the college', the coveringletter stated only that QAC 'offered thefollowing recommendations' while hoping thatstudent numbers would be sufficient to run thecourse again in the new academic year.Subsequent discussion in the AVA's QAC
evidenced concern that the college hadrecruited a further cohort of students in January2006 'without the course being approved',approval being related not to the initialvalidation but rather the requirementsconveyed to the college in the moderator'sreport. The team saw several subsequentcommunications sent to the college seekingclarification of whether it would be submittingrevised documentation to meet the AVA'srequirements, although no further cohort ofstudents had, in fact, been recruited. As late asJune 2006, the minutes of the AMG stated that'students had been enrolled on the un-validated(sic) course. The college had been written tomore than once but no reply had beenreceived'. The team subsequently soughtclarification from the AVA as to the currentstatus of the provision. The team was told thatthe college was no longer a part of the AVA'sapproved provision, and that this had comeabout because the college had not undertakenthe modifications to the programme that hadbeen required following the moderator's visit,but also because the college had been unableto recruit a further cohort of students. It wasconfirmed that the AVA did not have a formalprocedure, as such, for the withdrawal ofapproval. The team concluded that the AVAhad not made a distinction which wassufficiently clear to all parties, either in theseparticular circumstances or in its own formalprocedures, between: an approved course withconditions to be met; an approved course, butnot running; and a course where approval hadbeen formally withdrawn.
89 The AVA requires college providers topresent an annual report, and this is theresponsibility of the college coordinator. TheAVA does not currently specify a format for theproduction of coordinator reports and, whilethey generally covered key issues such as actionplans responding to moderators' reports, self-assessment of the provision and studentfeedback, there was some unevenness in thequality of reports presented in the audit trails.The AVA has recognised this as a weakness, andis proposing to introduce a new template forcoordinator reports (see paragraph 67).
Access to HE review report
page 18
90 Through the audit trails, it was possible totrack the transition from the original system ofexternal examiners and moderators to thecurrent system of team-based moderation. The audit trails provided confirmation thatmoderators fulfil their obligations, in some casesvisiting on more than the required number oftimes. The AVA has recently introduced newtemplates to support the twice-yearly reportswhich moderators are required to write. Fromthe examples the review team saw in the audittrails, this system appeared to be working well,and teams of moderators working in a specificcollege are taking the opportunity to producejoint reports on the college provision.
91 Overall, from the information provided bythe AVA in the audit trails, while the reviewteam saw examples of good practice in coursevalidation and revalidation and in moderation,it was not able to form a firm judgement aboutthe effectiveness of the quality assuranceframework and its application.
Conclusions92 AQA's position as a large national awardingbody provides it with obvious potentialadvantages for its role as an AVA. Not least, ithas very substantial experience in managingthe award of qualifications, and monitoring andmaintaining the academic standards of awards.It also has the structures and resources typicalof such an organisation, with extensive,specialist support services in a range of areas.Activity relating to Access to HE benefits fromsome of these services, for example, marketingand website development, financialmanagement, the development of informationmanagement systems to support theimplementation of the Access to HE Diploma,and the expertise available within theorganisation to provide sound analysis of theAVA's data in its annual report to QAA.
93 In some respects, an AVA may also bedisadvantaged by a situation in which Access to HE is a very small part of the organisation'stotal endeavour, and where the associatedactivity depends on a very small number ofindividuals, especially if that activity is outside
the main stream of the organisation's otheractivities. Clearly, there are dedicated,committed and able staff working in AQA'sAccess to HE section and in its partner colleges.The committees and moderators overseeing theactivity are also evidently committed toproviding a service for the benefit of Access toHE students. But AQA faces challenges, bothstructural and procedural, in ensuring thatmechanisms are in place to monitor anddevelop Access to HE activity and ensure thatequally rigorous procedures are followed toequally high standards for this, as for any other,part of its activity. It is a matter of somesurprise, and of particular concern, that thebenefits of AQA's awarding body experiencehave not always been evident in the qualityassurance of Access to HE, particularly inrelation to the key processes of coursevalidation and certification.
94 The last review report expressed concernsabout staffing levels and, in particular, aboutleadership and management for Access to HE.While the overall staffing complement hasincreased slightly, there have been recurrentchanges in staffing and management for Accessto HE in the period since the last review, withthe inevitable disruption and discontinuity tooperations that follow such change within asmall team. These changes have occurred at atime when major changes to the Access to HEqualification itself have occurred, as well aschanges in the AVA licensing requirements. Inthese circumstances, it is not surprising thatthere has been some slippage in the AVA'stimetable for the implementation of the Accessto HE Diploma. It is essential that this timetableis now reviewed and all necessary action takento ensure that all courses are consistent withthe specifications of the Access to HE Diplomaby 2008-09, if AQA is to be able to maintain its service to providers and students as alicensed AVA.
95 Although engagement in developmentactivity is now a regulatory requirement, it isnot foregrounded in AQA's work and littlenotable progress has been made since therevised licensing criteria were introduced.
Assessment and Qualifications Alliance
page 19
There are particular difficulties for AQA as itattempts to find an appropriate way of meetingthose AVA licensing criteria which require it tobe responsive to the development of regionalstrategies for widening participation in HE.However, while AQA describes itself as adoptinga 'flexible' approach, to provide for differentproviders' needs and demands, and respond todifferent opportunities at the local level, there islittle evidence to date that this has contributedto substantial or innovative development,regional or otherwise.
96 The last review report expressed concernsabout the AVA's governance arrangements.These have been addressed through thecreation of the Council Access Review Groupand a more direct and active link between theCouncil and the Access Management Group.These structures have clarified and confirmedthe Council as the appropriate locus ofresponsibility for the AVA licence, and improvedthe capacity of the Council to monitor Accessto HE activity. This has enabled Councilmembers to raise concerns, if they feel theneed to do so, and also to ensure that theCouncil is more fully informed about the AVA'swork and its responsibilities.
97 The continuing financial viability of Accessto HE as an area of work for AQA has been anatural and proper concern of the AQA Council.In its deliberations about the AVA's future, AQAwill, no doubt, consider the effectiveness ofmeasures taken to date, including any impacton the security of quality and standards, as wellas on the financial return from Access to HE. Inassessing any further potential for growth, theAVA may recognise three possibilities: attractingpre-existing provision from other AVAs;generating new provision in the areas in whichit is already active; or supporting developmentof provision outside its current area of activity.All three options are likely to require someinvestment of resources, for development andvalidation, if AQA is to be able to maintain thequality of provision and standards of the awardsit makes as an AVA.
The AVA licence
Review outcome
98 The review team recommends that AQA begranted a provisional renewal of its AVA licence,with conditions to be met by the datesspecified below and revisit in spring 2008.
99 The AVA's licence is restricted in relation tothe award of the Access to HE qualification,until Conditions iii and iv have been met toQAA's satisfaction. While the AVA may continueto award Access to HE certificates, it may notaward the Access to HE Diploma or indicate onits publicity that Access to HE Diplomas will beawarded until these conditions have been met.
The award of the Access to HE Diploma
Conditions
100The AQA AVA licence is renewed oncondition that it:
i carries out a review and analysis of itsstaffing arrangements for Access to HE, andreports on the outcomes of the review todemonstrate how it will ensure that itsresponsibilities as an AVA licence holder willbe met, including responsibilities for qualityassurance and development, actionsidentified in its action plan, actions relatingto the implementation of the Access to HEDiploma, and actions relating toimplementing the conditions of this report(paragraph 35)
ii reviews its approach to developing aregional strategy to ensure that it can meet its licence obligations in those regionsin which it operates, and reports, inparticular, on:
the regions in which it intends to operate,including any phasing of expansion
the means whereby it will engage and workcollaboratively with other organisations andinstitutions working to promote wideningparticipation in each region
Access to HE review report
page 20
its approach to the analysis of regionalpriorities and needs for Access to HEdevelopment
its approach to the promotion of Access toHE in each region
the role of its regional officers, regionalforums and bulletin boards in this strategy
the resources necessary to support the implementation of this strategy(paragraph 46)
iii reconsiders and revises its procedures forvalidation and revalidation and revise theAVA's formal documentation to clarify:
the role of standard AQA centre approvaland of the AVA's Memorandum ofAssociation, and requirements relating to these
the distinction between approval of acentre and the validation and approval ofparticular courses and pathways offered bya centre
composition and formal approval of thevalidation panel membership, and theprocess for ensuring that the panelmembership is appropriate for the range of courses and/or pathways which it is to consider
the point at which units are formallyapproved including the distinctive roles ofsubject specialists prior to the validationevent and the validation panel
matters to be addressed by a validationpanel, including requirements relating tothe Access to HE Diploma specification
expectations for the content and standardformat of the panel report, andresponsibility for its production
process for the confirmation of course andpathway validation
process and responsibility for monitoringconditions and recording that conditionshave been met and formally approved
processes for the formal withdrawal ofapproval, including notification to theprovider
the rationale for, and distinction between,different styles of validation event and thecriteria and process used to decide on thestyle for a particular event
how its proposed 'risk-based' approach to validation will be implemented(paragraph 62)
iv reviews and develops its plan for theintroduction of the Access to HE Diploma,to ensure that all its providers are able tooffer the Diploma from 2008 onwards, andto provide a revised schedule withconfirmed dates for all stages of validation,with particular reference to:
the revalidation schedule for existingproviders and for pathways
the incorporation of new providers into the schedule
the style of validation (provider or office-based) in each case
the resources necessary to implement thisplan, in particular
AVA officer support for providers
scrutiny of new and revised units
membership of, and officer support for,validation panels
the AVA's database of units
the training of moderators and subjectspecialists (paragraph 66)
v develops clear criteria, including curriculumexpertise, and process for the selection ofmoderators, including recommendation tothe QAC, or other appropriate committeefor appointment (paragraph 74)
vi implements the intention to makemoderator training mandatory (paragraph 77)
vii introduces a robust and effective system tomanage and record the issue of its Accessto HE certificates (paragraph 83).
Assessment and Qualifications Alliance
page 21
Conditions iii and iv to be met by 1 October2007.
Conditions i, ii and v-vii to be met by 1 December 2007.
Recommendations to the AVA
101The review team recommends that theAVA:
i specifies rules for the reappointment of members to the QAC when thecommittee's terms of reference are nextreviewed (paragraph 21)
ii reviews and updates job descriptions forstaff in the Access to HE section to ensurethat they reflect reality and, in particular,that reporting and line managementresponsibilities are clear (paragraph 30)
iii reviews its approach to strategic planningand makes explicit the articulation betweenplans and resources to support these plans(paragraph 39)
iv reviews its procedures for annual reviewreports from providers to ensure that that itmakes explicit how matters of concern areaddressed, how data on Access to HEcourses and students are formallyconsidered, the level of scrutiny by AMG,and how the AVA disseminates goodpractice identified as through annualreview, and to ensure that these proceduresare published to providers in the Handbookor in some other appropriate way(paragraph 67)
v develops an explicit policy statement toavoid situations of 'back-to-back'moderation arising (paragraph 72)
vi keeps under careful review the span ofsubject areas it expects moderators tocover, in order to ensure moderation ofstudent achievement is sufficiently rigorousand fair (paragraph 74)
vii incorporates a prohibition on subcontracting of duties in the letter of appointment for moderators(paragraph 75)
viii reviews the terms of reference of the QAC(or other appropriate body), in order toensure that responsibility for comparabilityof standards between providers is explicitlylocated within the AVA (paragraph 78)
ix codifies the process for the issue ofcertificates to students in the form of a set of written office procedures(paragraph 82).
Access to HE review report
page 22
Appendix
Aims and objectives of AVA review
The aims of the system of AVA review are:
i to provide the basis for an informed judgement by the ARLC about the fitness of the AVA tocontinue as a licensed agency
ii to promote public confidence in Access to HE as a properly regulated and respected route intoHE by assuring
the quality and adequacy of AVAs' systems and procedures
the quality, comparability and range of AVAs' operations
the adequacy and comparability of AVAs' standards for approval, moderation and monitoring ofprogrammes
consistency across AVAs in the operation of criteria for the granting of the Access to HE award
iii to stimulate reflective and self-critical perspectives within AVAs as an instrument to promotequality enhancement
iv to provide an opportunity to identify and disseminate good practice of AVA operations
v to provide a mechanism for ensuring necessary, and encouraging desirable, improvements anddevelopments in AVAs.
The objectives of each AVA review are:
i to examine, assess and report on:
the development of, and changes in, the AVA since its last review or initial licence, and its plansand targets for the future
the organisation's continuing viability and robustness and the ways in which the AVAdemonstrates sound governance
the efficiency and effectiveness of the AVA's operational and quality assurance systems
the range and scope of the AVA's activities, and the appropriateness and value of these activities
the ways in which the AVA approves and monitors programmes and the ways in which theseprocesses take account of the need for consistency and comparability
the ways in which the AVA satisfies itself of the adequacy and comparability of standardsachieved by students gaining the Access to HE certificate
the evidence available to indicate the AVA's success in achieving its aims and targets
ii to identify and report on:
strengths and good practice in procedures and operations
areas which would benefit from further development
areas requiring attention.
Assessment and Qualifications Alliance
page 23
RG
334 0
8/0
7
The Quality Assurance Agency for Higher Education Southgate HouseSouthgate StreetGloucesterGL1 1UB
Tel 01452 557000Fax 01452 557070Email [email protected] www.qaa.ac.uk
