What makes students satisfied? A discussion and analysis of the UK’s national student survey Article 

Accepted Version 

Bell, A. R. and Brooks, C. (2018) What makes students satisfied? A discussion and analysis of the UK’s national student survey. Journal of Further and Higher Education, 42 (8). pp. 1118­1142. ISSN 1469­9486 doi: https://doi.org/10.1080/0309877X.2017.1349886 Available at http://centaur.reading.ac.uk/68296/ 

It is advisable to refer to the publisher’s version if you intend to cite from the work.  See Guidance on citing  .

To link to this article DOI: http://dx.doi.org/10.1080/0309877X.2017.1349886 

Publisher: Taylor and Francis 

All outputs in CentAUR are protected by Intellectual Property Rights law, including copyright law. Copyright and IPR is retained by the creators or other copyright holders. Terms and conditions for use of this material are defined in the End User Agreement  . 

www.reading.ac.uk/centaur   

CentAUR 

Central Archive at the University of Reading 

Reading’s research outputs online

WhatMakesStudentsSatisfied?ADiscussionandAnalysisoftheUK’sNationalStudentSurvey

AdrianR.BellandChrisBrooksICMACentre,HenleyBusinessSchool,UniversityofReading,Whiteknights,

ReadingRG66BA,UK;tel:(+44)1183787809;

e-mails:A.R.Bell@reading.ac.ukandC.Brooks@reading.ac.uk

November2016

AbstractThispaperanalysesdatafromtheNationalStudentsSurvey,determiningwhichgroupsofstudents

expressedthegreatestlevelsofsatisfaction.Wefindstudentsregisteredonclinicaldegreesand

thosestudyinghumanitiestobethemostsatisfied,withthoseingeneralengineeringandmedia

studiestheleast.Wealsofindcontentmenttobehigheramongpart-timestudents,andsignificantly

higheramongRussellgroupandpost-1992universities.Wefurtherinvestigatethesub-areasthat

driveoverallstudentsatisfaction,findingteachingandcourseorganisationtobethemostimportant

aspects,withresourcesandassessmentandfeedbackfarlessrelevant.Wethendevelopamulti-

attributemeasureofsatisfactionwhichweargueproducesamoreaccurateandmorestable

reflectionofoverallstudentsatisfactionthanthatbasedonasinglequestion.

NOTICE:thisistheauthor’sversionofaworkthatwasacceptedforpublicationintheJournalofFurtherandHigherEducation.Changesresultingfromthepublishingprocess,suchaspeerreview,

editing,corrections,structuralformatting,andotherqualitycontrolmechanismsmaynotbe

reflectedinthisdocument.Changesmayhavebeenmadetothisworksinceitwas

submittedforpublication.

JELclassifications:C52,I21,I23Keywords:NationalStudentSurvey,studentsatisfaction.

AcknowledgementsWearegratefultoLisaSchopohlforexcellentresearchassistance.Wethankseminarparticipantsat

theUniversitiesofReadingandBathforinsightfulcomments.Wearegratefulforhelpfuldiscussions

2

withCherryBennett,MaxineDavies,NathanHelsby,EileenHyderandClaireMcCullogh.Wealso

thanktwoanonymousreferees,TonyMooreandJamesWalkerfordetailedcommentsonanearlier

draft.

1

1.Introduction

TheUKhighereducationsystemhasundergoneradicalchangesoverthepastdecade.Thefunding

formulahasalteredsignificantly,sothatthebulkofuniversities’incomesnowcomefromthefees

paidbythestudentsthemselvesratherthanfromagovernmentblockgrant.Theintroductionofthe

NationalStudentSurvey(NSS)intheUKhasintroducedaheightenedfocusonthe‘student

experience’(Gibbs,2010;2012),whichhasengenderedseveralimportantchangesinuniversities’

environmentsandmodusoperandi.First,studentsnowfeelasenseofempowermentaspaying

customersandtheyexpecthighqualityteaching,goodfacilities,andhighstandardsoforganisation

andofprofessionalismthroughouttheirexperience(Kay,DunneandHutchinson,2010).Second,

universitieshavebecomemorecorporateintheiroutlookandobjectives,entrepreneurially

establishingnewsubjectareasandprogrammeswiththeobjectiveofincreasingstudentnumbersto

generaterevenue(forinstanceseethetheUniversityofEastAnglia(UEA)Plan2016–20).1

Universitiescompeteinanincreasinglyinternationalmarketplacetoattracthighlyqualifiedstudents

(ChattertonandGoddard,2000),whointurnareincreasinglyawareoftherelativerankingsof

universitiesanddepartments,withthedangerthatsuchrankingsthenbecomeincreasinglyossified

andaself-fulfillingprophecy.Achievingahighandrisingpositionintherankingsisnowconsidereda

legitimate(andperhapsthemostimportantorevenonly)objectiveinitsownrightratherthanbeing

merelyapositivesideeffectofgoodperformanceonother,morespecificindicators.Consequently,

apoorerthanexpectedpositioningintherankingsislikelytoleadtoadmonishmentofdeansand

headsofdepartmentbyuniversityseniormanagers;theformerwillinturnpassontheir

disappointmenttotherankandfile,whoaretoldthatthingsmustimprove(Locke,2014).2Letcher

andNeves(2010)pointoutthatuniversitiesareinterestedinstudentsatisfactionfortwomajor

reasons:firstlyandpositively,thatitleadstogreaterretentionandacademicachievementbythe

studentsthemselves;andsecondlyandmoreselfishly,goodratingsofsatisfactionleadtogood

publicrankings,whichenableuniversitiestoenhancetheirprestige,recruitthebeststudentsand

fulfilltheirannualquotafornewstudents.3

Thepresentpaperaimstocontributetothedebateregardingtheextenttowhichstudentsare

satisfiedwiththeiruniversityexperiencebyconductingacomprehensiveanalysisoftheresultsof

theUK’sNationalStudentSurvey,completedbyover140,000undergraduatesin2014.Noother

surveyintheUKhassuchacomprehensivecoverageacrossboththesubjectandinstitutional

dimensionsandaswediscussedabove,theresultsfromtheNSSareofstrategicimportanceto

universitiessincetheyoccupysuchakeypositioninseveralrankings.TheexistenceoftheNSSdata,

whicharepubliclyavailable,providesauniqueenvironmentinwhichtoexaminestudentsatisfaction

onanationwidebasiscoveringallsubjectareastaughtbyeachuniversity.Weexaminesubjectand

regionaldimensions,andwealsofocusontheinterlinkagesbetweenthevariouscategoriesof

1https://portal.uea.ac.uk/documents/6207125/12506475/UEA+Plan+2016-2020_final.pdf/5ff04627-d120-4808-8fa1-196e57ca370d2Thisthenleadstoadditionalpressureonteachingstaffandincreasesthelikelihoodthatsomewillleave.Ofcourse,attritionfromtheteachingstaffisnotnecessarilybadifitinjectsadditionaldynamismandenergyintothefacultythrough“freshblood”,butthiswouldalsodependontheextenttowhichdepartingstaffweretheweakermembersoftheteamorthosewiththegreatestoutsideopportunities.Inaddition,researchsuggeststhatacademicsoftenworklonghoursforrelativelylowpay(Walkeretal.,2010).3Formanyreasons,recruitingmorestudentsfollowingaperiodofgoodNSSfiguresmayleadtogreaterdis-satisfactioninthefuture–notonlybecausetherewouldbemorecompetitiontosecurejobs,butalsosinceresourcesarespreadmorethinlyandclasssizesincreased.

2

questionswithintheNSS.Weexplorethisrichdatasourcealongvariousdimensionsandexamine

theextenttowhichscoresacrossthevariousareaswithintheNSSprovidecorrelatedassessmentsof

satisfaction.Finally,wealsoexamineamulti-attributeapproachtocalculatinganaggregaterating,

whichcouldbeusedtoreplacetheresultsfromtheoverallratingquestion,producingmorestable

andreliableestimatesofstudentsatisfaction.

Theremainderofthispaperisorganizedasfollows.Section2discussestheexistingliteratureonthe

driversofstudentsatisfaction,whileSection3presentstheNationalStudentSurveyandconsiders

thedebatesurroundingtheuseofsuchsurveysmoregenerally.Section4movesforwardsto

examinetheNSSdatafromvariousanglesandfinallySection5reflectsontheimplicationsofour

findingsandconcludes.

2.Whatarethedriversofstudentsatisfaction?

Theexistingevidenceintheacademicliteratureregardingstudentsatisfactionissomewhatsparse,

geographicallyverywidelyspreadandmainlyfocusedontheindividualstudentlevel.Thereare

severalexistingstudieswhichcovervariouspartsofouranalysis,however.Arelativelyearlypieceof

researchisthereportfortheHigherEducationFundingCouncilforEnglandbySurridge(2008).She

conductsacomprehensiveanalysisoftheearlyyearsoftheNSS,examiningthedriversofstudent

satisfactionforthe2005-07cohorts,includingstudentcharacteristics,broadsubjectareasand

severalinstitution-levelvariables.

Fieldingetal.(2013)conductananalysisofthe2006and2007NSSdataforninesubjectsinthe

scienceandengineeringareas.Theyusevariousstratifiedapproachestocaptureandcompare

satisfaction,reachingwide-rangingconclusionsbutcautioningonthedangersofcomparingbetween

subjectsandinstitutions.Hewson(2011)conductsananalysisoffourquestionsfromtheNSSfor19

subjectareasusingaBayesianMarkovChainMonteCarloapproach,highlightingsignificant

differencesbetweenlevelsofsatisfactionbysubjectareaforeachquestion.Healsoemphasisesthe

sample-selectionproblemsthatmayexistwherecertaingroupsofstudentsmaybemorelikelyto

completethesurveythanothers,thuspotentiallybiasingtheresults,althoughthisissuehasnot

beeninvestigatedfurtherinsubsequentwork.

Whenstudentsevaluatethequalityofacourseandreflectontheiroverallsatisfaction,they

undergoacognitiveprocessinwhichtheycomparetheirpriorexpectationsaboutthequalityofthe

deliveryandoutcomeswiththeirperceptionofthecorrespondingactualperformanceand

outcomes(Zeithamletal.,1993).Notonlycognitivefactors,butalsoemotionscanaffectsatisfaction

surveyresponses(Oliver,1981).AsMavondoetal.(2004)note,studentsarevariouslyseenas

customersreceivingaservice(Guolla,1999),asco-producersofknowledgewhoshareresponsibility

fortheirlearningwiththeiruniversity(Armstrong,1995)andasproducts(Guolla,1999)whichthe

universitythen‘places’inthejobmarket.Clearly,theperspectivefromwhichstudentsview

themselveswillinfluencehowtheyevaluatetheirsatisfactioninanessentiallyimmeasurable

fashion.

AsElliottandShin(2002)putit,“Studentsatisfactionisbeingshapedcontinuallybyrepeated

experiencesincampuslife.Moreover,thecampusenvironmentisseeminglyawebof

interconnectedexperiencesthatoverlapandinfluencestudents’overallsatisfaction.Whathappens

3

tostudentsintheclassroomisnotindependentofallotherexperiencesofcampuslife”(p.198).

Therefore,thequalityofthecourseandaspectsofthecurriculum(Browneetal.,1998),aswellas

thecampusenvironment(Borden,1995),areallarguedtobedriversofsatisfaction.4InaUSstudyof

undergraduatebusinessstudents,LetcherandNeves(2010)findthatthequalityofteachinginthe

specificsubjectmatterhaslittleornoeffectonstudentsatisfaction.Insteadtheyfindthatother

factorshaveagreaterimpact,includingself-confidence,extracurricularactivities,careers,andthe

generalqualityofteaching.Thestudyusesastudentevaluationformattoinvestigatewhatis

importanttoundergraduateswhenrecordingsatisfactionwiththeirprogrammesanduniversity

experience.ThereisaninterestingconundrumthatLetcherandNevesalsodiscuss,namelythat

otherresearchhasestablishedacademicperformancetobekeytostudentsatisfaction.Itisfairly

intuitivethatstudentswhoareperformingwellandreceivingpositivefeedbackwillrateamodule

morehighly.Butcausalitycouldgoinbothdirections:evenmoreintriguing,theyquotePike’s(1991)

conclusionthatsatisfactionexertsagreaterinfluenceonexamperformancethanacademic

performancedoesonstudentcontentment.Thushappystudentsperformwell,asopposedtohigh

performingstudentsbeinghappy.5

Studentswhoarereflectingonthequalityoftheeducationthattheyperceivetheyhavereceived

maybringawholehostofincidentalfactorsintotheevaluationprocess.Further,Merritt(2012)

studiesgeneralbiasesinevaluationsrelatedtoLawschoolsandstates,‘Thewayinwhicha

professorwalksintotheroomorsmilesattheclasscanaffectstudentratingsmuchmore

substantiallythanwhattheprofessorsaysorwritesontheblackboard’.Hecitesafamous

experimentbyNaftulinetal.(1973),‘TheDrFoxLecture’,whereanactordeliveredalectureof

nonsensebutwitha‘warmmanner'(pp.242and239),receivingglowingevaluations.

Insummary,theliteraturecitedinthissectionhasillustratedthediverserangeoffactorsthataffect

studentsatisfaction,butalsothatmostoftheexistingevidenceisbasedonsmallsamples,single

subjectsurveysorfocusgroups.Ourapproachisquitedifferentinthatweuseaverylargemulti-

disciplinedatabasewhichfacilitatesquantitativecomparisonsofsatisfactionbyvarious

categorisations.Weoutlinethepropertiesofthisdatabaseinthefollowingsection.

3.TheNSS–ADescriptionandDiscussion

TheNationalStudentSurvey(NSS)isaquestionnaire-basedmeasureofstudentsatisfaction

establishedin2005atthebehestoftheUKgovernment.Thesurveycanbetakenbyallthirdyear

undergraduatesatHigherEducationInstitutionsinEngland,Wales,NorthernIrelandandScotland.

TheaimsoftheNSSatthepointofitsestablishmentweretoauditthequalityofcoursesrunbyHEIs,

tomakethemmoreaccountableforquality,andtosupportthedecision-makingoffutureuniversity

applicants.6TheNSSisacumulativemeasureofsatisfaction(ParkerandMatthews,2001)thattakes

placetowardstheendofastudent’sexperienceandinvolvesrespondentsbalancingalargenumber

4Poorsatisfactioncanresultfromamismatchbetweendeliveryandexpectationsinanyoftheseareasbutithasbeenarguedthatsupportservicesarecommonlyperceivedaslesssatisfactorythantheacademicaspectsofthecourses(KotlerandFox,1995).5SeealsoDouglas,McClellandandDavies(2008)whoinvestigatestudentsatisfactionusingaCriticalIncidentTechniqueinordertodevelopanewmodel,arguingthat‘servicequalityisaprecursortostudentsatisfaction’(p.21);andArambewelaandHall(2009)whofoundthatbotheducationalandnon-educationalfactorsweresignificantvariablesinexplainingstudentsatisfaction.6HSwain,‘Ahotchpotchofsubjectivity’TheGuardian,19May2009.

4

offactorstoarriveatspecificsatisfactionmeasuresforeachcategoryofquestion.TheNSS

comprisesatotalof23questionssplitintosevencategories(theteachingonmycourse;assessment

andfeedback;academicsupport;organisationandmanagement;learningresources;personal

development;andfinallyoverallsatisfaction).Eachquestionasksstudentstoreflectontheir

experienceandrespondonaLikertscale(1-5)from“Verydissatisfied”to“Verysatisfied”.7These

Likertscaleresponsesarethendistilledintoabinaryvariableindicatingwhetherstudentsare

satisfied/verysatisfiedorunsatisfied.

Whileitisincreasinglybeingusedasapolicyinstrumentforbringingaboutchangesthatenhance

thestudentexperienceandasameansofcompetitioninrankingstables,studentsatisfaction

surveysingeneral,andtheNSSinparticular,havebeencriticisedonanumberofkeygrounds,both

philosophicalandoperational(Sabri,2013).Aparticularissuestrikesatthecoreofthepurposeof

thestudy–namelywhetherstudentsthemselvesareinthemostappropriatepositiontoassessthe

qualityoftheireducation,inparticularwhiletheyarestillstudyingandthesurveyresponseswill

probablybeweightedmoretowardsstudents’mostrecentexperienceswhichareattheforefronts

oftheirminds.Anentirelydifferentapproachwouldbetoaskalumnithisquestionseveralyears

aftertheygraduate(forinstanceaspublicationssuchasTheFinancialTimesdoestoinformBusiness

Schoolrankings)andhavehadtimetoreflectonwhetherwhattheylearnedwasusefulratherthan

merelyentertaining.Moreover,onecouldquestionwhetherstudentsareinfactthebestplaced

stakeholderstocommentonthequalityoftheireducationatUniversity.

Somedangersofun-scientificandtop-downmetricsofstudentsatisfactionarepointedoutby

Gruberetal.(2010)whodiscussanewmeasurementtoolforstudentcontentment.Theirstudywas

motivatedbythedecisiontointroducetuitionfeesinGermanuniversities(sincereversed)asthe

authorsbelievedthatinstitutionswouldnowhavetotreattheirstudentsascustomers.Theyfelt

thattheUKwasaleaderinthisareawhereasGermanyhadnotpaidattentiontoeithermeasuring

ortryingtoimprovethestudentexperience.Theypointoutthatduetotheuniquenatureofhigher

education,servicequalitycannotbemeasuredobjectively(p.107).Theyrightlyexplainthatthe

differentstakeholders–students,governmentandprofessionalbodies,forinstance–havevery

differentmeasuresofquality.Theyalso'regardservicequalityasanantecedenttosatisfaction'(p.

108).Intuitively,theydevelopthestudywiththebeliefthatuniversitiescanonlysatisfytheir

studentsiftheyknowwhattheyactuallywantratherthanbasingtheservicedeliveryonwhatthey

perceivethatstudentswant(p.108).Thismaysoundratherobvious,butGruberetal.referto

studiesdemonstratingthatacademicsandadministratorsprefertorelyontheirownviewofwhat

studentsneed.Thestudyactuallyfindsthatstudentsatisfactioniscorrelatedtothesatisfactionwith

lecturers,universityfacilityquality,andtherelevanceofteachingtopractice.Theirpaperdescribes

anexperimentwithtwosamplesofstudentsfromoneUniversityinGermanyandtheauthorsadmit

7Throughoutthispaper,weusearangeofparametricstatisticalapproachestotheanalysisofstudentsatisfaction.However,itisimportanttonotethatstrictly,themeanofaLikertscaleisundefinedandthereforeoneshouldinterpretthestatisticalinferencesundertakenonthesevariableswithcaution.Weobserve,though,thatacomparisonofthemeansandofthemedianspaintsaverysimilarpicture.Inaddition,ourunitofanalysisisthecourseorcoursecollectionataparticularuniversityandnottheindividualstudentscores.Assuch,allofthesatisfactionmeasuresweemployinthedatabasearecontinuousandnolongeronaLikertscaleastheyareaverages,albeitboundedbetweenoneandfive.

5

thereforethattheresultsarenotevenrepresentativeofthewholestudentpopulationinthat

country.

Methodologically,thevalidityoftheNSSasameasureofthestudentexperiencehasbeen

questioned(Yorke,2009).Thus,fromapracticalperspective,theNSShasnotbeenwithoutitscritics

regardingseveralaspects.First,thereisevidencethatuniversitiesmaybeproneto‘cheat’the

scores.8Second,ithasbeenarguedthatthesatisfactionscoresof‘thevastmajorityofinstitutions

fallwithinanarrowrangethatiscoveredbysamplingerror’andinsuchcircumstancestherankings

thatarebasedonthesurveywillhavelittlemeaning.Third,theneedto‘keepthecustomershappy’

mayengenderafallinstandardswherestudentsarespoon-fedandmarkingisundulylenientin

ordertoraisethescorestheeasyway.This,dovetailedwiththeheightenedemphasison

transferrableskillsandemployability,mayencourageuniversitiestoincreasinglyactastraining

colleges(TaylorandMcCaig,2014)attheexpenseofthedevelopmentofdeeperintellectualand

analyticalabilities.

ThereisthuslikelytobeconsiderablesamplingvariationinaverageNSSscoresforeachindividual

coursefromoneyeartothenextsinceeachgroupofstudentswillcompletethesurveyonlyonce.

Thereisalsoevidencethatstudentsmaymisinterpretquestionsorbelievethatspecificquestionsdo

notapplytothemgiventheirsubjectareaandthenatureoftheteachingtheyreceive(Yorke,2014,

p.557).Thusperceptionsofquality,andthereforerelativerankingpositions,mayvaryerratically

fromyeartoyeardespitethecoursestructure,teachingfaculty,facilitiesandassessment

approachesremainingostensiblythesame.9Itisclearthatwhilethescoresarecertaintovaryfrom

onecohorttoanotherforaspecificprogrammewithinagivenschool,academicsfearthattheir

institutionalmanagerswillexpectthemtoriseyear-on-year.

TheresultsfromtheNSSwereidentifiedasearlyas2007asbeingimportantforstudentselection

(AsthanaandBiggs,2007;Hewson,2011),andindeedthisisoneofitscorepurposes,althoughthe

overallrankingofinstitutionsismoreinfluentialforapplications(Gibbonsetal.,2015).Yetthereisa

cleardangerthattheinformationcontainedwithinthesurveyfindingsislikelytobeconsumedina

veryundiscerningwaybyprospectivedegreecourseapplicants,whohavenodetailonthecontexts

ofhoworwhyaparticularsetofsatisfactionscoresarose,andcouldasaresultmakeworsesubject

orinstitutionchoicesexpostthantheywouldhavedoneintheabsenceofanysatisfactioninformation.Anyattemptbytheinstitutionconcernedtoexplainorjustifylowratingsfora

particularcoursewillbesummarilydismissedasweakexcusesorsourgrapes.Theexistenceofthe

surveyisevenarguedtohavefundamentallychangedthestudent-teacherrelationship(Gornalland

Thomas,2014)andwithitstudents’notionsofwhatauniversityisfor(Collini,2012).

ItappearsthatthefutureoftheNSSitselfiscurrentlyupfordiscussion.10Itisexpectedthatthenew

TEF(TeachingEvaluationFramework)willsomehowhaveanaspectofstudentsatisfactionmetricto

8ItwasreportedthatLondonMetropolitanandKingstonUniversitiesmanipulatedthescoresattheirinstitutions–seeL.Harvey,‘Jumpingthroughhoopsonawhiteelephant:asurveysignifyingnothing’TimesHigherEducation,12June2008.9Naturally,UniversityteachersandadministratorsmayrespondtoapoorsetofNSSscores,althoughiftheconcernsofthedepartingcohortarenotconsistentwiththoseofthenewcohort,thereisadangerthatsuchintroducedchangesmaynotbewelcome.10TimesHigherEducation,23July2015.

6

feedintoitandevidencefromtheNSSwasusedtoinformthecurrentGreenPaperdiscussed

earlier.ThedirectionofthecurrentdebatesuggeststhattheNSS’daysinitscurrentformare

numberedandinsteadstudentengagementshouldbeafocusofthequestionnairealongside

studentsatisfaction.AreviewoftheNSSbyHEFCEfoundthatthesurveydoesnottakeaccountof

studentengagementwithlearning,andrecommends11newNationalStudentSurveyof

Engagement(NSSE)-stylequestionsby2017.11Areportentitled‘DimensionsofQuality’byGraham

Gibbsarguesthatqualitycangaugedbyvariousmeasuresofclasssize,teachingstaff,theefforts

studentsmakeandthequalityoffeedbacktheyreceive.Relatedly,the‘HEAUKEngagementSurvey’

waspilotedin2013andlargertrialtookplacein2014.YetthebroadprinciplesbehindtheNSShave

receivedstrongsupportbytheUKgovernment,whichhasarguedthatithas‘goodinternal

consistency’and‘doesnotneedradicalalterations’.12

TheStudentAcademicExperienceSurvey(SAES)isacomplementtotheNSSwhichasks

fundamentallydifferentquestions.TheSAESwasintroducedin2006toexaminetheimpactof

increasedfeesonstudents’perceptionsandpriorities(Buckleyetal.,2015).Thesurveyisnow

conductedannuallyandisrunjointlybytheHigherEducationPolicyInstituteandtheHEA.In2015,

around15,000studentscompletedthesurvey,representingamuchlowerresponserate(22%)than

thatachievedbytheNSS.WhilewedonothavedatafromtheSAES,itsfindingshaveinformedand

motivatedourresearchquestions.

Althougheachstudenthas(almostinvariably)onlyoneundergraduateuniversityexperience,and

maymaketheirjudgementsonadifferentbasisandusingdifferentordinalvaluesforagiven

receivedlevelofquality,thefactthateverystudentfacesthesamequestionsandmustprovide

answersonthesamescalemeansthatcomparisonsacrossfieldsandinstitutionsarepossiblewithin

theNSS.Weshouldnote,however,thatitmightbethecasethatcertainsubjectshaveanapproach

toteachingthatismorecloselyalignedwithwhattheNSSmeasures,whichwouldmake

comparisonsacrosssubjectshard(seeGibbs,2010,p.46).Thiscouldevenextendtocross-university

comparisonduetothedifferingsubjectmixeswithinthem(Fieldingetal.,2010,p.347).Many

criticismshavebeenlevelledattheNSS,butmostaregenerictoanymethodofgaugingstudents’

viewsoftheiruniversityexperienceandwithinitsowngenre,itcouldbeviewedasacomprehensive

androbustbarometerofstudentsatisfactionandthereforeworthyofquantitativeanalysis.

4.AQuantitativeAnalysisoftheResultsfromtheNationalStudentSurvey

4.1Howdoessatisfactionvarybysubjectarea?

Weemploydatafromthe2015NationalStudentSurvey.13Tables1and2summarisetheoverall

satisfactionmeasuresonthe1-5andpercentageofstudentssatisfiedscalesseparatedbyHigher

11TheNSSEisaUSinitiative,similarinspirittotheNSSbutfocusedonasmallernumberofspecificsubjectareas.TheNSSEasksalargernumberofmorepenetratingquestionsthantheNSSconcerningtheextenttowhichstudentshaveputeffortsintotheirstudiesandtheopportunitiestolearnthathavebeenmadeavailabletothem.12‘UKreviewofinformationabouthighereducation–NationalStudentSurvey–Aliteraturereviewofsurveyformandeffects,byDELNI,HEFCE,HEFCWandSFC,2015.13WeobtainthedatafromthePlanningOfficeatourownuniversity,buttheyarealsopubliclyavailablefromtheHigherEducationFundingCouncilforEngland’swebsite:http://www.hefce.ac.uk/lt/nss/results/2015/.

7

EducationStatisticalAgency(HESA)subjectarea.14Itisclearthatwhilethereisveryconsiderable

variationacrossindividualcourses,whenaggregatedacrossuniversitiestothesubjectarealevel,

thereismuchlessvariation.Studentsinthemediastudiesareaaretheleastsatisfiedofallon

average(scoreof4.02ona5pointscaleand79.77%ofstudentssatisfied),followedbygeneral

engineering(4.09and82.43%),whilestudentsofclinicaldentistry(4.65and96.38%)andthen

veterinaryscience(4.44and92.88%)arethemostcontented.Itwouldbetemptingtoconcludethat

highlyspecialised,vocationalcoursestendtobemostpopularwhilethosewhicharemuchbroader

inscopewithnoobviouscareerpatharelessfavoured;yettheaverageevaluationsintheclassics,

philosophyandtheologyfieldsarealsoveryhigh.Ingeneral,itthereforeseemsthatstudentsprefer

subjectswherethereisacorrectanswerratherthanhavingtoengageinadebate.Thisdisparityin

satisfactionmightadditionallyrelatetotheattractivenessofstudyingatthetypesofuniversities

thatofferthesesubjects(forexampleifclassicsisonlytaughtatpopularuniversitieswithgood

facilities),butthefindingisalsosuggestivethatamorenuancedexplanationofstudentsatisfaction

isrequiredthatexaminesarangeofpotentialfactorsasweattempttodointhisstudy.Almostno

existingresearchaddressesthisquestionwithwhichwecancompareourfindings.FocusingonArt

&Designspecifically,York,OrrandBlair(2013)alsousetheNSSandrevealthatitreceivedlower

evaluationsthanmostothersubjectsin2012,afindingwhichYorke(2014)associateswiththe

disciplinehavingadisproportionatelylargenumberofpart-timeteachingstaff.Althoughwealso

findthatthissubjectranksatthelowend,theothersthatwehighlight(e.g.,mediastudies)perform

evenworse.

ConsideringtheshapeofthedistributionsofsatisfactionwithineachsubjectareainTables1and2

revealssomefurtherinterestingpatterns.Therearesomespecificcoursesreceivingverypoor

evaluationsincludingthoseinnursing(minimumscore2.6,minimumjust29%ofstudentssatisfied),

healthandcommunitystudies(minimum2.4and24%),andelectricalengineering(2.5and17%);

theseindividualverylowevaluationsincreasedthespreadsofscoresandofpercentagessatisfied.At

theotherendofthescale,mostcourseareashadatleastone(measuredbythemaximum)or

several(measuredbythe95thpercentileofthedistributionofscores)coursesthatwereableto

achievea100%satisfactionrate.

4.2Istherearegionaldimensiontostudentsatisfaction?

InTable3wepresentsummarymeasuresforsatisfactionseparatedinto12regions.Itisplausibleto

expectdifferencesinsatisfactionlevelsacrossregions,arisingbothfromdifferencesinrelativecosts

oflivingbutalsoasaresultofthedifferentkindsoflifestylethateachregionmayoffer.Home-based

studentsinScotlandwillalsonotbepayingtuitionfeesastheywouldiftheystudiedsouthofthe

borderanditmightbethatthisenhancestheirfeelingofwell-being.Thefiguresinthetableindicate

astatisticallysignificantdifferenceintheaveragelevelofsatisfactionacrosstheregions,although

theabsolutemagnitudesofthedifferencesintheLikertscalemeasuresaremodest.Theresults

suggestthatthosestudyinginNorthernIrelandarethemostcontented,withover90%ofstudents

satisfiedwiththeircoursesoverall;attheotherendofthescaleareLondoners,whoaretheleast

satisfied.HerewecanassumethatwithinpartsoftheUKthemajorityofstudentsarehome,

14TheNSSrawdataincludes108separatedegreesubjectclassifications,whichisnotmanageableforanalysisandwethereforecollectthemintocognategroupingsaccordingtothe45‘costcentres’reportedbyHESA.

8

whereasinLondon,wecanexpectstudentsfromallovertheUK.Therefore,localstudentsin

NorthernIrelandpayjust£3,805(studentsfromtherestoftheUK,£9,000;fromotherpartsofthe

EU£3,805).15BelfastisalsothemostaffordablecityforlivingwithintheUKaccordingtothesame

source.Therearethereforetwopossibleenvironmentalfactorsthatdrivetheseresults,asstudents

maythinktheyarereceivinggoodvalueformoneyandalsotheycanhaveareasonablestandardof

livingontheavailablestudentloan.TheLondonresultmaybedrivenfromtheoppositedirection,

withmosthomestudentspaying£9,000feesandfindingithardtomakeendsmeetinanexpensive

placetolive.Inaddition,theirresidencemaypotentiallybealongtuberideawayfromthestudy

buildings,leadingtoafeelingofisolationfromtheirclassmatesandalackofcohesionwiththe

schoolandotherprogrammeparticipantsandstaff.16Indeed,inarecentsurveyLondonwas

reckonedtobeoneoftheworstplacestoliveintheUKbecauseofitshighrentsandlongworking

hours.17StudentsinLondonmayfindithardertobecomeinvolvedinrecreationalactivitiessuchas

sportswiththeircohort,andthismaynegativelyaffecttheirwell-being.18Inaddition,therewillbe

moreinternationalstudentswithintheLondonstudentbodywhoaredrawntostudyinginthe

capital,anditmaybepossiblethattheyevaluatetheirexperiencedifferentlyandperhapsmore

critically.

4.3Whichaspectsoftheirprovisionarestudentsmost(dis)satisfiedwith?

Ironically,whilethefocusofthegovernment(asdemonstratedthroughtherecentGreenPaperand

otherreports)appearstobesquarelyonuniversityteaching,thisaspectofdegreeprovisionappears

tobeworkingverywellandgeneratinghighlevelsofsatisfaction.Nationally,91%(4.2ona5-point

Likertscale)ofstudentsaresatisfiedthatstaffaregoodatexplainingthings,while89%(4.3/5)

agreedorstronglyagreedthatstaffareenthusiasticaboutwhattheyareteaching.Ontheother

hand,thescoresaremuchlowerforassessmentandfeedback,withonly69%(3.8/5)ofstudents

agreeingorstronglyagreeingthatfeedbackwaspromptand68%(3.8/5)agreeingthatfeedbackhad

helpedclarifymattersthattheyhadnotpreviouslyunderstood.Courseorganisationwasalsorated

relativelypoorly(77%satisfied;4.0/5),aswastheStudents’UnionorGuild(68%satisfied;3.8/5).19

Itispossiblethattherearespillovereffectsbetweenthecategoriesofquestionsusedinthesurvey

sothat,forexample,aparticularlybadexperiencewithaccommodationorevenatrivialmix-upwith

roombookingsataninopportunemomentclosetothesurveycompletiondatewillengendera

jaundicedviewoftheentireeducationalexperience,irrespectiveofhowgoodthequalityofcourse

deliveryhadbeen.20LandandGordon(2015)arguethatstudentsurveysunhelpfullyencourage

respondentstoconflateservicesatisfactionwithteachingexcellenceand‘drawheavilyonthelogic

15http://www.qub.ac.uk/home/StudyatQueens/UndergraduateStudents/FeesandFunding/.16Interestingly,however,theintroductionof£9000feesdidnotreducethenationwidepercentageofsatisfiedstudents,whichremainedat86inboth2014and2015(‘NSS2015:£9ktuitionfeesfailtodentsatisfaction’byC.Haveral,13August2015,TimesHigherEducationno.2216).17http://www.telegraph.co.uk/finance/economics/11976033/Forget-London-these-are-the-best-places-to-live-in-the-UK.html.18SeeGilman(2001)foradiscussionofthepositiveeffectsofrecreationandsportsonhighschoolstudents.19IntheUK,theStudents’Unionhasadualrole–bothasapoliticalcampaignerandactivistforstudentissueslocallyandnationally,andsecond,asanorganiserandvenueforstudentclubs,societiesandentertainment.20Thereverseproblemwherestudentsfocusonanarrowrangeofinformationwhenansweringthesurveyquestionshasbeentermed‘cherrypicking’–seeCallenderetal.(2014).

9

ofstudentasconsumer’(p.21).TheyproposethattheNSSmoreformallyseparatesthetwoissues

intoindividualsections.Thus,asthingsstand,thesatisfactionmeasuresoughttobeviewedas

generalindicatorsofoverallhappinesswiththeprovisionratherthanspecificcategorised

viewpoints.TheveryexistenceoftheNSShasfosteredandencouragedasenseofstudentactivism

sinc,thesurveycanbeusedasaweaponofrevengeforanydisaffectedstudentswhofeelthattheir

experiencehasfallenshortofexpectations.

Tables4and5presentPearsoncorrelationsbetweentheresponsestoeachsectionintheNSS,

employingtheLikertscaleandpercentagesatisfiedmeasuresrespectively.Asthetablesshow,and

asonemightexpect,theresponsestoeachsetofquestions,averagedacrossallstudentsonall

coursesandatallinstitutions,arehighlypositivelycorrelated,whichconfirmsasimilarfindingby

Fieldingetal.(2010)usingourupdatedsampleandwiderrangeofcourseareas.Forexample,the

correlationbetweenthescoreson‘teachingonmycourse’and‘learningresources’is0.645despite

theobviouspossibilitythatexcellentteachingcouldtakeplaceinsideadilapidatedpre-warbuilding.

Inthiscontext,theresourcesexplicitlyreferredtointhequestionsrelatetolibraryresources,IT,and

specialistresourcesrespectively.The‘personaldevelopment’scoreisalmostashighlycorrelated

with‘academicsupport’(nearly0.8)andwith‘organisationandmanagement’(0.75)asitiswith

teachingquality(0.83).ThecorrelationsareuniformlyslightlylowerinTable4,sinceconstructing

thesatisfactionvariableinthiswaydoesnottakeintoaccountstrengthoffeelingbecausethe

methodofconstructionofthisvariableeffectivelytranslatesitintoa0-1asstudentsareeither

satisfiedortheyarenot.ThenumbersinTable3arenonethelesshighagain,indicatingthat,byand

large,atthelevelofacourseorgroupofcourses,studentsareeitherhappywitheverythingor

unhappyacrossarangeofmeasures.Theonlyexceptionsarepresentinthefinalcolumnsofboth

tables,whichmeasurehowcontentedrespondentsarewiththeirStudents’UnionorGuild,and

whilestillalwayspositive,thisvariablehasmuchlowercorrelationswithothermeasures–typically

oftheorder0.25.

FollowingLanganetal.(2013),21Table6presentsthefindingsoftwoexploratoryregressionsto

determinewhichofthespecificareaswithintheNSSquestionnairestudentsappeartofocuson

whentheyanswerquestion22toexpresstheiroverallsatisfaction.Thisissueisofparticularinterest

since,asdiscussedabove,theoverallsatisfactionscoreistheonethatformsthebasisofmostofthe

rankingmeasures,andwhichismostdiscussedinthemediaandmosthighlightedonuniversityweb

sites.Theacademicaspectsofprovisionaretypicallywellreceived,butsupportservices(e.g.

careers)maybelesssatisfactory(KotlerandFox,1995)andstudentsmayevenconflatetheir

satisfactionwiththeirsociallivesintotheiroverallrating(ElliottandShin,2002).

TheregressionsthatgiverisetoTable6areconductedonall4465course-universitycombinationsas

describedabove;thedependentvariableistheoverallsatisfactionmeasure–eithertheLikertscore

(middlecolumn)orthepercentageofstudentswhoaresatisfied(right-handcolumn),whilethe

independentvariablesarethescoresforeachofthesections.Thisisalegitimatespecificationand

notatautologysincetheoverallsatisfactionscoresarisefromaspecificquestioninthesurveyand

arenotdirectaggregatesofthecomponentscores.Itisclearthatallofthecomponentscoresare

highlysignificantly(atthe0.1%levelorevenlower)andpositivelyrelatedtooverallsatisfaction,

21UnlikeLanganetal.,however,weretainOrdinaryLeastSquareswhereastheyusethemuchlesswellknown“randomforestsanalysis”approach.

10

althoughrespondentsseementirelyunfazedbythequalityorotherwiseoftheirStudents’Union,

whichisnotsignificantevenatthe10%levelandhasparametervalueswhichareseveralordersof

magnitudelower.Intermsoftheotherspecificsectionscores,studentsappeartoputmost

emphasisonteachingquality(asFieldingetal.,2010,p.360,noted),followedbyorganisationand

managementandthenpersonaldevelopment,whenarrivingattheiroverallrating,withthe

parameterestimateforthefirstofthesebeingmorethandoublethatofthelattertwo.Universities

mayberelievedtonotethat,whilestillhighlystatisticallysignificant,themagnitudesofthe

parametersonassessmentandfeedback,wheresatisfactionscoresaregenerallylower,areatenth

ofthatonteachingeffectiveness.22Similarly,theparameterestimateonlearningresourcesisof

smallsize,suggestingthatitplaysaminimalroleinoverallhappinessdespitethehugesumsthat

universitieshavespentoninfrastructure,whichsomehavedismissedasvanityprojects,inrecent

years.23ComparingourresultstothoseofLanganetal.(2013),weobservebroadlysimilarfindings

asfortheir(longer)three-yearsamplecoveringanarrowerrangeofsubjectareas,buttoagreater

extent.Specifically,weobserveanevenstrongerroleforteachingandevenweakerforassessment

andfeedback.

InTable7weproceedtosummarisethemeansatisfactionscores(1-5scale,lefthandpanel)andthe

percentagesatisfied(righthandpanel)splitinapairwisefashionforanumberofsub-samples,also

presentingWelch’st-testsofthedifferencebetweenthemeansofthetwosub-samples,allowingfor

unevensamplesizesandnotassumingequalvariances.Part-timestudentsaremoresatisfiedthan

theirfull-timecounterparts,sincetheirmeanscoreishigher,asistheaveragepercentageof

studentssatisfied,althoughthedifferencebetweenthetwogroupsisnotstatisticallysignificant(t-

value:-1.1,p-value0.3fortheaveragescoreandt-value:-0.8,p-value0.4forthepercentage

satisfied).Thiscouldbeexpectedasmanypart-timeundergraduatestudentsareeithermature

students,orstudentswhoenteruniversityfromanonA-levelroute.Suchstudentshavedifferent

motivationsfromthosewhohavemovedstraightintouniversityfromschool-perhapsseeingaccess

toUniversityasamajorlifelongachievementwhichthenimpactsontheirownstudentexperience.

Doinstitutionswithaneliteimagealwaysobtainhighsatisfactionscores?Aquickreviewofthe

2014NSSdataattheaggregatelevelshowsthatOxfordranked15th,Cambridge31standtheLSE

135th.Howcanthisbeexplained?24Eliteuniversitiesdonotchargehigherfeesthantheremainder

ofthesector,asalmostalluniversitieselectedtosetthematthecappedvalueof£9000forhome

undergraduatestudents.CouldwebefacingtwodifferentapproachestoimagewithinUK

universities?Underthisimageconstruct,someoftheleadinginternationallyrecogniseduniversities

intheUKareconcentratingontheirresearchbrand,potentiallytothedetrimentoftheirinterestin

studentsatisfaction,asthismeasuredoesnotimpactonthesetypesofinternationaluniversity

rankings.Inparallel,studentsselecttostudyattheseinstitutionsforsomethingotherthantobe

satisfiedwiththeirexperienceofteachingandlearning.Theywanttoreceiveadegreefroma

universitywithaninternationallyleadingimageandjoinaglobalnetworkofpeersandalumni.It

thereforefollowsthatsuchuniversitiesarenevergoingtohavetocompeteonsatisfactiontoattract

students.TheremainderoftheUKsectormustfightforstudentsandneedstoexcelnationallyin

22Itmaybethatstudentsarebasingtheirscoresonhowpositivetheirfeedbackhasbeen,ratherthanitsquality.23http://www.theguardian.com/education/2015/may/05/campus-universities-building-projects.24ItmayalsobethecasethatstudentsateliteUniversitieshavemuchhigherexpectationsduetoasenseofentitlement,whichmaytheneffectsatisfactionscores.

11

theleaguetablessuchasthoseofTheGuardianandTheCompleteUniversityGuide,andsuch

universitiesaremakingmuchmoreefforttoensurethatstudentswhochoosetheirprogrammes

leaveextremelysatisfiedwiththeirexperience.

YetifweexaminetheresultswithinthebroaderentireRussellgroup25inthesecondpanelofTable

7,overallsatisfactionattheseuniversitiesisstatisticallysignificantlyhigherbyaround0.06ona5-

pointscaleandbyoneinpercentageterms;theminimaandmediansarealsohigherforRussell

groupthanfornon-Russellgroupuniversities.Similarly,whenweseparateuniversitiesintotwo

groupsaccordingtowhethertheyareintheTop400QSWorldrankedornot(thefourthpanelof

Table7),weobserveasimilarpicturethatthemoreprestigiousuniversitiesengender(statistically

significantly)higherlevelsofsatisfaction,althoughalargernumberofuniversities(44)holdthis

designationcomparedwiththeRussellgrouping(24universities).

ThisresultisperhapssurprisingsinceitiscommonlybelievedthatthoseatRussellgroupuniversities

areexcessivelyfocusedontheirresearchtothedetrimentofstudentsandteachingactivities.Land

andGordon(2015,p.21)arguethatthereisa‘pronounceddisparity’betweenthefinancialrewards

forresearchexcellenceversusteachingexcellence,whichovertimeisboundtohaveaneffecton

thebehaviourandstrategicchoicesoffaculty.Itmaythusbethatnewuniversitiesthereforefocus

ondifferentkindsofrankingmeasureswheretheycanexcel,astheHuddersfieldexperience

documentedbyThornton(2014)shows.26Forexample,manyuniversitieshavenowestablished

targetsforhavingatleastaspecificpercentageoftheirfacultybeingprofessionallyqualified

teachersandgrowingthepercentageofqualifiedteachersisakeyaspectofthe‘professionalisation’

ofteaching.

Itispossiblethatthesehighscoresinstudentevaluationsforeliteinstitutionsaremerely

manifestationsofbrandloyalty,sothattheyarepunishedtoafarsmallerdegreeforfailingtomeet

expectationsthantheirlessrenownedcounterparts.Suchstudentsmayinternalisetheirproblems

withservicefailures,believingthatitmustbetheirownexpectationsorjudgementswhicharefaulty

sincethehallowedinstitution,withiconicheritagebuildings,mustbebeyondreproach.The

reputationofauniversityhasbeenshowntoaffectstudentretentionratesandloyalty(Eskildsenet

al.,1999;NguyenandLeBlanc,2001;HelgesenandNesset,2007).Brandandtheuniversity’s

positionintheleaguetablesarehighlycorrelatedandbothwillaffecttheabilityoftheinstitutionto

attracthighcalibrestudents(Palacioetal.,2002)andwillalsoimproveothermetricssuchas

employmentdatafollowinggraduation.

Wealsoseparateuniversitiesintowhethertheyarespecialistinstitutions,whichwedefineas

operatinginfourorfewersubjectareas,finding,perhapssurprisingly,thatmeanscoresandthe

percentageofstudentswhoaresatisfiedarebothlowerthanatmorebroad-baseduniversities,

althoughnotsignificantlyso.Thelargestdifferencebetweensub-groupsappearswhenweseparate

universitiesintotwosub-groupsaccordingtowhethertheywereestablishedbefore1992–theso-

called‘olduniversities’orwhethertheywereestablishedafterthatdate.Newuniversitieshave

25TheRussellgroupisamissiongroupof24UKUniversitiesrepresentingwhatmanywouldconsidertobetheeliteinstitutions.26TheUniversityofHuddersfieldset,andachievedatargetofhaving100%ofteachingstaffbeingprofessionallyqualifiedteachersortheequivalent,andatthesametime(althoughnocausalitycanbenecessarilyinferred)theirstudentsatisfactionscoresincreased.

12

highlysignificantlylowerlevelsofsatisfaction(amorethanthreepercentagepointdifferencein

studentsatisfaction),withtheformerhavinglowermean,medianandminimumlevelsof

satisfactionandhighervariances.

4.4Singleattributeversusmulti-attributesatisfactionmeasures

Wecouldcategorisetheapproachestomeasuringstudentsatisfactionaccordingtowhetherthey

attempttoevaluateitalongasingledimension(“Overall,areyousatisfiedwithyourexperience?”or

“Overall,howsatisfiedareyou?”)oralongseveralseparatedimensions–singleattributeandmulti-

attributemethodsrespectively.AsdescribedinSection3above,theNationalStudentSurvey

includesatotalof23questionscomprisingboththoseineachofsevencategoriesandalsoa

separatequestion22whichasksrespondentstoevaluatetheiroverallsatisfaction.Thus,fromthat

perspectiveitisaslightlyoddcreationsincearguably,giventhedetailedinformationineach

category,question22issuperfluous.Nevertheless,themajorityofleaguetables(andindeed

universities’ownadvertising)focusexclusivelyontheoverallquestionandentirelyignorethe

informationcontainedintheothersunlesstheyhaveaspecificinterestinoneoftheaspects(e.g.

improvingassessmentandfeedbackscores).

UsingasampleofstudentsfromanunnamedupperMidwestuniversityintheUSasanillustration,

ElliottandShin(2002)argueanddemonstratethatanaggregatescorewhichweightsasetof

individualattributesislikelytoembodymorevaluableinformationthanasingleoverallmeasure.

Theyarguethatthismayarisebecausetheorderingofquestionswithinasurveycouldinfluencethe

outcome.Forexample,theirresponsestothemostrecentlycompletedquestionsarelikelytobe

uppermostinstudents’mindswhentheycompletetheoverallquestionattheend.Askingstudents

toreflectontheiroverallsatisfactioninsuchanabstractmannermayencouragecherry-picking

wheretheydisproportionatelybasetheirevaluationononeortwohighlymemorableeventsrather

thantakingaholisticperspective;thisisfarlesslikelytobethecasewhenquestionsareframedina

morespecificwayandrelatetoasingleaspectofprovisionsuchasthequalityofthelibrary

resources.Respondentsmayalsosufferfromquestionnairefatiguesothattheiranswersbecome

lessandlesswellconsideredasthesurveyprogresses.27Afurtherpossibleissueistheoccurrenceof

errorswherestudents’intendedresponsesbasedontheiractualexperiencedonotmatchthe

answerstheyselect.Inallfoursuchcasesamulti-attributediagnosticthatincorporatestheresults

frommanyquestionsislikelytoprovideamoreaccuraterepresentationofstudents’trueunderlying

butlatentactuallevelsofsatisfaction.

Motivatedbythesearguments,wetakeupElliottandShin’ssuggestionandextendtheiranalysisto

themulti-subject,multi-universitydatasetontheNSSthatweanalyse.Wedonothaveaccesstothe

individualquestionnaireresponsescompletedbyeachstudent,andsoweadapttheirapproach

somewhattofocusatthelevelofaninstitutionandcoursecollection.Morespecifically,weruna

regression(lineofbestfit),wherethedependentvariableiseithertheLikert-scalesatisfactionscore

orthepercentageofstudentswhoaresatisfied–inbothcasesforquestion22.Theregression

includesaninterceptplusthescoresforeachofthecomponentareas,andthefittedlinethengives

theaveragerelationshipbetweentheresponsetoquestion22andthoseintheindividualarea

27Apossibleresponsetothisissuewhileretainingtheintegrityofthequestionnairewouldbetorandomisetheorderofthequestions.

13

coveredinthequestionnaire.WethentaketheregressionresultsreportedinTable6andemploy

theparameterestimatesasweightstoconstructsyntheticoverallmeasuresfortheLikertscale

variableandthepercentageofstudentssatisfied,whichareeffectivelythefittedvaluesfromthe

regression.Sowecouldwrite,forexample,foreachcoursecollectioniatagiveninstitution:

New_satisfaction_scorei=-1.073+0.489×teachingscorei+0.030×assessment&feedbacki+0.194×

academic_supporti+0.286×organisation&managementi+0.055×learning_resourcesi+0.212×

personaldevelopmenti+0.0004×Students’_Unioni

and

New_percent_satisfiedi=-15.179+0.524×teachingscorei+0.024×assessment&feedbacki+0.181

×academic_supporti+0.262×organisation&managementi+0.058×learning_resourcesi+0.159×

personaldevelopmenti-0.004×Students’_Unioni

Clearly,itwouldbepossibletouseadifferentsetofweights–forexample,bygivingeachgroupof

questionsthesameprominencesothatwetakeasimpleaverage.However,theapproachthatwe

haveoutlinedabovehastheadvantagethatthefunctionemployedtoartificiallyconstructoverall

measureswillpreservetherelativeimportancethatatypical(morespecifically,themean)student

assignstoeachcategory.Therefore,byconstruction,theoveralllevelofsatisfactionestimatedfrom

thisfunctionwillbeidenticaltotheiractuallevelofsatisfactionasexpressedintheirresponseto

question22.Forallothercourse-institutioncombinations,theestimatedlevelsofsatisfactionmay

differfromtheanswerstoquestion22bylesserorgreaterextents.Table8examinesthispointin

detail.Thefirstpanelofthetablepresentsthemean,standarddeviation,skewnessandkurtosisof

theproposednewmulti-attribute-basedsatisfactionmeasuresforboththeLikertscalescoreand

thepercentageofrespondentswhoaresatisfied.Itisevidentthattheweightingfunctionsachieve

thedesiredobjectiveofcreatingamorestablemeasurewhichhasanidenticalmeanbutamuch

lowerspreadofobservations,withalowerstandarddeviationandbothskewnessandkurtosisthat

areclosertozero.

Forthemajorityofcoursecollectionsateachinstitution,thedifferencebetweentheactualscores

andtheartificiallyconstructedcompositemeasuresremainverymodest,butinsomecasesthere

arespectacularchanges.TheremainderofTable8(PanelsBtoE)reportthetenlargestrisersand

fallersforthe1-5scoresandpercentageofstudentssatisfiedrespectively.Inthesecases,andmany

otherstoaslightlylesserextent,thedifferencesinthepicturespaintedbythecompositemeasure

andbytheresponsestoquestion22areverystark.Forexample,focusingontheverylargest

changesofall,OceanSciencesatSouthamptonSolentUniversityreceivedanaverageratingof4.2

ona1-5scalewith100%ofstudentssatisfiedorverysatisfiedoverallyetthemulti-attributemodel

wouldplacethemmuchclosertothemeanwitharatingof3.2andonly60%ofstudentssatisfied.

Ontheotherhand,FineArtattheUniversityofBedfordshirereceivedanaveragescoreof3.5with

55%ofstudentssatisfied,yetthemulti-attributeapproachwouldpresentthemwithascoreof4.3

and82%satisfied.

Furtherexaminingthislist,itisclearthatthebigmoversaresituationswhereanomalieshavearisen

wheretheoverallscoresdonottallywiththosefortheindividualcomponentareasbyabigmargin

andthereforethereitishighlylikelythattheoverallscorehasfailedtocapturethetruelevelsof

contentmentoftherespondents.Thecommonfeaturesofthislistofcoursecollectionsisthatthey

14

allhaverelativelysmallnumbersofrespondentscompletingthequestionnaires(albeitallstillwithin

theNSSreportingrequirementofatleast10respondentsrepresentingatleasta50%responserate).

Toillustratethisinmoredetail,Table9presentsthecomponentandoverallscoresfortwoofthe

course-institutionsfromTable8–althoughweconductthisanalysisonallcoursesandinstitutionsin

thesample,herewedrawouttwoextremecasesasexemplars.

OceanicEngineeringatSouthamptonSolentUniversityscored4.2overallwith100%satisfied,yet

notasingleoneofthecomponentscoresareanywherenearthislevel–forexample,teaching

satisfactionisat3.7(78%satisfied)andorganisationandmanagement,thesecondmostinfluential

sub-categoryaccordingtoourresultsinTable6,isatarelativelypoor2.7(33%satisfied).Ourmulti-

attribute-basedestimatewouldgiveanoverallsatisfactionof3.2(60%)satisfied,whichisfarlower

butwellwithintherangeofthecomponentscores.Turningnowtothebiggestchangeattheother

end,ElectronicandElectricalEngineeringattheUniversityofEastLondonreceivedanoverall

satisfactionratingof2.8(17%satisfied),yetnoneofthecomponentmeasuresarethislow(except

AssessmentandFeedback,withascoreof2.7butevenhere34%ofstudentsaresatisfied).Crucially,

theTeachingonMyCourseandOrganisation&Managementscoresare3.3and3.5respectively.

Again,theoverallresultappearsanomalousandthemulti-attribute-basedestimatewouldbe3.3

(51%satisfied),whichappearsmuchmoreplausibleandinlinewiththefiguresfortheconstituent

areas.

Whatunitesthesetwoillustrationsisthatthetotalpopulationofstudents(12and17)and

consequentlythenumberofrespondents(10and12respectively)areverylowandinsuch

situations,itiseasyforanerrorinunderstandingorcompletionbytwoorthreeindividualstudents

tohaveaprofoundeffectontheoutcome.Althoughthesearethemostextremeillustrations,they

arebynomeansisolatedexamples:theuseofourmethodologyresultsinchangesofmorethanten

inthepercentageofsatisfiedstudentsfor268coursecollections(6%ofthetotal),while1262

changebyatleastfivepercentagepoints(28%ofthetotal).Suchchangeswouldbetakenvery

seriouslybytheinstitutionsconcerned,andinallcaseswouldbelikelytorepresentthedifference

betweenrecriminationsandcelebrations.

Fromthefundamentalprinciplesofstatistics,thelawoflargenumbersoughttoimplythatan

average(weighted)measureofsatisfactionshouldbemorestableovertime(havealowerstandard

deviation)thanameasurebasedonasingleoverallscore.Inordertotestthis,werepeattheabove

analysistoconstructsyntheticoverallscoresbasedontheweightingsofthecomponentsforeachof

thesevenyears2009-2015forwhichdatafromtheNationalStudentSurveyareavailableona

comparablebasis.28Wethenconstruct,separatelyforeachcoursecollectionateachinstitution,the

standarddeviationofthesesyntheticcompositescoresandcompareitwiththestandarddeviations

oftheactualoverallscores(bothforthescoresmeasuredona1-5scaleandthepercentageof

studentssatisfied).Theresults,reportedinTable10,presenttheaverageofthesestandard

deviationsacrossallcoursecollectionsatallinstitutionsclearlyshowthatindeedthesynthetic

measureismorestableovertime.Forexample,inthecaseofthepercentageofstudentssatisfied,

thestandarddeviationforthesyntheticmeasureis5.95comparedwith6.91fortheoriginal

28Weemploythesameweightingsasaboveandapplythemtoallyears,althoughitwouldofcoursebepossibletooptimizeseparatelyforeachyear.

15

measure;thestandarddeviationofthesyntheticmeasureisalsolowerfortwothirdsofcourse

collectionsthantheresultsfromquestion22.

5.Discussion,ConclusionsandReflectionThisstudysystematicallyexaminesthecross-sectionalvariationinresponsestotheNationalStudent

Survey,coveringallsubjectareasanduniversitiesthatparticipated,andassuchithasabroader

subjectcoveragethanexistingwork.Initiallysummarisingthedata,wefindthatstudentsofclinical

subjectsorhumanitiesaremostsatisfied,whilethoseonmediastudiesandgeneralengineering

coursesaretheleastso.Fromageographicalperspective,studentsinNorthernIrelandandthe

NortheastarethemostcontentedandthoseinLondontheleast.Wenextexaminethe

interrelationshipsbetweenthesub-categoriesofquestionsaskedinthesurvey,findingperhaps

surprisinglyhighcorrelationsbetweenareaswherethequalityofservicemaynotnecessarilybe

expectedtogotogether.Weshowedthatwhendeterminingtheiroveralllevelofsatisfactionin

question22,asexistingresearchbyLanganetal.(2013)hasshown,studentsappeartoplacemost

emphasisonteaching,courseorganisationandpersonaldevelopmentwithverylittleonassessment

andfeedbackorhowhappytheywerewiththeirStudents’Union.Finally,weshowedthatin

pairwisecomparisonsofsub-groups,studentsonpart-timedegrees,atRussellGroupuniversities,at

non-specialistinstitutions,atuniversitiesintheQSTop400,andatolduniversities,weremore

satisfiedthanintheircounterparts.

Ourfindingsleadtoseveralinterestingpolicyimplications.First,manyuniversitieshaveactively

identifiedimprovingtheassessmentandfeedbackaspectsoftheirNSSasastrategicpriority,given

thatinmostcasesthesescoresarelowerthanforotherpartsofthesurvey.Whilstthisisundeniably

alaudableobjectiveinitsownright,itappearsthattheprimarymotivationforthisfocusinmany

instancesisinfactasanindirectwaytoimprovetheoverallscores.Ourregressionresultssuggest,

however,thattheroleoffeedbackandassessmentininfluencingthesummaryoutcomeis

extremelysmall(asFieldingetal.,2010,alsonote)andthusuniversitieswouldbebetterdirecting

theireffortsatfurtherincreasingthesub-scoresonteaching,andtoalesserdegree,course

organisationandpersonaldevelopment.Afurtherconcernisthatwhenstudentspresenttheir

scoresonthefeedbackthattheyhavereceived,theyareactuallyreportingonitspositivenessand

notitsqualityortimeliness(Boehleretal.,2006).

Second,ourresultssuggestthatpart-timestudentsaresignificantlymoresatisfiedthantheirfull-

timecounterparts,perhapsreflectingthatUniversitieshaveofferedavarietyofotherbenefits

includingaccesstocoursestomaturestudentswithfamiliesandlife-longlearningopportunities.Yet

part-timecoursesareincreasinglyunderpressureintheUKandiftheyarenotcarefullynurtured,

willcontinuetodeclineinnumber.29ForexampletheOpenUniversityhasreportedlossesfor

2014/15of£7milliononthebackofstudentnumbersfallingby7.2%.30Third,wesuggestthatfor

29See,forexample,theBBCwebsitenewsarticlebyA.Harrison,14March2013,“'Dramaticdecline'inpart-timeuniversitystudentsinEngland”.30https://www.timeshighereducation.com/news/open-university-posts-ps7m-loss-student-numbers-slump

16

universitieswhicharedesirousofimprovingtheirNSSscores,thereareprobablynosilverbullets

althoughencouraginghigherresponseratesisthequickestfix.31

Third,themulti-attributeapproachproposedbyLanganetal.(2013)andwhichweinvestigate

furtherhere,combiningtheresponsestoallquestionareasinthesurvey,hasadvantages–both

theoreticalandinpractice.Wefindthatthedifferencesbetweenthesinglequestionoverall

satisfactionscoresandthosebasedonthesyntheticcalculationapproachcombiningmany

attributesresultinveryconsiderablechangesinmeasuredsatisfactionformorethanaquarterof

thesample.Wewouldthereforerecommendthatuniversities’ownappraisalsofstudentsatisfaction

andthosethatfeatureaskeyingredientsinleaguetablerankingsshouldemploysuchameasureor

theycouldreportboththerawandthemodifiedoverallsatisfactionscores.

Itiscommonforscoresforagivencoursecollectionatagiveninstitutiontovaryfromoneyearto

another,sometimessubstantially.Yettypically,thestructureofcoursesandtheirorganisation,the

teachingstaff,andthefacilitiesallchangerelativelyslowlyovertimeinmostcases.Theonlyfactor

changingsystematicallyfromoneyeartoanotheristhestudentbodyitselfandsoperhapsthese

year-to-yearvariationssaymoreaboutthestudentsthanthequalityofprovision.Despitethis,

anecdotalevidenceappearstosuggestthatsenioruniversitymanagersarefailingtograspthatthe

resultsfromtheNSSareexamplesofrealisationsofastochasticprocesssothatscoresmayriseand

fallovertimeasaresultofessentiallyrandomfactors.Hencemodestfallsareconsidereddisasters,

asevidencethatthingsaregoingbackwardswhenthismaynotbethecase,andtheunreliabilityof

thesinglequestionoverallsatisfactionmeasurewillsurelyexacerbatethis.Wehaveshownthat

broad-basedaggregatemeasuresaremorestableovertimethanthosebasedontheresponsestoa

singlequestion.Suchadditionalstabilitywouldbehighlywelcomewithininstitutions,andwould

enablethemtoassesstheunderlyingqualityoftheirprovisionwithgreaterprecisionandplanmore

effectively.

Finally,weshouldnotethatitmaybeinappropriatetodrawtoostrongasetofprescriptivesfrom

ouranalysis,however,sincestudents’subjectiveperceptionsofthequalityoftheirprogrammewill

beevaluatedrelativetotheirpriorexpectationsandthelatterisunobservable.Therefore,students

showingtheleastsatisfactionmayhavereceivedahighqualitylearningexperiencebutnevertheless

bethemostdisappointedastheirlevelsofanticipationweresohigh.Suchaneffectmightexplain

why,forexample,studentsintheUKwerelesssatisfiedonaveragealongalldimensionsthanthose

intheUS(Mai,2005,p.874).32Relatedly,itiswellestablishedthatstudentdemographicsthemselves

mayaffecttheirperceptionswhencompletingthequestionnaire.Forexample,researchhas

suggestedthattheyoungestandoldeststudentstendedtobehappiest,withthoseinbetweenthe

leasthappy(BlanchflowerandOswald,2004)althoughthegenderoftherespondentdoesnothave

adiscernibleeffect(Chanetal.,2005).

Further,itmaybethecasethatthedifferencesbetweenlevelsofsatisfactionacrosssub-groupsmay

merelybeseparatemanifestationsofthesamephenomenon–forexample,thatnewuniversities

31Unreportedresultsshowthatincreasingtheresponserateshasasignificantpositiveeffectonsatisfaction,consistentwiththenotionthatthemostaggrievedstudentsarethekeenesttocompletethesurvey,whilethosewithmoremoderateorpositiveviewsaremorelikelytorequireprompting.32FurtherlightcouldbeshedonthisbytheintroductionofentrysurveystomeasurestudentexpectationsoftheirforthcomingUniversityexperience.

17

haveweakerreputations(notbeingmembersoftheRussellgroupandlesscommonlywithintheQS

Top400thanolduniversities)orthataremorelikelytoofferthelesspopularsubjectssuchasmedia

studies.Therefore,furtherresearchcouldfruitfullyemployamultipleregression-typemodelthat

attemptstoteaseouttheseparateeffectsofthesevariousfactors(e.g.,universityreputation,class

sizes,demographicsofthestudentbody,subjectarea,region,demographicsandqualificationsof

thefaculty,etc.).

18

ReferencesAlves,H.andRaposo,M.(2010)Theinfluenceofuniversityimageonstudentbehaviour.

InternationalJournalofEducationManagement24(1),73-85.Arambewela,R.andHallJ.(2009)Anempiricalmodelofinternationalstudentsatisfaction,Asia

PacificJournalofMarketingandLogistics,21(4),555–569.Armstrong,J.S.(1995)Thedevil’sadvocateresponsestoMBAstudents’claimsthatresearchharms

learning.JournalofMarketing59,101-106.Asthana,A.andBiggs,L.(2007)Studentspaymorebutreceiveless,TheObserver,11February.Blanchflower,D.andOswald,A.J.(2004)Well-beingovertimeinBritainandtheUSAJournalof

PublicEconomics61,359-381.Boehler,M.L.,Rogers,D.A.,Schwind,C.J.,Mayforth,R.,Quin,J.,andWilliamsRG,Dunnington,G.

(2006)Aninvestigationofmedicalstudentreactionstofeedback:arandomizedcontrolled

trialMedicalEducation32,367-369.Borden,V.M.(1995)Segmentingstudentmarketswithstudentsatisfactionandprioritiessurvey.

ResearchinHigherEducation36(1),73-88.Browne,B.,Kaldenberg,D.,Browne,W.andBrown,D.(1998)Studentascustomers:factors

affectingsatisfactionandassessmentsofinstitutionalquality,JournalofMarketingforHigherEducation,8(3),1-14.

Buckley,A.,Soilemetzidis,I.,andHillman,N.(2015)The2015StudentAcademicExperienceSurvey.HigherEducationPolicyInstituteandHigherEducationAcademy.

Callender,C.,Ramsden,P.andGriggs,J.(2014)ReviewoftheNationalStudentSurveyHigherEducationFundingCouncilforEngland.

Chan,G.,Miller,P.andTcha,M.(2005)HappinessinuniversityeducationInternationalReviewofEconomicsEducation4(1),20-45.

Chatterton,P.andGoddard,J.(2000)TheResponseofhighereducationinstitutionstoregional

needs.EuropeanJournalofEducation35(4),475-496.Collini,S.(2012)WhatareUniversitiesFor?Penguin,London.Douglas,J.McClelland,R.DaviesJ.(2008)Thedevelopmentofaconceptualmodelofstudent

satisfactionwiththeirexperienceinhighereducation.QualityAssuranceinEducation16(1),19-35.

Elliott,K.M.andShin,D.(2002)Studentsatisfaction:analternativeapproachtoassessingthis

importantconcept.JournalofHigherEducationPolicyandManagement24(2),197-209.Eskildsen,J.,Martensen,A.,Gronholdt,L.andKristensen,K.(1999)Benchmarkingstudent

satisfactioninhighereducationbasedontheECSImethodology.ProceedingsoftheTQMfor

HigherEducationInstitutionsConference:HigherEducationInstitutionsandtheIssueof

TotalQuality30-31,August,Verona,pp.385-402.

Fielding,A.,Dunleavy,P.J.,andLangan,M.InterpretingcontexttotheUK’snationalstudent

(satisfaction)surveydataforsciencesubjectsJournalofFurtherandHigherEducation34(3),347-368.

Gibbons,S.,Neumayer,E.andPerkins,R.(2015)Studentsatisfaction,leaguetablesanduniversity

applications:evidencefromBritain.EconomicsofEducationReview48,148-164.Gibbs,G.(2010)DimensionsofQuality.HigherEducationAcademy,York.

Gibbs,G.(2012)Implicationsof‘DimensionsofQuality’inaMarketEnvironment.HigherEducationAcademy,York.

19

Gilman,R.(2001)TheRelationshipbetweenLifeSatisfaction,SocialInterest,andFrequencyof

ExtracurricularActivitiesamongAdolescentStudentsJournalofYouthandAdolescence30(6),749-767.

Gornall,L.andThomas,B.(2014)Professionalworkandpolicyreformagendasinamarketised

highereducationsystem,in:Gornall,L.,Cook,C.,Daunton,L.,Salisbury,J.andThomasB.

(eds.)AcademicWorkingLives:Experience,PracticeandChange.Bloomsbury,London.

Gruber,T.Fuß,S.Voss,R.andGläser-ZikudaM.(2010)Examiningstudentsatisfactionwithhigher

educationservices,InternationalJournalofPublicSectorManagement,23(2)105–123.Guolla,M.(1999)Assessingtheteachingqualitytostudentsatisfactionrelationship:applied

customersatisfactionresearchintheclassroom.JournalofMarketingTheoryandPractice7(3),87-97.

Hattie,JandMarsh,H.W(1996),Therelationshipbetweenresearchandteaching:Ameta-analysis,

ReviewofEducationalResearch,vol.66,no.4,pp.507-542.Helgesen,Ø.andNesset,E.(2007),Images,satisfactionandantecedents:driversofstudentloyalty?

AcasestudyofaNorwegianuniversitycollege.CorporateReputationReview10(1),38-59.Hewson,P.(2011)PreliminaryanalysisofthenationalstudentsurveyMSORCommunications11(1),

25-28.

Kay,J.,Dunne,E.,andHutchinson,J(2010),Rethinkingthevaluesofhighereducation-studentsaschangeagents?,QualityAssuanceAgency,London.

Kotler,P.andFox,K.(1995)StrategicMarketingforEducationalInstitutions,secondedition,PrenticeHall,EnglewoodCliffs,NewJersey.

Land,R.andGordon,G.(2015)TeachingExcellenceInitiatives:ModalitiesandOperationalFactors.HigherEducationAcademy,York.

Langan,A.M.,Dunleavy,P.,andFielding,A.(2013)Applyingmodelstonationalsurveysof

undergraduatesciencestudents:whataffectsratingsofsatisfaction?EducationSciences3,

193-207.

Letcher,D.W.andNeves,J.S.(2010)Determinantsofundergraduatebusinessstudentsatisfaction,

ResearchinHigherEducationJournal,6,1-26.Locke,W.(2014)ShiftingAcademicCareers:ImplicationsforEnhancingProfessionalisminTeaching

andSupportingLearning.HigherEducationAcademy,York.

Mai,L-W.(2005)AcomparativestudybetweenUKandUS:Thestudentsatisfactioninhigher

educationanditsinfluentialfactorsJournalofMarketingManagement21,859-878.Mavondo,F.T.,Tsarenko,Y.andGabbott,M.(2004)Internationalandlocalstudentsatisfaction:

resourcesandcapabilitiesperspective.JournalofMarketingforHigherEducation14(1),41-60.

Merritt,D.J.(2012)Bias,theBrainandStudentEvaluationsofTeaching,StJohn’sLawReview,82(1),Article6.

Naftulin,D.H.Ware,J.E.Jr.,andDonnelly,F.A.(1973)TheDoctorFoxLecture:AParadigmof

EducationalSeduction,JournalofMedicalEducation,48,630-635.Nguyen,N.andLeBlanc,G.(2001)Imageandreputationofhighereducationinstitutionsinstudents’

retentiondecisions.InternationalJournalofEducationManagement15(6),303-311.Oliver,R.L.(1981)Measurementandevaluationofsatisfactionprocessesinretailsettings,Journalof

Retailing57(3),25-48.Parker,C.andMatthews,B.P.(2001)Customersatisfaction:contrastingacademicandconsumers’

interpretations.Marketing,IntelligenceandPlanning19(1),38-44.

20

Palacio,A.,Meneses,G.andPérez,P.(2002)Theconfigurationoftheuniversityimageandits

relationshipwiththesatisfactionofstudents.JournalofEducationalAdministration40(5),486-505.

Pike,G.(1991)Theeffectsofbackground,coursework,andinvolvementonstudents’gradesand

satisfaction.ResearchinHigherEducation,32(1),15-31.Sabri,D.(2013)StudentEvaluationsofTeachingas‘Fact-Totems’:theCaseoftheUKNational

StudentSurvey.SociologicalResearchOnline18(4),1-15.

Surridge,P.(2008)Thenationalstudentsurvey2005-2007:findingsandtrendsAReporttotheHigherEducationFundingCouncilforEngland

Taylor,C.andMcCaig,C.(2014)EvaluatingtheImpactofNumberControls,ChoiceandCompetition:AnAnalysisoftheStudentProfileandtheStudentLearningEnvironmentintheNewHigherEducationLandscape.HigherEducationAcademy,York.

Thornton,T.(2014)Professionalrecognition:promotingrecognitionthroughtheHigherEducation

AcademyinaUKhighereducationinstitution.TertiaryEducationandManagement20(3),225-238.

Walker,J.T.,Vignoles,A.,andCollins,M.(2010)HigherEducationAcademicSalariesintheUK,

OxfordEconomicPapers,62,(1),pp.12-35.Yorke,M.(2014)Theimpactofpart-timestaffonArt&Designstudents’ratingsoftheir

programmes.JournalofHigherEducationPolicyandManagement36(5),557-567.Yorke,M.(2009)Studentexperiencesurveys:somemethodologicalconsiderationsandanempirical

investigation.AssessmentandEvaluationinHigherEducation34(6),721-739.Yorke,M.,Orr,S.andBlair,B.(2013)Hitbyaperfectstorm?Art&Designinthenationalstudent

survey.StudiesinHigherEducation39(10),1788-1810.Zeithaml,V.A.,Berry,L.L.andParasuraman,A.(1993)Thenatureanddeterminantsofcustomer

expectationofservice.JournaloftheAcademyofMarketingScience21(1),1-12.

21

Table1:AverageNSSScoresbySubjectAreabasedonOverallSatisfaction

Notes:Thistablepresentssummarymeasureswithineachsubjectarea(HESAcostcoding)usinganunweightedaverageacrossallHEIsoperatingwithinthatcostcodebasedontheconversionoftheLikertscaleresponsesforoverallsatisfaction(1=stronglydisagree,…,5=stronglyagree)toapointsscore.

SubjectArea Mean Standarddeviation

Min. 5thPercentile Median 95thPercentile Max.

(145)Mediastudies 4.02 0.37 2.6 3.4 4.1 4.56 4.8(115)Generalengineering 4.09 0.36 3.1 3.5 4.1 4.7 4.8(120)Mechanical,aero&productionengineering 4.12 0.32 3.4 3.6 4.1 4.6 4.8(121)IT,systemssciences&computersoftwareengineering

4.13 0.31 3 3.64 4.1 4.66 4.9

(125)Areastudies 4.13 0.22 3.8 3.8 4.1 4.45 4.5(143)Art&design 4.14 0.31 3.2 3.68 4.1 4.6 4.9(144)Music,dance,drama&performingarts 4.14 0.38 2.8 3.4 4.2 4.7 5(117)Mineral,metallurgy&materialsengineering 4.15 0.37 3.2 3.53 4.15 4.55 4.6

(119)Electrical,electronic&computerengineering 4.15 0.36 2.5 3.66 4.2 4.6 4.9

(129)Economics&econometrics 4.16 0.29 3.5 3.61 4.2 4.5 4.8(131)Socialwork&socialpolicy 4.16 0.32 3.3 3.6 4.2 4.6 4.8(103)Nursing&alliedhealthprofessions 4.17 0.32 2.6 3.6 4.2 4.6 4.8(137)Modernlanguages 4.18 0.3 3.2 3.6 4.2 4.6 4.8(133)Business&managementstudies 4.19 0.29 2.8 3.7 4.2 4.6 4.9(134)Catering&hospitalitymanagement 4.19 0.2 3.8 3.92 4.2 4.56 4.6(116)Chemicalengineering 4.21 0.3 3.6 3.62 4.3 4.5 4.7(135)Education 4.21 0.35 2.7 3.6 4.2 4.7 4.9(123)Architecture,builtenvironment&planning 4.22 0.3 3.4 3.69 4.3 4.7 4.8(132)Sociology 4.22 0.32 2.7 3.7 4.3 4.6 4.7(104)Psychology&behaviouralsciences 4.23 0.26 3.2 3.7 4.2 4.6 4.9(105)Health&communitystudies 4.23 0.42 2.4 3.52 4.3 4.7 4.8(118)Civilengineering 4.23 0.31 3.2 3.5 4.3 4.6 4.7(128)Politics&internationalstudies 4.23 0.31 3.1 3.7 4.3 4.6 4.7(108)Sportsscience&leisurestudies 4.24 0.3 3.2 3.8 4.2 4.7 4.8(138)Englishlanguage&literature 4.26 0.3 2.9 3.71 4.3 4.6 4.9(127)Anthropology&developmentstudies 4.27 0.28 3.8 3.8 4.35 4.67 4.7(130)Law 4.27 0.24 3.5 3.8 4.3 4.6 4.8(101)Clinicalmedicine 4.28 0.31 3.5 3.63 4.3 4.7 4.8(110)Agriculture,forestry&foodscience 4.28 0.31 3.3 3.68 4.3 4.65 4.7(112)Biosciences 4.28 0.28 3.5 3.8 4.3 4.7 4.9(124)Geography&environmentalstudies 4.3 0.26 3.4 3.8 4.4 4.6 4.8(111)Earth,marine&environmentalsciences 4.31 0.3 3 3.9 4.3 4.8 4.9(122)Mathematics 4.32 0.22 3.7 3.9 4.3 4.7 4.7(113)Chemistry 4.33 0.24 3.7 3.92 4.4 4.64 4.7(106)Anatomy&physiology 4.34 0.23 3.6 4 4.4 4.7 4.8(107)Pharmacy&pharmacology 4.34 0.26 3.9 3.9 4.4 4.7 4.8(126)Archaeology 4.35 0.36 2.9 3.7 4.35 4.8 4.9(139)History 4.35 0.28 3.4 3.8 4.4 4.7 5(140)Classics 4.35 0.22 3.8 4 4.4 4.69 4.7(114)Physics 4.37 0.27 3.6 3.8 4.4 4.7 4.8(141)Philosophy 4.38 0.19 3.9 4.1 4.4 4.6 4.8(142)Theology&religiousstudies 4.41 0.28 3.4 4 4.5 4.7 4.9(109)Veterinaryscience 4.44 0.17 4.3 4.3 4.4 4.7 4.8(102)Clinicaldentistry 4.65 0.14 4.4 4.48 4.7 4.83 4.9

22

Table2:AverageNSSScoresbySubjectAreabasedonPercentageSatisfied

Notes:Thistablepresentssummarymeasureswithineachsubjectarea(HESAcostcoding)usinganunweightedaverageacrossallHEIsoperatingwithinthatcostcodebasedonthepercentageofstudentswhoaresatisfiedoverall.

SubjectArea MeanStandarddeviation Min. 5th

PercentileMedian 95th

PercentileMax.

(145)Mediastudies 79.77 12.74 30 56.9 82 99.1 100

(121)IT,systemssciences&computersoftwareengineering

83.45 10.02 38 69 83 100 100

(144)Music,dance,drama&performingarts 82.6 12.17 34 59.3 84 98.7 100

(117)Mineral,metallurgy&materialsengineering

82.58 13.96 50 58.8 84.5 98.35 100

(143)Art&design 82.76 10.08 50 66 84.5 100 100

(115)Generalengineering 82.43 11.07 52 64 85 97.8 100

(125)Areastudies 85.42 6.08 71 77.05 85 93 93

(134)Catering&hospitalitymanagement 85.16 6.13 67 75 85 93 94

(103)Nursing&alliedhealthprofessions 85.32 9.96 29 68 86 100 100

(120)Mechanical,aero&productionengineering

83.48 10.4 58 63.65 86 96 100

(129)Economics&econometrics 83.87 10.27 52 64.15 86 95.95 100

(131)Socialwork&socialpolicy 84.21 10.49 50 64.9 86 97 100

(108)Sportsscience&leisurestudies 86.45 9 52 70.05 87.5 98.95 100

(119)Electrical,electronic&computerengineering

84.15 14.18 17 66.65 88 98.9 100

(123)Architecture,builtenvironment&planning

86.14 10.04 46 65.85 88 100 100

(133)Business&managementstudies 86.04 9.56 25 69 88 99.55 100

(135)Education 85.92 11.03 25 66.9 88 100 100

(137)Modernlanguages 85.83 9.69 50 66.9 88 98.2 100

(104)Psychology&behaviouralsciences 86.8 8.42 53 69.5 89 97 100

(105)Health&communitystudies 85.29 13.78 24 63.4 89 100 100

(116)Chemicalengineering 87.26 9.83 69 70 89 100 100

(130)Law 87.96 7.85 56 72.55 89 98 100

(132)Sociology 86.69 10.74 38 71 89 98 100

(138)Englishlanguage&literature 87.35 9.32 39 70 89 100 100

(111)Earth,marine&environmentalsciences 88.62 9.44 44 73.6 90 100 100

(112)Biosciences 88.16 8.84 58 71.35 90 100 100

(118)Civilengineering 87.14 9.06 56 67.5 90 97.25 100

(122)Mathematics 89.78 6.5 73 75.75 90 100 100

(124)Geography&environmentalstudies 89.28 7.91 57 74 90 100 100

(127)Anthropology&developmentstudies 86.86 10.93 57 67.75 90 98.95 100

(128)Politics&internationalstudies 87.2 10.25 52 68.8 90 100 100

(101)Clinicalmedicine 87.98 9.27 64 67.25 91 99.75 100

(107)Pharmacy&pharmacology 90.11 6.76 75 76.8 91 99.2 100

(113)Chemistry 89.74 7.42 67 75.6 91 98.8 100

(126)Archaeology 88.23 11.37 40 65.9 91 100 100

(106)Anatomy&physiology 91.11 6.33 73 75.65 92 99.35 100

(110)Agriculture,forestry&foodscience 88.22 10.86 52 68.95 92 100 100

(114)Physics 90.28 7.12 70 75.65 92 100 100

(139)History 89.72 8.57 53 73.85 92 100 100

(140)Classics 90.21 5.92 77 78.6 92 99.25 100

(141)Philosophy 90.75 6.67 77 79 92 100 100

(109)Veterinaryscience 92.88 3.56 88 88.7 93 97.3 98

(142)Theology&religiousstudies 90.86 9.47 59 75.2 93.5 100 100

(102)Clinicaldentistry 96.38 2.68 90 91.5 97 99.25 100

23

Table3:SatisfactionbyRegion MeanScore PercentSatisfiedSub-sample N Mean Std.

dev.Min. Median Max. Mean Std.

dev.Min. Median Max.

Southeast 582 4.261 0.303 2.6 4.3 4.9 87.74 9.59 25 90 100EastMidlands 321 4.249 0.295 2.4 4.3 4.9 86.89 9.72 24 89 100London 653 4.114 0.331 2.5 4.1 4.9 82.62 11.21 17 84 100NorthWest 512 4.195 0.309 2.8 4.2 4.9 85.72 9.89 34 87 100Yorks&Humber 407 4.245 0.270 3.3 4.3 5.0 87.50 8.72 50 89 100WestMidlands 370 4.235 0.291 3.2 4.3 4.9 87.11 9.28 55 89 100Southwest 325 4.198 0.349 2.9 4.2 4.9 85.44 11.01 40 88 100East 229 4.253 0.289 3.2 4.3 5.0 87.07 9.04 53 88 100Northeast 168 4.270 0.269 3.1 4.3 4.9 88.11 8.09 50 90 100Wales 299 4.184 0.350 2.8 4.2 5.0 84.95 11.89 25 87 100Scotland 491 4.220 0.327 2.6 4.2 4.9 86.07 10.48 29 88 100NorthernIreland 107 4.377 0.377 3.4 4.4 4.8 90.50 7.87 52 92 100F-testofequalmeaninallregions F-statistic:11.325(p-value:0.000) F-statistic:11.944(0.000)Entiresample 4465 4.216 0.315 2.4 4.3 5.0 86.13 10.20 17 88 100

Notes:Thistablereportssummarystatisticsforoverallsatisfaction(basedonresponsestoquestion22inthesurvey)byregion.ThemiddlepanelshowstheresultsbasedontheLikertscaleresponseswhiletherightpanelisbasedonthepercentageofstudentswhoaresatisfied.ThepenultimaterowreportstheresultsfromanF-testforequalsatisfactionacrossallregionswhilethefinalrowpresentsoverallsatisfactionsummarystatisticsfortheentiresample.

24

Table4:CorrelationbetweenSub-Categories–Average(1-5)Scores

Assessment&feedback

Academicsupport

Organisation&

managementLearningresources

Personaldevelopment

Overallsatisfaction

SatisfactionwiththeStudents’

UnionTeachingonmycourse 0.657 0.752 0.599 0.312 0.645 0.827 0.187

Assessment&feedback 0.694 0.565 0.258 0.531 0.666 0.223

Academicsupport 0.649 0.413 0.665 0.797 0.275

Organisation&management 0.334 0.425 0.749 0.242

Learningresources 0.396 0.407 0.311

Personaldevelopment 0.680 0.287

Overallsatisfaction

0.263

Notes:ThistablepresentsthePearsoncorrelationsbetweentheaveragescoresawardedtoeachsectionoftheNSSquestionnaire,basedontheconversionoftheLikertscaleresponses(1=stronglydisagree,…,5=stronglyagree)toapointsscore.

25

Table5:CorrelationbetweenSub-Categories–PercentSatisfied

Assessment&feedback

Academicsupport

Organisation&

managementLearningresources

Personaldevelopment

Overallsatisfaction

SatisfactionwiththeStudents’

UnionTeachingonmycourse 0.565 0.678 0.555 0.240 0.537 0.768 0.185

Assessment&feedback

0.639 0.508 0.182 0.444 0.569 0.186

Academicsupport

0.584 0.338 0.578 0.707 0.261

Organisation&management

0.263 0.358 0.684 0.250

Learningresources

0.326 0.318 0.277

Personaldevelopment

0.567 0.276

Overallsatisfaction

0.236

Notes:ThistablepresentsthePearsoncorrelationsbetweentheaveragescoresawardedtoeachsectionoftheNSSquestionnaire,basedonThepercentageofstudentswhoaresatisfiedineachcase.

26

Table6:WhichIndividualAreasDriveOverallSatisfaction?

ExplanatoryVariable SatisfactionScore

PercentSatisfied

Constant -1.073(0.055)***

-15.179(1.563)***

Theteachingonmycourse 0.489(0.016)***

0.524(0.021)***

Assessmentandfeedback 0.030(0.011)***

0.024(0.012)**

Academicsupport 0.194(0.017)***

0.181(0.020)***

Organisationandmanagement 0.286(0.010)***

0.262(0.013)***

Learningresources 0.055(0.010)***

0.058(0.013)***

Personaldevelopment 0.212(0.015)***

0.159(0.017)***

SatisfactionwiththeStudents’Union 0.004(0.006)

-0.004(0.006)

R2 0.83 0.72N 4465 4465Notes:Thistablepresentstheparameterestimates(withstandarderrorsinparentheses)forregressionswherethedependentvariableisoverallsatisfaction(theresponsetoquestion22inthesurvey),basedoneithertheLikertscale(middlecolumn)orpercentageofstudentswhoaresatisfied(righthandcolumn).Theexplanatoryvariablesaretheaveragescoresforeachsectionontheform.Eachdatapointrepresentsacourseorcoursecollectionatahighereducationinstitution.*,**and***denotesignificanceatthe10%,5%and1%levelsrespectively.

27

Table7:PairwiseComparisonsofNSSScoresbySub-groupings

MeanScore PercentSatisfiedSub-sample N Mean Std.

dev.Min. Median Max. Mean Std.dev. Min. Median Max.

Part-time 201 4.242 0.337 2.6 4.3 4.9 86.75 11.31 29 89 100Full-time 4264 4.215 0.314 2.4 4.3 5.0 86.10 10.14 17 88 100t-test:part-timevsfull-time t-statistic:-1.113(p-value:0.267) t-statistic:-0.800(p-value:0.425)Russell-group 1024 4.261 0.279 2.7 4.3 5.0 87.66 8.88 38 90 100NotRussellgroup 3440 4.202 0.323 2.4 4.2 5.0 85.68 10.52 17 88 100t-test:RussellvsnotRussell t-statistic:-5.721(p-value:0.000) t-statistic:-5.992(p-value:0.000)Specialistinstitution 77 4.190 0.324 3.2 4.2 4.9 84.75 10.36 45 86 100Notspecialistinstitution 4387 4.216 0.315 2.4 4.3 5.0 86.16 10.19 17 88 100t-test:specialisedversusnot t-statistic:0.698(p-value:0.486) t-statistic:1.184(p-value:0.238)UniversityinQSTop400 1662 4.273 0.280 2.7 4.3 5.0 88.03 8.90 25 90 100UniversitynotinQSTop400 2802 4.182 0.329 2.4 4.2 5.0 85.01 10.74 17 87 100t-testinQStop400vsnot t-statistic:-9.824(p-value:0.000) t-statistic:-10.133(p-value:0.000)NewUniversity 2318 4.165 0.333 2.4 4.2 4.9 84.54 10.94 17 86 100OldUniversity 2146 4.271 0.284 2.7 4.3 5.0 87.85 9.02 25 90 100t-testnewuniversityvsold t-statistic:11.469(p-value:0.000) t-statistic:11.061(p-value:0.000)Entiresample 4465 4.216 0.315 2.4 4.3 5.0 86.13 10.20 17 88 100

Notes:Thistablereportssummarystatisticsforoverallsatisfaction(basedonresponsestoquestion22inthesurvey)forvariouspairwisesamplesplitstogetherwiththeresultsoft-testsofthenullhypothesisthatthemeansofthetwosub-samplesareequalineachcase.Thefinalrowpresentsoverallsatisfactionsummarystatisticsfortheentiresample.

28

Table8:SummaryStatisticsofSyntheticOverallResponsesComparedwithActualValuesPanelA:SummaryStatistics ActualScore(1-5) SyntheticScore(1-5) Actual%Satisfied Synthetic%SatisfiedMean 4.216 4.216 86.132 86.132Std.dev. 0.315 0.286 10.196 8.680Skewness -0.859 -0.463 -1.441 -0.974Kurtosis 1.711 1.258 3.642 1.884PanelB:TopTenFallersonActualScoreInstitution(University)

Course

SyntheticValuefor

Q22

ActualValuefor

Q22

Difference

SouthamptonSolent OceanSciences 3.2 4.2 -1.0LondonMetropolitan HumanResourceManagement 4.1 4.6 -0.5EastLondon MolecularBiology,BiophysicsandBiochemistry 3.3 3.8 -0.5Wolverhampton Finance 4.0 4.5 -0.5Hull TheologyandReligiousstudies 4.1 4.6 -0.5Staffordshire FineArt 3.8 4.3 -0.5Stirling HumanResourceManagement 4.3 4.7 -0.4Roehampton HumanResourceManagement 3.9 4.3 -0.4Liverpool AnimalScience 3.8 4.2 -0.4Canter.ChristChurch OthersinBiologicalSciences 4.1 4.5 -0.4PanelC:TopTenRisersonActualScoreBournemouth HumanandSocialGeography 3.7 2.9 0.8Bedfordshire FineArt 4.3 3.5 0.8DeMontfort Anatomy,PhysiologyandPathology 3.1 2.4 0.7Greenwich Nutrition 4.0 3.4 0.6Highlands&Islands Building 4.2 3.6 0.6Chester SocialPolicy 3.9 3.3 0.6EdinburghNapier Economics 4.1 3.5 0.6ManchesterMetrop. Dance 3.4 2.8 0.6Ulster AuralandOralSciences 4.0 3.4 0.6Lancaster Music 4.0 3.4 0.6PanelD:TopTenFallerson%SatisfiedSouthamptonSolent OceanSciences 60 100 -39.8Uni.ofArts,London Managementstudies 61 82 -20.6SheffieldHallam ElectronicandElectricalEngineering 78 98 -20.2Liverpool AnimalScience 74 94 -20.2Hull TheologyandReligiousstudies 80 100 -19.8SheffieldHallam Planning(Urban,RuralandRegional) 80 100 -19.6Liverp.JohnMoores ElectronicandElectricalEngineering 80 100 -19.6EastLondon MolecularBiology,BiophysicsandBiochemistry 53 72 -18.6Brighton Politics 81 100 -18.6Surrey Dance 82 100 -18.4PanelE:TopTenRiserson%SatisfiedEastLondon ElectronicandElectricalEngineering 51 17 33.8Bournemouth HumanandSocialGeography 70 40 29.5Sheffield TheologyandReligiousstudies 90 62 28.5Bedfordshire FineArt 82 55 27.1WalesTrinitySt.Dav. Accounting 51 25 25.6ManchesterMet. Dance 59 34 24.6ArtsBournemouth Mediastudies 71 47 24.4Canter.ChristChurch Publicitystudies 54 30 24.1NewcastleuponTyne Nutrition 76 52 23.7Chester SocialPolicy 77 53 23.7

29

Table9:AnAnalysisoftheComponentandOverallSatisfactionScoresfortheBiggestChangesbetweentheActualandSyntheticOverallScores

AspectunderEvaluation SatisfactionScore

PercentSatisfied

SatisfactionScore PercentSatisfied

OceanSciences,SouthamptonSolentUni.

ElectronicandElectricalEngineering,Uni.OfEastLondon

Theteachingonmycourse 3.7 78 3.3 53Assessmentandfeedback 3.4 58 2.7 34Academicsupport 3.4 64 3.4 53Organisationandmanagement 2.7 33 3.5 61Learningresources 4.1 87 3.7 67Personaldevelopment 3.2 47 3.6 51SatisfactionwiththeStudents’Union 2.4 40 3.0 50ActualOverallScore(Q22) 4.2 100 2.8 17Numberofrespondents 10 10 12 12TotalPopulation 12 12 17 17ProposedSyntheticOverallScore 3.2 60 3.3 51

Table10:AverageoverallCoursesandInstitutionsoftheStandardDeviationovertimeinActualandSyntheticScores

Syntheticoverallscore

Actualoverallscore(Q22)

%ofcoursecollectionsforwhichthesyntheticoverallscorehasalowerstandarddeviation

Std.dev.ofscores 0.196 0.208 57.6Stddev.of%satisfied 5.95 6.91 66.0