Final Report of Delphi Study

Final Report of Delphi Study

E3M Project - European Indicators and Ranking Methodology for University Third Mission

This project has been funded with support from the European Commission.This publication reflects only the views of the authors, and the Commission cannot be held

responsible for any use which may be made of the information contained therein.

Grant Agreement Number: 2008 - 3599 / 001 – 001

2

3

TABLEOFCONTENTS

1. Introduction

2. ObjectivesoftheDelphistudy

3. Methodology

3.1Methodologyusedforthe1stand2ndDelphirounds

3.2Methodologyusedforthe3rdDelphiround

4. ResultsobtainedwiththeDelphiprocess

4.1FrameworkofCE,TTIandSEprocesses

4.2IndicatorsselectedalongtheDelphiprocess

5. Conclusionsandfinalcomments

6. ProjectParticipants

7. Acknowledgements

8. Annex.Modelofonlinesurveys

4

1.Introduction

TheValenciaUniversityofTechnology(UniversidadPolitécnicadeValencia,UPV)coordinatedthethreeyearsresearchproject(2009‐2012)European Indicators and Ranking Methodology for University’s Third Mission (E3M) co‐financedby theEuropeanCommission’sLifelongLearningProgramme. Themainobjectiveof theproject, involving partners from eightEuropeanHigherEducation Institutions and seven countries,was togenerateacomprehensiveinstrumenttoidentify,tomeasureandtocompareThirdMissionactivitiesofHEIs,inpartthroughtheuseofanarrayofindicatorsofThirdMissionactivityandperformance.

Inordertoreachitsobjective,theE3MprojecthasdevelopedaDelphistudy.Delphiisasurveymethodusedforobtaining theopinionofexperts inanumberofconsecutiverounds.The informationobtained inaround isusedasabasisforthequestionnaireofthenextround.Inthisstudythreeroundsweredevelopedinawaythatallowed the experts to lookat individualdimensions separatelyduring the firstand second rounds,and todevelopamoreglobalviewofthewholesetof indicators forthethreedimensions inthethirdround.Thesedimensionsthatwere identified inpreviousphasesoftheE3Mprojectareconsideredtoberepresentativeofthethirdmissionandwerenamedaccordingly:ContinuingEducation(CE),TechnologyTransfer&Innovation(TTI)andSocialEngagement(SE).ThisdocumentpresentsthemethodologicalapproachappliedinthestudyandthemainresultsoftheDelphiprocessobtainedtherein.

5

2.ObjectivesoftheDelphistudy

TheDelphimethodologywas applied to achieve a consensus about a previous set of indicators that couldaccuratelyandeconomicallydescribe theThirdMissionofHEIs,analysingeach indicator indetail.Throughthismethodology,workingasanorganizeddiscussion,indicatorswereanalysedindividuallyandasaset.TheE3MprojectsetseveralobjectivesfortheDelphistudy:

1. Incorporateexpertsopinionaboutdefinitionsandcharacteristicsofthevariousindicators2. Feedbackontheprocessesidentifiedineachdimension3. AgreementaboutasetofindicatorssuitabletodescribetheThirdMissionofHEIs4. Analysis of the properties of these indicators, mainly relevance and feasibility but also validity,

reliabilityandcomparabilityBycreatingthissetofrelevantindicators,attheendoftheprojectwewillbeabletoofferanewapproachontheconceptofmethodologiestoevaluateThirdMissionactivitiesofHEIs.

6

3.Methodology

GeneralBackground

TheDelphi technique isamethod forobtaining consensus. It consistsofa seriesofquestionnaires thataredevelopedandrefinedinsequentialstagesuntilconsensusisachieved.Inthisprojectwetakeadvantageofoneofthestrengthsofthemethodwhichistheabilitytogatheropinionsfromexpertsfromdifferentbackgroundsand use it to get a selected set of indicators from a broad collection, in this case formeasuring theThirdMissionactivitiesofHEIs.ADelphisurveyisastructuredgroupinteractionprocessorganisedinseveralroundsofopinioncollectionandfeedback.Opinion collection is achieved by conducting a series of surveys using questionnaires.During thethreeroundsofourstudyatotalofsevenquestionnaireswere launched.Threewereelaborated forthe firstroundinaccordancetothethreedimensionsconsideredinthethirdmission,anotherthreequestionnairesforthe second round and finally a unique general questionnaire for the last and third roundwhere the threedimensionswereincluded.

Selectionofexperts

TheexpertpanellistswhoparticipatedintheDelphistudywereproposedbyprojectpartners.TheyproposedanumberofspecialistsintheareasofCE,SEandTTI.Oncetheprojectcoordinationreceivedalltheproposals,aselection of these experts was made and a final list of experts was defined. Two criteria were mainlyconsidered:theexpert’sprofileandtheDelphineeds.Theproposalsreceivedincludedthefollowingdataforeachpanellist:name,institution,fieldofexpertiseandcontactdata.Theconsideredexpertsshouldhavemetthefollowingrequirements:

‐ Technicalknowledgeandprofessionalexperienceinatleastoneofthethreedimensionsoftheproject.‐ Willingnessandabilitytoparticipateduringthetimeofthesurvey.‐ Tobeneutralintheirassessmentandtomaintainconfidentiality.‐ Toagreeinparticipatinginsuchprocedure.

Theexpertpanelwas finallycomposedof twentypanellists fromdifferentgeographicalregions,EuropeandUSA. In the invitationemail, they receivedgeneral informationabout theE3Mprojectand somecontextualinformationabouttheworktheyhadtodo.More informationwasavailable forthem intheprojectwebsitewww.e3mproject.eu.Theyalsoreceivedadetailedscheduleofthethreerounds.

As mentioned before, the role of the experts was to answer a series of questionnaires. Through everyquestionnairethepropertiesoftheproposedindicatorswereevaluated.Theexpertsprovidedtheiropinionsonthedescriptionofindicatorsaswellasageneraloverviewonthewholesetofindicatorsinordertoachieveaconsensusonthebest indicatorstouse incharacterising thirdmissionactivities.Dependingon their fieldofexpertise, the experts contributed toone, twoor threedimensions,whichweredeveloped in threedifferentquestionnaires. In the first round,expertsalsohadanopportunity to suggestadditional indicators tocover

7

possiblegapsintheoriginalproposal.Thestrategywastoselectasetofindicatorsfromabroadinitialbasketandgivetheexpertsthepossibilitytocreateamorerelevantgroupofthem.Table1showsthetemplateusedforthedescriptionoftheindicatorsandtheinformationprovidedwiththem.

Codeoftheindicator NameoftheindicatorPurpose ThereasonwhytheindicatorisselectedDefinition BriefdescriptionoftheindicatornatureInterpretation ThemeaningandresultofthedirectionoftheindicatorMeasurement ThetypeofunitformeasuringtheindicatorFormula(ifapplies) Ifitisneeded,howtocalculatetheindicatorLevelofdatacollection Institution,Faculty/Department,ProgrammeTypeofdatasource Institutional data,surveydataTimereference Lastyear,xyear’s average…RelevanceValidityReliabilityFeasibilityComparability

ImportanceforthemeasurementofthirdmissionactivitiesAbilityoftheindicatortomeasurewhatreallyhastobemeasuredAmeasureoftheabsenceofrandomerrorassociatedwiththeindicatorExpectedfacilityofobtainingtheinformationPossibilityofmakingadequatecomparisonsbetweendifferentHEIs+(high/good)–(low/poor)

Table1

Questionnairesweresentbye‐mail. DuringeveryroundoftheDelphiprocessthenumberandthequalityofthe answersweremonitored and several reminderswere sent out in order to promote participation (seeAnnexes).

Delphiprocedure

TheDelphiprocedurehadthreeroundsofquestionnairesandwascarriedoutinthefollowingsevenstages:

1. Implementationofthefirstroundquestionnaire.2. Analysisof1stroundresponses.3. Implementationofthesecondroundquestionnaire.4. Analysisofthe2ndroundresponses.5. Implementationofthethirdroundquestionnaire.6. Analysisofthe3rdroundresponses.7. Finalreport.

8

3.1Methodologyusedforthe1stand2ndDelphirounds

The first and second roundswere carried out using the email as the communication channel and awebapplication forthesurvey.ThewebapplicationusedwasLimeSurvey.Eachpanellistwasaskedthroughthisonlinesurveytoevaluatethesetofindicatorsproposedforthedifferentdimensions.LimeSurveyfacilitatedtheinputandcollectionoftheresponsesfromallpanellists.

FirstDelphiround

TheaimofthefirstroundwastodeterminethelevelofconsensusabouttheindicatorsunderthedimensionsofCE,TTIandSEandgroupedintoprocesses.Oneofourprioritieswastoachieveaconsensusabouttheselectionanddefinitionoftheindicators.Theideawastoidentifythemostrelevantindicatorsfromaninitialsetofmorethanonehundred.Threequestionnaireswereelaboratedforthefirstround,accordingtothethreedimensionsthat the project had identified as part of the ThirdMission activities. Expertswere also asked to proposeadditionalindicatorsthattheyconsideredimportantandtheywerenotincludedintheinitiallist.

Duringtheanalysisofthefirstroundthefollowingcriteriawereadopted:

a. Treatmentofmissingvalues

Someof thequestionsproposed in the surveywerenotansweredby several experts.Given that the rateofpartial non‐responses was minimal and not focused on a specific item, it was decided not tomake anycorrectionactionandthencalculatethedescriptivestatisticandthedispersionexcludingthemissingvalues.

b. Criteriafortheindicatorsselection:organizingtheindicatorsinfivecategories

The indicatorswereclassifiedintofivegroupsorcategoriesonthebasisoftheevaluationcarriedout by the experts. All observations were considered, specially the answers to the attribute“Relevance”. This attribute, as well as “Validity”, “Reliability”, “Feasibility” and “Comparability”weredescribedinaLikertScaleoffourpoints,from“Unimportant”to“Veryimportant”.TheLikertScale is an ordered, one‐dimensional scale fromwhich respondents choose the option that bestalignswiththeirview.

Inordertoclassifytheindicators,thepercentageinwhichtheattributewasmarkedas“Important”and “Very important” was calculated. With these values the following decision criteria wasestablished:

1) The indicatorwas initiallymaintained ifat least66%oftheexpertshaveanswered intheattributeof“Relevance”thatitis“Important”and“Veryimportant”.Otherwise,therestoftheattributes(validity,reliability,feasibilityandcomparability)andallcommentsmadebythe expertswere consideredand revised carefully inorder todecide if the indicatorwasfinallymaintainedornot.

2) Inthecasethattheindicatorwasmaintained,all itsattributeswereagainrevisedsothattheindicatorcouldbekeptwithorwithoutmodifications.

Withthesecriteria,thefollowingcategorieswereproposed:

9

Category1‐Unchanged:Theindicatorismaintainedwithoutchanges Category2‐Modified:Theindicatorismaintainedwithsomechanges Category3‐Doubtful:Theindicatorisstillundecided Category4‐Deleted:Theindicatorhasbeenremoved Category5‐Added:Anewindicatorisproposed

Thoseindicatorsclassifiedincategory4wereremovedandnotevaluatedinthenextround.Theattributesofthecategory3wererevaluatedsinceaconsensushadnotbeenreachedyetamongtheexperts.Theindicatorsfrom the category 5 were those new indicators that have been proposed by some of the experts. As aconsequence, theopportunity for furtherevaluationswasgiven inorder toknow if these indicatorswillbefinallyintroducedornotinthewholesetofindicators.

c. Correctionsintheformulationoftheindicators

Allthesuggestionsandcommentsmadebytheexpertsrelatedtotheimprovementoftheformulationaswellas the terminologyused in the indicatorswere taken into consideration.Theappropriate correctionsweremadeinthoseindicatorswhichwerenotexcludedpermanently.Thenewchangescomparingtothefirstroundresultsweremarkedinlightbluecolour.

Table2showstheinitialandfinalnumberofindicatorsconsideredperdimensionduringthefirstround.

FirstroundQuestionnaires Initialnumberofindicators Finalnumberofindicators

ContinuingEducation 28 21TechnologyTransfer&Innovation 31 23SocialEngagement 36 19

Table2

SecondDelphiround

Thesecondroundwasalsocomposedof threedifferentquestionnaires forCE,SEandTT&I.Thegoalof thesecond roundwas to further examine the indicators obtained during the first round.Here a decisionwasachieved regarding those indicatorswhereaconsensuswasnot reachedduring the first round.Thismeansthat the indicators that were doubtful or modified were again assessed. In addition, the new indicatorsproposedinthefirstroundwereaskedtobeevaluated.Theobjectivewastoachieveanagreement,especiallyonthose indicatorsthatshouldbemaintainedconsideringtheirrelevance,validity,reliability, feasibilityandcomparability.

The responsesobtained in the second roundwereanalysedand summarized inorder tobe circulated lateragainamongtheexperts.

A veryhighdegree of consensuswasachieved.Themajorityof the commentsmade by thepanellistswererelatedtotheterminologyandtheinterpretationofsomeoftheindicators.Table3showstheinitialandfinalnumberofindicatorsconsideredperdimensionduringthesecondround.

10

SecondroundQuestionnaires Initialnumberofindicators Finalnumberofindicators

ContinuingEducation 21 18TechnologyTransfer&Innovation 23 20SocialEngagement 19 16

Table3

3.2Methodologyusedforthe3rdDelphiround

The third round of theDelphiwas structured differently in comparison to the first and second rounds. Inpreviousrounds theexpertpanellistsevaluated independently the indicators foreachdimension throughanonlinesurvey. Incontrast, in thisround theywererequested togiveusaglobalviewandopinionabout thewholesetofindicatorsforallthreedimensions:CE,SEandTTI.Expertsassessedtheimportanceandfeasibilityof every indicator using a rating scale of 1 to 7, from the least to themost important and feasible. Theimportancewas used as the base to identify the relative significance of each indicator, and the feasibilityprovidedacontrastelementforfurtherphasesofthestudy.

Similarly to the preceding rounds, a total of 19 questionnaires were received and evaluated. The resultsachievedfromthethirdDelphiroundwerefurtherprocessedforevaluation.Thecriteriaappliedforanalysingtheresultsofthisroundimpliedcomputingthemeanofthevaluesobtainedbytheexpertsforeachindicator.Thesemeans could be interpreted easily for every indicator, considering the same scale of 1 to 7 of thequestionnaire,fromtheleasttothemostimportantandfeasible.Table4showstheinitialandfinalnumberofindicatorsconsideredperdimensionduringthethirdround.

ThirdroundQuestionnaires Initialnumberofindicators FinalnumberofindicatorsContinuingEducation 18 18TechnologyTransfer&Innovation 20 20SocialEngagement 16 16

Table4

Figure1 showsadiagrampresenting the evolution of thenumberof indicators in the three rounds of theDelphistudy.

11

SECOND ROUND

THIRD ROUND

FIRST ROUND 36i SE

8i Un‐changed

8iModified

2iDoubtful

18iDeleted

1iAdded

19i SE

16iMaintained

3iDeleted

16i SE

31i TTI

8i Un‐changed

10iModified

2iDoubtful

11iDeleted

3iAdded

23i TTI

20iMaintained

3iDeleted

20i TTI

28i CE

12i Un‐changed

6iModified

0iDoubtful

10iDeleted

3iAdded

21i CE

18iMaintained

3i Deleted

18i CE

Figure1

12

4.ResultsobtainedwiththeDelphiprocess

4.1FrameworkofCE,SEandTTIprocesses

Dimension1:ContinuingEducation(CE)

CE2: Implementation of Continuing Education

Activities

CE3: Information and Advertising

CE5: Financial Management

CE4: Application and Admission

Management

CE6: Teaching and Learning

CE7: Quality Evaluation

CE1: Analysis of the Demand and Curriculum

Design

CE0: Institutional Involvement in Continuing

Education

CE8: Final Assessment and Follow-up

Figure2

Dimension2:TechnologyTransfer&Innovation(TTI)

TTI0: Institutional Involvement in Technology Transfer & Innovation

Specific Processes

Generic Processes

TTI1: Licensing of university patents to companies

TTI2: Formation of start-ups & spin-offs

companies

TTI3: Non-patent & software

innovations in public domain-

creative commons &

social innovation

TTI4: Problem solving

cooperation in R&D

TTI5: Public space –sharing

space/facilities/ equipment/services/

networking

TTI6: People – mobility

and education

Figure3

13

Dimension3:SocialEngagement(SE)

SE0: Institutional Involvement in Social Engagement

SE1: Non-discipline volunteering

SE2: Expert advisory engagement

SE3: Services and facilities to

community

SE4: Educational outreach/collaboration

and widening participation

Local National International

Individual

Institutional

Figure4

4.2IndicatorsselectedalongtheDelphiprocess

ContinuingEducation

ThefollowinglistofindicatorswasselectedfortheCEdimension:

CEIndicatorsCE0‐I1:PRESENCEOFCEINTHEMISSIONOFTHEHEICE0‐I2:PRESENCEOFCEINTHEPOLICYAND/ORTHESTRATEGYOFTHEHEICE0‐I3:EXISTENCEOFANINSTITUTIONALPLANFORCEINTHEHEICE0‐I4:EXISTENCEOFQUALITYASSURANCEPROCEDUREFORCEACTIVITIESCE1‐I1:CEPROGRAMMESACTIVEFORIMPLEMENTATIONCE1‐I2:CEPROGRAMMESDELIVEREDWHICHHAVEAMAJORAWARDUNDERHIGHEREDUCATIONSYSTEMCE1‐I3:PARTNERSHIPWITHPUBLICANDPRIVATEBUSINESSCEPROGRAMMESDELIVEREDINTHATYEARCE1‐I4:INTERNATIONALCEPROGRAMMESDELIVEREDCE1‐I5:FUNDEDCETRAININGPROJECTSDELIVEREDCE1‐I6:CREDITSOFTHEDELIVEREDCEPROGRAMMESCE4‐I1:CREDITSENROLLEDCE4‐I2:REGISTRATIONSINCEPROGRAMMESCE4‐I4:CECREDITSENROLLEDREFERREDTOTHETOTALCREDITSENROLLEDCE6‐I1:QUALIFICATIONSISSUEDREFERREDTOTOTALCEREGISTRATIONSCE7‐I1:STUDENTSSATISFACTIONCE7‐I2:KEYSTAKEHOLDERSATISFACTIONCE7‐I3:COMPLETIONRATEFORALLPROGRAMMESCE8‐I1:CEPROGRAMMESWITHEXTERNALACCREDITATIONS

14

Figure5

Figure5showsthatalltheCE indicatorsareconsideredtobesignificantly importantandfeasible.Everyexpertratedtheimportanceandthefeasibilityofalltheseindicatorsabovethemedian.The CE indicators, which belong to the CE0 process, were considered to be the most feasible and/or mostimportantindicatorsfromtheentiresetoftheCEindicatorsexaminedinthethirdDelphiround.ThetablesbelowshowthedescriptionsoftheCEindicatorsselected:

CE0‐i1: CE IS INCLUDED IN THE MISSION OF THE HEI

Purpose Tomeasure thestatusofCEwithintheHEIaswellas thecommitmentof theHEItowardsCE

Definition InclusionofCEinthedefinitionofthemissionoftheHEIInterpretation This indicatormeasures theextentof theHEI’s institutionalcommitment towards

CEonalongtermbasisMeasurement Binary Levelofdatacollection InstitutionFormula(ifapplies) ‐‐ Typeofdatasource InstitutionaldataTimereference Lastyear

15

CE0‐i2: CE IS INCLUDED IN THE POLICY AND/OR THE STRATEGY OF THE HEI

Purpose Tomeasure thestatusofCEwithintheHEIaswellas thecommitmentof theHEItowardsCE

Definition InclusionofCEinthepolicyand/orstrategyoftheHEIInterpretation This indicatormeasures theextentof theHEI’s institutionalcommitment towards

CE on a long term basis. A policy/strategy plan dedicated to CEwith indicatorsreflects the fact that CE is taken into account on theHEI’smanagerial level andfinancialplansaswell

Measurement Binary Levelofdatacollection InstitutionFormula(ifapplies) ‐‐ Typeofdatasource InstitutionaldataTimereference Lastyear

CE0‐i3: EXISTENCE OF AN INSTITUTIONAL PLAN FOR CE IN THE HEI

Purpose Tomeasurethepracticalimplementation‐ organisation,goalsandmeasures– ofCEactivitiesintheHEI.MeasurestheHEI’sinvolvementinCEinpractice

Definition ExistenceofaninstitutionalactionplanforCEintheHEIInterpretation ThisindicatormeasurestheextentoftheactualimplementationofCEintheHEI.An

actionplanwouldrevealorganisationalandadministrativearrangementsaswellasfinancialandintellectualresourcesallocatedforCE


CE0‐i4: EXISTENCE OF QUALITY ASSURANCE PROCEDURE FOR CE ACTIVITIES

Purpose TomeasurethequalityassuranceeffortoftheinstitutionDefinition ExistenceofasetofqualityassuranceproceduresInterpretation IfCEactivitieshaveaqualityassurancesystem,the importancegiventoCEbythe

institution ishigh,andtheperformanceandconsistencyofCEactivitiestendtobehigher


CE1‐i1: CE PROGRAMMES ACTIVE FOR IMPLEMENTATION

Purpose TomeasurethelevelofactivityinCEDefinition TotalnumberofCEprogrammesactiveintheyearofreferenceInterpretation ThisindicatordescribestheoverallCEactivityMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_CEprogrammes Typeofdatasource InstitutionaldataTimereference Lastyear

CE1‐i2: CE PROGRAMMES DELIVERED WHICH HAVE A MAJOR AWARD UNDER HIGHER EDUCATION SYSTEM

Purpose TomeasuretheacademicleveloftheCEactivityDefinition Total number of CE programmes delivered which have a major award under

EuropeanHigherEducationsystemInterpretation ThisindicatormeasurestheacademicleveloftheCEactivityMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_Degree

programmesTypeofdatasource Institutionaldata

Timereference Lastyear

16

CE1‐i3: PARTNERSHIP WITH PUBLIC AND PRIVATE BUSINESS CE PROGRAMMES DELIVERED

Purpose TomeasurethequantitativeoutputsandtheamountofpartnershipinCEactivityDefinition Total number of partnership CE programmes with public and private business

designedandapprovedforimplementationwithanyexternalpartnerintheyearofreference

Interpretation This indicator assesses the activity level, having the focus on partnershipprogrammeswithpublicandprivatebusiness

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_Partnership

programmesTypeofdatasource Institutionaldata


CE1‐i4: INTERNATIONAL CE PROGRAMMES DELIVERED

Purpose TomeasurethequantitativeoutputsandtheinternationalizationofCEactivityDefinition Percentage of international CE programmes designed and approved for

implementationintheyearofreferenceInterpretation ThisindicatorassessestheCEactivityhavingthefocusonprogrammestargetedfor

internationalmarketsandstudentsMeasurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) (N_International

CE programmes /TotalProgrammes)*100

Typeofdatasource Institutionaldata


CE1‐i5: FUNDED CE TRAINING PROJECTS DELIVERED

Purpose To measure the quantitative outputs and the access to external funding by CEactivity

Definition Percentage of funded CE training projects delivered in in the year of referencereferredtothetotalnumberofprogrammes

Interpretation This indicator assesses the effectiveness of the CE activities, having the focus ontraining projects which receive project funding through application or tenderprocedures

Measurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) (N_fundedCE

trainingprojects/TotalProgrammes)*100



CE1‐i6: T CREDITS OF THE DELIVERED CE PROGRAMMES

Purpose TomeasuretheextentoftheCEprogrammesactiveintheyearofreference,viathetotalECTSdeliveredintheseprogrammes

Definition TotalnumberoftheECTScreditsoftheactiveCEprogrammesInterpretation This indicator assesses the activity having the focus on the total workload of

students(ECTScredits)inCEprogrammesMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_ECTS Typeofdatasource InstitutionaldataTimereference Lastyear

17

CE4‐i1: CREDITS ENROLLED

Purpose TomeasurethetotalvolumeofCEactivitiesinaHEIDefinition TotalnumberofECTScreditsoftheenrolledstudentsInterpretation This indicator measures the quantity of CE activities only considering the total

numberofECTScreditsMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_ECTScreditsof

theenrolledstudents



CE4‐i2: REGISTRATIONS IN CE PROGRAMMES

Purpose TomeasurethetotalnumberofpeopleregisteredinCEactivitiesDefinition TotalnumberofregistrationsofstudentsintheCEactivities(notjustthenumberof

students)intheyearofreferenceInterpretation This indicatormeasures the totalnumberofpeopleregistered inCEprogrammes.

This indicator can be interpreted together with CE4‐i1 in order to describe thequantityandintensityofCEactivitiesinaHEI

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_registrationsin

CEprogrammesTypeofdatasource Institutionaldata


CE4‐i4: CE CREDITS ENROLLED REFERRED TO THE TOTAL NUMBER OF CREDITS ENROLLED

Purpose TomeasuretherelativeimportanceofCEactivitiesDefinition PercentageofCEECTSenrolledreferredtothetotalECTSenrolledintheHEIsInterpretation The percentage of the total ECTS from CE activities indicates the relative

importanceofCEfortheHEIactivitiesMeasurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) (CEECTSenrolled

/totalECTSenrolled)*100



CE6‐i1: QUALIFICATIONS ISSUED REFERRED TO TOTAL CE REGISTRATIONS

Purpose TomeasuretheresultsoftheCEprogrammesDefinition PercentageofqualificationsissuedreferredtototalCEregistrationsInterpretation This indicatormeasures the relativeamountof thequalifications inCE,providing

informationabouttheresultsoftheCEprogrammesMeasurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) N_qualifications

issued/N_CEregistration



18

CE7‐i1: STUDENTS SATISFACTION

Purpose TomeasuretheglobalstudentsperceptionabouttheinstitutionDefinition SatisfactionlevelofstudentsInterpretation Aspartoftheobjectivesoftheinstitution,thesatisfactionofthestudentsmakes,for

thethirdmission,aroleasimportantascustomersatisfactionforamanufacturingcompany. This satisfaction must be measured by directly asking the differentstudentsaboutit,usingasurveymethodology

Measurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) %of3‐4answersina4point

scaledegreeofsatisfactionquestion(0%=completelydissatisfied,100%=completelysatisfied)

Typeofdatasource Surveydata


CE7‐i2: KEY STAKEHOLDER SATISFACTION

Purpose TomeasurethekeystakeholdersperceptionabouttheinstitutionDefinition SatisfactionlevelofkeystakeholdersInterpretation Thekeystakeholderssatisfactionmustbemeasuredbydirectlyaskingthedifferent

studentsaboutit,usingasurveymethodologyMeasurement Percentage Level of data

collectionInstitution

Formula(ifapplies) %of3‐4answersina4pointscaledegreeofsatisfactionquestion(0%=completelydissatisfied,100%=completelysatisfied)

Type of datasource

Surveydata


CE7‐i3: COMPLETION RATE FOR ALL PROGRAMMES

Purpose TomeasuretheefficiencyoftheprogrammesforattendantsDefinition AveragecompletionrateforallprogrammesInterpretation Therecanbemanycausesforattendantsfailingincompletingaprogramme,andthere

will be always a certain non‐completing rate. Among this causes, the lack ofcorrespondencebetweenattendant interestsandprogramme characteristics isoneofthe most important. In any case, completion rate can be considered also as anevaluationoftheinterestleveloftheprogramme

Measurement Percentage Level of datacollection

Institution

Formula(ifapplies)

Ni =Number of attendants to programme i CRi =Completion rate of programme i (in percentage) k = number of programmes considered

Type of datasource

Institutionaldata


19

CE8‐i1: CE PROGRAMMES WITH EXTERNAL ACCREDITATIONS

Purpose TomeasurethequalityoftheCEprogrammesDefinition Percentage of accredited programmes by national or international agencies and

officialbodiesInterpretation Accreditation needs a big effort of theHEI. If theHEI has a large proportion of

accreditedprogrammessotheimportanceofCEfortheHEImustbealsohighMeasurement Percentage Level of data

collectionFaculty/Department/CEcentre

Formula(ifapplies) (Accreditedprogrammeswithexternalaccreditations/NTotalprogrammesoffered)*100

Type of datasource

Survey data, programme lists,programmefolderandwebsites


TechnologyTransfer&Innovation

ThefollowinglistofindicatorswasselectedfortheTTIdimension:

TTIIndicatorsTTI0‐i1:PRESENCEOFTTIINTHEMISSIONOFTHEHEITTI0‐i2:PRESENCEOFTTIINTHEPOLICYAND/ORSTRATEGYOFTHEHEITTI0‐i3:EXISTENCEOFANINSTITUTIONALACTIONPLANFORTTIINTHEHEITTI1‐i1:LICENSES,OPTIONSANDASSIGNMENTS (ACTIVEANDEXECUTED,EXCLUSIVEANDNON‐EXCLUSIVE)TOSTART‐UPSORSPIN‐OFFSANDEXISTINGCOMPANIESTTI1‐i2:BUDGETCOMINGFROMREVENUESFROMCOMMERCIALISATIONOFHEIKNOWLEDGETTI2‐i1:START‐UPSANDSPIN‐OFFSESTABLISHEDTTI3‐i1:CREATIVECOMMONSANDSOCIALINNOVATIONPROJECTSTHATHEIEMPLOYEESAREINVOLVEDINTTI4‐i2: R&D SPONSORED AGREEMENTS, CONTRACTS AND COLLABORATIVE PROJECTSWITH NON‐ACADEMICPARTNERSTTI4‐i3:BUDGETCOMINGFROMINCOMEOFR&DSPONSOREDCONTRACTSANDCOLLABORATIVEPROJECTSWITHNON‐ACADEMICPARTNERSTTI4‐i4:CONSULTANCYCONTRACTSTTI4‐i5:POSTGRADUATESTUDENTSANDPOSTDOCTORALRESEARCHERSDIRECTLYFUNDEDORCO‐FUNDEDBYPUBLICANDPRIVATEBUSINESSESTTI5‐i1:CREATED(CO‐FUNDED)ORSHAREDLABORATORIESANDBUILDINGSTTI6‐i2:COMPANIESPARTICIPATINGINCONTINUOSPROFESSIONALDEVELOPMENTCOURSES(CPD)TTI6‐i3:HEIEMPLOYEESWITHTEMPORARYPOSITIONSOUTSIDEOFACADEMIATTI6‐i4:NON‐ACADEMICEMPLOYEESWITHTEMPORARYPOSITIONSATHEISTTI6‐i5:POSTGRADUATETHESESORPROJECTSWITHNON‐ACADEMICCO‐SUPERVISORSTTI6‐i7:JOINTPUBLICATIONSWITHNON‐ACADEMICAUTHORSTTI6‐i8:ACADEMICSTAFFPARTICIPATINGINPROFESSIONALBODIES,NETWORKS,ORGANIZATIONSANDBOARDSTTI6‐i9: EXTERNAL ORGANIZATIONS OR INDIVIDUALS PARTICIPATING AT ADVISORY, STEERING, VALIDATION,REVIEWBOARDSTOHEIS,INSTITUTES,CENTRESORTAUGHTPROGRAMMESTTI6‐i10: PRESTIGIOUS INNOVATION PRIZES AWARDED BY BUSINESS AND PUBLIC SECTOR ASSOCIATIONS ORFUNDINGAGENCIES(NATIONALANDINTERNATIONAL)

20

Figure6Figure 6 shows that TTI indicators examined in the thirdDelphi roundwere rated highly above themedianregardingtheimportance.Ingeneral,alltheindicatorsareabove4inthefeasibility,excepttheindicatorsTTI6‐i3andTT3‐i1thattheexpertsratedbelowthemedian.AsintheCEdimension,theindicatorsundertheTTI0processwerethemostrelevantandfeasibleones.ThedescriptionsoftheTTIindicatorsselectedarepresentedinthetablesbelow:

TTI0‐i1: TTI IS INCLUDED IN THE MISSION OF THE HEI

Purpose TomeasurethestatusofTTIwithintheHEIaswellasthecommitmentoftheHEItowardsTTI

Definition InclusionofTTIinthedefinitionofthemissionoftheHEIInterpretation This indicatormeasures theextentof theHEI’s institutionalcommitment towards

TTIonalongtermbasisMeasurement Binary Levelofdatacollection InstitutionFormula(ifapplies) ‐‐ Typeofdatasource InstitutionaldataTimereference Lastyear

21

TTI0‐i2: TTI IS INCLUDED IN THE POLICY AND/OR STRATEGY OF THE HEI

Purpose To measure the status of TTI within the HEI as well as the commitment of the HEI towards TTI

Definition Inclusion of TTI in the policy and/or strategy of the HEI

Interpretation This indicator measures the extent of the HEI’s institutional commitment towards TTI on a long term basis. A policy/strategy plan dedicated to CE with indicators reflects the fact that TTI is taken into account on the HEI’s managerial level and financial plans as well

Measurement Binary Level of data collection Institution

Formula (if applies) ‐‐ Type of data source Institutional data

Time reference Last year

TTI0‐i3: EXISTENCE OF AN INSTITUTIONAL ACTION PLAN FOR TTI IN THE HEI

Purpose Tomeasure the practical implementation‐ organisation, goals andmeasures – ofTTIactivitiesintheHEI.MeasurestheHEI’sinvolvementinTTIinpractice

Definition ExistenceofaninstitutionalactionplanforTTIintheHEIInterpretation This indicatormeasurestheextentoftheactual implementationofTTI intheHEI.

An action planwould reveal organisational and administrative arrangements aswellasfinancialandintellectualresourcesallocatedforCE


TTI1‐i1: LICENSES, OPTIONS AND ASSIGNMENTS (ACTIVE AND EXECUTED, EXCLUSIVE AND NON‐EXCLUSIVE) TO START‐UPS OR SPIN‐OFFS AND EXISTING COMPANIES

Purpose Tomeasurea specificmechanismofTT&Iwhich isdirectlyaimedatcommercialisingHEIknowledge

Definition Number of licenses, options and assignments (active & executed, exclusive & non‐exclusive)tostart‐ups/spin‐off&existingcompanies

Interpretation Thisindicatormeasuresallkindoflicenses,optionsandassignmentstocompaniesMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies)

(N_licenses+N_options+N_assignments)tostart‐upsorspin‐offandexistingcompanies



TTI1‐i2:BUDGET COMING FROM REVENUES FROM COMMERCIALISATION OF HEI KNOWLEDGE

Purpose To measure a specific mechanism of TT&I which is directly aimed atcommercialisingHEIknowledge

Definition PercentageoftotalbudgetgeneratedfromcommercialisationofHEIknowledge,e.g.licensingincome,totalearnedroyaltyincome(ERI)

Interpretation This indicator measures the percentage of the total budget coming fromcommercialisationofHEIknowledge

Measurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) (Totalrevenuefrom

commercialisationofHEIknowledge/TotalHEIbudget)*100

Type ofdatasource Institutionaldata


22

TTI2‐i1: START‐UPS AND SPIN‐OFFS STABLISHED

Purpose To measure a specific mechanism of TT&I which is directly aimed atcommercialisingHEIknowledge

Definition Totalnumberofstart‐upsandspin‐offsestablishedInterpretation Thisindicatormeasuresatotalnumberofstart‐upsandspin‐offsestablishedMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) (N_start‐ups+N_spin‐offs)

establishedTypeofdatasource Institutionaldata


TTI4‐i2: R&D SPONSORED AGREEMENTS, CONTRACTS AND COLLABORATIVE PROJECTS WITH NON‐ACADEMIC PARTNERS

Purpose To measure problem solving activities/cooperation in R&D with non‐academicpartners. This process could be viewed as the input to the technology developmentand/orimprovement

Definition NumberofR&Dsponsoredagreements,contractsandcollaborativeprojectswithnon‐academicpartners

Interpretation This indicator measures a number of R&D sponsored agreements, contracts andcollaborativeprojectswithnon‐academicpartners

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies)

(N_R&Dsponsoredagreements+N_contracts+N_collaborativeprojects)withnon‐academicpartners



TTI3‐i1: CREATIVE COMMONS AND SOCIAL INNOVATION PROJECTS THAT HEI EMPLOYEES ARE INVOLVED IN

Purpose To measure the engagement of HEI staff in non‐patent public domainentrepreneurialactivities,includingcreativecommons&socialinnovation

Definition NumberofcreativecommonsandsocialinnovationprojectsInterpretation Thisindicatormeasuresanumberofnon‐patentinnovationprojectsofHEIstaffMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) (N_creativecommons+

N_socialinnovationprojects)thatHEIemployeesareinvolved



23

TTI4‐i3: BUDGET COMING FROM INCOME OF R&D SPONSORED CONTRACTS AND COLLABORATIVE PROJECTS WITH NON‐ACADEMIC PARTNERS

Purpose To measure problem solving activities/cooperation in R&D with non‐academicpartners.Thisprocesscouldbeviewedasthe inputtothetechnologydevelopmentand/orimprovement

Definition PercentageofHEIbudgetcomingfromR&Dsponsoredcontractsandcollaborativeprojectswithnon‐academicpartners

Interpretation ThisindicatormeasurestheimportanceofincomeofR&Dsponsoredcontractsandcollaborativeprojectswithnon‐academicpartnersfortheHEI

Measurement Percentage Levelofdatacollection Institutionand/orfaculty

Formula(ifapplies) 100*((TotalincomeofR&Dsponsoredcontracts+Totalincomeofcollaborativeprojectswithnon‐academicpartners)/TotalHEIbudget)



TTI4‐i4: CONSULTANCY CONTRACTS


Definition Numberofconsultancycontractswithnon‐academicpartnersInterpretation This indicator measures a number of consultancy contracts with non‐academic

partnersMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_consultancycontracts Typeofdatasource InstitutionaldataTimereference Lastyear

TTI4‐i5: POSTGRADUATE STUDENTS AND POSTDOCTORAL RESEARCHERS DIRECTLY FUNDED OR CO‐FUNDED BY PUBLIC AND PRIVATE BUSINESSES


Definition Percentage of postgraduate students/postdoctoral researchers directly funded orco‐fundedbypublicandprivatebusinesses

Interpretation Thisindicatormeasuresthedegreeofcooperationofpublicandprivatebusinesseswithuniversitiesinthetrainingofresearchers

Measurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) (N_postgraduatestudents+

N_postdoctoralresearchersdirectlyfundedorco‐fundedbypublicandprivatebusinesses/Totalnumberofpostgraduateandpostdoctoralstudents)*100



24

TTI6‐i2: COMPANIES PARTICIPATING IN CONTINUOS PROFESSIONAL DEVELOPMENT COURSES (CPD)

Purpose Tomeasureinteractionswithnon‐academicpartners.Thisprocesscouldbeviewedastheinputtothetechnologydevelopmentand/orimprovement

Definition NumberofcompaniesparticipatinginCPDcoursesInterpretation Thisindicatormeasuresthecooperationbetweenpublicandprivatecompaniesand

theuniversityinCPDcoursesMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_companiesparticipatinginCPD

coursesTypeofdatasource Institutionaldata

and/or surveydata


TTI6‐i3: HEI EMPLOYEES WITH TEMPORARY POSITIONS OUTSIDE OF ACADEMIA

Purpose Tomeasuremobilityofacademicstaff.Thisprocesscouldbeviewedastheinputtothetechnologydevelopmentand/orimprovement

Definition NumberofHEIemployeeswithtemporarypositionsoutsideacademia–sabbaticalsInterpretation Thisindicatormeasuresthemobilityofacademicstaffprovidinginformationabout

therelationshipbetweenacademiaandtheexternalenvironmentMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_HEIemployeeswithtemporary

positionsoutsideofacademia/TotalHEIemployees



TTI5‐i1: CREATED (CO‐FUNDED) OR SHARED LABORATORIES AND BUILDINGS

Purpose Tomeasure joint access toR&D space/facilities/equipment/services. This processcouldbeviewedastheinputtothetechnologydevelopmentand/orimprovement

Definition Numberofcreated(co‐funded)and/orsharedlaboratories/buildings/facilitiesInterpretation Thisindicatormeasuresthedegreeofcooperationofthepublicandprivatebusiness

withuniversityinsharingfacilitiesMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_created(co‐funded)orshare

laboratories+N_created(co‐funded)orsharebuildings


Timereference Over5years

25

TTI6‐i4: NON‐ACADEMIC EMPLOYEES WITH TEMPORARY POSITIONS AT HEIS

Purpose Tomeasuremobilityofnon‐academicpartners.Thisprocesscouldbeviewedastheinputtothetechnologydevelopmentand/orimprovement

Definition Numberofnon‐academicemployeeswithtemporarypositionsattheHEI,e.g.part‐timelecturerand/ordoingtheirmasterordoctorate

Interpretation This indicator measures the mobility of non‐academic employees providinginformationabouttherelationshipbetweenacademiaandtheexternalenvironment

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_non‐academicemployeeswith

temporarypositionsatHEIs/TotalHEIemployees



TTI6‐i5: POSTGRADUATE THESES OR PROJECTS WITH NON‐ACADEMIC CO‐SUPERVISORS

Purpose Tomeasurecollaborationwithnon‐academicpartners.Thisprocesscouldbeviewedastheinputtothetechnologydevelopmentand/orimprovement

Definition Numberofpostgraduatesthesisorprojectswithnon‐academicco‐supervisorsInterpretation This indicator measures the degree of collaboration of non‐academic partners in

researchactivitiesMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_postgraduatetheses +

N_postgraduateprojectswithnon‐academicco‐supervisors



TTI6‐i7: JOINT PUBLICATIONS WITH NON‐ACADEMIC AUTHORS

Purpose Tomeasurecollaborationwithnon‐academicpartners.Thisprocesscouldbeviewedastheinputtothetechnologydevelopmentand/orimprovement

Definition Numberofjointpublicationswithnon‐academicauthorsInterpretation All kind of publications in peer‐reviewed journals, professional magazines and

conferenceproceedingsMeasurement Numerical Levelofdatacollection Institution

and/or publicdatasets

Formula(ifapplies) N_jointpublicationswithnon‐academicauthors

Typeofdatasource Institutionaldataand/orbibliometricdata


26

TTI6‐i8: ACADEMIC STAFF PARTICIPATING IN PROFESSIONAL BODIES, NETWORKS, ORGANIZATIONS AND

BOARDS

Purpose Tomeasurecollaboration/mobilitywithnon‐academicpartners.Thisprocesscouldbeviewedastheinputtothetechnologydevelopmentand/orimprovement

Definition Percentage of academic staff participating in professional bodies, networks,organizationsandboards

Interpretation Thisindicatormeasurestheinvolvementofacademicstaffinexternal,professionalandscientificorganizations

Measurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) (N_academicstaffparticipatingin

professionalbodies,networks,organizationsandboards/Totalacademicstaff)*100

Typeofdatasource Institutionaldata, public datasets and/orsurveydata


TTI6‐i9: EXTERNAL ORGANIZATIONS OR INDIVIDUALS PARTICIPATING AT ADVISORY, STEERING,

VALIDATION, REVIEW BOARDS TO HEIS, INSTITUTES, CENTRES OR TAUGHT PROGRAMMES

Purpose Tomeasurecollaboration/mobilitywithnon‐academicpartners.Thisprocesscouldbeviewedastheinputtothetechnologydevelopmentand/orimprovement

Definition Number of external organizations/individuals participating atadvisory/steering/validation/review boards to HEIs/institutes/centres/ taughtprogrammes

Interpretation This indicator measures the involvement of external organizations in HEorganizationalstructures

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies)

(N_externalorganizations+N_individuals)participatingatadvisory,steering,validation,reviewboardstoHEIs,institutes,centresortaughtprogrammes

Typeofdatasource Institutionaldata or surveydata


TTI6‐i10: PRESTIGIOUS INNOVATION PRIZES AWARDED BY BUSINESS AND PUBLIC SECTOR ASSOCIATIONS

OR FUNDING AGENCIES (NATIONAL AND INTERNATIONAL)

Purpose To measure collaboration with non‐academic partners. This process could beviewedastheinputtothetechnologydevelopmentand/orimprovement

Definition Number of prestigious innovation prizes awarded by business & public sectorassociations/fundingagencies(national/international)

Interpretation RecognitionofoutstandingTT&IcontributionsMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_prestigiousinnovationprizes

awardedbybusinessandpublicsectorassociationsorfundingagencies(nationalandinternational)

Typeofdatasource Institutionaldata and/orPublicdatasets


27

SocialEngagement

ThefollowinglistofindicatorswastheselectedfortheSEdimension:

SEIndicatorsSE0‐i1:PRESENCEOFSEINTHEMISSIONOFTHEHEISE0‐i2:PRESENCEOFSEINTHEPOLICYAND/ORSTRATEGYOFTHEHEISE0‐i3:EXISTENCEOFANINSTITUTIONALACTIONPLANFORSEINTHEHEISE0‐i4:BUDGETARYASSIGNMENTTOSESE2‐i1:ACADEMICSINVOLVEDINVOLUNTEERINGADVISORYSE3‐i1:EVENTSOPENTOCOMMUNITY/PUBLICSE3‐i2:RESEARCHINITIATIVESWITHDIRECTIMPACTONTHECOMMUNITYSE3‐i4: COST OF STAFF/STUDENT HOURS MADE AVAILABLE TO DELIVER SERVICES AND FACILITIES TOCOMMUNITYSE3‐i5:PEOPLEATTENDING/USINGFACILITIESSE4‐i1:PROJECTSRELATEDTOEDUCATIONALOUTREACHSE4‐i2:ACADEMICSTAFFANDSTUDENTSINVOLVEDINEDUCATIONALOUTREACHACTIVITYSE4‐i4:BUDGETUSEDFOREDUCATIONALOUTREACHSE4‐i5:COMMUNITYPARTICIPANTSINEDUCATIONALOUTREACHACTIVITYSE4‐i7:ACTIVITIESSPECIFICALLYTARGETINGDISADVANTAGEDSTUDENTS/COMMUNITYGROUPSSE4‐i9:COMMUNITYREPRESENTATIVEONHEBOARDSORCOMMITTEESSE4‐i11:GRANTS/DONATIONS/CONTRACTSARISINGFROMENGAGEDPARTNERSHIPS

Figure7

28

Figure7showsthatthesetofSE indicatorshadanacceptable levelof importance(morethan4 ina1‐7ratingscale),but in comparison toCEandTTI indicators,SE indicatorswere less feasible. In this case the indicatorswhichbelong toprocessSE0were themost importantandat the same timemost feasible indicators,with theexceptionofSE0‐i4.

ThedescriptionsoftheSEindicatorsselectedinthestudyareshowninthetablesbelow:

SE0‐i1: SE IS INCLUDED IN THE MISSION OF THE HEI

Purpose Tomeasurethecommitment oftheHEItowardsSEDefinition InclusionofSEinthedefinitionofthemissionoftheHEIInterpretation ThisindicatorevaluatesthecommitmentoftheHEIattheadministrationleveland

onalongtermbasisMeasurement Binary Levelofdatacollection InstitutionFormula(ifapplies) ‐‐ Typeofdatasource InstitutionaldataTimereference Lastyear

SE0‐i2: SE IS INCLUDED IN THE POLICY AND/OR STRATEGY OF THE HEI

Purpose Tomeasure thestatusofSEwithin theHEIaswellas thecommitmentof theHEItowardsSE

Definition InclusionofSEinthepolicyand/orstrategyoftheHEIInterpretation ThisindicatormeasurestheextentoftheHEI’sinstitutionalcommitmenttowardsSE

onalongtermbasis.Apolicy/strategyplandedicatedtoSEwithindicatorsreflectsthefactthatSEistakenintoaccountontheHEI’sadministrativelevelandfinancialplansaswell


SE0‐i3: EXISTENCE OF AN INSTITUTIONAL ACTION PLAN FOR SE IN THE HEI

Purpose Tomeasurethepracticalimplementation ‐ organisation,goalsandmeasures– ofSEactivitiesintheHEI.MeasurestheHEI’sinvolvementinSEinpractice

Definition ExistenceofaninstitutionalactionplanforSEintheHEIInterpretation ThisindicatormeasurestheextentoftheactualimplementationofSEintheHEI.An

actionplanwouldrevealorganisationalandadministrativearrangementsaswellasfinancialandintellectualresourcesallocatedforCE


29

SE0‐i4: BUDGETARY ASSIGNMENT TO SE

Purpose TomeasuretheactualSEcompromiseoftheHEIDefinition PercentageofthetotalHEIbudgetassignedtobudgetaryassignmenttoSEInterpretation Thisindicatorevaluatestheactual andspecificlevelofinvolvementoftheHEIinSEMeasurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) (Budgetary

assignmenttoSE/TotalHEIbudget)*100



SE2‐i1: ACADEMICS INVOLVED IN VOLUNTEERING ADVISORY

Purpose To measure the involvement of academics in volunteering advisory activitiestowardsthecommunity

Definition Percentageofacademics(intermsofFTE)involvedinvolunteeringadvisoryInterpretation This indicator evaluates the extent and engagement of the academics in

volunteeringadvisorytowardsthecommunityMeasurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) (N_academics

involvedinvolunteeringadvisory/Totaln_academics)*100

Typeofdatasource Institutionaldata/Surveydata


SE3‐i1: EVENTS OPEN TO COMMUNITY/PUBLIC

Purpose To measure the numbers of events held by the HEI open to the general public(excludinginvitation‐onlyevents)

Definition NumbersofeventsheldbytheHEIopentothegeneralpublic(excludinginvitation‐onlyevents)

Interpretation Eventsorganisedordeliveredby theHEI, freeor charged,whichareopen to thegeneralpublicwithoutneedinganinvitationtoattend(e.g.concert;artexhibition;lectures;opendays)

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_eventsperyear Typeofdatasource InstitutionaldataTimereference Lastyear

30

SE3‐i2: RESEARCH INITIATIVES WITH DIRECT IMPACT ON THE COMMUNITY

Purpose Tomeasure the level of community‐based research and researchwith a policyimplicationand/orbenefitforthecommunity

Definition The level of community‐based research and research with an explicit policyimplicationand/orexplicitbenefitforthecommunity

Interpretation Researchmust be carried outwith a stated benefit for the broader community,whetheritiscollaborativeresearchorHEI‐drivenresearch

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_research

projectsTypeofdatasource Institutionaldata


SE3‐i4: COST OF STAFF/STUDENT HOURS MADE AVAILABLE TO DELIVER SERVICES AND FACILITIES TO

COMMUNITY

Purpose Tomeasure thecostof staff/studenthoursmadeavailable todeliver servicesandfacilitiestocommunity

Definition The costof staff/studenthoursmadeavailable todeliver servicesand facilities tocommunity

Interpretation Thehumancostoffacilitiesbeingmadeavailabletothepublic(e.g.costoflifeguardandadminstaffatHEIswimmingpoolwhenopentopublic;costofoptometrystaffandstudentsofferingfreeeyetests;adminsupport/buildingsmaintenancestaffforroomhire)

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_staffxhoursx

hourlycost,plusN_studentsxhoursx¼ofequivalentstaffhourlycost



SE3‐i5: PEOPLE ATTENDING/USING FACILITIES

Purpose TomeasuretheextentofprovisionofservicesbytheHEIandtheirrelevancetothepublicbyquantifyingattendance

Definition Numberofpeopleattending/usinglow‐cost/freefacilitiesofferedbyHEIInterpretation TheextentofprovisionofservicesbytheHEIandtheirrelevancetothecommunity

byquantifyingattendanceMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_people

attending/usingfacilities



31

SE4‐i2: ACADEMIC STAFF AND STUDENTS INVOLVED IN EDUCATIONAL OUTREACH ACTIVITY

Purpose To measure the effort of academic staff and students in Educational Outreachactivities

Definition The number of academic staff and students that declare to have undertaken anEducationalOutreachactivityinthepasttwelvemonths

Interpretation “Academic staff and students” refers to all the HEI personnel and the enrolledstudents involved in an activity suchasEducationalOutreach project could havealsoacomponenttargetedtoHEinstitutionalbeneficiariesbutmostoftheactivityissupposedtohaveexternaltargets

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_academicstaff

andstudentsTypeofdatasource Institutionaldata/Surveydata


SE4‐i4: BUDGET USED FOR EDUCATIONAL OUTREACH

Purpose TomeasuretheeffortofHEIinsupportingEducationalOutreachactivitiesthroughinternalresources

Definition PercentageofHEIbudgetusedforEducationalOutreachInterpretation IncaseofprojectwithaEducationalOutreachcomponentitreferstotheamountof

budgetspecificallydedicatedtoitMeasurement Percentage Levelofdatacollection InstitutionFormula(ifapplies) (Internalamount

offundingallocatedbyHEItoEducationalOutreach/TotalHEIBudget)*100



SE4‐i1: PROJECTS RELATED TO EDUCATIONAL OUTREACH

Purpose TomeasuretheactivityofEducationalOutreachprojectsonnon‐studentpopulationDefinition Number of EducationalOutreach project targeting non‐institutional beneficiaries

outsidetheHEorganizationInterpretation An Educational Outreach project could have also a component targeted to HE

institutional beneficiaries but most of the activity is supposed to have externaltargets

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_projectsrelated

toEducationalOutreach



32

SE4‐i5: COMMUNITY PARTICIPANTS IN EDUCATIONAL OUTREACH ACTIVITY

Purpose To measure the ability of HEI in attracting and mobilizing external citizens inEducational Outreach activities. It quantifies the number of people outside HEIsinvolvedinEducationalOutreachactivity

Definition ThenumberofpeopleoutsideHEIs that takeadvantageofEducationalOutreachactivityinthereferenceyear

Interpretation Ifapersonparticipatesinseveralactivitiesitwillcountasmanytimesasthepersonparticipatesinsuchactivities

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_community

participantsTypeofdatasource Institutionaldata


SE4‐i7: ACTIVITIES SPECIFICALLY TARGETING DISADVANTAGED STUDENTS /COMMUNITY GROUPS

Purpose To measure the effort of HEIs in developing activities specifically designed fordisabledorsociallydisadvantagedcommunities

Definition Number of activities specifically designed for disabled or socially disadvantagedcommunities in order to enable betteraccess to knowledge, economicalmeansoreducationalopportunities

Interpretation Ameasureoftheinvolvement withcommunityMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_activities

specificallytargetingdisadvantagedstudents/communitygroups



SE4‐i9: COMMUNITY REPRESENTATIVE ON HE BOARDS OR COMMITTEES

Purpose Tomeasure the extent of involvement of local institutions in theHE activities ingeneral

Definition NumberofcommunityrepresentativeinHEboardsorcommitteesInterpretation Ifacommunityrepresentativesitsinmorethanonecommitteecountsthenumberof

committeescoveredMeasurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) N_community

representativeonHEboardsandcommittees



33

SE4‐i11: GRANTS/DONATIONS/CONTRACTS ARISING FROM ENGAGED PARTNERSHIPS

Purpose Tomeasurethecommitmentto,andsuccessin,gettingfundingforSEPartnershipsDefinition ValueperyearoffundingfrompartnershipsinSEactionsInterpretation ItisthevalueperyearoffinancingfrompartnersinSEactions.Itreflectsthedegree

ofmotivationcreatedbyeachHEI in thepartnersofSEactions. It issomehowanindexofthecapacitytoengagepartnersinSEactionspromotedbyHEIs

Measurement Numerical Levelofdatacollection InstitutionFormula(ifapplies) Fundsgainedfor

SEactionscomingfromexternalsources(notcommunitypartners)



34

5. Conclusionsandfinalcomments

As a result of the implementation of theDelphi technique a set of relevant indicators that describe the thirdmissionactivitiesofHEIshasbeenobtained.Moreover,itwasobservedthattheDelphimethodologyhasservedto:

‐ Provetheusefulnessofthemethodfortherefinementoftheinitialcollectionofindicators.

‐ Demonstratethevalueoftheexperts’opinionintheprocessofselectingasetofrelevantinformationfor

theevaluationfortheThirdMissionactivity.

‐ Show that all final indicators have been rated above themedian in relation to importance. Thiswas

expectedconsideringthethreeroundprocessusedandthenatureoftheDelphimethod.Thiscontributes

toachievingrobustnessoftheresults.

‐ DemonstratethatthereisageneralagreementonthefactthatCEindicatorsarethemostfeasible.Onthe

otherhand,therearesomedoubtsaboutthefeasibilityofsomeoftheSEindicators.

‐ Identify that the indicatorsofprocess0 (related to institutionalcommitment toThirdMission)are the

mostsignificantinthethreedimensions.

‐ Noticethatalthoughall indicatorsareconsideredvery important forthestudy,notallare inthesame

wayeasytomeasureandquantify.

‐ Show thatdifferentpropertiesofthe indicators, likerelevanceand feasibility,havedemonstrated tobe

usefulforratingdifferentaspectsofthevalueoftheinformationhandled.

35

6. Projectparticipants

The consortium formed for this Project consists of the following institutions and coordinators:

UniversitatPolitècnicadeValència,Spain

José‐MiguelCarot

AndrésCarrión

UniversityofHelsinki;Finland

KaukoHämäläinen

Donau‐UniversitätKrems,Austria

AttilaPausits

UniversityofMaribor,Slovenia

MarkoMarhl

UniversidadedoPorto,Portugal

AlfredoSoeiro

IstitutoSuperioreMarioBoella,Italy

StefanoBoffo

DublinInstituteofTechnology,Ireland

MikeMurphy

UniversidaddeLeón,Spain

JavierVidal

External experts:

Christopher Padfield, University of Cambridge

José‐Ginés Mora, Institute of Education, University of London

36

7. Acknowledgements

We would like to thank for their contribution to the project to the following:

Vesa Harmaakorpi Professor, Lappeenranta University of Technology, Lahti School of Innovation

Asta Manninen Director of City of Helsinki, Centre of Urban Facts

Jussi Onnismaa Adjunct professor, University of Helsinki, Palmenia Centre for Continuing Education

Mika Tuuliainen Liaison Manager, University of Helsinki Career Services

Fabio Di Pietro Professor, University of Sassari

Giunio Luzzatto Professor, University of Genova

Marino Regini University of Milan

Tom Collins Vice President for External Affairs and Dean of Teaching and Learning, National University of Ireland

Maynooth

Dermot Coughlan Director, Centre for Life Long Learning & Outreach, University of Limerick

Josephine Boland Senior Lecturer, School of Education, National University of Ireland

Jeroen Huisman Professor of Higher Education Management, Director of the International Centre for Higher Education

Management, University of Bath

Jan Sadlak President of IREG Observatory on Ranking and Academic Excellence and Vice‐Rector for International

Cooperation, Warsaw School of Social Sciences and Humanities

Gero Federkeil Manager in Charge of Rankings, CHE‐Centre for Educational Development

Pat Davies Former EUCEN Executive Secretary and Projects Director

Kari Seppala Director of the Centre for Extension Studies, University of Turku

Francesc Pedró Senior Policy Analyst, Centre for Educational Research and Innovation (CERI), OECD

37

Lars Miikki Senior Consultant at Tomi Järvelin Design Oy

Guy Haug International expert in Higher Education

Uwe Brandenburg CHE Consultant

38

8. Annex.Modelofonlinesurveys

39

40

41

Date post:	28-Dec-2016
Category:	Documents
Upload:	vuthuan
View:	220 times
Download:	2 times

Final Report of Delphi Study

Documents