1

SABER – Systems Approach for Better Education ResultsTertiary Education Governance

World Congress of Colleges and PolytechnicsHalifax, May 26, 2012

Quentin Wodon, World Bank

2

Structure of presentationSABER goals, scope, and methodologyContext for tertiary education governanceSystem-wide and institution-level indicatorsExample of data collection for institutions:

MENAAnalysis of MENA data on autonomy

Financial autonomyHR autonomyAcademic autonomy

Conclusion

3

What SABER isNew initiative to advance Learning for All

First detailed, disaggregated database of education policies/institutions in core areas

Open data tool for empowering stakeholders Two key areas:

Maps out policies/institutions Links to implementation data

4

Goal: Make visible what’s underwaterInputs and (some) outcomes

Everything else:• Policies• Institutions• Implementation• Effects of interventions

5

Domain development: Key stepsWhat Matters paperIndicators and scoring rubricData-collection instrumentData collectionAnalysisData validationPublication of data & analyses

6

What SABER provides(not just ratings)

Analytical framework for thinking about domainDescriptive data on policies/institutionsEvaluative judgments (ratings, not rankings)

“Latent” (poor performance)“Emerging” (insufficient performance)“Established” (adequate performance) “Advanced” (outstanding performance)

 

7

SABER informs policy choices & diagnoses gaps in implementation in each

domain

Catalog & assess quality of policy

framework

POLICIES IMPLEMENTATION+ OUTCOMESCollect &

analyze data on policy execution

• Framework• Collection instrument• Rubric• Manual

• Survey data (e.g. PETS, QSDS, Absence, Household)

• Proxy indicators (e.g. economy-wide metrics, survey data from other countries)

SAB

ER

Too

lkit

Country, regional, and policy domain reports with interpretation, including expert judgment

Tools to benchmark policies Tools to assess implementation based on available data

Online knowledge base

8

Tertiary education governanceLarge variation in performance of higher education

institutions – governance as a key determinantDifferences in objectives: research, teaching, contribution

to local economy, etc. Not one “best” modelBenchmarking as step towards monitoring performanceThree levels of analysis

Institutions level: Identification of Strengths and weaknesses, base line for comparison overtime

Country level: Identification of models, differences between institutions, correlation between models and performance

International level: Comparisons between models, correlations between models and performance

9

System Level Context, Mission and Goals

Management Autonomy Accountability Participation

Government-driven

Government-defined missions and policies

Government-appointed president

Centrally managed budgetCentral control for new programs and curriculumCentral HR management

Central auditsCentral QA National driven curriculumLow accountability-no links between performance and rewards

Mainly on consultation basis

Autonomous- Government-steered

Mission-oriented InstitutionsStrategic plans prepared by Institutions

Governing boards led

Competitive funds allocationAutonomy to introduce new programs and set curriculumHR autonomy

External auditsIndependent external QAPerformance-based salaries

High participation of stakeholders throughout the decision-making process

10

Context, Mission and Goals

Management Orientation

Autonomy Accountability Participation

Corporate Mission-oriented Decentralized Results-based

High autonomy in all three areas, academic, financial and HR

High accountability in financial and HR

Academic

Mission-oriented- Defined in consultation with academic staff

High academic autonomy

High internal academic accountability

High participation of academic staff

Representational High external accountability

High participation of stakeholders

Trustee

Mission-oriented-Defined in consultation with trustee

Results-based High internal accountability

11

System-wide Policy GoalsGoal #1: Vision The country or government has a vision and plan for

the tertiary education sector, a willingness to translate its vision into a concrete action plan, and an ability to implement and monitor reforms

Goal #2: Regulatory Framework The tertiary education system is governed by an appropriate regulatory framework including for private providers

Goal #3: Leadership The TEA has an appropriate policy on the role and functions of the boards of tertiary education institutions, as well as for the selection of the leadership of tertiary education institutions, and the respective responsibilities of the Board and leadership

Goal #4: Financial Autonomy and Equity The regulatory framework provides enough financial autonomy to tertiary education institutions while still promoting equity 

12

System-wide Policy GoalsGoal #5: Staffing Autonomy The regulatory framework

provides enough staffing autonomy to tertiary education institutions  

Goal #6: Academic Autonomy The regulatory framework provides enough academic autonomy to tertiary education institutions 

Goal #7: Performance-based Funding The TEA negotiates performance targets and uses financing as incentives for institutions to achieve the targets.  

Goals #8: Quality assurance and transparency The TEA has an independent quality assurance and accreditation agency for both public and private institutions. Institutions are held to specific standards of transparency around financial health, fraud, student engagement and employment of graduates.

13

A Note on QA and quasi-corruptionQuestion Respons

eShare of household with positive value for annual official school cost 95.7%Share of household with positive value for annual unofficial school cost 44.1%Annual official school cost 1.2MAnnual unofficial school cost 0.4MDid you or anyone in your family make unofficial payments to get admission? Yes 27.2% No 72.8%It is common for parents to make some “unofficial payments” to gain admission? Never 1.1% Seldom 2.2% Sometimes 43.0% Frequently 47.3% Always 6.5%When unofficial payments are required, how is it done? A school official indicates or asks for a payment 30.3% The parent or family member offer a payment on his/her own accord 37.1% It is known before hand how to pay and how much to pay, so it is not discussed 32.6%

14

Institution-level: MENA University Governance Screening CardTool to assess to what extent Universities are

following good Governance practices aligned with their Institutional Goals, but also to allow Universities monitor their progress and compare themselves with other institutions

Inspiration:Guidelines and Good Practice Codes that have been

revised by OECDAutonomy Score Card- European University AssociationCUC in the UKBenchmarking guidelines- Australian UniversitiesWest Coast Guidelines, USA

15

Institution-level screening cardDIMENSION 1: CONTEXT, MISSION and GOALS

Are the missions of the University formally stated?

DIMENSION 2: MANAGEMENT Are the management mechanisms results-based or traditional?

DIMENSION 3: AUTONOMY What is the degree of academic, HR Management, and financial

autonomy?

DIMENSION 4: ACCOUNTABILITY AND PARTICIPATION How much is the university held responsible vis à vis its stakeholders?

Do the stakeholders have a voice in decision making?

16

Example of Autonomy - FinancialFinancial autonomy - ability of universities to:

set tuition feesaccumulate reserveskeep surplus on state fundingborrow moneyinvest money in financial or physical assetsown and sell the land and buildings they occupydeliver contractual services;attract funds on a competitive basis.

17

Example of Autonomy - AcademicAcademic autonomy

Responsibility for curriculum designExtent to which universities are autonomous to

introduce or cancel degree programs and to determine academic structure

Overall number of studentsAdmissions criteriaAdmissions per discipline;Evaluation of programs;Evaluation of learning outcomesTeaching methodologies.

18

Example of Autonomy – HRHuman Resources autonomy

Recruitment procedures for appointment of academic and other staff – hiring and firing

Status of employees (whether they are considered civil servants or not)

Procedure for determining salary levels, salary incentives, and workloads

Human resources policiesCareer development policiesPerformance management.

19

MENA Case study: Sample sizeEgypt: 12 universities 6 Public 6 PrivateMorocco: 9 universities 8 Public 1 PrivatePalestine: 9 universities 2 Public 7 Private Tunisia: 10 universities 7 Public 3 Private

Statistical AnalysisMCA for construction of indices of autonomyComparisons between countries/types of

universitiesAssessment of correlation structure & causality

20

MENA Case study: MCA resultsCategories % Coord.

Financial Autonomy

Has autonomy to define revenue structure of the University (No) 0.08 1.994

Has autonomy to set the level of fees (No) 0.074 1.724

Has autonomy to set the level of fees (Yes) 0.074 -1.724

Has the autonomy to run a deficit (No) 0.07 1.678

Has the autonomy to run a deficit (Yes) 0.07 -1.678

Has autonomy to set bonuses to be paid to private owners (Yes) 0.063 -1.969

Allowed to own Financial Assets (No) 0.056 1.928Allowed to own Land (No) 0.055 2.497

Has autonomy to define revenue structure of the University (Yes) 0.053 -1.329

21

Normalized Indices of autonomyIndices of autonomy

CountryEgypt Morocco Palestine Tunisia

Academic Autonomy 0.62 0.57 0.74 0.29Human Resources Autonomy 0.76 0.27 0.75 0.36Financial Autonomy 0.69 0.57 0.81 0.55All Dimensions of Autonomy 0.80 0.49 0.86 0.42

Indices of autonomy Status AllPublic PrivateAcademic Autonomy 0.48 0.65 0.55

Human Resources Autonomy 0.30 0.88 0.55

Financial Autonomy 0.52 0.84 0.65

All Dimensions of Autonomy 0.46 0.90 0.65

22

Correlations between indices

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2

-2

-1.5

-1

-0.5

0

0.5

1

1.5

Academic Autonomy

Staff

ing

Aut

onom

y

0 1 2 3 4 5 6

-2

-1.5

-1

-0.5

0

0.5

1

1.5

Autonomy Self Assessment

Staff

ing

Aut

onom

y

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

Academic Autonomy

Fina

ncia

l Aut

onom

y

-2 -1.5 -1 -0.5 0 0.5 1 1.5

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

Staffing Autonomy

Fina

ncia

l Aut

onom

y

-2 -1.5 -1 -0.5 0 0.5 1 1.5

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

Staffing Autonomy

Fina

ncia

l Aut

onom

y

0 1 2 3 4 5 6

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

Autonomy Self Assessment

Fina

ncia

l Aut

onom

y

23

Importance of various forms of autonomyVARIABLES Autonomy index Subj. Autonomy

AutonomyAcademic autonomy index 1.72*** 1.17***Staffing autonomy index 2.69*** 1.27***Financial Autonomy index 1.86*** 0.86***Size of the universityLog of the number of students -0.05 0.04Countries (ref= Egypt)Morocco -0.19 0.34*Palestine -0.24 0.19Tunisia -0.19 0.29*Type of program (ref=PhD)Undergraduate -0.21 -0.22Under & Graduate 0.01 0.09Status (ref=Public)Private -0.02 0.13Constant -2.13*** 1.40***Observations 40 40

24

ConclusionSABER: New effort and framework to document

and assess policy frameworksSystem-wide data and institutio-level data

Institution-level data helpful for implementation, but also calibration (weights) for system-wide indicators

Institution-level tool available for deployment in case of interest among participants at World Congress

Thank you!