Date post: | 27-Mar-2018 |
Category: |
Documents |
Upload: | doankhuong |
View: | 240 times |
Download: | 7 times |
EDUCATION STAFF DEVELOPMENT PROGRAM
Assessing School Autonomy and Accountability (SAA) Policies
An Overview of SABER-SAA
Angela Demas Senior Education Specialist
The World Bank November 2014
Overview of presentation
1. Introduce the SABER-SAA Tool for analyzing
policy intent and the enabling environment
2. Methodology and data collection instrument
3. The SABER-SAA country report
4. Implementing SABER-SAA
3
SABER Domains or Policy Areas
Education inputs
Student enrollment
4
Expanding the Education Knowledge Base
to help countries assess, benchmark, and improve their education policies and governance
Quality of policies &
institutions Education
inputs
Quality of policy
implementation Learning for all
Quality and quantity of education delivered
5
SABER Policy Implementation
SABER Policy intent
School & household
surveys
EMIS EMIS, student assessments
Expanding the Education Knowledge Base
to help countries assess, benchmark, and improve their education policies and governance
Impact Evaluation
What is SABER-SAA?
SABER-School Autonomy and Accountability (SAA) analyzes school-based management policies in view of the level of managerial autonomy delegated to the school level, the existence and use of assessment, and the extent of accountability in the education system.
6
What does it do? • Classifies and analyzes education systems according to 5 policy
goals critical for enabling effective school-based management. What’s the goal? • To identify policy areas and actions that support better alignment
of managerial responsibilities at the school level, assessment of results, and use of assessments to promote accountability with an end view to increasing education quality and student learning.
What does it produce? • Country reports and other knowledge products that diagnose how well a country’s policies support school autonomy and accountability.
SABER-School Autonomy & Accountability
Motivation of SABER-SAA: How can we improve the performance of the education system?
More quantifiable issues: Access Internal efficiency Quality of learning Less quantifiable issues: Policies Institutional arrangements Processes Enabling environment
0%10%20%30%40%50%
8
Focus of SABER SAA
What matters for success in SAA?
Three Components of SBM: 1. Managerial autonomy, 2. The assessment of results, and 3. Use of assessment to promote accountability among stakeholders
When the 3 components of School-based Management are in balance with each other, the system performs better.
Teacher Quality EMIS
School Council Autonomy
Local decision-making based on evidence
Accountability Responsibility for school
and student performance
Assessment Measuring school and student performance
School Autonomy & Accountability The 3 A’s
SABER School Autonomy & Accountability: 5 Policy Goals
Policy Goals for SAA
1. Level of autonomy in planning and management of the school budget.
2. Level of autonomy in personnel management
3. Role of school council in school governance
4. School and student assessment
5. Accountability
5 Policy Goals for SAA
12 12
Personnel Autonomy
Student/School Assessment
Accountability
Participation
Budgetary Autonomy
1A Legal authority over the management of the operational budget
1B Legal authority over the management of the non-teaching staff salary
1C Legal authority over the management of the teacher salary
1D Legal authority to raise additional funds for the school
1E Collaborative budget planning
2A Autonomy in teacher appointment and deployment decision
2B Autonomy in non-teaching staff appointment and deployment decision
2C Autonomy in school principal appointment and deployment decision
3A Participation of the school council in budget preparation
3B Participation of the school council in financial oversight
3C Participation of the school council in personnel management
3D Participation of the school council in school activities
3E Participation of the school council in learning inputs
3F Transparency in community participation
4A Existence and Frequency of school assessments
4B Use of school assessments for making school adjustments
4C Existence and Frequency of standardized student assessments
4D Use of standardized student assessments for making school adjustments
4E Publication of student assessments
5A Guidelines for the use of results of student assessments
5B Analysis of school and student performance
5C Degree of Financial accountability at the central level, regional, municipal, local and school level
5D Degree of Accountability in school operations
5E Degree of learning accountability
Policy Action
SABER-School Autonomy & Accountability: How Data Are Collected
• Data collection instrument (questionnaire) organized around the 5 policy
goals and links to policy rubric.
• Collect documentation: Legislation, laws, policies • Conduct interviews: Policy-makers, officials
• Data and sources: researcher provides justification and reference materials to support answers to questionnaire.
Snapshot of Data Collection Instrument
Indicator 4. School and student assessment Sub indicator 4A. Existence and frequency of school assessments Justification References
Q1 Are schools assessed using the Ministry of Education criteria?
1. No
2. Yes
Q2 If yes, how often will it be assessed?
1.Not on a regular basis
2.Every few years
3.Every year
4.Every year including in-depth on a special topic (e.g. student poverty, school nutrition, migration)
Policy Goal 4: School and Student Assessment Policy Goal has 5 Policy Actions These two questions are for policy action 4A – Existence and
Frequency of School Assessments.
Producing Scores
Q1. --------? Q2. --------? Q3. --------? Q4. --------? Q5. --------?
Policy Action
Answers for each
Policy Action are analyzed
using Scoring Rules to produce a score from
1 to 4
Scoring Rules are defined by
the SABER-SAA Rubric, which describe a system’s level of development in relation to global good
practice on a scale from 1 to 4
Scoring Rules
Scores at the policy action level are then aggregated to the Policy Goal level.
SABER-SAA: Example of a Rubric Policy Goal 2: The level of autonomy in personnel management
Policy Action
Variable name Latent Emerging Established Advanced
2A Autonomy in teacher appointment and deployment decisions
Teachers must be appointed and deployed by the central government level under a union or civil service agreement.
Regional or municipal governments have legal authority to appoint teachers under union or civil service agreements. Appointments are subject to final review by central authorities.
Regional or municipal governments have legal authority to appoint and deploy teachers under union or civil service agreements without review by central authorities.
Schools (school principals, school council, parent association etc.) have legal authority to appoint teachers. Union and civil service agreement may or may not regulate the appointments.
2B Autonomy in non-teaching staff appointment and deployment decisions
Non-teaching staff must be appointed and deployed by central government level under civil service agreement.
Regional or municipal governments have legal authority to appoint non-teaching staff under civil service agreements.
Regional or municipal governments have legal authority to appoint and deploy non-teaching staff. Civil service agreement may or may not regulate the appointments.
Schools have legal authority to appoint non-teaching staff. Civil service agreement may or may not regulate the appointments.
2C Autonomy in school principal appointment and deployment decisions
Principals are to be appointed and deployed by the central level. Their performance is evaluated centrally and they can be transferred or fired by Central authorities.
Principals are to be appointed and deployed by the central level. Their performance is evaluated regionally or by municipal inspectors, which determines their tenure, transfer, or removal by Central authorities.
Principals are to be appointed and deployed by regional or municipal/local authorities, who are also responsible for their evaluation and have the authority for determining tenure, transfer, or removal.
Principals are to be appointed and deployed by municipal/local authorities in consultation with the School Council/ stakeholders at school level, or by the School Council alone. Municipal/local authorities are responsible for the principal's evaluation to determine tenure, transfer, or removal.
Classification of Scores
Latent
Emerging
Established
Advanced
Reflects policy not in place or limited engagement
Reflects some good practice; policy work still in progress
Reflects good practice, with some limitations
Reflects international best practice
The Country Report Shows where a country stands in relation to global good
practice as defined by the rubric
It serves as a platform to deepen dialogue and build consensus on system reform
It’s clear, concise, and neutral style conveys key messages to a broad range of stakeholders
The content is structured to ensure systematic coverage of all elements of the conceptual framework
Layout of SABER-SAA Country Report
Standard cover page that provides a brief overview of the results by policy goal. Inside the report (5-6 sections):
1. Introduction and country background information 2. The case for SAA – what matters (standard) 3. The country’s policy analysis results by policy goal 4. Policy recommendations for improvement 5. Optional – comparison of countries 6. Acknowledgements, Acronyms, References
Bulgaria Country Report
What did the policy analysis tell us about the state of SAA policies and their potential impact for improving learning?
Bulgaria’s Scores
21
1. School Autonomy in the planning and management of the school budget
Established
2. School Autonomy in Personnel
Management
Established
3. Role of the School Council in School Governance
Latent
4. School and student assessment Emerging
5. Accountability Emerging
SABER - School Autonomy & Accountability Assessment of Bulgaria
Score1 Autonomy in Budget Planning and Approval - Established
Municipalities receive funding for their schools through a per capita government transferSchool director in consultation with pedagogical staff has autonomy to prepare school budgetSchool director can set teacher salaries and bonuses following rules determined at central level. Board of Trustees, parents and community are not consulted nor have input to the budget. Schools have authority to raise additional funds from all sources - parents, NGOs, private sector, etc.
2 Autonomy in Personnel Management - EstablishedSchool director has full authority to appoint and deploy teachers using criteria determined at the school level and central level guidelines.School director has autonomy to hire and fire non-teaching staff following Bularian labor code. School Directors are hired and evalutated by the Regional Education Inspectorate and the municipality.
3 Participation of the School Council in School Governance - LatentBoard of Trustees (BOT) is not empowered to participate in budget preparation, school policies and decisionsBOT only has legal standing as a non-profit organization, no voice, and no oversight on budget issuesNo laws or systems to promote community participation in personnel matters, school activities and learning No provisions for the open election of BOT members and for calling general assemblies
4 Assessment of School and Student Performance - EmergingSchool inspections are not scheduled regularly and only assess for compliance, not for outcomesResults do not contribute to pedagogical, personnel or operational adjustmentsStudent assessments are conducted annually at grades 4, 7 and 12 There is no policy obligating schools to use data to make adjustments but schools do this in practice. Regions and schools receive data on student exam results, but not the public.
5 Accountability to Stakeholders - EmergingNo guidelines or mandates for use of or comparative analysis of student assessments results Rules and regulations guide financial management and transparency at all levels Some regulations for accountability in school operations exist but do not link rewards to performanceNo requirement to explain and simplify student assessment results to the public
Policy Goals
SABER-SAA application Other examples and how did it help?
• Informed Project Design • Nigeria – Disbursement Linked Indicators were developed
using information from the SABER assessments. Assisted project to craft relevant indicators for monitoring progress.
• Senegal – analysis used for new project design
• Informed policy for Education Sector Strategy • 3 Pacific countries – Samoa, Solomon Islands, Papua New
Guinea (SABER-SAA applied in partnership with AusAid)
• Deepened policy dialogue • Bulgaria and Kazakhstan – SABER-SAA combined with SABER
student assessment to understand better where the system could be improved for better learning outcomes.
Patterns Typical Approach to SBM
SBM is introduced through a pilot in a region(s) before going to scale
Often starts with policies that call for establishing School Councils.
Next step is to build capacity, some training on how to develop a School Improvement Plan (SIP)
Usually a school grant is involved – often for oversight of school maintenance or infrastructure
• Implementing autonomy or accountability alone has no additional effect on learning
• School autonomy must be complemented with school accountability to promote academic excellence
• Highest PISA scores come from countries where autonomy and accountability are implemented together
Autonomy and Accountability
Implementation Process
Getting Started
Step 1: Orientation and Training
Step 2: Develop Implementation Plan
Data Collection
Step 3: Carry out documentary evidence review and conduct interviews with credible informants to close knowledge gaps and corroborate evidence
Data Verification
Step 4: Validate data through a series of meetings
Step 5: Complete and submit DCI to the SABER-SAA Team for scoring
Report Preparation
Step 6: Prepare Country Report and obtain validation of information.
Key Players
• Data collection, analysis & reporting
• Technical support, scoring & feedback
• Validation, policy and its implementation
• Dialogue & operations
World Bank Country TTL
Government counterparts
Principal Investigator
World Bank SABER-SAA
team
EDUCATION STAFF DEVELOPMENT PROGRAM
THANK YOU
Contact World Bank Team SABER-SAA: TTL: Angela Demas ([email protected])