Monitoring and Evaluation in Perspective Lessons from International Experience Delivering on the...

transcript

Monitoring and Evaluation in Perspective

Lessons from International Experience

Delivering on the Promises of e-government

PREM WeekWashington DC – 26 April 2005

Bruno LANVINCITPO/GICT

The World Bank

Four points

• e-gov, e-strat, k-strat, d-strat• M&E as a strategic tool (WB approach)• Some lessons from experience • Avenues for action

Point 1

Global Information Infrastructure

Applications

Digital Divide

Content

Knowledge

Empowerment

1985 1988 1991 1994 1997 2000 2003

Shedding different lights at a complex set of issues

Missing Link(Maitland Report)

Infrastructure

Regulatory aspects

E-Strat, D-Strat and M&E StratRenewed ‘IS’ context(Information Society approach)

WSIS (Declaration and Plan of Action)

MDGs (Goals and specific

targets under Goal 8)

E-STRATEGY

Monitoring and Evaluation

OUTCOME(e-readiness, connectivity,

usage .. etc..)

Changes(PRSP, CAS, ..)

New ‘E’ context

Is ICT important

for the success of the country’s

D-Strategy ?

Evidence from ICT4D experienceDOT Force, etc ..

E-strategies and Development :from D to E to IS

Development StrategyD-STRATEGY

(PRSP, CAS, ..)

Monitoring and Evaluation

OUTCOME(growth, competitiveness,poverty, health, education,

etc ..)

Traditional ‘D’ context

Point 2

Monitoring & Evaluation Toolkit for E-strategies Results

METER: background and objectives

• Consideration of 50+ e-strategies• Identification of goals pursued and ways to

assess progress • Identification of linkages between ICT objectives

and sectoral/strategic needs (WSIS, MDGs e.g. )• Proposal for an M&E methodology and specific

implementation tools

Assumptions and risks

Resources

Actions

Key initiatives

Strategic priorities

Policy goals

From LogFrame to M&E

Indicators

Impact

Outcomes

Outputs

Deliverables

Background and rationaleBackground and rationale

MethodologyMethodology

Framework

• E-readiness• Security• Digital Divide

Infrastructureand ICT sector

• Rationale• Definition• Overview• M&E Tables• Assumptions and risks

Indicators and quantificationIndicators and quantificationAnnexes

Module 2

Module 1

Toolkit Structure

E-government

E-business

E-learning

E-health

• Rationale• Definition• Overview

• M&E Tables• Assumptions and risks

Module 3 Modules 4 & 5

SectoralModules(6 & 7)

Create efficient, responsive, transparent

government

Publish: Provide valuable

on-line information

Interact: Engage society to

improve government

Contract: Offer cost-effective

online services

M&E : the 3 levels of E-government

to incl

-govern

Prerequisite Concern Activity (typical indicators)

Access Infrastructure, costs, competition/ regulation (hence includes proper regulatory and competition frameworks)

Equipment (PCs, kiosks, community centers) Teledensity Rule of law Pro-competitive ICT regulation (tariff and non-tariff barriers, competition in the ICT sector) Cost (fixed line calls and Internet access) Access for disadvantaged or excluded

Basic Skills Basic education, vocational training, ICT awareness

Literacy (alphabetization rates) E-literacy ratios per age/group/sex/region Vocational training

Content Value to government and citizens Questionnaires on value to users/citizens and government Content in local languages

Desire Political leadership and will to reform

Public statements/decisions Laws & regulations (perceptions of quality of legal system)

Engagement Commitment of all components of civil society

Broad involvement of civil society (questionnaire/survey) Local awareness of ICT potential for development (questionnaire/survey)

” le

Pyramid Layer

Objective Indicator Data source

Policy goals Create an efficient, responsive and transparent government

Perception of overall administrative burden Perception of government effectiveness

Office of Government and Ministry of Local Government

Strategic priorities

Bring valuable information online to the public, anytime anywhere a

Perception of government online presence % pop. using govt. sites Usage growth rate

E-government CIO’s Office

Key initiatives

Roll-out of online information services b Raise public awareness through online and offline channels

No. of agencies with web sites % of agencies with web sites % of information services rolled-out on time

E-government CIO’s Office

Actions Establish an independent central e-government group and M&E unit c Establish selection guidelines for information/content to be posted, including nature and volumeAssess technology and organizational needs/requirementsDevelop online information platformsOffer information services on line, including local language contentDevelop publicity campaign to promote new e-government initiativesSolicit feedback on usability and usefulness of online government services

Central e-government team and M&E unit established by month A Guidelines for M&E established by month B Relevant information sources identified by month C System functional requirements completed by month D Mid-term implementation review conducted by month E Public awareness survey results

Project team or Central M&E unit

(a) There should be a clear linkage between the type of content that is brought on-line and larger development goals, such as economic development, anti-corruption, and attracting FDI.(b) Criteria used for information publication should be related to cost and time savings and envisaged productivity gains.(c) Central e-government group responsible for interoperability and inter-agency consistency of e-government services, security, consolidation of records, and M&E.

METER: next steps

• Gather comments on the toolkit• Establish and launch METER website• Develop new modules (in partnership)• Launch revised version of toolkit at WSIS• Offer toolkit in different languages• Continue to contribute to continuous exchange

of best practices in the area of e-strategies M&E• Reflect outcome in WB practices• Identify, disseminate and promote best practices

Point 3

ICT &Knowledge Economy,Innovation

ICT & sectoral improvement

(e.g. educationhealth, etc..)

ICT access(infrastructure,

equipment,connectivity)

Measurable

Strategic

ICT’s Impact on DevelopmentMeasurable or strategic ?

ICT &Job creation,Productivity,

Competitiveness,Poverty

reduction

Readiness Access Usage Value

- Enhance produc- tive uses of ICTs in economy and ICT awareness in society- Foster SME competitiveness at home and abroad

- Promote visible E-government services with measurable impact on SMEs- Foster regional integration through on-line services to rural areas

- Establish a one- stop portal for SME registration- Allow and promote on line request/issuance of land titles and birth certificates through commu-nity access points

PolicyD- Strategy

E- Strategy

- Adapt legal syst. for on-line business registration (OLBR)- Create/promote portal for OLBR - Establish CAPs- Create/promote Adm. Doc. portal

Initiatives

Actions

Typical M&E indicators/ time horizon

# of SMEs regis- tered on-line# of CAP created

Example of a possible ‘e-gov’ implementation chain

Relevant M&E indicators

- Pursue a ‘knowledge-based’ competitiveness strategy-Improve image and efficiency of government- Enhance involvement of private sector in key areas- Redress local imbalances (social, geographic, eg)

SME rating of e-govRegional disp.index

ICT awarenessSME

competitiv.

GDP growth% P.S. in GDP

Inequality index

# of hits to site (awareness) for SMEs, titles,..

A pre-requisite for e-government: Internet Access

15 20 25 30 35 40 45 50 55 60 65 70

Internet subscribers

per 100 inhabitants

Average cost for 20 hours

(day + evening) in $ PPP

GERAUT

CDN UK

MEXGRE

HUNCZE

The ‘Nordic’ cluster

The ‘mature markets’ cluster

The ‘Emerging Europe’ cluster

Source : World Bank, based on OECD data

Internet Costs and Diffusion(OECD – 2002)

Governments’ functions

Governments’ roles

•E-government- Customs

- Taxes- Procurement

•Statistics/macro-ec managmnt•E-democracy/local government

•Fiscal and monetary policy•Trade/FDI policy•Diplomacy/defense•Education•Legal/regul/compet. policies•Justice/order etc. (“regalian”)

Informatization +Decentralization

Globalization +Market efficiency

CivilSociety +

Demand/Supply for IT purchases

Technology supply, markets for IT

Limits to IT imports/exports

Labor supply for IT sector

Incentives/limits for IT market

Leadership +

Facilitation

E-strategy, national ambition/social project pursued through IT

Demand for IT solutions/leading through example

OLD (diminishing role)

NEW (increasing role)

Environment Readiness Usage

Provide and promote vision (KISS)• Address ‘Digital Divides’

• domestically• internationally

• Give signals to markets• ICT as a national priority• large projects or objectives

• Promote and defend national interests in international and global forums

Education policy• curricula/life-long learning• ICT training facilities• Wiring/networking of schools

New roles for governmentsG5

E-government• services on line• procurement• trade facilitation• civil society participation• good governance

G2Provide proper environment• Macro-economic environment• Fiscal policies (cost, innovation,

investment, VC, PPP)• Legal/regulatory environment for

ICT (competition, independent regulator, rule of law)

Provide access (univ serv)• Lay out ICT infrastructure• Produce ICT equipment• Finance Public R&D

Leaders

y = 0.6839x + 1.93992

R = 0.3991

NICBAN

SAFSLV

SRICZE

JAPEST

NETAUL

SPADEN

SWIICE

SWEUSA

2.00 3.00 4.00 5.00 6.00 7.00

Note : the ‘Government’ variable of the GITR index has been used as proxy for Government leadership (x), whereas ‘Competition in telecoms’ has been used as an indicator of the effectiveness of Governments as facilitators (y)

First circle (top performers)

Second circle (the contestants)

Third circle (ready or not)

e-government for ‘new’ government

Point 4

The A,B,C,D,E of the Digital Divide

Access

Basic skills

Content

Desire

Engagement

- Infrastructure, costs, competition/regulation

-Basic education, vocational training, entrepreneurship- Local value, languages

- Local will to reform

-Commitment of all components of civil society

Direct effets

• Increased public sector efficiency– Savings for governemnt (lower administrative costs)– Better management of public resources

• Better access to public services by firms and citizens– Savings for users (time and money) – Extended coverage (geographic, social, timewise: 24/7)– New services offered (e.g. itineraries, doc search & comparison,.)

• Improved economic governance– Data on traffic (flows of goods & services, payments, tracking, ..)

is more precise, quasi-instantaneous and cross-referentiable– Public service staff can find renewed motivation– Cooperation improved between public sector, private sector and

citizens

Indirect effects

• Transparency/governance– e-procurement (on-line tendering)– Selectivity systems in customs and other inspection-based services– Involvement of citizens in policy debates and decisions

• Business competitiveness– Faster services (registrations, licenses, authorizations,..)– Timely access to strategic data (prices, markets, laws/regulations,,..)

• IT knowledge and litteracy– learning-by-doing, incremental improvements to software, etc…– Breaking psychological barriers (public access points, ..)

• Building information societies– Turning local knowledge into value and competitiveness (culture, modus

operandi,..)– Attracting external partners (trade, investment, ..)– Contributing to global development efforts (MDGs)

e-government vs better government

• Consider and promote e-government as a government-centered effort

• Consider and promote e-government as a technology-driven effort

• Replace every paper-based process by an electronic process

• Offer ‘ministry-specific’ e-solutions

• Launch, measure, punish/reward

• Focus the e-government process on users (citizens & businesses)

• Focus e-government strategy on people (both on the government and on the users side); favor technology-neutral choices

• Use e-government as a tool to foster changes in attitudes and thinking, and as an instrument for reforms

• Rally government-wide energies and competence around common procedures and standards (back office), and an ‘all-of-government’ approach (front office)

• Adopt an early common approach to monitoring and evaluation (accountability, ownership, results)

Ways to fail Ways to go

Thanks for your attention

blanvin@worldbank.org

Monitoring and Evaluation in Perspective Lessons from International Experience Delivering on the...

Documents