Workshop, 29 of june Evaluation tools and criteria for eGov services _________________ Leda Guidi...

Workshop, 29 of june

Evaluation tools and criteria for eGov services

_________________

Leda Guidi

Responsible of Communication services to citiezens - Iperbole civic network

BOLOGNA MUNICIPALITY

Knowledge Society Forum - Telecities Summer Event

Bologna, 28-30 June 2007

Comune di Bologna

Knowledge Society Forum - Telecities Summer Event Bologna, 28-30 June

workshop aim

The aim of the workshop is to compare different evaluation

systems of e-gov services sustainability in order to share

approaches and methodologies in:

•ROI (qualitative and quantitative) measurement

•definition of indicators (financial, social, political, etc.)

•achievements of success as well as weak points

Bologna will present the tool developed in partnership

with Deloitte within the framework of the EUROCITIES «e-

Citizenship for All» prize.


eGov in Bologna

• The Bologna Municipality has always been open to the use of the most advanced information and communication means both in the administrative and internal managing action as well as in the relation with citizens and the local community.

• Through the last ten years the Municipality has been constantly focusing its government action towards a considerable increase in the use of new networking technologies for the delivery of information and services to community, the two way communications and participation trials.

• Despite the efforts spent to fight digital barriers, a part of Bologna population still risks to be cut off from e-participation processes (age, gender, social-economic situation, etc.)


• To make citizens proactive and fully aware of its e-rights the key is to invest in e-Citizenship at all levels of multichannel delivery: services information, interaction, participation in decision making processes.

Local public administrations need to build consensus around shared choices, based on wider community/ies participation in the ongoing decision-making processes.

• A success factor is inter-institutional, multi-level cooperation, partnerships, in order to achieve

• resource effectiveness • generate synergies• standardise approaches, formats and languages

eGov and eCitizenship


What “business model” for e-gov?

• Sustainability for e-service and e-democracy

• Impact at a medium/long term

• New methodologies to “measure” the e-gov benefits

• The efficacy and efficiency of an eGov project is not measurable in the short term (break even!)

• Only after months / years it becomes possible to give estimations of its impact on the social growth of the community and evolution.

• It is necessary to use techniques more related to “social balance” than to “analytic book-keeping”.

• Complex mix of indicators both qualitative and quantitative to evaluate the ROI also in term of innovation

eGov Evaluation


Aim of the Assessment System

The aim of the tool developed is to provide the Municipality with a

qualitative indicator that will support assessment of actions

required to implement e-government.

The final indicator value is obtained through assessment of the

following drivers :

Organizational

Economic

Social

Financial

Qualitative (vis-à-vis the user)


Content Index

• Introduction

– Aim of the Assessment System

– Index of Content

– Method Adopted

– Guidelines

• Qualitative Assessment Criteria

– Qualitative Indicators

– Assessment Drivers

– Comments

• Economic Assessment Criteria

– Economic Indicators

• Overall Assessment of the Initiative

– Final Score

• Assessment Tool

– IT-based Assessment


Method adopted

The Assessment Model to evaluate e-government initiatives entails the following steps:

Defining a set of e-gov project objectives: assessment indicators

Defining a scale of values so as to assign a score system to each driver

Identifying assessment drivers for each indicator

Developing a computerised system for the automatic calculation of a summary assessment index.

Developing a questionnaire regarding the qualitative aspects of the initiative


Guidelines

• The assessment criteria must be deployable for any e-government initiative;

• The system must be able to take into account the Administration’s weighting of the various achievable end points;

• The system must be able to produce a result that is weighted for the importance attributed by the Administration to the financial impact of the e-gov initiatives;

• The indicators must be assessed by attributing value judgements to the various assessment drivers;

• All value judgements must be motivated;

• The indicator must be measurable by means of a scoring system;

• The system should be operational without excessive staff requirements;

• The final overall assessment should be concise and accurate.

The guidelines underpinning the assessment system are as follows:


Qualitative Assessment Criteria


Qualitative IndicatorsQualitative Assessment Criteria

Qualitative indicators derive from the objectives the Administration intends to achieve with e-gov initiatives:

Image

Initiative assessment

Economic savings

Advantages for societyQuality of

services

Performance


Assessment DriversQualitative Assessment Criteria

Qualitative indicators are evaluated by assessment drivers:

INDICATOR ASSESSMENT DRIVER

Administration’s image •User awareness of subject matter•Press awareness of subject matter•User pool for the service offered•Degree of “sophistication” of service offered•Attention capturing capacity

Service performance •Potential to cut resource requirements•Potential to reduce throughput times•Potential to reduce workload•Potential to reduce errors

Economic savings •Potential to replace workforce•Potential to reduce workstations•Potential to reduce use of materials•Potential to reduce public counters

Quality of services •Type of technology used•Cutting throughput times•Cutting waiting (queues)•Potential to replace “traditional” procedure

Advantages for society •Cutting user transfers•Increased transparency/user oversight of administrative practices•Integration of disadvantaged groups•Economic development of disadvantaged areas•Employment gains


Score Systems and WeightingsQualitative Assessment Criteria

Drivers are qualitatively assessed, each judgement corresponding to a value

Depending on driver evaluation, indicators will have a score ranging from 0 to 10 and will also be weighted for the importance attributed by the Administration

Weighting attributed by Administration to e-gov objectives

PWeightingEnhance Administration’s image 0,2Enhance service performance 0,2Obtain economic saving 0,2Increase quality of service 0,2Advantages for Society 0,2

0

2

4

6

8

10

Enhance Administration’s image

Enhance service performance

Obtain economic savingIncrease quality of service

Advantages for Society

With the weighting system, overall assessment of qualitative aspects is expressed by a score from 0 to 10

The sum of the weighting must be 1


Economic Assessment Criteria


Economic IndicatorsEconomic Assessment Criteria

Economic indicators consider the financial requirement for initiative implementation:

Computer systems

Office equipment and furniture

General plant and equipment

Operational and management

costs

Training

Cost for: Total financial requirement assessed by scoring system

From: To SCORE

0 2.500 102.501 5.000 95.001 10.000 8

10.001 20.000 720.001 40.000 640.001 80.000 580.001 160.000 4

160.001 320.000 3320.001 640.000 2640.001 oltre 1

Economic aspects expressed by a scoring system from 0 to 10


Overall Assessment of the Initiative


Final ScoreOverall Assessment of the Initiative

The final score is the sum of the overall qualitative assessment and the economic assessment, weighted for the importance attributed by the Administration to the economic factors

Economic Score

Economic Value

X

Weighting

attributed

(from 0 to 1)

+

Qualitative

Score

Qualitative Value

X

Weighting

attributed

(from 0 to 1)The sum must be 1

=Overall assessment

of the initiative

Score range:

1 to 10


Assessment Tool


IT-based Assessment Assessment Tool

A computer spread sheet automatically attributing a value to each indicator

Economic Aspect Qualitative Aspect

Input of estimated financial investment requirement

Input of Administration’s weighting of financial aspect compared to qualitative aspect

Value judgement given to each driver indicated by putting “X” in questionnaire fields provided.

Value judgement is motivated

Inclusion of weighting for each indicator

Each initiative receives an overall score (range 0 to 10) that will support the Municipality in assessing action to be implemented

Date post:	19-Dec-2015
Category:	Documents
View:	215 times
Download:	1 times

Workshop, 29 of june Evaluation tools and criteria for eGov services _________________ Leda Guidi...

Documents