In search of the chimera

Measuring the impact of information Measuring the impact of information on developmenton development

In search of the chimera

Maartje op de CoulMaartje op de CoulNew media evaluation, Oneworld InternationalNew media evaluation, Oneworld International

maartje.opdecoul@oneworld.orgmaartje.opdecoul@oneworld.org

Sarah CummingsSarah CummingsInformation manager, Information and Library Services, Information manager, Information and Library Services,

Royal Tropical Institute (KIT)Royal Tropical Institute (KIT)s.cummings@kit.nls.cummings@kit.nl

Content

Chimera = impactChimera = impact Oneworld case studies (impact assessment)Oneworld case studies (impact assessment) Smart tools (performance evaluation)Smart tools (performance evaluation) Some conclusionsSome conclusions And a question….And a question….

Why ‘chimera’ = impact

• Mythical creature with three heads... but also unachievable illusion

• Child of Typhon, a dynamic force like the Internet

• Impact: socioeconomic development or long-term behavioural change

• Performance evaluation

• ICT4D: wireless, mobile phones, telecentres

• Information services: QAS, newsletter, SDI, libraries etc

Oneworld case studies: overview

20032003 20+ ICT4D projects visited20+ ICT4D projects visited Three continentsThree continents Link between ICT4D and povertyLink between ICT4D and poverty Building Digital Opportunities (DFID, SDC Building Digital Opportunities (DFID, SDC

and DGIS) plus implementing agenciesand DGIS) plus implementing agencies

Oneworld case studies:methodology Impact =Impact =

socio-economic developmentsocio-economic development long-term behavioural changeslong-term behavioural changes ‘ ‘any change at the level of organizations’any change at the level of organizations’

Indicators: opportunity, empowerment and Indicators: opportunity, empowerment and security security

Oneworld case studies: findings 1

OpportunitiesOpportunitiesSkill building, access to markets and e-businessSkill building, access to markets and e-business

EmpowermentEmpowerment- - Individual level: self-confidence, self-esteem, emancipation, Individual level: self-confidence, self-esteem, emancipation, changing power relations, visibilitychanging power relations, visibility- Organizational level: efficiency, networking, effective - Organizational level: efficiency, networking, effective lobbying, access to public services, transparency and lobbying, access to public services, transparency and accountability of governmentsaccountability of governments

SecuritySecurityLife saver, warning fires, empowermentLife saver, warning fires, empowerment

Oneworld case studies: findings 2

SustainabilitySustainabilityFinancial, human resourcesFinancial, human resources

Regional differencesRegional differencesSouth Asia: varied infrastructure, involvement of private South Asia: varied infrastructure, involvement of private sector, rural areas, little M&E, technical creativitysector, rural areas, little M&E, technical creativitySouthern Africa: low commitment of government and Southern Africa: low commitment of government and private sector, infrastructure limited, early stageprivate sector, infrastructure limited, early stageCentral America: high commitment government and Central America: high commitment government and private sector, sophisticated content, Spanish, good M&Eprivate sector, sophisticated content, Spanish, good M&E

Tecnoclub/Paniamor – San José, Costa Rica

Agronegocios – El Salvador

Video SEWA - Ahmedabad, India

Oneworld case studies: conclusions

Cautious optimismCautious optimism Compatible technologiesCompatible technologies Combination with traditional technologiesCombination with traditional technologies Financial baseFinancial base

LEAP IMPACT 1

Dgroup on evaluation of ICTs and information Dgroup on evaluation of ICTs and information servicesservices

Smart tools: CTA, IICD and KIT plus 40-50 Smart tools: CTA, IICD and KIT plus 40-50 individual partners: writers, testers, thinkers, individual partners: writers, testers, thinkers, coordinators, reviewerscoordinators, reviewers

Self-help, resource poor environmentSelf-help, resource poor environment 4 modules of ‘Smart tools’4 modules of ‘Smart tools’

- - Preparatory workPreparatory work- Process tools- Process tools- Data collection and analysis- Data collection and analysis- Activity tools- Activity tools

LEAP IMPACT 2

Testing of the activity tools: 18 testers in Testing of the activity tools: 18 testers in the South, and some in the Norththe South, and some in the North

Testing to be completed OctoberTesting to be completed October Revision based on testingRevision based on testing Handbook end 2004Handbook end 2004 Publication: medio-2005Publication: medio-2005 End of a three year project........what next?End of a three year project........what next?

Conclusions

In different ways, either self-motivated or donor In different ways, either self-motivated or donor motivated trying to understand the impact of motivated trying to understand the impact of ICT4D and performance of information servicesICT4D and performance of information services

How do you show that your newsletter or website How do you show that your newsletter or website has ‘impact’?has ‘impact’?

Is impact assessment a three-headed monster...?Is impact assessment a three-headed monster...? Impact assessment essential if investments in Impact assessment essential if investments in

‘information’ are to be compared with other ‘information’ are to be compared with other development interventions....development interventions....

Sustainability of the ‘information hype’Sustainability of the ‘information hype’

A question…..

• How to evaluate How to evaluate communities?communities?

• KIT has 3 new KIT has 3 new communities, one on communities, one on iiManagementManagement

• Standards and approaches?Standards and approaches?