7/24/2019 Idrc Ia for Ict4d Compendium
1/167
Compendium on ImpactAssessment of ICT-for-
Development Projects
2008
Richard Heeks & Alemayehu Molla
7/24/2019 Idrc Ia for Ict4d Compendium
2/167
he authors1 moral ri$hts are retaied i this licece*
7/24/2019 Idrc Ia for Ict4d Compendium
3/167
Table of Contents
Introduction to the Compendium.................................................................................... iii
COMPENDIM O!E"!IE#$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$%
* A3456R5'674'MA)A++6++M634R')9*********************************************************
1A. Guiding Model The ICT4D Value Chain................................................................2
1B. Classi!ing the "#erall Impact o an ICT4D $ro%ect.................................................4
2* A3456R5'674')9 R4:6)'MA)A++6++M63RAM674R;+*************************/
2A. Comparing IA &rame'or(s B! Method.....................................................................)
IMPACT A&&E&&MENT '"AME#O"(&$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$)
* )4+.
7/24/2019 Idrc Ia for Ict4d Compendium
4/167
Introduction to t/e Compendium
7/24/2019 Idrc Ia for Ict4d Compendium
5/167
Compendium Overvie0
%$ An Overvie0 of Impact Assessment for ICT*D
As !ith ay i-esti$ati-e process, t!o Fuestios dri-e ')9 impact assessmet%
7hat do !e ot ko!, that !e eed to ko!G
Ho! are !e $oi$ to fid that outG
+pecifically, impact assessmet of ')9 proDects ca be based aroud siE Fuestios (seei$ure "%
Why% !hat is the ratioale for impact assessmetG
For whom% !ho is the iteded audiece for the impact assessmetG
What% !hat is to be measuredG
How 1% ho! are the selected idicators to be measuredG
When% at !hat poit i the ')9 proDect lifecycle are idicators to be measuredG
How 2% ho! are impact assessmet results to be reported, dissemiated ad usedG
#/45
'or
#/om5
#/at5 o05
6%7
#/en5
o05
687
Pilot 9
Action
7/24/2019 Idrc Ia for Ict4d Compendium
6/167
'cludes Fuestios o !hether idicators are reported as is, or commuicated -ia causalmodels, case sketches, stories, etc*
7/24/2019 Idrc Ia for Ict4d Compendium
7/167
%A$ ,uidin1 Model 3 T/e ICT*D !alue C/ain
he basis for uderstadi$ the assessmet of ')9 proDects is the ')9 -alue chai, sho! i i$ure 2*
'i1ure 82 T/e ICT*D !alue C/ain
his builds o a stadard iputIprocessIoutput model to create a seFuece of liked ')9 resources ad processes* 't is di-ided ito fourmai tar$ets for assessmet%
Readiness% e.readiess assessmet typically measures the systemic prereFuisites for ay ')9 iitiati-e e*$* presece of ')
ifrastructure, ') skills, ') policies, ad so o* 4e could also assess the strate$y that turs these precursors ito proDect specific iputs,ad the preseceabsece of those iputs*
Availability% implemetatio of the ')9 proDect turs the iputs ito a set of ta$ible ') deli-erables# oe ca assess the presece ada-ailability of these itermediate resources*
Uptake% assessmet typically measures the eEtet to !hich the proDect1s ') deli-erables are bei$ used by its tar$et populatio*
7/24/2019 Idrc Ia for Ict4d Compendium
8/167
Impat% as the ame su$$ests, oly this focus actually assesses the impact of the proDect ad !e ca di-ide it ito three sub.elemets%
o "utputs% the micro.le-el beha-ioural cha$es associated !ith the ')9 proDect*
o "utcomes% the specific costs ad beefits associated !ith the ')9 proDect*
o
De#elopment Impacts% the cotributio of the ')9 proDect to broader de-elopmet $oals*
o some eEtet ad particularly i relatio to outputs, outcomes, ad de-elopmet impacts as you mo-e from ri$ht to left alo$ the -aluechai, assessmet becomes more difficult, more costly but also more -aluable* hat mo-e also represets somethi$ of a chroolo$y* hus, asidicated i i$ure @, iterest i assessi$ differet aspects of the ')9 -alue chai has cha$ed o-er time, !ith the stro$ diffusio of ')9proDects o! creati$ most particular iterest i assessmet of impacts, as opposed to uptake, a-ailability or readiess* ' this )ompedium, themai focus is o assessmet of impacts rather tha other -alue chai sta$es*
'i1ure ;2 C/an1in1 'ocus of ICT*D Assessment Over Time
Time
Readiness
. A!areess
. 'frastructure
. i$ital i-ide
Uptake
. emad
. sa$e
. se i-ide
Impat
. 6fficiecy
. 6ffecti-eess. 6Fuity
Availability
. +upply
. Maturity +ta$e
Level of ICT*D
Activit4
9
7/24/2019 Idrc Ia for Ict4d Compendium
9/167
/
7/24/2019 Idrc Ia for Ict4d Compendium
10/167
%+$ Classif4in1 t/e Overall Impact of an ICT*D Project
7e ca classify the o-erall impact of a ')9 proDect ito oe of the fi-e follo!i$cate$ories%
!otal "ailure% the iitiati-e !as e-er implemeted, !as implemeted but
immediately abadoed, or !as implemeted but achie-ed oe of its $oals*
#argely unsuess"ul% some $oals !ere attaied but most stakeholder $roups did
ot attai their maDor $oals ador eEperieced si$ificat udesirable outcomes*
$artial suess%partial "ailure% some maDor $oals for the iitiati-e !ere attaied
but some !ere ot ador there !ere some si$ificat udesirable outcomes #argely suess"ul% most stakeholder $roups attaied their maDor $oals ad did
ot eEperiece si$ificat udesirable outcomes*
!otal suess% all stakeholder $roups attaied their maDor $oals ad did ot
eEperiece si$ificat udesirable outcomes*MaDor $oals are the mai obDecti-es a $roup !ated to achie-e !ith the ')9
proDect (!hich mi$ht typically relate to outputs ador outcomes ador de-elopmet
impacts"# udesirable outcomes are ueEpected outcomes that a $roup did ot !at tohappe but !hich did happe*
7/24/2019 Idrc Ia for Ict4d Compendium
11/167
8$ An Overvie0 of ICT*D Project Impact Assessment
'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
12/167
he )ompedium offers a syopsis of frame!orks !ithi four of the siE cate$ories,as summarised i able *
!ype Sub(!ype Fous *ompendium
+o,
?636R') CostBeneit Anal!sis
$ro%ect Goals 2
'+)'='36.+6)'') )ommuicatio+tudies
Communicationsor
De#elopment
@
e-elopmet +tudies Capa3ilities-en 9
0i#elihoods &rame'or( /
'formatio +ciece Inormation +conomics B
Inormation ,eedsMapping C
+ociolo$y CulturalInstitutional 8
'++6.+6)'') +nterprise 5Gro'th6 >a (5ariables">b (Relatios">c (5alue )hai
Gender 0
A=')A'43.+6)'')
Telecentres
Table %2 ICT*D Impact Assessment 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
13/167
8A$ Comparin1 IA 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
14/167
1 2 @ 9 / B C 8 >a >b >c 0
*ost(
-ene"it
Analysis
$ro%ectGoals
Communicationsor
De#elopment
Capa3ilities5-en6
&rame'or(
0i#elihoods&rame'or(
Inormation+conomics
Inormation,eedsMapping
CulturalInstitutional
&rame'or(
+nterprise5Varia3les6
+nterprise5*elations
6
+nterprise5ValueChain6
Gender Telecentres
$rimary%Seondary MiEed rimaryypicallyReFuired
rimaryypicallyReFuired
rimaryReFuired
rimaryReFuired
rimaryReFuired
rimaryReFuired
rimaryReFuired
rimaryReFuired
rimaryReFuired
rimaryReFuired
rimaryReFuired
rimaryReFuired
'ata(&athering
)ethodsMultiple Multiple Multiple Multiple Multiple Multiple Multiple Multiple Multiple Multiple 'ter-ie!s
lusMultiple Multiple
$artiipatory 3ot likely ossible Rarely ossible ossible 3ot likely ossible ossible ossible Rarely 3o ypical ossible
uasi(/0perimental ossible ossible ypical ossible ossible ossible 3ot likely Rarely ossible ossible Rarely Rarely ossible
uantitative%ualitative
Juatitati-e 6ither 6ither 6ither 6ither
7/24/2019 Idrc Ia for Ict4d Compendium
15/167
Impact Assessment 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
16/167
he basic elemets i )
7/24/2019 Idrc Ia for Ict4d Compendium
17/167
Sensitivity analysis%future estimates of costs ad beefits caot be ko! for certai*
o accout for such risks, a sesiti-ity aalysis (for eEample, see ;umar 2009" ca be
udertake, !hich models -arious !hat if scearios looki$ at differet possibilities(e*$* !hat if fe!er cliets pay for the ser-ice tha aticipatedG 7hat if maiteace costsare hi$her tha aticipatedG"*
4e ca also iclude%
6With6 versus 6without6 analysis%to better uderstad the impact of a ')9 proDect
usi$ )
7/24/2019 Idrc Ia for Ict4d Compendium
18/167
7/24/2019 Idrc Ia for Ict4d Compendium
19/167
"eferences
;umar, R* (2009" e)houpals% a study o the fiacial sustaiability of -illa$e 'teret
ceters i rural Madhya radesh,Inormation Technologies and InternationalDe#elopment, 2(", 9/.C@http%!!!*mitpressDourals*or$doipdf0*B2/99C/209@>CB
)oA (200Ba",=and3oo( o Cost Beneit Anal!sis, )ommo!ealth of Australia, )aberra
http%!!!*fiace*$o-*aufiframe!orkdocsHadbookLofL) A CostBeneit +#aluation -tud!, )etre for 6lectroic
?o-erace, 'dia 'stitute of Maa$emet, Ahmedabadhttp%!!!*iimahd*eret*ie$o-documets$yadoot.e-aluatio*pdf A )
7/24/2019 Idrc Ia for Ict4d Compendium
20/167
!ariants
* *ost(/""etiveness Analysis* his is a techiFue used !here costs ca be measured but itis hard to assi$ a fiacial -alue to beefits* )ost.effecti-eess aalysis measures the costof deli-eri$ a particular impact# typically compari$ the costs of differet approaches (e*$*!ith or !ithout ')9"* or eEample, ;helladi (200" see summary belo! compares thecost effecti-eess of fi-e differet alterati-es for coecti$ t!o millio midlo!.icome+al-adorias to the 'teret* he alterati-es ra$e from a basic 0.) telecetre up to a full.ser-ice 20.) telecetre* hey share some fiEed costs but the -ary o other setup costs* 4this basis, calculates that the full.ser-ice 20.) telecetre !ill be most cost.effecti-e i termsof cost per )* +ee also otashik & Adkis (>>B" see
7/24/2019 Idrc Ia for Ict4d Compendium
21/167
E=amples of se 3 *ost(-ene"it Analysis
Cost-+enefit Anal4sis
E=ample %27helladi
Comment "eference
airly detailed lists of direct cost ad beefit items* A simple
report ad estimatio of re-eues ad costs ad calculatioof future profits* A rather premature eE.post e-aluatio ofthe cost.effecti-eess of itroduci$ telecetres(ifocetros"*
;helladi, Y* (200" The Inocentros Telecenter Model, 7orld Resources
'stitute, 7ashi$to, )http%!!!*eEtbillio*etfiles'focetros*pdf'mpact assessmet report# 4pe Access# 29 pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio telecetres i 6l +al-ador that aim
to icrease midlo!.icome users1 access to the'teret
'mpact le-el ')9 proDect
Research Resource 3o clearly specified method of data
collectio ad data source*
rimary +ome iter-ie!s !ith ifocetros officials*
+ecodary rasactio ad performace records from
fi-e ifocetros*
4ther )ross.sectioal# Juatitati-e (i terms of cost
compariso ad re-eue estimatio"# 3ot participatory
+mphasis% eE.post assessmet
Cost identiication and #aluation% oly direct ad ta$ible costs
Beneit identiication and #aluation% oly re-eue streams from
pro-idi$ ser-ice
Discount rate% ot applied
Decision rule% a simple cost aalysis ad compariso !ith re-eue
-ensiti#it! anal!sis% ot icluded
7ith #ersus 'ithout anal!sis% ot udertake
Dept/ of Met/od ,uidance Causal Lin< to ICT*D 'indin1s on ICT*D Impact
5ery limited detail o ho! to collect data, ho! toidetify costs ad beefits ad ho! to -alue them*air o the summary of data to sho! costestimatio*
+imple lik of ')9 to selli$ ') ser-ices* 5ery !eaklik to sho! the actual -alues of usi$ the ifocetros fromthe beeficiaries1 perspecti-e* Makes claim that proDect !asistrumetal i buildi$ huma capacity ad creati$techolo$ical a!areess*
'focetros are ecoomically sustaiable ad !ould break e-e i 8
moths*
remature e-aluatio of the proDect (fi-e moths after implemetatio"
did ot eable obser-atio of outcomes ad social de-elopmet impactof the ifocetros o the user commuity*
here is a time.la$ factor to obser-e ')9 impacts*
)ommuity.based cotet ad ser-ice ad public.pri-ate partership
are critical for the fiacial sustaiability of ifocetros*
'come $eerati$ (cost.reco-ery" ')9 proDects should ha-e a soud
re-eue model beyod access char$es to make them fiaciallysustaiable*
+aseline:Counterfactual !alue C/ain &ta1e6s7
3o baselie ad o !ith!ithout couterfactualaalysis*
Mai focus is o 4utput (facilities ad ser-ices" ad4utcome (ecoomic rets $eerated for the ifocetros as aresult of commuities1 use of the cetre1s outputs" rather thaimpact of ')9 proDects*
C
http://www.nextbillion.net/files/Infocentros.pdfhttp://www.nextbillion.net/files/Infocentros.pdfhttp://www.nextbillion.net/files/Infocentros.pdf7/24/2019 Idrc Ia for Ict4d Compendium
22/167
Cost-+enefit Anal4sis
E=ample 827umar
Comment "eference
A -ery $ood aalysis of the potetial for the sustaiabilityof -illa$e.iteret kiosks (e)houpals" i 'dia* 3ote ideaso tria$ulatio ad -alidatio of cost ad beefit
estimates ad ecessary assumptios* 5ery $oodcosideratio of discout rates ad applicatio ofsesiti-ity aalysis*
;umar, R* (2009" e)houpals% a study o the fiacial sustaiability of -illa$e 'teretceters i rural Madhya radesh,Inormation Technologies and InternationalDe#elopment, 2(", 9/.C@
http%!!!*mitpressDourals*or$doipdf0*B2/99C/209@>CBRefereed Doural article# 4pe Access# 2> pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio -illa$e 'teret kiosks
i rural 'dia supporti$ tradi$ ofsoybeas
'mpact le-el ')9 proDect
Research Resource 4e idepedet researcher for >
days
rimary ormal ad iformal iter-ie!s !ith o!i$
compay persoel ad maa$ers, e)houpals operators,farmers, traders ad -illa$ers usi$ the e)houpals*?roup discussios i social places*
+ecodary rasactio data from e)houpals
4ther )ross.sectioal# Juatitati-e ad Fualitati-e#
3ot participatory
+mphasis% eE.post sustaiability e-aluatio
Cost identiication and #aluation% direct ad ta$ible costs
Beneit identiication and #aluation% oly uses trasactio cost sa-i$s ad
impro-emet i procuremet Fuality beefits
Discount rate% both base ad risk adDusted discout rate applied
Decision rule% et preset -alue ad payback period
-ensiti#it! anal!sis% the effect of se-eral risk elemets o the profitability of the
proDect is aalysed
7ith #ersus 'ithout anal!sis% compares the cost of trasactios (tradi$" !ith ad!ithout the proDect
Dept/ of Met/od ,uidance Causal Lin< to ICT*D 'indin1s on ICT*D Impact
4e pa$e o data collectio adtria$ulatio procedure* ?ood detail oassumptios as !ell as calculatios ofre-eues* 3o appediE of iter-ie!protocol*
A idirect lik of ') beefits (throu$h eabli$ soybeatradi$" to ecoomic sustaiability* 7eak lik to socialde-elopmet of beeficiaries*
5illa$e iteret kiosks (e)houpals" are ecoomically sustaiable ad ca reduce
tradi$ costs*
+ource of ecoomic sustaiability lies i the -alue.added ser-ices they offer ( i*e*
ite$rati$ ') i the process of a$ricultural product tradi$ rather tha from the') aloe"*
re-aili$ socio.cultural ad political structures of a commuity ca mediate the
social ad de-elopmetal impact of fiacially sustaiable telecetres*+aseline:Counterfactual !alue C/ain &ta1e6s7
)ompares the cost of trasactios !ithad !ithout usi$ the proDect to sho!the beefit (impact" of the proDect*
Mai focus is o 4utput (facilities ad ser-ices" ad4utcome (ecoomic rets $eerated for the e)houpals adthe o!i$ compay" rather tha impact o the li-es of
farmers ad -illa$ers*
8
http://www.mitpressjournals.org/doi/pdf/10.1162/1544752043971161http://www.mitpressjournals.org/doi/pdf/10.1162/15447520439711617/24/2019 Idrc Ia for Ict4d Compendium
23/167
Cost-+enefit Anal4sis
E=ample ;2#obo 8
-alakrishnan
Comment "eference
3ot cost.beefit aalysis but Dust beefit aalysis, of !hichthis is a -ery $ood eEample related to e.$o-ermetser-ices -ia kiosks* etailed descriptio of report card
methodolo$y i user beefits aalysis* Rather arro!defiitio of beefits* oes ot Fuatify (-alue" all beefititems* A $ood eEample for coducti$ Fuasi.eEperimetal!ork*
=obo, A* &
7/24/2019 Idrc Ia for Ict4d Compendium
24/167
ICT*D Impact Assessment 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
25/167
Met/odolo1ical &ummar4
$ro5et &oals Framework$rimar!-econdar!9 rimary
ypicallyReFuired
' order to Dud$e the achie-emet of specific$oals
DataGathering
Methods9
Multiple 3ot pre.determied depeds o proDect $oals
$articipator!9 ossible )ould iclude discussio of stakeholder $oals adidicators, icludi$ meta.aalysis of $oals
:uasi+;perimental9 ossible 6*$* compare commuity ')9 users -s* o.users
:uantitati#e:ualitati#e
9
6ither 3ot pre.determied depeds o proDect $oals
MultiDisciplinarit!9 ossible 3ot pre.determied depeds o proDect $oals
Timing9 6ither
7/24/2019 Idrc Ia for Ict4d Compendium
26/167
!ariants
*-roadening o" &oal Analysis* o pro-ide some $reater cosistecy, oe ca broade outthe assessmet of impact to co-er a defied set of !ider $oals*
7/24/2019 Idrc Ia for Ict4d Compendium
27/167
E=amples of se 3$ro5et &oals
Project ,oals E=ample
%2-allantyne
Comment "eference
A -ery clear eEample of assessi$ a ')9
proDect a$aist its $oals* ro-ides more aeEample rather tha ay $ood practice$uidace, ad otes difficulties of assessi$post hoc !he $oal idicators ad data.$atheri$ methods ha-e ot pre-iously beethou$ht throu$h*
7/24/2019 Idrc Ia for Ict4d Compendium
28/167
Project ,oals E=ample 82-athelor 8
+orrish
Comment "eference
3ot assessmet of a idi-idual proDect,but descriptio of a assessmetframe!ork for ')9 pilot proDects* he
approach is Fuite $eeric, !ith relati-elylittle that is ')9.specific, but !ouldpro-ide a useful o-erall frame!ork*
7/24/2019 Idrc Ia for Ict4d Compendium
29/167
ICT*D Impact Assessment 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
30/167
proDect users eed moey, skills, moti-atio, cofidece, trust ad ko!led$e i order toaccess, assess ad apply the processed data they $et from the ')9 system*
Ation Resoures%
7/24/2019 Idrc Ia for Ict4d Compendium
31/167
Met/odolo1ical &ummar4
*ommuniations("or('evelopment Framework$rimar!-econdar!9 rimary
ypicallyReFuired
' order to Dud$e cha$es i idi-idualbeha-iour
DataGathering
Methods9
Multiple Most studies use sur-eys, but all othermethods could be icorporated
$articipator!9 Rarely )ould be potetial but appears little used sofar i practice eEcept i 5ariat 2
:uasi+;perimental9 ypical +tro$ emphasis i may studies ocompari$ those eEposed -s* ot eEposed tocommuicatio
:uantitati#e:ualitati#e
9
6ither
7/24/2019 Idrc Ia for Ict4d Compendium
32/167
"eferences
Chttp%her*oEfordDourals*or$c$ireprit29/BC
Heeks, R*
7/24/2019 Idrc Ia for Ict4d Compendium
33/167
!ariants
* *ommuniation as a $roess* he )9 model focuses o ')9 proDect actors asrecipiets of commuicated data* Ho!e-er, ')9 proDects ca also be assessed by seei$actors as commuicators !ho are themsel-es trasmitti$ data* here are t!o mai !ays thisca be doe%
uctioalist% looki$ particularly at the !ay i !hich ')9 cha$es the Fuatitati-e
ad Fualitati-e ature of the commuicatio process* or eEample, :a$u et al1s (200C"study of mobile telephoy1s impact i reduci$ commuicatio costs ad risks, butreiforci$ eEisti$ structures of commuicatio*
+ociolo$ical% seei$ commuicatio as a performed practice !ithi a social coteEt* oreEample, Mosse & 3ielse1s (2009" study !hich sees commuicatio as fuctioal butalso as symbolic (performed to preset ad le$itimiOe a ratioal or$aiOatio to eEteralcostituecies" ad ritualistic (performed as a meas to reiforce membership of a
particular commuity"* (+ee also )ompedium etry o )ultural.'stitutioalrame!ork*"
2*$artiipatory%Soial *hange Approah* he )9 model outlied abo-e comes from thebeha-ioural cha$e traditio of commuicatios.for.de-elopmet* Ho!e-er, there aremay other strads to )9 (6ldis *d*# 7aisbord >>>"* ' particular, there is a participatory,social cha$e strad that sees the beha-ioural cha$e approach as arro!, top.do!,
pateralistic ad idi-idualistic* his strad istead seeks empo!ermet for commuities ascollecti-es to defie !hat iformatio they reFuire, to seek out appropriate commuicatioschaels, ad ultimately to cotrol, o! ad maa$e their commuicatio processes(i$ueroa et al 2002"*
7/24/2019 Idrc Ia for Ict4d Compendium
34/167
arks, 7*, ?ray.elder, *, Hut, :* &
7/24/2019 Idrc Ia for Ict4d Compendium
35/167
E=amples of se - *ommuniations
Communications
E=ample %2 *hesterton
Comment "eference
Assesses impact of data commuicated -ia
traditioal ot di$ital ')s* +tro$est elemetis compariso of eEposed -s* o.eEposed$roups* oes pro-ide $ood detail o methods*
)hesterto, * (2009"+#aluation o the Meena Communication Ini tiati#e, 3')6, ;athmadu
http%!!!*uicef*or$e-aldatabasefilesR4+AL2009L800LMeeaL)ommL'itiati-e*pdf'mpact assessmet report# 4pe Access# 0B pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio multi.chael (radio, 5,
ad-erts, etc" iitiati-e i
7/24/2019 Idrc Ia for Ict4d Compendium
36/167
Communications E=ample 82
)eekers et al
Comment "eference
A $ood model for ri$orous aalysis of commuicatiosimpact usi$ a positi-ist, mass before.ad.after sur-eymodel, !hich also measures impact of differet
idepedet -ariable (le-el of commuicatio" o typical)9 itermediate (attitudes, beliefs" ad depedet(beha-iour" -ariables*
Meekers, *, A$ha, +* & ;lei, M* (200/" he impact o codom use of the00T :eue social marketi$ pro$ram i )ameroo, ?ournal o Adolescent=ealth, @B, /@0*e./@0*e2
Refereed Doural article# Restricted Access# 2 pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio multi.chael (radio, ma$aOie,
ad-ertisemet, peer educatio sho!" adolescetreproducti-e health commuicatio iitiati-e(00T :eue" i )ameroo
'mpact =e-el idi-idual recipiets
Research Resource oOes of paid researchers
!orki$ o-er a period of c*2 E t!o !eeks
rimary
7/24/2019 Idrc Ia for Ict4d Compendium
37/167
Communications E=ample
;2-ertrand et al
Comment "eference
3ot assessmet of a idi-idual proDect, but are-ie! of 29 other commuicatios impactassessmet studies* seful i maki$ the )9
model eEplicit, ad i offeri$ $uidace o $oodpractice i )9 impact assessmet*
C
http%her*oEfordDourals*or$c$ireprit29/BCRefereed Doural article# 4pe Access# @ pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio ot a idi-idual study, but
a meta.re-ie! of 29 other studies ofimpact of (mass" commuicatio proDectso H'5.rele-at beha-iour i de-elopi$coutries
'mpact =e-el idi-idual recipiets
Research Resource clear for idi-idual
sur-eys but must i-ol-e research teams!orki$ for some days!eeks
rimary
7/24/2019 Idrc Ia for Ict4d Compendium
38/167
ICT*D Impact Assessment 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
39/167
I*!s and *apabilities
3ot eEplicit !ithi +e1s !ork but ca iterpret ')s as a commodity (a $ood or ser-ice" !ith
a -alue oly i terms of !hat it helps idi-iduals to do or to be (adapted from he$ &7alsham 200C, from Robeys 2000"%
')s therefore ha-e $eeral characteristics (processi$ ad commuicati$ di$ital data" butthe lik to actual achie-emets is mediated at t!o sta$es%
)o-ersio of ')s1 characteristics ito capabilities for a idi-idual is shaped by factors
that may be persoal (e*$* disability, a$e, $eder"# social (ifrastructure such as health,educatio# istitutios of formal policies ad iformal orms-alues# ad relatioships of
Commodities
e$1$ ICT
and t/eir
c/aracteristics
e$1$ di1ital data
processin1:
communication
Capabilities
>
!ectors of
potential
functionin1s
e$1$ di1ital
communication
One vector
of ac/ieved
functionin1s
e$1$ faster and
easier access to
better
information
$ersonal social
and en#ironmentalcon#ersion actors
Choice
)eans to
Ahieve
Freedom to
Ahieve
Ahievement
e.g. literac!
telecominrastructure
go#ernment
censorship
e.g. personal
preerencesneeds or social
pressures
ICT
7/24/2019 Idrc Ia for Ict4d Compendium
40/167
7/24/2019 Idrc Ia for Ict4d Compendium
41/167
Anal4sis
Strengths ro-ides a ori$ial perspecti-e o e-aluatio
Reco$ises each idi-idual perso% their aspiratios, eeds ad choices
A-oids both social ad techolo$ical determiism% reco$ises that techolo$y ca create
e! freedoms but also eEplais !hy same techolo$y leads to differet outcomes idifferet situatios
rame!ork is !ell.reco$ised by de-elopmet a$ecies ad practitioers
seful focus o o.usa$e (urealised fuctioi$s" ad o costraits to actio
(ufreedoms"
Weaknesses
=imited usa$e of frame!ork to date for ')9 proDects, so o cosistet approach for 'A
ReFuires iterpretatio to apply for ')s% ori$ial frame!ork says othi$ eEplicit ad is
Fuite academic ad fleEible (i*e* uclear"
ReFuires defiitio (e*$* participati-e" of !hat aspects of freedom are -alued# e*$* ')s
ofte pro-ide the freedom to access poro$raphy* 's that a de-elopmetal freedomG
ReFuires uderstadi$ of the potetial freedoms 34 chose, as !ell as the actualfreedoms chose
)ompleEity that capabilities are both iputs to ad outputs from ay ')9 proDect
otetial for Dust addi$ a complicated foudatio to other!ise simple issue of ho!
')9 users actually use ad do1t use the techolo$y
Met/odolo1ical &ummar4
7/24/2019 Idrc Ia for Ict4d Compendium
42/167
Met/od "ecommendations
7orth further i-esti$atio for ')9 'A !ork, particularly if there are iterests i
huma de-elopmetempo!ermet issues of idi-iduals ador i o.usa$e or failure todeli-er of ')9
'clude cosideratio of both costraits to usa$e ad o.usa$e of ')s*
?ood !ork reFuires relati-ely i.depth data.$atheri$ from idi-iduals*
4f three literature items, Alampay pro-ides most i.depth usa$e but he$ & 7alsham
pro-ide clearest model*
4-erall, reFuires more precursory !ork to be doe to de-elop a clear ')9 'A
methodolo$y from this frame!ork*
"eferences
Robeys, '* (2000"An n'or(a3le Idea or a $romising Alternati#e9 -ens Capa3ilit!
Approach *ee;amined, iscussio aper 00*@0, )eter for 6coomic +tudies, i-ersityof =eu-e, 8/" 7ell.bei$, a$ecy ad freedom, The ?ournal o $hilosoph!, =ZZZ''(9",
B>.22
+e, A* (>>>"De#elopment as &reedom* 3e! York, ;opf
+iblio1rap/4
Mado, +* (2009"+#aluating +Go#ernance $ro%ects in India> A &ocus on Micro0e#el
Implementation, 7orki$ aper o*29, 'formatio +ystems ept, =+6, =odohttp%is2*lse*ac*uk!ppdf729*ses +e1s cocepts (freedoms, opportuities, capabilities, fuctioi$s" to colour ae-aluatio of ;erala proDects R'63+ ad Akshaya# but pro-ides o frame!ork or
http://www.econ.kuleuven.ac.be/ces/discussionpapers/Dps00/DPS0030.pdfhttp://is2.lse.ac.uk/wp/pdf/WP124.PDFhttp://www.econ.kuleuven.ac.be/ces/discussionpapers/Dps00/DPS0030.pdfhttp://is2.lse.ac.uk/wp/pdf/WP124.PDF7/24/2019 Idrc Ia for Ict4d Compendium
43/167
!ariants
* *ombined #ivelihoods%*apabilities Framework* ?i$ler (2009" pro-ides such aframe!ork, !hich basically substitutes capabilities idea for li-elihood strate$ies i the +=frame!ork* (Ar$uably, capabilities are more aki to the iteractio of assets adstructuresprocesses, !ith fuctioi$s bei$ the actual strate$ies adopted*" ?i$lerdisti$uishes bet!ee idi-idual capabilities (!ith siE dimesios% iformatioal,
psycholo$ical, social, ecoomic, political ad cultural" ad $roupcommuity capabilities(!ith siE dimesios% iformatioal, or$aisatioal, social, ecoomic, political ad cultural"*6ach of these is liked to a set of outcome idicators that could be measured accordi$ to ')impact*
7/24/2019 Idrc Ia for Ict4d Compendium
44/167
E=amples of se - *apabilities
Capabilities 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
45/167
Capabilities 'rame0or Measuring the Impact o Inormation on De#elopment, *Mc)oell (ed*", 'R), 4tta!a
http%!!!*idrc*caee-.>@C2.20..4L4')*htmlre.impact assessmet report# 4pe Access# c*20 pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio )ommuity
iformatio cetres i three Africacoutries (Mala!i,
7/24/2019 Idrc Ia for Ict4d Compendium
70/167
Information
Needs:Mappin1 E=ample
;2-eardon et al
Comment "eference
+tro$ly iformatio.based approach* 5ery $ood foreed idetificatio ad usi$ that as a iput fordesi$i$ the structure ad process of commuicatioiter-etio proDects* 'detifies processes ad po!er
issues that affect the percei-ed -alue ad usefulessof iformatio* oes offer a list of commuity.specific impact idicators de-eloped based obottom.up participatio*
7/24/2019 Idrc Ia for Ict4d Compendium
71/167
Information Needs:Mappin1
E=ample *2'unombe 8
Heeks
Comment "eference
oes ot cosider specific impact of ')9 butmaps !ith!ithout ') iformatio eeds adflo!s of small eterprises* ro-ides usefulchecklists ad method o eterprise iformatio
eeds (-ia demadIsupply $aps", ad detail omethodsistrumets*
ucombe, R** & Heeks, R*
7/24/2019 Idrc Ia for Ict4d Compendium
72/167
ICT*D Impact Assessment 'rame0or>0%@"* hey may be formal sactios(re$ulatios such as rules, la!s or cotracts" or they may be iformal sactios (such as theorms, -alues ad meai$s boud up i the otio of culture"* 7e ca summarise therelatioship to techolo$y ad huma beha-iour i the follo!i$ !ay (adapted from4rliko!ski >>2"%
Institutional
Conte=t2
Regulations?*ulture
'mpacts'flueces
7/24/2019 Idrc Ia for Ict4d Compendium
73/167
' $eeral, thou$h, these ha-e so far bee see maily as atioal.le-el ad ha-e bee usedmaily to model static iputs to proDects, ot impacts from proDects*
'mpact assessmet could therefore use these frame!orks but mi$ht eed to eEted them it!o !ays* irst, looki$ at a broader ra$e of cultural orms ad -alues* +ecod, addi$ iimpacts o the other type of istitutios formal re$ulatory costraits% rules, la!s adcotracts*
Anal4sis
Strengths ' its full form, pro-ides a !ay to assess ho! ')9 affects the key iflueces o all
huma beha-iour# particularly the soft ofte rather hidde ifluece of culture
ro-ides a i.depth, o.techo.cetric meas to uderstad real -alues ad practices o
')9 proDect
Weaknesses
ifficulty that istitutioal forces such as culture are both cause ad effect i relatio to
')9, ad may ')9 assessmets to date see oly oe side especially seei$culture as a static ifluece o (i*e* iput to, ot impact of" ')9 proDect implemetatio
)ulture particularly is ofte see as a atioal set of -alues, rather tha somethi$ that is
commuity., e-e idi-idual., specific
here are specific models ad methods for i-esti$ati$ static, atioal.le-el, cause.
orieted perspecti-es o culture, but little specific $uidace to date i ')9 research othe recommeded dyamic, micro.le-el, impact.orieted perspecti-e o istitutios
7/24/2019 Idrc Ia for Ict4d Compendium
74/167
Met/odolo1ical &ummar4
*ultural(Institutional Framework
$rimar!-econdar!9 rimaryReFuired
'f aalysi$ cha$es to cultural orms ad -alues*ormal istitutioal re$ulatios may be recordedi documetatio, thou$h their impact i practice!ill ot
DataGathering
Methods9
Multiple Most studies use i.depth iter-ie!s, but all othermethods could be icorporated
$articipator!9 ossible ?i-e the importace of idi-idual perceptiosad -alues* Also a umber of eEamples of
participatory research i*e* researcher !orked o')9 proDect*
:uasi+;perimental9 Rarely Maily adopts a case study approach
:uantitati#e:ualitati#e
9
ypicallyJualitati-e
7/24/2019 Idrc Ia for Ict4d Compendium
75/167
"eferences
afoulas, ?* & Macaulay, =* (200" '-esti$ati$ cultural differeces i -irtual soft!are
teams,+lectronic ?ournal o Inormation -!stems in De#eloping Countries, C(9", .9http%!!!*eDisdc*or$oDs2ideE*phpeDisdcarticle-ie!ile@C@C
=icker, * (200" A $ift from the $odsG )ompoets of iformatio techolo$ical
fatalism, determiism i se-eral cultures,+lectronic ?ournal o Inormation -!stems inDe#eloping Countries, C(", .http%!!!*eDisdc*or$oDs2ideE*phpeDisdcarticle-ie!ile@9@9
3orth, *)* (>>0"Institutions Institutional Change and +conomic $erormance,
)ambrid$e i-ersity ress, )ambrid$e, ;
4rliko!ski, 7*:* (>>2" he duality of techolo$y% rethiki$ the cocept of techolo$y
i or$aiOatios, "rganiation -cience, @(@", @>8.92C
http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/37/37http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/34/34http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/37/37http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/34/347/24/2019 Idrc Ia for Ict4d Compendium
76/167
!ariants
*Institutional Systems and 'ualism* his ar$ues that, rather tha seei$ istitutioal
forces, such as cultural -alues ad re$ulatios, as idi-idual free.floati$ iflueces, !eshould reco$ise that there are istitutioal systems (or istitutioal et!orks" ofelemets that are self.reiforci$ ad self.reproduci$ (Heeks & +atos 200C see summary
belo!"* hese self.reiforci$ systems cosist of sets of formal ad iformal istitutioalforces, of huma beha-iours, ad of or$aisatioal structures created by the beha-iour adshaped by the forces*
7he a ')9 proDect is itroduced, it ofte bri$s t!o differi$ istitutioal systems ito
cotact -ia the ')9 applicatio% first, the istitutioal system of the proDectdesi$ersimplemeters# secod, the istitutioal system of the proDect users* his is asituatio of istitutioal dualism t!o differi$ istitutioal systems that o! itersect*he outcome may be domiatio of desi$ers1 istitutios, or domiatio of the users1istitutios, or some hybrid outcome*
I*!@' $ro5et Appliation%'stitutio & 4r$aisatio
Institutions'
4rganisations'
+actios & )apacities
InstitutionsU
4rganisationsU
+actios & )apacities
Institutional
&4stem D
Institutional
&4stem
7/24/2019 Idrc Ia for Ict4d Compendium
77/167
Mimetic isomorphism% arisi$ from the copyi$ of other proDects i order to reduce the
ucertaity about ho! to proceed !ith ')9*
3ormati-e isomorphism% arisi$ from the orms !ithi the professioal $roupi$s to
!hich ')9 proDect staff percei-e themsel-es to belo$*A eEample of usi$ this to aalyse a ') proDect i a de-elopi$ coutry (thou$h i a bakrather tha a traditioal ')9 setti$" ca be foud i 8@" he iro ca$e re-isited% istitutioal isomorphism
ad collecti-e ratioality i or$aiOatioal fields,American -ociological *e#ie', 98, 9C.B0
Heeks, R*
7/24/2019 Idrc Ia for Ict4d Compendium
78/167
E=amples of se 3 *ultural(Institutional Framework
Cultural-Institutional
E=ample %2 &riswold et al
Comment "eference
=ooks Dust at culture, ot at all istitutios*ro-ides o frame!ork or checklist, but
discusses the cultural orms ad -aluesassociated !ith ')* Relati-ely limited i the$uidace for 'A practice that it offers*
?ris!old, 7*, Mcoell, 6*M* & Mcoell, *6* (200B" ?lamour ad hoor% $oi$olie ad readi$ i 7est Africa culture,Inormation Technologies and
International De#elopment, @(9", @C./2http%!!!*mitpressDourals*or$doipdf0*B2itid*200C*@*9*@CRefereed Doural article# 4pe Access# B pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio a -ariety of iteret cafes
ad school.based 'teret access i partsof Accra, =a$os, ad some other parts ofurba 3i$eria
'mpact =e-el idi-idual recipiets
Research Resource three idepedet
researchers for se-eral moths
rimary 4bser-atio i cybercafes,
iter-ie!s !ith a fe! cybercafe maa$ers,ad focus $roups i three schools (uclearho! may i-ol-ed"
+ecodary 3ot stated
4ther )ross.sectioal# Maily Fualitati-e,
thou$h some implicit Fuatificatio of ho!time is spet# 3ot participatory
'mplicitly uses the tria$ular model focuses o a particular huma a$ecybeha-iour (readi$"!hich it sees as re$ulated by local culture* hat culture, i tur, may be impacted by theitroductio of ')9* (oes also complete the tria$le by reco$isi$ ho! readi$.typebeha-iours are part of the usa$e of ')s*"
' looki$ at the impact of techolo$y o istitutios (culture", cotrasts the cha$e -s*reiforcemet -ie!s*
'mplicitly cosiders some otio of istitutioal systems (see 5ariat "% reiforci$ $roups oforms, -alues ad practices*
Dept/ of Met/od ,uidance Causal Lin< to ICT*D 'indin1s on ICT*D Impact
:ust a si$le para$raph* ocus $roup, butot iter-ie!, Fuestios pro-ided*
Reco$ises the duality bet!ee culture(istitutios" ad ')% that particular ormsad -alues come to be associated !ith ')#both affecti$ ad bei$ affected by ')*
?oi$ olie eats ito time for phoe calls, tele-isio, letter.!riti$, ha$i$ !ith frieds*
7/24/2019 Idrc Ia for Ict4d Compendium
79/167
Cultural-Institutional
E=ample 82)osse 8
+ielsen
Comment "eference
A eEample of ')9 actio research !ith auseful three.!ay characterisatio of practices(beha-iourQistitutio combiatios" that couldbe used o other 'A* oes reiforce the -alue of
lo$itudial ad i.depth research i order tostudy impact of culture ad other istitutios*
Mosse, 6* & 3ielse, * (2009" )ommuicatio practices as fuctios, rituals adsymbols,+lectronic ?ournal o Inormation -!stems in De#eloping Countries, 8(@",.Chttp%!!!*eDisdc*or$oDs2ideE*phpeDisdcarticle-ie!ile22Refereed Doural article# 4pe Access# C pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio study of commuicatio
practices i oe MoOambica healthdistrict at time of itroduci$ e! ').based health iformatio system
'mpact =e-el idi-idual recipiets
Research Resource !o researcher.
implemeters for four moths
rimary 'ter-ie!s ad discussios#
4bser-atio of actual practices# articipatobser-atio (oe researcher !as also a traierad facilitator for the proDect"
+ecodary roDect documets
4ther Juasi.lo$itudial (shortish period
but spaed the itroductio of ')9system"# Jualitati-e# articipatio i proDectof oe researcher
'mplicit use of the tria$le model% sees practices huma beha-iour $uided by a miE ofistitutioal forces as falli$ ito three types%
uctioal% to achie-e ratioal purposes e*$* sedi$ a email to commuicate iformatio to
assist a decisio
+ymbolic% to preset ad le$itimiOe a ratioal or$aiOatio to eEteral costituecies e*$*
usi$ ')s to be see as moder ad effecti-e by doors
Ritualistic% as a meas to reiforce membership of a particular commuity e*$* cotributi$
to a d?roup to be see as part of the membership of that $roup+ees these practices (beha-iour.istitutio combiatios" as both affecti$ ad affected by')s*
'mplicit, -er$i$ o eEplicit, use of the istitutioal dualism model, cotrasti$ traditioalpractices !ith differet istitutioal forces aroud, ad iscribed ito, ') bei$ itroduced*
Dept/ of Met/od ,uidance Causal Lin< to ICT*D 'indin1s on ICT*D Impact
hree para$raphs o method* 3o detailsof research istrumets*
Reco$ises the duality bet!ee practices(beha-iourQistitutio" ad ')% that particularorms, -alues ad beha-iours impact !hetherad ho! ') is used, but are i tur impacted!he ') is used*
escribes the three types of practices at health facility, district ad pro-icial le-el* oreEample, at pro-icial le-el, typi$ health data recei-ed ito health '+ ad produci$ reportsca be uderstood three !ays%
uctioal% to produce a$$re$ated health statistics*
+ymbolic% to demostrate to Miistry ad door officials the credibility of pro-ice staff ad
the autheticity of their !ork*
Ritualistic% to reiforce membership of the health '+ team*
')s do ot ha-e a determiistic impact o these three, but ca see ho! they mi$ht coflict
e*$* that ')s could ehace fuctioal ad symbolic practices but dama$e ritualistic practices,ad so ot be fully utilised*
+aseline:Counterfactual !alue C/ain &ta1e6s7
3o formal baselie sur-ey but proEied byobser-i$ pre. ad post.') practices*
)outerfactual obser-ed -ia thosesetti$s yet to ha-e ')s or resisti$iteded implemetatio of ')s*
;ey focus o se (both '). ad paper.basedpractices" but also cosideratio of 4utcomes
(impact o fuctioal, symbolic ad ritualisticpractices"*
C/
http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/112/112http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/112/112http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/112/1127/24/2019 Idrc Ia for Ict4d Compendium
80/167
Cultural-Institutional
E=ample ;2Heeks 8
Santos
Comment "eference
Rather dese academic style of !riti$, ad at least half thepaper focuses o ho! to eforce adoptio of ')9*
7/24/2019 Idrc Ia for Ict4d Compendium
81/167
ICT*D Impact Assessment 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
82/167
I*! $roess% these are itermediati$ -ariables that ca be take as proEies or precursors tothe impact of ')9 o eterprise performace* 6Eample -ariablesimpact idicatorsiclude%
ICT A#aila3ilit!% the eEtet to !hich etrepreeurs are able to access ')s ICT "'nership% the eEtet to !hich etrepreeurs themsel-es o! ')s
ICT se% the eEtet to !hich etrepreeurs actually use ')s
ICT sage% the actual uses to !hich etrepreeurs put ')s
+li$htly harder to assess, ucombe & Heeks (200" talk of eterprises $oi$ throu$h aiformatio trasitio poit* his is the poit or, more accurately, the process oftrasitio from etirely iformal to more balaced formaliformal iformatio systems* 'tis reFuired !he the eterprise reaches the capacity ad compatibility limits of its iformaliformatio systems* 4e impact, therefore, of ')9 may be to usefully haste eterprisesthrou$h the trasitio poit*
$roess% these are a set of -ariables that measure the processes !ithi a eterprise* ' somesese, they itermediate bet!ee the precursors ad the actual performace of the eterprise*hus the impact of ')9 o these, also, may be see as itermediate to the actual
performace of the eterprise* ' more detail, the -ariables are (adapted from =efeb-re &=efeb-re >>B"%
#/oll4
Informal
Information
#/oll4
'ormal
Information
Maki$ the rasitio
7/24/2019 Idrc Ia for Ict4d Compendium
83/167
i3ilit!% the de$ree to !hich the eterprise ca cope !ith cha$e i terms of its
processes, labour, eFuipmet, ad maa$emet* )ha$es could be reFuiredcustomisatio, or competi$ productsser-ices, or sales fluctuatios (e*$* a lar$e e!
order", or other challe$es such as loss of staff* he typical itetio is that ') !ouldicrease fleEibility*
A detailed list of possible idicators allo!i$ assessmet of ') impact o these -ariables ispro-ided i AppediE 6 of =efeb-re & =efeb-re >>B (http%!!!*idrc*caict9de-.@[email protected]')*html"*
Capa3ilities% the skills, ko!led$e ad attitudes boud up !ithi the eterprise* he
typical itetio is that ') !ould eEpad capabilities* 4e particular case is that oftecholo$ical capability !hich ca be read as a measure of io-atio* his is a
process idicator but is so critical i lo$er.term eterprise performace that it is oftere$arded as a performace measure* ')91s impact !ould be assessed as positi-e to theeEtet it eabled mo-emet up the cate$ories sho! belo! (ad re$ardless of !hether orot the fial sta$e is attaied"*
&cale of ,eneral Tec/nolo1ical Capabilit4
#evel 1, +on(prodution operational apabilities
a% si$ the mai productio techolo$y i-ol-ed i produci$ the eterprise1s $oods or ser-ices b% )hoosi$ the techolo$y
c% raii$ others to use the techolo$y
#evel 2 +on(prodution tehnial apabilities
2a% 'stalli$ ad troubleshooti$ the techolo$y
#evel B Adaptation without prodution
@a% Modifyi$ the fiished $ood or ser-ice to meet local cosumer eeds
http://www.idrc.ca/ict4d/ev-30737-201-1-DO_TOPIC.htmlhttp://www.idrc.ca/ict4d/ev-30737-201-1-DO_TOPIC.htmlhttp://www.idrc.ca/ict4d/ev-30737-201-1-DO_TOPIC.htmlhttp://www.idrc.ca/ict4d/ev-30737-201-1-DO_TOPIC.html7/24/2019 Idrc Ia for Ict4d Compendium
84/167
Income-ales% ')91s impact o the o-erall re-eues of the eterprise# these mi$ht be
proEied i terms of sales or the o!er1s icome from the eterprise* A e-e simplerproEy may be umber of customers or freFuecy of orders*
?o3s% ')91s impact o the umber of people employed i the eterprise* 'f
appropriate, oe could also look at employee icome, ad also at more Fualitati-emeasures the skill le-els of Dob, Dob security, !orki$ hours, ad !orki$ coditios*
Assets% ')91s impact o the assets (e*$* eFuipmet, accommodatio" o!ed by the
eterprise*=ooki$ at the impact of ')9 o profitability may be of iterest but is ot ofte possible$i-e data limitatios* =ike!ise impact of ')9 o eEports may apply oly i a limitedumber of cases*
Anal4sis
Strengths
ro-ides a comprehesi-e ad systematic meas to uderstad the impact of ')9 o
eterprise*
7/24/2019 Idrc Ia for Ict4d Compendium
85/167
Met/odolo1ical &ummar4
/nterprise .ariables
$rimar!-econdar!9 rimaryReFuired
less eterprise keep, ad !ill allo! access to,$ood !ritte records
DataGathering
Methods9
Multiple ?i-e the !ide -ariety of differet possible-ariables that could be studied
$articipator!9 ossible or eEample, $etti$ etrepreeurs to defie !hat-ariables are importat to them# perhaps
particularly !hat eterprise performace-ariables they -alue
:uasi+;perimental9 ossible or eEample compari$ ') users -s* o.users,or compari$ differet le-els of ') usa$e (e*$*see ucombe & Heeks (200""
:uantitati#e:ualitati#e
9
7/24/2019 Idrc Ia for Ict4d Compendium
86/167
"eferences
oer, :* (200C" )ustomer acFuisitio amo$ small ad iformal busiesses i urba
'dia,+lectronic ?ournal o Inormation -!stems in De#eloping Countries, @2(@", .B
http%!!!*eDisdc*or$oDs2ideE*phpeDisdcarticle-ie!ile9B92@2 ucombe, R** & Heeks, R*
Anal!tical Tools and Models, e-elopmet 'formatics !orki$ paper o*@0, 'M,
i-ersity of Machester, ;http%!!!*sed*machester*ac*ukidpmresearchpublicatios!pdidiL!p@0*htmAlso cotais ideas about other possible eterprise assessmet models*
=efeb-re, 6* & =efeb-re, =*A* (>>B"Inormation and Telecommunication Technologies>
The Impact o Their Adoption on -mall and Medium-ied +nterprises, 'R), 4tta!ahttp%!!!*idrc*caict9de-.>@[email protected]')*html)hapter 9 discusses impact of ')s# Appedices 6, ad ? su$$est possible idicatorsfor assessi$ ') impact* here is o particular focus o de-elopi$ coutries*
http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/464/232http://www.sed.manchester.ac.uk/idpm/research/is/ictsme/summary/summary.pdfhttp://www.sed.manchester.ac.uk/idpm/research/publications/wp/di/di_wp30.htmhttp://www.idrc.ca/ict4d/ev-9303-201-1-DO_TOPIC.htmlhttp://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/464/232http://www.sed.manchester.ac.uk/idpm/research/is/ictsme/summary/summary.pdfhttp://www.sed.manchester.ac.uk/idpm/research/publications/wp/di/di_wp30.htmhttp://www.idrc.ca/ict4d/ev-9303-201-1-DO_TOPIC.html7/24/2019 Idrc Ia for Ict4d Compendium
87/167
!ariants
*/ntrepreneur%/nterprise *ategorisation* he impact assessmet approach described
abo-e has bee blid to the particular moti-atios of the etrepreeur* Ho!e-er, !e caidetify three differet types of etrepreeur (Heeks 2008"%
-ur#i#alistsare those !ho ha-e o choice but to take up the icome.$eerati$ acti-ity
because they ha-e o other source of li-elihood* 'come pro-ided may be po-erty.lie ore-e sub.po-erty.lie* Most etrepreeurs i de-elopi$ coutries are of this type, adMead (>>9" describes them as supply.dri-e% forced ito eterprise by push factorsrelated to their po-erty ad lack of opportuity*
&l!ersare true etrepreeurs !ho ha-e take up eterprise because they see opportuities
for $ro!th* 'come le-els may meet more tha basic eeds, ad eterprises may $raduateto the medium.scale cate$ory* 4ly a -ery small proportio of de-elopi$ coutry smalletrepreeurs fall ito this cate$ory* Mead describes them as demad.dri-e% pulledito eterprise by factors such as the opportuity for profit*
Trundlersfall i bet!ee the t!o other $roupi$s ad represet those !hose eterprise
turo-er is rou$hly static ad !ho sho! o $reat desire or o $reat capacity to eEpad*'come pro-ided !ill be eou$h to meet basic eeds* hese form the secod.lar$est$roup of small etrepreeurs i de-elopi$ coutries, ad their stasis reflects the relati-e
lack of stro$ eEteral pushpull factors*As already see from this descriptio, differet patters of -ariables !ill be rele-at to thesedifferet $roups* herefore ')9 impact assessmet !ould focus o differet -ariables forthese differet etrepreeur cate$ories, or !ould $eeralise !ithi but ot bet!ee cate$ories*hese cate$ories are sometimes proEied by icome, or by the inormalormal dichotom!(typically uderstood i terms of !hether eterprise is re$istered or ot"* +ee also 6sselaar etal (200C" cate$orisatio*
7/24/2019 Idrc Ia for Ict4d Compendium
88/167
)ompared to the eterprise performace -ariables listed abo-e, this $i-es a fuller picture ofthe actual impact that ')9 has had o the li-es of idi-idual etrepreeurs ad eterpriseemployees* +ee )ompedium 6try o =i-elihoods rame!ork for further details*
9*/nterprise #i"eyle *ategorisation* he impact assessmet approach described abo-ehas bee blid to the particular lifecycle sta$e of the eterprise* Ho!e-er, the impact of')9 may be differet at differet sta$es* ypically, the sta$es are di-ided ito three%
Birth% focusi$ o the impact of ')9 o the creatio of e! eterprises* o the
-ariables listed abo-e, oe !ould at least add assessmet of ')9 impact o umbers of
e! eterprises formed* +;istence% focusi$ o the impact of ')9 o the $ro!th of eEisti$ eterprises* o the
-ariables listed abo-e, oe !ould at least add assessmet of ')9 impact o cha$e ieterprise performace -ariables o-er time*
Death% focusi$ o the impact of ')9 o the sur-i-al of eEisti$ eterprises* o the
-ariables listed abo-e, oe !ould at least add assessmet of ')9 impact o lo$e-ityof eterprises*
E=istence
6,ro0t/7
+irt/
6Creation7
Deat/
6&urvival7
7/24/2019 Idrc Ia for Ict4d Compendium
89/167
+irt/ "ates ,ro0t/ &urvival
re-ious maa$erial eEperiece Q Q G
Marketi$ eEperiece G QG G
+eE QG G G
Membership of immi$rat or othermar$ialised $roup
QG G G
raii$ QG QG G
)ultural factors PG PG G
A$e 0 0G G
Enterprise 'actors
+oud eterprise fiaci$ G Q Q
roduct ad customer ra$e G 0G Q
ositioi$ ad io-atio G Q Q
6terprise $ro!th *a* *a* Q Q
7/24/2019 Idrc Ia for Ict4d Compendium
90/167
6terprise soud fiace, positioi$ ad io-atio, busiess plai$, youth, maa$er
recruitmet, multiple fouders
+ome $o-ermet policies
-ur#i#al>
he eEistece of market demad ad the perceptio of $reater tha curret eari$s
Miimisatio of barriers to fiace, skillslabour, techolo$y, iformatio, other iputs
6terprise soud fiace, breadth of productcustomer ra$e, positioi$ ad io-atio,
$ro!th, siOe
/*Setoral *ompetitive Advantage* Most ')9 impact assessmet !ill focus o a fe!micro.eterprises i oe locatio* A alterati-e !ould be to focus o oe sector !ithi acoutry (e*$* the teEtile sector or the food processi$ sector"* orter1s (>>0" diamod ofcompetiti-e ad-ata$e determiats could the be applied, looki$ at !ays i !hich ')9had affected those determiats ad, i tur, had affected measures of sectoral competiti-ead-ata$e* +uch measures are $eerally cofied to sectors that are si$ificat eEporters*+ee Heeks (200B" for further details of applyi$ this model*
actor)oditios
irm +trate$y,+tructure ad
Ri-alry
emad)oditios
Measures of)ompetiti-eAd-ata$e%
7/24/2019 Idrc Ia for Ict4d Compendium
91/167
5au$ha, =*J* & a$ue.+utcliffe, :* (>>C" Measuri$ the impact of iformatio o
de-elopmet% a ='+R6=.based study of small busiesses i +ha$hai,?ournal o theAmerican -ociet! or Inormation -cience, 98(0", >C.>@
Richard Heeks
7/24/2019 Idrc Ia for Ict4d Compendium
92/167
E=amples of se 3/nterprise .ariables
Enterprise !ariables
E=ample %27aranasios 8
-urgess
Comment "eference
ocuses oly o a limited, specific part of the6terprise 5ariables Model ') rocess,
thou$h does $i-e percei-ed impact oeterprise performace* +paces model ismore a simple checklist tha a model*4-erall, does ot offer breadth or depth of$uidace for 'A of 6terprise 5ariables*
;araasios, +* & Refereed Doural article# 4pe Access# 2 pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio use of ')s by 9 tourism
small eterprises (_20 staff" i urba,rural ad semi.rural Malaysia
7/24/2019 Idrc Ia for Ict4d Compendium
93/167
Enterprise !ariables
E=ample 82/sselaar et
al
Comment "eference
rustrati$ presetatio of fidi$s!ith key impacts ot discussed ordiscussed obscurely* As a basic model,thou$h, looks a potetially.iteresti$
!ay of follo!i$ a Fuatitati-eapproach to 'A of ')9 oeterprises* seful cate$orisatio ofeterprise by formality*
6sselaar, +*, +tork, )*, 3di!alaa, A* & ee.+!array, M* (200C" ') usa$e ad its impact o profitabilityof +M6s i @ Africa coutries,Inormation Technologies and International De#elopment, 9(", 8C.00http%!!!*mitpressDourals*or$doipdf0*B2itid*200C*9**8C'mpact assessmet report# 4pe Access# 9 pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio ')s i o-er @,/00
small ad medium eterprises (_/0staff" i urba locatios i @ Africacoutries
'mpact =e-el idi-idual eterprises
Research Resource 'depedet
research team# uclear time spa
rimary Juestioaire sur-ey
+ecodary clear
4ther )ross.sectioal#
Juatitati-e# 3ot participatory
' terms of the 6terprise 5ariables Model, this focuses o the impact of ')9 o%
') rocess% based o three rati$ scale idices* a" 4!ership (of phoes, faE, 4 boE, ), 'teret"#
b" se (of each ') type either to commuicate !ith cliets or to order supplies"# ad c" sa$e itesity(calculated from di-idi$ use ideE by o!ership ideE"
rocess (labour producti-ity"
6terprise erformace (profitability# sales turo-er"
ses a three.!ay eterprise cate$orisatio based o re$istratio ad record.keepi$% iformal# semi.
formal, formal*
Dept/ of Met/od ,uidance Causal Lin< to ICT*D 'indin1s on ICT*D Impact
!o pa$es o method* Appears to assume causal liks toeterprise performace that are otappropriate to statistical associatiospreseted*
oes ot pro-ide direct data o o!ership ad use (i*e* !hat T of +M6s ha-euse a mobile or ha-euse a'teret coectio"* oes ot pro-ide data o relatio bet!ee ')s ad profits* idi$s that arepreseted%
Mobiles are by far most !idely.used ')*
4!ership ad use of all ')s icreases from iformal to semi.formal to formal, but usa$e itesity is
the opposite*
') usa$e is positi-ely correlated !ith hi$her labour producti-ity,* ') eEpediture is positi-ely
correlated !ith hi$her sales turo-er* )oclusios appear to make u!arrated leap from correlatio tocausatio*
;ey obstacle to further diffusio is hi$h cost of ')s proportioately hi$her for iformal tha formal
eterprises*
+aseline:Counterfactual !alue C/ain &ta1e6s7
3o baselie* )outerfactual co-ered bydifferi$ le-els of ') o!ership aduse*
Mai focus o 4utputs (') processidices ad producti-ity" ad4utcomes (turo-er"*
8>
http://www.mitpressjournals.org/doi/pdf/10.1162/itid.2007.4.1.87http://www.mitpressjournals.org/doi/pdf/10.1162/itid.2007.4.1.877/24/2019 Idrc Ia for Ict4d Compendium
94/167
ICT*D Impact Assessment 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
95/167
)otet% of the iteractio or ifluece, icludi$ ay material or immaterial items
trasacted
)oteEt% the broader ature of social ad po!er relatios boud up !ith iteractio or
ifluece icludi$ its eFuality or ieFuality, ad the syer$y or coflict of iterestsrepreseted*
Anal4sis
Strengths
articularly !ell.attued to the commuicatio ad trasactioal capacities of ') (as
opposed to its data processi$ capacities"*
Weaknesses
4f itself says relati-ely little about the impact of ')9 o the performace of the
eterprise*
Home.based, iformal eterprises ca be hard to idetify ad iclude*
As su$$ested by oer1s (200C" research, as yet, ')s e-e mobiles may be ha-i$
relati-ely little impact o the relatioships of the maDority of urba micro.eterprise
Met/odolo1ical &ummar4
/nterprise Relations
$rimar!-econdar!9 rimaryReFuired
he ature, cotet ad coteEt of iteractiosad ifluece are rarely recorded i secodarysources
7/24/2019 Idrc Ia for Ict4d Compendium
96/167
idi-idual etrepreeurs
Generalisa3ilit! &rom
"ne $ro%ect9
=imited less proDects ecompass a lar$e umber ofeterprises
Compara3ilit! Across$ro%ects9
air +o lo$ as the same stakeholders ad samerelatio defiitios are used
Met/od "ecommendations
Make use of call lo$s ad other forms of recordi$ the ') compoet of relatios*
+eek to compare ').eabled -s* o.').eabled relatioship for the same eterprise*
)osider ho! home.based ad other iformal eterprises !ill be idetified ad icluded*
or more formalised eterprises, cosider !hether oe of the competiti-eess -ariats
(see belo!" !ould offer $reater isi$hts*
7/24/2019 Idrc Ia for Ict4d Compendium
97/167
!ariants
* *ompetitive Strategy* his focuses solely o the impact of ')9 o the relatios
bet!ee the eterprise ad its competitors* oes the ')9 iter-etio make the eterprisemore or less able to competeG orter (>8/" idetifies three mai competiti-e strate$ies%
=o! cost% does ')9 help the eterprise reduce its costs (ad, hece, the price of its
products or ser-ices"G
ifferetiatio% does ')9 help the eterprise produce some uiFue -alue that
competitors do otG
3iche% does ')9 help the eterprise more successfully focus o ad meet the particular
eeds of a arro! market se$metG
+ome care eeds to be take, thou$h, i applyi$ this -ery 3orther eterprise model e*$* tode-elopi$ coutry micro.eterprises*
2* *ompetitive Fores* his pro-ides a busiess.orieted checklist for assessi$ the cha$esi relatios !ith ecoomic istitutios that ')9 may bri$ about* 't particularly looks at!hether or ot ')9 alters the competiti-eess of a eterprise by cha$i$ the status of,ad relatios !ith other stakeholders* or further details, see orter (>C>"*
otetial6trats
Threat o ,e'
+ntrants.
7/24/2019 Idrc Ia for Ict4d Compendium
98/167
E=amples of se 3/nterprise Relations
Enterprise "elations
E=ample %2'onner
Comment "eference
ocuses oly o relatios !ith customers, otother stakeholders* ro-ides four dimesios
for assessi$ relatios* oes ot directlyfocus o ho! ')s ha-e cha$ed relatios,thou$h idirect e-idece su$$ests they ha-eot*
oer, :* (200C" )ustomer acFuisitio amo$ small ad iformal busiesses i urba 'dia,+lectronic ?ournal o Inormation -!stems in De#eloping Countries, @2(@", .B
http%!!!*eDisdc*or$oDs2ideE*phpeDisdcarticle-ie!ile9B92@2Refereed Doural article# 4pe Access# B pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio ') use by micro.
eterprises i Hyderabad, 'dia
'mpact =e-el idi-idual
etrepreeureterprise
Research Resource se-eral paid field
researchers, uclear for ho! lo$
rimary iter-ie! sur-ey of Dust o-er @00
etrepreeurs o use of ')s for customercommuicatio
+ecodary 3ot stated
4ther )ross.sectioal# Maily Fuatitati-e
e*$* relatioal -ariables !ere rated o
scales# 3ot participatory
3o eEplicit frame!ork but dra!s o ideas about the importace of social capital ad trust ide-elopi$ coutry microeterprise busiess relatios, ad the role of ') (esp* mobile phoes"i mediati$ relatios i urba e-iromets*
ocuses o oly oe stakeholder $roup customers* =ooks at relatios !ith those stakeholdersi terms of%
+iOe (i*e* umber of relatios"
MultipleEity (e*$* combii$ busiess ad social elemets i a si$le relatio"
amiliarity (i*e* of customer prior to iteractio" ?eo$raphic distace
)otrols for both eterprise locatio (urbasuburba" ad icome (see as a proEy forsur-i-alist, trudler, flyer cate$orisatio see 6terprise 5ariables etry"*
Dept/ of Met/od ,uidance Causal Lin< to ICT*D 'indin1s on ICT*D Impact
About oe pa$e o methods* 3oistrumets but the results $i-e a $oodsese of the Fuestios asked*
3o clear impact of ')9 study did ot askdirectly !hat ')s !ere actually used for*
irst cotact !ith customers !as e-er -ia ')s most !ere !alk.is# the remaider referralsor direct approach by o!er* >CT of customer cotact !as face.to.face# remaider by phoe*or all customers, oly @0T are ko! -ery !ell ad B2T represet a busiess.olyrelatioship*
7/24/2019 Idrc Ia for Ict4d Compendium
99/167
ICT*D Impact Assessment 'rame0orb"*
he $eeric -alue chai model cosists of t!o parts (de-eloped from =audo & =audo 200Cafter Michael orter1s ori$ial"%
/nterprise value hain% related to a idi-idual eterprise i !hich the impact of ')9
is bei$ assessed*
Setoral value hain(or supply chai"% relati$ to the chai of eterprises that supply to,collaborate !ith, ad purchase from, the focal eterprise*
Admiistratio ad Maa$emet(e*$* iace, lai$, )ommuicatio"
Huma Resource Maa$emet
echolo$y Maa$emet
-upport
Acti#ities
7/24/2019 Idrc Ia for Ict4d Compendium
100/167
he eterprise -alue chai is basically a checklist of areas to i-esti$ate for impact of ')9%
$rimar!% here are primary acti-ities that directly create -alue for the eterprise% Has') affected the !ay materials ad other items purchased by the eterprise are recei-edad stored (iboud lo$istics"G [ Has ') affected the !ay marketi$ ad sales areudertakeG [ Has ') affected after.sales ser-iceG
-upport% here are the support acti-ities that eable the primary acti-ities to take place%
Has ') affected the !ay that fiace is maa$edG Has ') affected the !ay plas aremadeG [Has ') affected the !ays purchases are madeG
o mo-e beyod mere descriptio, some aalytical frame !ould eed to be added* or
eEample, Fuatitati-ely, this could look at the fiacial ad time costs of each -alue chaiprocess* Jualitati-ely, this could look at the Fuality of particular processes*
he sectoral -alue chai is similarly Dust a descripti-e checklist i its ra! form, to !hichsome aalytical frame must be added* A$ai, Fuatitati-ely, oe ca look at the fiacial adtime trasactio costs of -alue chai iteractios* Jualitati-ely, oe could use a approachsimilar to the 6terprise Relatios Model%
$rocess% has ') affected the supply chai processG '*e* the !ay i !hich materials,
ser-ices, iformatio, moey, etc are passed from oe stakeholder to the eEt* Content% has ') affected !hat is eEcha$ed bet!ee the stakeholdersG
*elations% has ') affected the ature of relatios !ith other stakeholdersG '*e* ot Dust
the !ay they iteract ad freFuecy of iteractio, but also the social ad po!er relatiosboud up !ith their iteractio*
' particular, there !ould be a focus o ho! ')9 impacts the supply of iputs ito theeterprise (labour, ra! materials, eFuipmet, moey, iformatio", ad the ability to accesscustomer markets for eterprise outputs* hese are the key elemets i the sustaiability of
7/24/2019 Idrc Ia for Ict4d Compendium
101/167
Met/odolo1ical &ummar4
/nterprise .alue *hain
$rimar!-econdar!9 rimaryReFuired
' order to uderstad the specific impact of '),!hich !ill almost al!ays be udocumeted
DataGathering
Methods9
'ter-ie!slus
'ter-ie!s ormally reFuired to $et depth ofuderstadi$ of impact o -alue chai elemets*May supplemet !ith other methods such asobser-atio or diary*
$articipator!9 3o ypically the idea of the -alue chai is imposedtop.do! as a frame!ork for assessmet
:uasi+;perimental9 Rarely eds to take a more case study approach:uantitati#e:ualitati#e
9
7/24/2019 Idrc Ia for Ict4d Compendium
102/167
!ariants
*/nterprise $roess .ariables* he -alue chai ca be combied !ith the 6terprise
5ariables Model, usi$ the process -ariables to create a aalytical matriE checklist ofpossible effects of ')9* or eEample, Dust for the eterprise -alue chai%
$rodutivit
y
*ost 9in,
transation
osts:
uality
9in,
time:
'ependabilit
y
Fle0ibility *apabilities
9in,
tehnologia
l
apabilities:
Admin H
Mgmt=* Mgmt
Tech Mgmt
$rocuremen
t
In 0ogistics
"perations
Mr(ting H
-ales
"ut
0ogistics
-er#ice
7/24/2019 Idrc Ia for Ict4d Compendium
103/167
E=amples of se 3/nterprise .alue *hain
Enterprise !alue C/ain
E=ample %2Annamalai 8
Rao
Comment "eference
Appears a little rose.tited i places but, basedaroud the -alue chai, pro-ides a systematic
aalysis of impact of ') o a$ro.eterprisetrade* A useful model*
Aamalai, ;* & Rao, +* (200@"ITCs +Choupal and $roita3le *ural Transormation, 7orldResources 'stitute, 7ashi$to, ) http%!!!*eEtbillio*etfilese)houpal*pdf
'mpact assessmet report# 4pe Access# @@ pa$es
'ocus and Level Met/od 'indin1s on ICT*D Impact
Applicatio a et!ork of 'teret.
liked )s set up by pri-ate firm ') irural farmi$ commuities i 'dia
'mpact =e-el idi-idual users
(farmers" ad sector (-alue chai"
Research Resource 3ot stated
rimary clear but did i-ol-e some
field iter-ie!s
+ecodary +ome use of other e.)houpal
articles
4ther Juasi.lo$itudial (charts before
ad after ') situatio"# MiE of Fuatitati-e(e*$* trasactio costs" ad Fualitati-e (e*$*ature of -alue chai processes"# 3otparticipatory
5alue chai before%
5alue chai after%
armers beefit from more accurate !ei$hi$, faster processi$ time, ad prompt paymet, adfrom access to a !ide ra$e of iformatio, icludi$ accurate market price ko!led$e, admarket treds, !hich help them decide !he, !here, ad at !hat price to sell* armers usi$the system $et a a-era$e 2*/T hi$her icome (thou$h uclear if this is due to hi$her price orlo!er trasactio costs"* he total beefit to farmers icludes lo!er prices for iputs ad other$oods, hi$her yields, ad a sese of empo!ermet*
') also sa-es o commissio ad trasport costs, ad $ets a more direct lik to farmers* ')also sells more iputs ad $oods to farmers -ia the system*
' systems are also used for accessi$ other iformatio, for email, for studet use*
=osi$ out from the e! -alue chai model are commissio a$ets, labourers at the o.').eabled $o-ermet markets, ad shops ear those markets* 3o !ome ha-e access to thesystem*
Dept/ of Met/od ,uidance 'rame0or< Application
Absolutely oe* he primary acti-ities -alue chai is a core ofthe frame!ork for uderstadi$ ho! ')9has affected sales from a$ro.eterprises*Althou$h ot eEplicit, the aalytical checklistthe used for assessi$ ')9.eabled cha$eto the -alue chai processes is%
iacial cost
ime cost
Juality (e*$* accuracy of !ei$hi$"
epedability (e*$* ability of farmers to
trust the process -ia its trasparecy"here is also cosideratio of empo!ermet-ia the iformatio produced ad choicesfacilitated*
+aseline:Counterfactual
>
http://www.nextbillion.net/files/eChoupal.pdfhttp://www.nextbillion.net/files/eChoupal.pdf7/24/2019 Idrc Ia for Ict4d Compendium
104/167
ICT*D Impact Assessment 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
105/167
(Ramilo & )ico 200/"* ?6M is summarised as follo!i$ se-e steps, as illustrated i thefi$ure*
' some !ays, ?6M is similar to other o-erarchi$ ')9 'A frame!orks* hus, foreEample, it does ot prescribe !hat criteria to assess or ho! to measure those criteria but,istead, pro-ides $uidace o ho! to choose idicators ad ho! to choose appropriatemethods for measuremet* 7hat it does do, thou$h, is a" pro-ide a cotiuous remider toiclude $eder.rele-at elemets at each sta$e of impact assessmet implemetatio# b"
pro-ide eEamples ad real.!orld illustratios of $eder.rele-at elemets# ad c" seek tofollo! its particular priciples e*$* i ad-ocati$ a participati-e approach*
oudatioal cocepts that shape a $edered approach iclude%
riple role of !ome reproducti-e (childbeari$ ad domestic", producti-e (icome.
$eerati$ !ork", ad commuity (maiteace of collecti-e resources"* ')s mayimpact ay or all of these*
l l l f 1 i t t ti l d (f lfilli i ti t i l l i l di
* efie
'teded sead 'tededsers
2* 'detify
?eder ad ')'ssues
@* ialise
6-aluatioJuestios
9* +et ?eder
ad ')'dicators
/* +elect ata.?atheri$Methodsools
B* Aalyse atafrom a ?edererspecti-e
C*'corporate=eari$ ito7ork
7/24/2019 Idrc Ia for Ict4d Compendium
106/167
Anal4sis
Strengths
' ?6M, pro-ides a clear o-erall frame!ork that has bee de-eloped throu$h practice here are clear $roups, !orkspaces ad idi-iduals !ho ca pro-ide ad-ice ad support
i aalysi$ ')9 proDects from a $eder perspecti-e (e*$*http%!!!*apc!ome*or$$emGFPforum2/"
A ready direct audiececostituecy for such 'A !ork but also some recepti-ity ad
uderstadi$ of the approach !ithi de-elopmet a$ecies
se of ?6M ca, of itself, lead to $eder de-elopmetal impacts (see
7/24/2019 Idrc Ia for Ict4d Compendium
107/167
Met/odolo1ical &ummar4
&ender
$rimar!-econdar!9 rimaryReFuired
' order to uderstad the differetial eEperiecesof !ome ad me, their roles, relatios, adcoteEtual iflueces
DataGathering
Methods9
Multiple +tudies use Fuite a !ide -ariety of methods#typically some form of seE.disa$$re$ated lo$ orsur-ey of use plus some more participati-eapproach icludi$ focus $roups!orkshops
$articipator!9 ypical At least for those e-aluatios !hich are follo!i$
a ?6M.type methodolo$y* articipatio mayiclude ot Dust e-aluatio desi$ adimplemetatio but also data e-aluatio
:uasi+;perimental9 Rarely artly due to emphasis o participati-e adFualitati-e methodolo$y
:uantitati#e:ualitati#e
9
7/24/2019 Idrc Ia for Ict4d Compendium
108/167
recommeds a feasibility sca first prior to $oi$ do! this route* Ramilo (200@" otesthe eed for pre.sesitisatio traii$ i most situatios*
'detify !hether applicatio of ?6M is reFuired, or Dust basic $eder.sesiti-ity, !ith
seE.disa$$re$atio of key data ad i-esti$atio of some deeper foudatios for $ederdifferetiatio*
4-erall, some form of $eder les is a critical part of ay ')9 impact assessmet*
"eferences
"#er#ie' *eport,
7/24/2019 Idrc Ia for Ict4d Compendium
109/167
!ariants
*$roess as 4utome* 3ot so much a -ariat as taki$ seriously oe lesso of may
$eder.sesiti-e impact assessmets% that the implemetatio process stro$ly shapes theoutcome* or eEample, a $eder.blid implemetatio is ulikely to produce $eder.positi-eoutcomes* hus oe ca see oe elemet of a full impact assessmet bei$ to assess theimplemetatio process i $eder terms*
4e eEample is +!amy (200C", looki$ at a rural ')9 proDect* his focuses o the(positi-e" $eder implicatios of this proDect1s implemetatio process%
)ollecti-e o!ership of proDect space by !ome
)ollecti-e participatio of !ome i proDect implemetatio )ollecti-e rather tha idi-idual empo!ermet throu$h $roup ')9.related acti-ities
Allo!i$ for eEteral iformatio iputs ad proDect ideas, but rooti$ ')9 -alue i
$oals defied by the !ome
Allo!i$ !ome1s $roup to appropriate the techolo$y to aid sustaiability ad
istitutioalisatiohis focus $i-es a stro$ sese of factors that !ill determie lo$.term $eder impacts,thou$h it does eed to be set alo$side assessmet of the more direct impacts of ')9*
2* Women>B"* uller details of the measures used are pro-ided i the article but, i
brief, the ei$ht scaled items are%
Mobility% local facilities !ome had e-er -isited*
6coomic security% o!ership of assets ad sa-i$s
Ability to make small purchases% cooki$ oilspices, persoal items, !ith!ithout ma1s
permissio ad moey
7/24/2019 Idrc Ia for Ict4d Compendium
110/167
E=amples of se 3 &ender
,ender E=ample %2
Ramilo
Comment "eference
A eEample of usi$ ?6M to assess impact ofa telecetre* ro-ides some detail o methodsused but does ot offer a $uide to ho! ?6M!as actually applied i practice* =ittle deptho actual impact*
Ramilo, )*?* (ed*" (200@" Gender +#aluation Methodolog! or Internet and ICTs, A)73+,=odo http% !!!*apc*or$e$lishcapacitypolicymmtkL$ederLictpolL$emLpublicatio*pdf?uidace report# 4pe Access# C pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio summarises siE case
studies but mai focus of this summaryis t!o telecetres i rural areas of thehilippies
'mpact =e-el idi-idual recipiets
(ad small metio of commuity$roups"
Research Resource small ?6M team for
t!o days but proDect staff ad users !eremai e-aluators
rimary +eE.disa$$re$ated lo$s of
telecetre use# focus $roups ad iformaliter-ie!s o ho! telecetre used, ad tocreate persoal stories# reflecti-e Douralsfor proDect staff
+ecodary 3ot stated
4ther =o$itudial# MiE of Fuatitati-e(e*$* use lo$s" ad Fualitati-e (e*$* stories"#articipatory (ic* data e-aluatio"
ses the ?6M approach to e-aluate thou$h i a much.simplified maer* Relati-ely fe! detailso actual applicatio* ' terms of ?6M priciples, it is $eder.sesiti-e, adopts a participatoryapproach, ad roots i real eEperieces* oes ot particularly focus o coteEt, or reco$ise theo.eutrality of e-aluatio*
oes ot eEplicitly cosider !ome1s triple role (thou$h stories do so implicitly", or the impacto precursors, iputs, implemetatio, a-ailability ad use*
akes e-aluatio up to about =e-el 2*
Dept/ of Met/od ,uidance Causal Lin< to ICT*D 'indin1s on ICT*D Impact
4e pa$e of methods ad o istrumetspro-ided*
3ot discussed thou$h implicit direct causalrelatio assumed bet!ee ')9 adoutcomes*
7/24/2019 Idrc Ia for Ict4d Compendium
111/167
,ender E=ample 82
Ha"kin
Comment "eference
A pre.?6M $eder aalysis ofifoe- proDects* 3o method detailsad Dust limited $uidace o Fuestiosasked ad impacts see*
Hafki, 3*:* (2002" Are ')s $eder eutralG A $eder aalysis of siE case studies of multi.door ')proDects` paper preseted for ,I,-T*A7 Virtual -eminar -eries on Gender and ICT, .2 :ulyhttp%!!!*u.istra!*or$edocs$ederLadLictHafki*pdf'mpact assessmet report# 4pe Access# C pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio siE ifoe- proDects
co-eri$ i all# this summary focuseso t!o rural ') proDects i eru(a$ric* iformatio system" ad 'dia(health !orker use of As"
'mpact =e-el idi-idual recipiets,
ad !ome as a $roup
Research Resource 4e researcher
for each proDect
rimary roDect staff ad user
iter-ie!s
+ecodary ypical absece of user
lo$s or seE.disa$$re$ated records
4ther )ross.sectioal# Jualitati-e#
3ot participatory
alks oly of -ie!i$ proDects throu$h a $eder les !ith fi-e focal areas%
'detifyi$ $eder issues that affected implemetatio ad results
Ho! the proDect outputs affected !ome1s situatio
'f ad ho! !ome beefited from the proDect
Ho! !ome could ha-e beefited more from the proDect
7hat lessos could be leared by other proDects about the i-ol-emet of, ad beefits to, !ome
' terms of ?6M priciples, it is $eder.sesiti-e, ad roots i real eEperieces* 't has a smallcosideratio of coteEt* At least as preseted here, it does ot appear to be participatory, or to reco$isethe o.eutrality of e-aluatio*
oes ot eEplicitly cosider !ome1s triple role or the impact o precursors, iputs, implemetatio,
a-ailability ad use (thou$h does cosider the impact of $eder o some of these"*
akes e-aluatio up to about =e-el .plus focuses o impact o !ome, ot comparati-e impact o!ome ad me, but does touch sli$htly o deeper $eder issues*
Dept/ of Met/od ,uidance Causal Lin< to ICT*D 'indin1s on ICT*D Impact
3othi$ stated other tha field studies* Reco$itio that ') plus broadercoteEtual factors determie impacts*
roDect schedules ad $eder.blidess (e*$* Doit.seE meeti$s ad traii$" disad-ata$ed !ome, adme !ill typically beefit more tha !ome from ')s uless specific steps are take*+ome $ai i skills ad leadership roles for !ome i eru* +ome $ai of ko!led$e, self.esteem adstatus for !ome i 'dia*+aseline:Counterfactual !alue C/ain &ta1e6s7
3o formal baselie, thou$h doescosider the $ais made from thatbaselie* 3o couterfactual*
Mai focus o 'mplemetatio ad-ery limited cosideratio of 'mpact*
0C
http://www.un-instraw.org/en/docs/gender_and_ict/Hafkin.pdfhttp://www.un-instraw.org/en/docs/gender_and_ict/Hafkin.pdf7/24/2019 Idrc Ia for Ict4d Compendium
112/167
,ender E=ample ;2
Rihardson et al
Comment "eference
+ho!s the ability to uco-er $eder.rele-atimpacts by aski$ some fairly simpleFuestios (i*e* !ithout usi$ a specific $ederframe!ork but by bei$ $eder.sesiti-e"*
Richardso, *, RamireO, R* & HuF, M* (2000" Grameen Telecom@s Village $hone $rogramme in *uralBangladesh> a MultiMedia Case -tud! , ele)ommos e-elopmet ?roup, ?uelph, 43http%!!!*telecommos*com-illa$ephoefialreport*pdf'mpact assessmet report# 4pe Access# 09 pa$es
'ocus and Level Met/od 'rame0or< Application
Applicatio 5illa$e ay hoe
proDect i >"
4ther )ross.sectioal# Juatitati-e (e*$*
usa$e rates, cosumer surplus" adJualitati-e (e*$* creatio of phoe
culture"# 3o.participatory
3o eEplicit $eder frame!ork for aalysis but focuses o differeces i usa$e patters see !ith male-s* female phoe operators# ad specific impacts o actios, status ad empo!ermet of !ome*
' terms of ?6M priciples, it is $eder.sesiti-e, roots i real eEperieces, ad there is si$ificatcosideratio of coteEt* 't does ot appear to be participatory, or to reco$ise the o.eutrality ofe-aluatio*
oes ot eEplicitly cosider !ome1s triple role* +ome cosideratio of impact o precursors (culture",a-ailability ad use*
akes e-aluatio up to about =e-el 2.plus limited differetiatio of impacts o !ome -s* me icosumer surplus terms# some cosideratio of $eder roles ad resources*
Dept/ of Met/od ,uidance Causal Lin< to ICT*D 'indin1s on ICT*D Impact
our pa$es of detail o researchmethods* !o pa$es o ho!cosumer surplus !as calculated*ull list of all data items (i*e*Fuestios" for sur-ey*
irect Fuatitati-e lik see bet!ee ')(phoe" ad creatio of cosumer surplus# likassumed to other outcomes status, culture,icome*
sa$e fidi$s% he $eder of the 5illa$e hoe operator ad the physical placemet of the phoe!ithi a $edered -illa$e coteEt ca either ihibit or impro-e !omes access to phoes* [ me tedto use telephoes o!ed by male operators WBT of ?ramee
7/24/2019 Idrc Ia for Ict4d Compendium
113/167
ICT*D Impact Assessment 'rame0or
7/24/2019 Idrc Ia for Ict4d Compendium
114/167
!eleentre *ategorisation%elecetres eEist i -arious forms, shapes ad ames*)ate$orisatio of the telecetre(s" uder assessmet ca help idetify particular coteEtualad other -ariables of importace to impact assessmet idicators* +ee 5ariat for more
details o implicatios of cate$orisatio by sophisticatio*
Criterion Cate1ories
0in(age 'ith
de#elopment polic!
'solated, ali$ed, ite$rated
"'nership ublic($o-ermet", commuity (Doit", ci-il society, pri-ate
-tartup und ublic, pri-ate, commuity, door, public.pri-ate, door.pri-ate,public.pri-ate.door
Business model 3o.char$e, miimal fee, cost.reco-ery, profit.orieted, social.
eterprise-ophistication
7/24/2019 Idrc Ia for Ict4d Compendium
115/167
+ecod, i terms of key reFuiremets of sustaiable ')9 proDects (Heeks 200/"%
Capacit!% does the telecetre ha-e a-ailable the ecessary resources o a o$oi$ basis#
these iclude moey, skills, data ad techolo$yG his makes a proDect usable* tilit!% !ill the telecetre keep meeti$ the eeds of at least some stakeholdersG 't must
cotiue to be useful to someoe, for this is !hat makes a proDect used*
+m3edding% for lo$.term sustaiability, !ill the telecetre become istitutioalised
embedded i the rules ad orms, culture ad -alues of its setti$G his makes a proDectused as a matter of routie*
4e key Fuestio rarely made eEplicit i cosideri$ telecetre sustaiability is !hether
iformatio should be re$arded as a pri-ate or public $ood* he as!er !ill lead to -erydifferet -ie!s o !hether or ot telecetres should be subsidised*
-, Framework(-ased !eleentre Impat Assessment
Rather tha approachi$ matters i terms of idicators, this approaches impact assessmet iterms of frame!orks, dri-e by the particular obDecti-es of assessmet* hose obDecti-es ca
be related to the ')9 -alue chai* May of the models are co-ered i other etries i the
)ompedium* +pecific eEamples that start from this perspecti-e iclude Molla & Al.:a$houb(200C", based o the li-elihoods frame!ork, ad ;helladi (200, based o the cost.beefitaalysis frame!ork*
ptake ad4utput
4utput 4utcome 'mpact
')09(5alue
)hai
7/24/2019 Idrc Ia for Ict4d Compendium
116/167
Weaknesses
rame!orks su$$ested for telecetre impact assessmet ted to be $eeric, ad focus o
the ratioale, plai$ ad process of coducti$ the assessmet rather tha the cotet of
assessmet* hus, i may !ays, this approach still lea-es ope the Fuestio of !hat,actually, is to be doe*
e-elopi$ ad a$reei$ idicators ca be -ery time cosumi$* ' additio dra!i$ the
causal relatioship bet!ee telecetre use ad its impact is compleE ad frau$ht !ith aumber of eEo$eous iflueces*
7/24/2019 Idrc Ia for Ict4d Compendium
117/167
Met/od "ecommendations
?uide idicator selectio o the basis of pro$rammeassessmet $oals, -alue chai sta$e
ad time horiOo*
' $eeral, udertake telecetre assessmet later rather tha sooer* 6sure the eeds ad -ie!s of key stakeholders are icorporated ito both desi$ ad
data.$atheri$*
articularly if plai$ assessmet of multiple telecetres, use telecetre cate$orisatio as
a $uide (see abo-e ad also 5ariat "*
6stablish early o !hether a more participatory, leari$ approach to telecetre 'A is
appropriate (see 5ariat 2"*
7/24/2019 Idrc Ia for Ict4d Compendium
118/167
pro-ided# acility maa$emeto!ership# iacial sustaiability# Rele-ace adaccessibility* oes ot pro-ide frame!ork or idicators, ad barely a setece o method*
6tta, *6* & ar-y.7amahiu, +* (200@" The +;perience 'ith Communit! Telecentres,
'R), 4tta!a ocuses o issues of telecetre a-ailability (resources ad ser-ices", uptake (patters ofuse, use(r" ad o use(r" profiles, user satisfactio" ad outcome (fiacial sustaiability"*
Harris, R*7* (>>>" 6-aluati$ telecetres !ithi atioal policies for ')s i de-elopi$
coutries* '% Telecentre +#aluation, R* ?omeO & * Hut (eds", 'R), 4tta!a, @.@8http%!!!*idrc*cauploadsuser.+02992989@0arhills*pdf )oceptualises the role of telecetres i the coteEt of atioal ') diffusio adoffers iput ad output idicators oly* 'put idicators iclude resources
(accommodatio, eFuipmet, ad people" ad ser-ices* 4utput idicators focus ocommuity.based idicators (such as socio.ecoometrics ad stories" ad sustaiability(o!ership, fiace ad replicability"*
=e$yel, ?*, 6rausO, 6*, leki, *, =ricO, =* & +ikls, 5* (200B" he )ser]fa
eEperimet% o the attempt to deploy computers ad 'teret i a small Hu$aria -illa$e,?ournal o Communit! Inormatics, 2(@" http%ci.Doural*etideE*phpcieDarticle-ie!2>B2B Relati-ely uiFue i focusi$ o telecetre impact o idi-idual1s li-es, both positi-es
(e*$* icrease i ko!led$e, skills ad aspiratios" ad e$ati-es (e-y ad frustratio"* Miller, 3*=* (2009" Measuri$ the cotributio of 'foplaOas to 'teret peetratio ad
use i aama,Inormation Technologies and International De#elopment, 2(2", .2@http%!!!*mitpressDourals*or$doipdf0*B2/99C/2099>@99@ ocuses more o precursors, adoptio ad use ad less o impact* 4ffers a oe pa$edescriptio of methods but does iclude sur-ey Fuestioaire*
Rotheber$.Aalami, :* & al, :* (200/"*ural Telecentre Impact Assessments and the
$olitical +conom! o ICT or De#elopment 5ICT4D6,
7/24/2019 Idrc Ia for Ict4d Compendium
119/167
!ariants
*Assessment by !eleentre !ype* As oted abo-e, telecetre typecate$orisatio ca partly
$uide idicator selectio* o $i-e a eEample of this, !e choose sophisticatio of telecetres*
7/24/2019 Idrc Ia for Ict4d Compendium
120/167
E=amples of se 3 !eleentres
Telecentre E=ample %2
7yabwe 8 7ibombo
Comment "eference
ocus almost all o pre.implemetatio issues, ot impact*ro-ides ri$orous approach !ith stro$, clear methodolo$y* oesot