UNIFEM Strategic Plan 2008‐2011 Evaluability Assessment€¦ · 2 This Assessment ... 3.4 The...

UNIFEMStrategicPlan2008‐2011EvaluabilityAssessment

FinalReport

EvaluationOffice

March2011

2

ThisAssessmentwasconductedbyindependentevaluationconsultants

fromIODPARC,andmanagedbytheUNWomenEvaluationOffice.

EvaluationTeam:IODPARCisthetradingnameofInternationalOrganisationDevelopmentLtd//CathyGaynorandJuliaBettsOmegaCourt362CemeteryRoadSheffieldS118FTUKwww.iodparc.comTheviewsexpressedinthispublicationarethoseoftheauthor(s)anddonotnecessarilyrepresenttheviewsofUNWomen,theUnitedNationsoranyofitsaffiliatedorganization

3

Contents

ExecutiveSummary 6

1.Introduction 11

1.1TheStrategicPlan,itsresultsframeworksandassociatedsystems 11

1.2Originalremitofthestudy 11

1.3Re‐orientation 12

2.Approachandmethodologyofthestudyincludingre‐orientation 14

2.1Elementsofthemethodology 14

2.2Samplebase 14

2.3Limitations 15

2.4Currentinstitutionalrelevance 15

2.5Structureandaudience 16

3.Keyfindings 17

3.1TechnicalrobustnessoftheStrategicPlananditsresultsframeworks 17

3.2AccompanyingsystemsandtheirroleinsupportingStrategicPlandelivery

23

3.3Theexperienceofimplementation 27

3.4TheUNIFEMmandateregardingnormative/operationalactivityandUNcoordination

35

3.5Theinformationbasetosupportanyfutureevaluation 38

4.OverallConclusionsandLessonsLearned 42

4.1OverallConclusions 42

4.2LessonsLearned 44

5.TableofRecommendations 46

Annexes(separatedocument)Annex1:TermsofReferenceAnnex2:AnalyticalOverviewAnnex3:TechnicalAnalysisTheoryofChangeAnnex4:TechnicalAnalysisStrategicPlanandResultsFrameworksAnnex5:IndividualsConsultedAnnex6:DocumentsConsulted

4

Acknowledgements

Theevaluatorswouldliketothankallthosewhoparticipatedinthisstudy,includingallrespondents,theglobalReferenceGroup,andinparticular:ShravantiReddy,BelenSanzandLauraGonzalesofEvaluationUnit;thestaffofUNIFEMAndeanandCEEsub‐regionaloffices;andColombiaandAlbaniacountryoffices.

MembersoftheGlobalReferenceGroupforthisstudyincluded:

1. JoanneSandler,DeputyDirectorofProgrammes2. S.K.Guha,InstitutionalDevelopmentAdvisor,IDT3. ElenaMarcelino,ProgrammeSpecialist,IDT4. LinetOtieno,MonitoringSpecialist,AfricaSection5. ZinaMounla,ChiefofCEE/CISSection6. RachelDoreWeeks,CoordinationSpecialist.GenderandPeacebuilding,Governance,PeaceandSecurity7. LeeWaldorf,HumanRightsAdviser8. MitushiDas,DonorRelationsSpecialist9. RobertaClarke,RPDforCaribbeanSub‐regionalOffice10. GitanjaliSingh,NPO,SouthAsiaSub‐regionalOffice11. SagipaJusaeva,ProgrammeSpecialist,CISSub‐regionalOffice12. LucieLuguga,ProgrammeSpecialistandHeadofJubaSub‐office,Sudan

5

Acronyms

CEDAW Convention on the Elimination of All Forms of Discrimination Against Women

CEE Central and Eastern Europe

CIS Commonwealth of Independent States

CO County Office

CS Country Strategy

DAW Department for the Advancement of Women

DRF Development Results Framework

ECOSOC Economic and Social Council

EVAW Ending Violence Against Women

GEWE Gender Equality & Women’s Empowerment

GRB Gender Responsive Budgeting

HRBA Human Rights Based Approach

HQ Headquarters

INSTRAW International Research and Training Institute for the Advancement of Women

IRF Integrated Results Framework

M&E Monitoring and Evaluation

MRF Management Results Framework

MYFF Multi-Year Funding Framework

OSAGI Office of the Special Adviser on Gender Issues and Advancement of Women

PAC Project Approval Committee

PCM Programme/Project Cycle Management

RBM Results Based Management

SP Strategic Plan

SRO Sub Regional Office

SRS Sub Regional Strategy

UNCT United Nations Country Team

UNDAF United Nations Development Assistance Framework

UNDP United Nations Development Programme

UNFPA United Nations Population Fund

UNIFEM The United Nations Development Fund for Women

6

ExecutiveSummary

1. Studybackgroundandapproach

Thisstudy,whichwasconductedfromMay2010toFebruary2011,isananalysisoftheUNIFEMStrategicPlan(2008‐2011),itsassociatedresultsframeworksandinstitutionalsystems.ItassessesUNIFEM’sexperienceofimplementingtheStrategicPlan,withaviewtoinformingthenewstrategicplanningprocessofUNWomen.ItisaimedatallmembersofUNWomenandrelevantpartners,atheadquarters,sub‐regionalandcountrylevels,plusanyinterestedexternalpartners.TheimplementationofResolutionA/RES/64/289inJuly2010,whichmandatedtheformationoftheUN’sneworganizationdedicatedtogenderequalityandtheempowermentofwomen(subsequentlyUNWomen)ledtoadecisiontomodifythestudyandaconsequenttwo‐monthpauseinitsimplementation.Ratherthanpursueaclassicevaluabilityassessment(giventhatevaluationofUNIFEM’sStrategicPlanwasnowunlikely),therevisedpurposeofthestudywas:

AqualitativeanalysisoftheStrategicPlan’sbasicparametersanditsmonitoringandreportingsystems…Aformativeandforwardlookingexerciseaimedatcapturingbestpractices,challengesandlessonslearnedfromtheUNIFEMStrategicPlanexperiencetodateforreflectionandlearning.

Thestudycomprisedarangeofmethods:technicalappraisalofcorporate,thematic,regional,sub‐regionalandcountryStrategicPlandocumentsandresultsframeworks;reviewofawiderangeofinternalandexternaldocumentation,includingworkplansandannualreports;interviewswithUNIFEMstaffandexternalinformants;workshopswithandfeedbackfromaReferenceGroup;programmeandcountrysamplingandselectionleadingtoin‐depthanalysisofselectedsub‐regional(8)andcountry(8)officesandfieldvisitstotwosub‐regionaloffices(AndeanandCentral&EasternEurope)andtwocountryoffices(Colombiaand

Albania)conductedinNovember2010.Despiteexperiencinganumberofchallenges,whicharesetoutinthefullreport,thestudyhasidentifiedsomerelevantandinterestinglessonsontheformulationoftheUNIFEMStrategicPlan,itssystemsandtheexperienceofitsimplementation.Consequently,itprovidesausefulbodyofevidenceonhowthelearningfromUNIFEM’sStrategicPlanexperiencecouldbeusedtosupportanyfutureinstitutionalplanningprocessesofUNWomen.

2. Keyfindingsandconclusions

Thekeyconclusionsofthereportarepresentedbelowinsummaryform,followedbylessonslearnedinsection3andrecommendationsinsection4.Thefullreportalsocontainsfivefindingssections,whichsetoutfromwhichtheretheseconclusionshavebeenderived.

Conclusion1:TheStrategicPlananditstechnicalrobustness

TheUNIFEMStrategicPlanandtheunderstandingofchangeonwhichitisbasedprovidedarelevant,appropriateandconceptuallysoundarticulationofUNIFEM’scoremandateontheachievementofgenderequalityandwomen’sempowerment(GEWE).However,therearesometechnicalweaknesses,whichconstraineditseffectiveimplementation.Theseinclude:

• ThereisinsufficientdevelopmentoftheunderstandingofchangeandresultslogicwithintheStrategicPlanandinadequate

recognitionofpotentiallydifferentpathwaysofchange;

• Thereisverylimitedacknowledgment

withintheStrategicPlanoftheassumptionsandrisksthatunderlietheprocessesofchange,suchasthepolitical

7

ExecutiveSummary

factorsthatdriveorhinderthis;

• Therolesofkeystrategies(e.g.knowledge

management,capacitydevelopment)inlinkingoutcomesandprogressingchangearenotsufficientlyelaboratedwithinthe

StrategicPlan;

• Despitetheconsiderableimprovementfrompreviousyears,specificresultsand

neutralandmeasurableindicators/concretetargetsarenotconsistentlyappliedwithintheStrategic

Planframework,nordoesithaveabaselinetosetastartingpoint;

• WhiletheStrategicPlanplacesemphasisonimplementationatnationallevel,theprimacyofcountrystrategyplanninghasbeenonlyarelativelyrecentdeparturewithinthestrategicplanningprocess.

Conclusion2:AccompanyingsystemsandtheirroleinsupportingStrategicPlandelivery

ThesystemsdevelopedforStrategicPlanimplementationrepresentamajortransformationinthewayUNIFEMsoughttointroducearesultsbasedcultureandtocollectandanalysedataforperformancemonitoring.However,implementationdidnotdelivertheresultsorientedcultureenvisaged.Specifically:• Guidancematerialtosupportsectionsand

officestodeveloplinkedstrategiesisgoodpracticebutcanbeimprovedonintermsoftimelinessandcomprehensiveness;

• Theresultstrackingsystemenabledcomprehensiveresultsreportingatgloballevelbutitsmaingearingtowards(centralised)annualreportingconstraineditspotentialutilityformorelocallyrelevant

performancemanagementandtrendanalysis;

• Therewaslimitedanalysisoftrends/progressanddownwardfeedbackloops,whichconstrainedtheflowanduseofvaluableinformation;

• Therewasaneedtoenablereportingbeyondfixedandsometimesnarrowglobalindicatorssothatofficesandsectionscouldreportonsignificant,widerchanges;

• Considerableprogresswasmadeonresultsmeasurementbutsystemshadnotyetbeengearedtosupportcomprehensiveresultsmanagement(throughoutthefullprogrammecycle).Inparticularmonitoringwasnotwelldeveloped,thoughsignificantprogresshasbeenmadetomovetowardsresults‐focusedevaluation.

Conclusion3:Theexperienceofimplementation:UNIFEM’stake‐upanduseoftheStrategicPlananditssystems

TheStrategicPlan,itsresultsframeworksandassociatedsystemsprovidedaclearorganisingframetomakeexplicitUNIFEM’sworkandstrategicfocustostaffandpartners.Theysupportedcoherenceandconsistencyacrosstheorganisation.However,theirpotentialvaluewasnotbeingfullyrealised(especiallyatsub‐regionalandcountrylevels):• Theunderstandingofhowchangewould

besupportedthroughtheStrategicPlanwasnotbeensufficientlytestedandvalidatedatlocallevels;

• TheStrategicPlanwasprovidingaconceptualumbrellaforthematicandstrategiccoherence,ratherthanactingasastrategicdriverforoperations;

8

ExecutiveSummary

3.LessonsLearned

Thestudyfindsthefollowinglessonslearnedwhichmighthavebroaderrelevancebeyondthisprocess.Moredetailoneachlessoniscontainedinthefullreport:

• Anorganisation‐widestrategicplanresultslogic,whichsetsouttheunderlyingbeliefinhow

• Theresultstrackingsystemwasnotadequatelysupportinglocalreportinganddecision‐making;

• UNIFEM’sstatusintheUN,inadequatestaffingandinsufficientpredictabilityofresourcesconstrainedeffectiveimplementation,andwereoutofsyncwiththeaimsoftheStrategicPlan;

• TheprocessofdevelopmentoftheStrategicPlanandrelatedstrategies(suchascountrystrategies)tookplacerapidly,andwithoutthetimeperiodrequiredforfullengagementanddiscussionwithstaffandpartners.

Conclusion4:TheroleoftheStrategicPlaninsupportingthedeliveryofUNIFEM’sremitaroundnormative/operationalactivityplusUNco‐ordinationprocesses

UndertheStrategicPlanaconsiderablevolumeofworktookplacetoaddressnormativeandoperationallinkagesandtosupportthemainstreamingofGEWEwithinUNcoordinationprocessescentrallyandatfieldlevel.However,thiswasnotbeingconductedtoitsfullpotential:

• WhileUNIFEM’smandatewasgenerallyunderstoodbypartners,theStrategicPlandocumentwasnotgenerallywellknownorcommunicated;

• Thereisaneedforclearerdefinitionoftheroleof‘driver’withinUNcountryteams,afocusonimpactaswellasprocessincoordinationwork,corporatecommitmenttoagreedarrangementssecured(e.g.providinghumanandfinancialresourcestocarryoutanagreedco‐ordinationrole)anddistillationandsharingofgood/promisingpractice;

• WhiletheStrategicPlanfacilitatedstafftomakeexplicittheconnectionsbetween

normativeandoperationalwork,furtherworkisneededtostrengthentheselinkages,includinggenerationofanevidencebaseonwhatisworking/notworking.

Conclusion5:TheroleoftheStrategicPlanindeliveringaninformationbasetosupportanylaterevaluation

ThemeasurestakentoprovideevidenceofStrategicPlanimpacthasenabledthegenerationofamorefocusedandcoherentevidencebaseforfutureevaluationthanwasthecasepreviously.Inparticulartherearenowup‐to‐dateandalignedperformancedata,whichareaggregatedfromcountrytocorporatelevels.However:

• Thelackofclarityonstartingposition(baseline);anabsenceofcleartargets(incountrystrategiesaswellascorporateStrategicPlan)andinadequatemonitoringsystemsandcapacity,severelylimitsrobustandcomprehensiveperformancemeasurement;

• Theevaluationevidencebaseisgraduallydevelopingbutlacks(i)asystematicapproachtogeneratingevaluationinformationaroundareasofstrategicinstitutionalinterest;(ii)feedbackloopsbetweenevaluationreportsandstrategicplanningandoperations;andlongitudinalstudies.

9

ExecutiveSummary

changehappensandthecause‐effectrationaleoftheplan,needstobeclearandarticulatedinappropriatedetail;

• However,itisimportantforacorporatestrategicplantoprovideastrategicframeworkthatgivesdirectiontobutdoesnotconstraincountrylevelflexibilitytorespondtocontext.Thisimpliesabroadframeworkatcorporatelevel,including,e.g.clearoutcomestatementsandtargets,butwhichallowscountriestodemonstratethroughtheirownresultsframeworkshowtheywillcontributetothesechanges;

• Developmentandvalidationofastrategicplanthroughconsultationandacomprehensiveplanningprocesshelpsbuildtechnicalquality,ownershipandinstitutionalbuy‐in.Thisimpliesabroad‐basedparticipatoryprocessinvolvingrelevantstaffacrosstheorganisationand,whichtakesaccountofbottom‐upandtop‐downplanning;

• Astrategicplan,whichincludeshigh‐levelambitions,requiresconcomitantinvestmentinresources–humanandfinancial.Thereneedtobeveryclearandexplicitlinksbetweenastrategicplanandstaffing,professionaldevelopment,andelapsedtimetoconductaparticipatorystrategicplanningprocessandtoensureassimilationacrosstheinstitution;

• Akeymessageistheprimacyofcountry‐levelforchange.Evenwithastrategicplanwhichisclearthatthelocusofchangeisatnationallevel,successfulimplementationatcountrylevelrequiresanumberofconditionsrelatedtostructures,systems(planning,appraisal,monitoring&reporting,qualityassuranceandguidanceandevaluation),capacities,thecontextualisationofthestrategicplanandtheoryofchange,andhumanandfinancialresourcing;

• Theimplementationofastrategicplanneedstobeaccompaniedbyanorganisation‐wideshiftfromresultsmeasurement/trackingtoresultsplanningandmanagement.Thisrequiressignificantinvestmentoftimeandresources.Elementsinclude:clearfeedbackloops;

investmentinsupport/guidance,monitoringandevaluationsystems(includingatthecountrylevel);therecruitmentandcapacitydevelopmentofdedicatedstaffand/orexternalexpertisetosupportthesesystems;andtoallowforreportingongainsbeyondindicators,recognisingtheprocess‐basednatureofchange,thefluidityofcontextspecificity,andtheneedtoaggregateupovertime;

• Astrategicplancanprovidestaffwithavaluabletooltopromotetheorganisation’sremittostrengthennormative‐operationalconnectionsandtobe‘adriverofgenderequality’withintheUN.However,spaceandperformancearedependentonanumberoffactorsincluding:coordinationmechanisms;strategiesforengagingwiththese,andguidanceprovided;andthestatusandcapacitiesofstaffandofofficeswithintheUNsystem;

• Buildinganevidencebasetosupportresults‐basedmanagementandfutureevaluationisdemandingand,realistically,needstohappenincrementallyratherthanallatonce.Keybuildingblocksincludethesettinginplaceofanevaluationpolicyandstrategy;amonitoringandreportingpolicy;lessonsfromexperiencetoguiderefinementofperformancemeasurement;andthenecessaryfeedbackloopstosupportresultsmanagement.Forbaselines,clearinstitutionaldemandandclarityofpurposeandpragmatismareneeded.

4. Recommendations

Basedontheseconclusions,thereportmakesanumberofrecommendationstoUNWomenwiththeaimofhelpingto(i)improveanynewplanasaresults‐basedmanagementtooland(ii)supportanynewstrategicplanningprocessinordertostrengthenboththeplanandinstitutionalbuy‐in.Thestudyalsoprovidesspecificproposalsforhowtherecommendationsmightbeoperationalised,withdueregardforthecurrentinternalcontext,i.e.theprocessofconfirmingtheinstitutionalarrangementsofUNWomen.Thesearecontainedinsection5ofthefullreport.

10

ExecutiveSummary

4.1Summaryrecommendation

Basedonthebodyofevidence,thestudyproposesthefollowingheadlinerecommendationtoUNWomen:

Developacorporatestrategicplanbasedaroundacentralframework,includingcorporateleveloutcomesandtargets,butwhichallowscountriesandsectionstoflexiblyidentifytheircontributionstothis.Countriesandsectionsshouldthendevelopcontextspecificresultsframeworkswithclearstartingpoints(baseline);outcomeandoutputtargets(results);andthepathway(processandmilestones)towardsachievingthis.

4.2Supportingrecommendations

Tohelpimplementthis,thereportmakesthefollowingsupportingrecommendations:

TowardsarobustStrategicPlan

• Buildon,makemoreexplicitandvalidatetheresultslogicofanyfutureStrategicPlanthroughdevelopingfurthertheunderstandingofchange,totakeaccountofmovementovertimeandtoclarifyrelationshipsbetweencorporateandcountrylevelresultslogic(includinganyin‐betweenlevels).

• Furtherstrengthenthegoal,outcomeandoutputstatementsinlinewiththeresultslogic,includingalong‐termgoalthataddressesthevisionofGEWE(toembedthelonger‐termgoalofGEWEintotheStrategicPlanresultsplanningandlogic).

• RequirethedevelopmentofbaselineswithinoneyearofStrategicPlandevelopment.Developmentresultsbaselinesshouldbeestablishedprincipallyatcountrylevelwhileamanagementresultsbaselineneedstobedevelopedcorporatelyandatcountrylevel.

1) Thestrategicplanningprocess

• Recognisethattheprimacyofchangeisatthe

countrylevel–somergebottom‐upwithtop‐downplanningprocesses.TimingandsequencingofplanningwillneedtotakeaccountofinstitutionaldemandsforaUNWomenStrategicPlanwithinashorttimeframe(adraftinApril2011forJunesubmissiontotheExecutiveBoard).

• DevelopacommunicationsstrategytoinformstakeholdersatalllevelsoftheUNWomenmandate,strategicplanningprocessandStrategicPlanincludingfurtherclarificationofthenormativeandoperationallinkagesandroleasa‘driver’ofgendermainstreaminginUNcountryteams.

SystemstosupportimplementationoftheStrategicPlan

• Developandstrengthenthesystemsneededtosupportresultsmanagementthroughthefullprogramme/strategycycle(includingevaluation),keepingthecountryastheprimaryunitofchange–andsetandupholdminimumstandardsforthis.

• Developandrefinetheresultstrackingsystemtosupportlocallevelmonitoringaswellassystematiccentralisedresultsreporting.

• GiventhecomplexityofsocialtransformationrequiredforGEWE,introducecomplementaryapproachestoresultstrackingthattakeaccountofnon‐linearchangeandthepossibilityofunplannedconsequences(positiveandnegative).

IncreasingreadinessofanyfutureStrategicPlantobeevaluated

• RetaintherequirementforanevaluationplanintheStrategicPlan,includingcountrystrategies,butbasethisonstrategicprogrammeinformationneeds,ratherthanbeingdominatedbyprojectordonorrequirementsandrequirecompliance.

11

Introduction

This study,whichwasconductedduring2010, isananalysisoftheUNIFEMStrategicPlan,itsassociatedresults frameworks and institutional systems. Itreflects on UNIFEM’s experience of implementingtheStrategicPlan,withaviewtoinformingthenewstrategicplanningprocessofUNWomen.Thestudyhasbeensubjecttoasignificantshiftinitsparameters during its design and implementation.Thisintroductionsetsouttheoriginalandfinalbasisandrationaleforthestudy,andexplainstheeffectsofthere‐orientationontheresultingreport.

1.1 TheStrategicPlan,itsresultsframeworksandassociatedsystems

TheUNIFEMStrategicPlanconstitutedthecorporateprogrammaticframeworkfortheperiod2008‐2011(laterextendedto2013)followingtheMulti‐YearFundingFramework2004‐2007(MYFF).ItwasalignedwiththestrategicplansofUNDPandUNFPA,andwasintendedto‘providestrategicpolicyandmanagementdirectionforUNIFEMtoincreaseitsdevelopmenteffectiveness,strengthenstrategicpartnershipsandmobiliseresourcesinthefouryearperiod.’1TheStrategicPlanwasbuiltonUNIFEM’sdualmandateto(i)provideinnovativeandcatalyticprogrammingandfinancialsupporttocountriestoachievegenderequalityinlinewithnationalpriorities;and(ii)tomainstreamgenderequalityacrosstheUNsystem.ItsdevelopmentwasinformedbyanevaluationofthepriorMYFF(2004‐2007);andbystakeholderconsultations(MemberStates,UNorganisationsandkeypartners),aswellasregionalandcountryanalysis.TheStrategicPlanidentifiedanumberofkeyaspectstowardsfulfillmentofthemandate,whichincludepresenceatcountryandregionallevels;strategicpartnerships;adequateanddedicatedresources;the

identificationofwhatworksandbuildingcapacities.ItprioritisedthreethematicareasforUNIFEM’swork:Women’sEconomicSecurityandRights;PrevalenceofViolenceagainstWomenandHIV&AIDS;andGenderJustice.TheStrategicPlanconstitutedthemainmechanismsforUNIFEMprogramming,reportingandaccountabilitytotheUNDP/UNFPAExecutiveBoard.Itispremisedonthebasisthataholisticapproachtoadvancinggenderequalityandwomen’sempowerment(GEWE)requiressupportingchangeatmacro,mesoandmicrolevels.Itincludesagoal,purposeandeightintendedoutcomes,withasupportinginterventionlogicsettingouttheanticipatedresultsareas.Tosupportimplementationandtomonitorprogress,aDevelopmentResultsFramework(DRF),ManagementResultsFramework(MRF)andanIntegratedResourcesFramework(IRF)weredeveloped,inlinewithResultsBasedManagement(RBM)principles.2Anonlineresultstrackingsystemfacilitatedprogressreporting;thisrequiredfieldofficesanddivisionsatheadquarterstoreportonannualprogressagainstindicatorsviathecentralmechanism.TheUNIFEMevaluationssystemalsorequiredthealignmentofevaluationswithStrategicPlanoutcomes.

1.2 Originalremitofthestudy

InApril2010,UNIFEM’sEvaluationUnitcommissionedanEvaluabilityAssessmentoftheorganisation’sStrategicPlan2008‐2013.AnEvaluabilityAssessmentisdefinedas‘Aqualitativeanalysisofaproject,programmeorplantodeterminewhetheritmeetsthepreconditionsforitsevaluationand,ifso,howtheevaluationshouldbedesignedtoensuremaximumutility’.3

1EvaluabilityAssessmentTermsofReferenceversion1(December2009)2TheDevelopmentResultsFramework,ManagementResultsFramework(MRF)andIntegratedResourcesFramework(IRF)areavailableinAnnex43Rossietal(2004)

12

Introduction

Specifically,thisstudywasoriginallycommissionedasa‘qualitativeanalysisof[theStrategicPlan’s]basicparameterstoascertainwhetherameaningfulevaluationof[theStrategicPlan]isfeasibleandwillprovideusefulinformationatalaterstagebothintermsoftheresultsoftheplan,aswellastheprocessesthatleadtotheseresults’.4Asoriginallyintended,thestudyaimedtoprovideinformationon‘fitness’oftheStrategicPlanforitsfutureevaluation.Consequently,itscomponentsincluded:thetechnicalrobustnessandinternal/externalcoherenceofthestrategyandtheunderstandingofchangeonwhichitwasbased;thesystemsandresourcessetinplaceforitsimplementation;theextent,natureandlocationofdatatosupportafutureevaluation;gapsevident;anyactionsrequiredinpreparationforcarryingouttheevaluationcommittedtointhestrategyandtheparametersforanyfuturefinalevaluation.ThestudycommencedinMay2010basedontheseparameters.Howeverasplansfordevelopmentofthenewgenderentityevolved,andinparticularduringfollow‐uptoResolutionA/RES/64/289(July2010),itbecameclearerthatUNIFEM’sStrategicPlanwaslikelytobeovertakenbyafuturestrategicplanforthenewgenderentityin2011.Consequently,thecurrentStrategicPlanwouldbeunlikelytobeevaluated.InlateJuly2010,thestudywaspausedfortwomonthsinordertore‐orientittothenewinstitutionalcontext.

1.3 Re‐orientation

Oncethefuturedirectionofthestudywasclear,arevisedtermsofreferencewasdeveloped.Thisincludedachangedpurpose,asfollows:

Aqualitativeanalysisofthe[StrategicPlan’s]basicparametersanditsmonitoringandreportingsystems…Aformativeandforward

lookingexerciseaimedatcapturingbestpractices,challengesandlessonslearnedfromtheUNIFEMStrategicPlanexperiencetodateforreflectionandlearning

There‐orientedstudythereforehasmoreincommonwithanevaluationofastrategy,initsfocusonsystems,itsemphasisontheinformationneedsofevaluationusersanditsintendedrelevancetodecision‐makers–anareawhereevaluationisjuststartingtobetested.5

Fortherevisedstudy,thefollowingobjectiveswereidentified:

• ToassesstheStrategicPlan’stheoryofchange6anditsresultsframeworksandidentifystrengths,weaknesses,challengesandlessonslearnedinregardstoformulationofresultsandindicatorsandproviderecommendationsforstrengtheningit;

• ToassesstheutilityoftheStrategicPlanasaguidingframeworkforthedevelopmentofthematic,regional,sub‐regional,countrystrategiesandprogrammesandproviderecommendationsforenhancedutility;

• ToassessthedatacollectionandinformationsystemsfortrackingStrategicPlanresultsintermsoftheirrelevance,effectivenessandcoherenceandtheabilitytoaggregateresultsfromcountrytocorporatelevelandovertimeandproviderecommendationonhowtheycanbeimproved;

Atheoryofchangedescribestheunderstandingofhowchangehappens.Itunderpinsany

interventiontobringaboutchange,evenifitisnotexplicit.Experienceindicateshoweverthatbeingexplicithelpstotestunderstandingandtoimprovethelikelihoodofbringingaboutthe

desiredchange.

4TermsofReferenceversion1(December2009)5SeeQuinnPatton,M.andPatrizi,P.A.(2010),Strategyasthefocusforevaluation.NewDirectionsforEvaluation,2010:5–28.doi:10.1002/ev.343

6Atheoryofchangedescribestheunderstandingofhowchangehappens.Itunderpinsanyinterventiontobringaboutchange,evenifitisnotexplicit.Experienceindicatesthatitisbesttobeexplicitasthishelpstotestunderstandingandimprovethelikelihoodofbringingaboutthedesiredchange.

13

Introduction

• ToassesstheextenttowhichtheresultsandindicatorsintheStrategicPlananditsresultsframeworkscontributetoenhancedmonitoring,reportingandlearningaboutUNsystem‐widecoordinationandaccountabilityforresultsonGEWE,particularlyatthecountrylevelandthroughMRFOutput2,andproviderecommendationsonhowtostrengthenthisaspect;

• ToassesstheStrategicPlananditsresultsframeworksintermsofhowitenhancesandclarifieslinkagesbetweennormativeandoperationalareasofworkandproviderecommendationsonhowtoimprovethisaspect;

• ToassesstheextenttowhichtheStrategicPlanwouldallowformeaningfulfinalevaluationthatwouldprovideusefulinformationintermsoftheachievementofresultsorlackthereof,aswellastheprocessesthathaveledtotheachievement/non‐achievementofresultsandtoproviderecommendationsonhowtostrengthentheStrategicPlan’sevaluability;

• ToassesshowlearningtodateontheStrategicPlananditsresultsframeworks,datacollectionandinformationsystemscancontributetofutureUNWomenstrategicplanningprocessesandproviderecommendationstothiseffect.

ThefulltermsofreferenceareavailableinAnnex1.Thestudyisthereforemuchmoreformativethanoriginallyintended(thoughthefocusonrequirementsforfutureStrategicPlanevaluationhasbeenretained).ThescopeofworkisalsomuchmorestronglygroundedinunderstandingtheexperienceoftheStrategicPlananditssystemsatsub‐regionalandcountrylevels,witharequirementfortwofieldstudiesatsub‐regionalandtwoatcountrylevel.

14

Approachandmethodologyofthestudyincludingre‐orientation

2.1 Elementsofthemethodology

Thestudywasconductedbyatwo‐personteam,fromMay2010toFebruary2011.FollowingtheapproachadvocatedbyPattonandQuinninrelationtotheevaluationofstrategy,abroadmixofmethods,wasadopted,includingcomprehensivedocumentaryanalysisandinterviews,reviewoftheonlinetrackingsystems,twofieldtripsandavalidationvisit.Samplingandanalyticalframeworksweredevelopedfordatagathering/analysis,includingthefieldtrips,andweresharedwithEvaluationUnitandthestudy’scross‐institutionalReferenceGroup.Therewasastrongfocusontheeventualusersofthestudy.Essentially,themainelementsofthemethodologywere:

• Aninceptionphase,includingabriefingmeetingwithEvaluationUnitinNewYorkplusaworkshopandinterviewswithacrosssectionofUNIFEMstaff(centreandregionally‐based);

• Twoanalyticalframeworkstoguideresearch–

onefortheoverallconceptualapproachtostudyasawhole,andamorespecificonetoguidedocumentanalysis;

• AsamplingframeworkagreedwithEvaluation

Unitfortherangeofdocumentationandsystemsfordatacaptureincludingtheon‐linetrackingsystem,theintranetandinternetsitesforassessment;

• AtechnicalappraisaloftheStrategicPlan

document,theunderlyingtheoryofchange/understandingofhowchangehappensanditsresultsframeworks,especiallytheDRFandMRF;

• Asamplingmethodologyfortheselectionof

regionaloffices(ROs),sub‐regionaloffices(SROs)andcountryoffices(COs)forstudy(subsequentlyamendedontheguidanceofEvaluationUnitandtheReferenceGrouponthebasisofinformationavailabilityandthepracticalitiesoffieldvisitscheduling);

• Documentaryreviewandphoneinterviewsto

assesstheexperienceofeightSROsandeightCOs,coveringallregions(seefollowingpageforsample);

• Sitevisits(oneweekeach)toSRO‐COgroupings

ofCEE/AlbaniaandAndean/Colombia,guidedbytheBriefingNote.

InterimoutputsincludedanEmergingFindingspresentationandnotes(December2010)whichelicitedfeedbackfromtheReferenceGroup,followedbyaDiscussionPaperandValidationmeetinginJanuary2011whichelicitedfeedbackonfindingsandproposedfuturedirections.

2.2 SampleBase

AfulllistofdocumentationsurveyedandpersonsconsultedisavailableinAnnexes5and6.Inadditiontoawiderangeofinstitutionalandsystemsinformation(resultstracking,evaluationprocesses,ATLAStrackingandcodes),plusexternaldocumentationsuchasdonorinstitutionalandthematicreviews,themainsampleframeforthestudyislistedintable2.1onthenextpage.Additionally, key informants from each office orsectionwere interviewedbytelephoneorSkype;aswellasarangeofinterviewsbeingconductedduringvisits to headquarters, e.g. during the inceptionstage.

15

2.3 Limitations

Thestudyexperiencedanumberoflimitationsduringitsimplementation.Theseincluded:thelossofmomentumduetotheneedtopauseandre‐orientthestudy,whichmeantthedevelopmentofnewanalyticalframeworks(andthere‐analysingofsomedata)plustheinitiallossofsomeinstitutionaltraction;thedesignofthesamplingframeforSROandCOselection,whichwasconstrainedbycompetingdemandsaroundinformationavailabilityandfeasibilityofvisits;dataconstraintsaroundprogrammedocumentsinparticular‐mucheffortwasrequiredtoidentifyandsourcerelevantinformation(plusdocumentationfromprogrammesdesignedafterthe2008StrategicPlanwaslimited)andfinally,thelimitedscopeforexplorationoftheexternalcontext,duetothestateoffluxinthe

institutionalcontext–consequentlyexternalperspectivesonUNIFEM’sStrategicPlanweregleamedmostlyfromsecondarydocumentedsourcesorfrominterviewsduringfieldvisits.

2.4 Currentinstitutionalrelevance

Despitethesechallenges,thebodyofevidencedevelopedprovidessomeverysignificantinsightsintoUNIFEM’sexperiencewiththeStrategicPlan2008‐2011.SomerelevantandinterestinglessonshaveemergedontheStrategicPlanitself,thesystemsassociatedwithit,andtheexperienceofitsimplementation.Thestudyhasbeenabletoidentifywhathasworkedwell,whatlesswell–andwhatmoreisneededforUNIFEManditssuccessorUNWomentomoreeffectivelymonitorperformance,

AreasAddressed

Thematic(Thematicstrategiesplusassociatedsection/individualworkplans)

GenderResponsiveBudgetingEndingViolenceAgainstWomen

Governance,PeaceandSecurityEconomicSecurityandRights

Funds EliminatingViolenceAgainstWomenGenderEqualityFund

Regional(RegionalStrategies;1programmeperregionplusindividualworkplans)

Africa

LatinAmericaandtheCaribbean

Asia,PacificandArabStates

CEE/CISGeo‐section

Sub‐regional(SubRegionalStrategies,2programmesperSRO,officeandindividualworkplans)

WestAfrica

EastandHornofAfrica

Andean

Caribbean

SouthAsia

ArabStates

CEE

CIS

Country(CountryStrategies/equivalent,2programmesperCOplusoffice/individualworkplan;relevantUNDAFs)

Liberia

Sudan

Colombia

Haiti

Pakistan

Palestine

Albania

Moldova

FIELDVISITS(Asabove,plusarangofadditionalprogrammingandstrategicdocumentationperoffice)

AndeanSRO

ColombiaCO

CEESRO

AlbaniaCO

Approachandmethodologyofthestudyincludingre‐orientationc

Table2.1:SampleFrame

16

trackresultsandbeabletoreportonorganisationalachievements.Thestudydoesnotaimtoprovidedetailedrecommendationsonpreciselywhatanynewstrategicplan,theoryofchange,outcomes/outputs/indicators,institutionalsystems,etc.shouldconsistofunderUNWomen.Firstly,thisisbecausethelearningpresentedhereisbasedonUNIFEM’sexperienceonly.Secondly,atthetimeofwriting,UNWomen’sownnewinstitutionalstructureisnotyetinitsfinalform.Consequently,thestudyfocusesonhighlightinghowthelearningfromthisexperiencecanbeusedtosupportthestrategicplanningprocessofUNWomen,inparticularthroughhighlightinglessonslearned,goodpracticeandprovidingtechnicaladviceforfutureimprovements.Inparticular,itaimstofirmlygroundtheexperienceoftheStrategicPlananditssystemsinthecountrycontext,inorderthatgoingforwardUNWomen’srevisedplansandsystemscanbedesignedontheterrafirmaoflessonslearnedandevidencefromrealexperience.

2.5 Structureandaudience

Thisreportisstructuredalongthefollowingaxes(whichalsocorrespondtothequestionsofthetermsofreference):

• Technicalrobustness:HowdotheStrategicPlananditsresultsframeworksfunctionasinstrumentsforstrategicplanning?

• Accompanyingsystems:WhataretheirrolesinsupportingStrategicPlandelivery?

• Theexperienceofimplementation:WhathasbeentherealityoftakingupandusingtheStrategicPlananditssystems?

• TheUNIFEMmandate:HowhastheStrategicPlansupportedthedeliveryofUNIFEM’sremitaroundnormative/operationalactivityplustheUNcoordinationprocesses?

• Theinformationbase:Towhatextenthasthe

StrategicPlandeliveredaninformationbasetosupportanylaterevaluation?

ThereportisaimedatallmembersofUNWomenandrelevantpartners,includingtheformerUNIFEM,DAW,OSAGIandINSTRAW,atheadquarters,sub‐regionalandcountrylevels.Itmayalsoberelevanttoexternalpartnersandfunders.

Approachandmethodologyofthestudyincludingre‐orientation

17

KeyFindings

Section3outlinesthemainfindingsofthestudy.Eachsectionsummarisestheseintermsof:organisationalprogress,challengesidentifiedandsummarymessages.Conclusions,lessonslearnedandrecommendationsfollowinsections4and5.

3.1Technicalrobustness:HowdotheStrategicPlananditsresultsframeworksfunctionasinstrumentsforstrategicplanning?

ThestudyhasconsideredfirstthetechnicalrobustnessoftheStrategicPlananditsassociatedresultsframeworks(particularlytheDRFandMRF)asatoolforstrategicplanning.Thisincludesassessingthefollowing:

• Theuseofaclearmappingofhowchangehappens(internalandexternalcoherence)

• Theuseofrelevant,clearandvalid

objectives(goal,outcomes,outputs)andindicatorstomeasureresults(relevance,clarityandvalidity)

• Thefeasibilityoftheobjectivestobe

attainedandtheplausibilityoftheintendedchangetooccur(feasibilityandplausibility)

• Theinclusionofclearandexplicitnormative‐

operationallinkagestosupporttheachievementofGEWE

Afulltechnicalappraisaloftheunderstanding/theoryofchangeandoftheStrategicPlanandtheDRF,MRFandIRFarecontainedinAnnexes3and4.Theinformationbelowsummarisesthesefindings.

3.1.1Organisationalprogress

ThestudyhasfoundthattheStrategicPlananditsassociatedresultsframeworkswererelevantandappropriatetothecoreUNIFEMmandateandsubsequentoperations.Thisisevidencedthrough:

Anunderstandingofchange(theoryofchange),whichprovides:

• Aconceptuallysoundandvalid(thoughlimited)analysisofwhatisneededtoachievegenderequalityandwomen’sempowermentandoftheUNIFEMresponsetothisneed.Emphasisisontheneedforchangetohappenatthelevelofmacro(normativeframework);meso(institutionalframeworkandpractices)andmicro(voiceandinfluenceofwomenandwomen’sgroupsandcommunity‐levelattitudesandpractice).Outcomesbuildonthisunderstandingtoindicatewhatchangeneedstohappen(Annex6oftheStrategicPlan‐Theinterventionlogic);

• Adegreeofflexibilityandresponsiveness(i.e.

notwhollyprescriptive),theStrategicPlanallowsscopeforprioritisationofoutcomesandforselectionofstrategies;

• Relevanceforthematicandfieldlevel

operations(regional,sub‐regionalandcountry)‐nodivergingdescriptionsofhowchangehappens(theoriesofchange)havebeenidentifiedacrossacomprehensivedocumentationanalysis.Someoffices,e.g.SouthAsiaRO,emphasizedtheutilityoftheStrategicPlandescriptionofchangeinhelpingcommunicateUNIFEM’srationaleforactionacrossteamsandwithpartners;

• Stronginternalcoherence,with,e.g.Strategic

Planthemesfittingwellwiththeunderstandingofchangeandoutcomes.TheCISoffice,forexample,foundthedescriptionofchangeveryusefulinhelpingtoincreasestaffunderstandingonhowthedifferentStrategicPlanoutcomesarelinkedtogether,e.g.DRF5‐6‐7(capacitybuildingofstakeholders)supportingresultsinDRF1‐4(thenormativeframework);

• Anexplicitconnectionbetweenthenormative

andoperationalaspectsofworktoachieveGEWEplusclearandoperationalmicro‐meso‐

18

macrolinks(seesection3.4).WithintheStrategicPlananditsresultsframeworks,agoal,outcomesandoutputswhichprovide:

• Asinglegoalwhich–whilelowinambition‐wasgermanetotheUNIFEMcoremandateandwhichrelatestoalevelofchangethatUNIFEMfeltcomfortabletoaddress.Bycontrast,theUNDPandUNFPAStrategicPlans(developedatthesametimeandusingsimilarapproaches)have,respectively,fourandthreedevelopmentgoalsininterlinkedfocusareas;

• Consistencyandalignmentofoutcomeswith

thedescriptionofchange(internalcoherence),andaclearfocusonthenationallevel–asalignedwiththeParisDeclaration/ManagingforDevelopmentResultsagendas;

• ClearandexplicitoutputstatementsatDRFand

MRFlevels,withsometargetsinplaceparticularlyatoutcomelevel(seeAnnex4);

• Arecognitionofaprocessapproachto

deliveringonthecoremandateatoutputlevel(e.g.throughtheemphasisongeneratingabodyofevidence,effectivepolicydialoguemechanisms;improvingskills/capacitiesandservices/systems).

Indicators,whichsupporttheprocessthrough:

• Ahelpfulshiftfromthepreviouslyquantitativefocustoensuretherecognitionofqualitativeapproachestodeliveringresults.Whilechangingindicatorsisalwaysproblematic,therevisedindicatorswerefelttobeclearertoreportagainst,forexamplereplacing‘Numberofnewincentivestructuresintroducedtoenhanceactionongenderequality(e.g.‘gendersensitiveperformancemeasures’with‘Extenttowhichkeypolicyandservicedeliveryinstitutionshaveincreasedbudgetsforpromotinggenderequalityandwomen’shumanrights’(new7b);

• Clarityandexplicitnessatgoallevel,withfull

inter‐linkagetothethreeStrategicPlanthemes(thoughthereisscopeforanadditionalindicator

tomeasure‘genderjustice’);• AdequatebreadthandscopeatDRFandMRF

leveltoallowforappropriatecontextualisation,evenifthiswasnotalwaysmaximised.

3.1.2Challengesidentified

However,intermsofitstechnicalrobustness,theStrategicPlanandtheresultsframeworkshaveanumberofshortcomings,whichwouldneedtobeaddressedinanyfutureplanningprocess.Figure1providesasummaryoverviewofwhatispresentandwhatisabsentfromamoredevelopedmodelandAnnexes3and4providemoredetail.

KeyFindings

19

KeyFindings

20

Themappingofhowchangehappens(theoryofchange)asitstandsisverycompressedandneedsgreaterexplication,asfollows:

• TheGoaliscurrentlyinadequatetoreflectUNIFEM’sobjectives.ItsemphasisonrealizingnationalcommitmentstoGEWEisnotasufficientreflectionofUNIFEM’saimofachievingthis,anditcontainsanumberofassumptionsabouttheroleoftheState.TheGoalalsostopsshortoftheendoftheresultschain(reducedinequality)andisveryprocess‐oriented;itdoesnotspelloutthechangesenvisagedforwomenandmenthroughUNIFEM’sinterventions.Goingforward,thenewentitywouldneedtoconsiderdevelopingalong‐termgoalstatement,whichreflectstheambitionstowhichUNWomenwillaimtocontributeovertime;

• Theexpectationofhowchangewillhappen

containsanumberofassumptionsandrisksthatmaywellaffectcausalitybutwhicharenotmadeexplicit,suchastheassumptionthataconduciveenvironmentwillleadtotangiblechange;

• Thedriversofchange–conditions,motivations,

incentivesintheinternalandexternalenvironment‐areallfactorsingenerating,shapingandintensifyingorblockingchange,butarenotelaboratedinthedocument;

• Thereisnoattentionwithinthedescriptionof

howchangehappens(thetheoryofchange)topossiblealternativeexplanations/pathwaysforachievingresults–anumberofwhichwereindicatedbyUNIFEMstaff(seesection3.3);

• Thetheoryofchangeassumesthattheroleof

UNIFEMasacatalystwascentraltoachievingthedesiredresults.Itdoesnotexplorenorchallengethis(e.g.throughconsideringtheassumptionswithinit);

• TheStrategicPlanindicatesthatUNIFEMresultsare‘owned’nationallybutthereisnoexplanationaroundspecificstrategiesforthisotherthanviaundefined‘strategicpartnerships’–leavingquestionsaroundaccountability;

• Thereisageneralisedassumptionthatthe

barriersandchallengestoGEWErestheavilyintechnicalcapacityandcapability.Theroleofthewiderpoliticalenvironmentandthefunctionofthepoliticaleconomyarenotexplored–yetveryoftenthesedeterminethepolicymakingenvironment.

Outcomesandoutputsarebasicallyrobustbutlimited,particularlyintermsofupwardsconnections,articulationofthestartingpoint(baseline)andtargets:

• ThereisalackofclaritywithintheStrategicPlanaroundthestrategiesenvisagedforachievingtheobjectives(goal,outcomesandoutputs),particularlyatDRFlevel.AnotableabsencerelatestotheroleofmeninbringingaboutGEWE8andalackofdevelopedstrategiesintheStrategicPlanandrelatedprogrammedocumentsrelativetothis.Non‐discussionofstrategiesintheStrategicPlanmaybedeliberateinordertoenablethesetobedeterminedincontext,buttherestillneedstobesometestingofkeyapproaches(e.g.partnership;knowledgemanagement)relativetohowchangeisexpectedtohappen(asnowexistsintheCapacityDevelopmentStrategy2010).DiscussionofstrategiesisalsoabsentfromStrategicPlanguidancematerials;

• TheDRFandMRFareinadequatelylinked

together‐forexampleOutcomes3,5,6,8are“functionality”outcomes(theyreflecthowwellsystemsareoperating),butthesearenotcarriedthroughtotheMRFbeyondconsiderationof“cultureandvalues”aroundresults‐basedmanagement;

KeyFindings

8WhiletheauthorsrecognisethemanydifferingmodelsandstrategiesforaddressingtheroleofmenwithinGEWE,wecontendthatthisroleneedstobeatleastrecognisedandacknowledgedwithintheStrategicPlanandthetheoryofchange,ifnotdefinedindetail(toallowforflexibilityincontext)

21

• Thereisalackofarticulationaroundtheinter‐

linkagesbetweenoutcomesandoutputs(whichhasimplicationsforreporting–see4.2below).Oneofficestatedanoriginalintentiontofocusonfouroutcomes–butwhenoperationalisingthis,endedupactuallyworkingonseven.Manystaffrelayedthechallengesandinappropriatenessofdissectingactivity–of‘splicinganddicing’inamannerthatisnotobviouslycogentorreflectiveofrealactivityontheground;

• Targetsarenotalwaysspecified,andthereis

insufficientarticulationofresultsovertime(meaningchallengesforcapturingtheprocessandpaceofchange).Notspecifyingoutputtargetsatthecorporatestrategylevelisappropriateasitallowsflexibility–however,withsomeexceptions(seebelow),context‐specificoutputtargetshavenotbeenelaboratedinsub‐regionalandcountrystrategies–thusmakingmeasurementofprogresschallenging;

• Theoutcomes–whiletheyrelatetothenational

levelasabove–arecurrentlyframedasstepsintheprocesstowardsoutcomes,ratherthanstandingasoutcome‐levelstatementsthemselves,whichwouldreflectactualchangesandbenefitsanticipatedforwomenandmen;

• Atoutputlevel,itisnotclearthattheidentified

outputs,ifrealised,willleadtotheexpectedoutcomes.SomeoutcomeshaveonlytwooutputsandtheStrategicPlandoesnotincludediscussionofotherrequirementstorealisetheexpectedoutcome.InOutcome6,forexample,theoutcomeisreallyjustthesumofthepartsofthreeoutputs,ratherthanrepresentingthenextlevelofchange(seeAnnex4).Onceoutputsareidentifiedtheyneedtobesufficienttoreasonablydeducethattheywillleadtotheintendedoutcome.Outcomesresultingfromoutputsalsoneedtoreflectthenext/higherlevelofchange;

• ItisalsounclearinthecorporateStrategicPlan

whatUNIFEMintendedtoactuallycontributetoDRFoutputachievementbeyondformingstrategicpartnershipsand,insomecases,carryingoutactivities,whichareidentifiedinthetext.

Indicatorsarenarrowattimesandpotentiallydifficulttomeasure;

• Therevisedindicators,whilemorequalitative,arealsolesstangibleandconsequentlymorechallengingtomeasure.Examplesinclude:1a,1band3a.While1ahasatargetitisunclearhowthiswillbemeasured(i.e.howsuccessful“incorporationofgender”wouldbedefined.)And,while3aalsohasanumericaltarget,itisunclearifthisreferstonumberofreformsornumberofcountrieswithreforms;

• Therearesomeinstancesofdisconnectbetween

expectedresults(outcome/output)andassociatedindicators.Anexampleincludesindicator5.3a,thesingleindicatorforthisoutput:‘MainstreampolicymakingforumsinwhichUNIFEMhassecuredincreasedinfluenceforgenderequalityadvocates.’Alone,thiswillnoteffectivelymeasureOutput5.3:‘Governmentalandnon‐governmentalgenderequalityadvocatesparticipateeffectivelyinmainstreamingintegratinggenderequalityintopolicyprocesses,’norisit,onitsown,ausefulproxyindicator;

• Astheyarecurrentlyformulated,indicatorsmay

havelimitedreportingagainstprocess‐typechangesuchasqualitativechangeinrelationshipsorhowissuesarediscussedandtalkedabout(seesection3.3ontheexperienceofimplementation);

• OverallthelevelofSMART9nessisvaried,with

thebiggestchallengesbeingmeasurabilityandextenttowhichindicatorsaretime‐bound.Not

KeyFindings

9Specific,Measurable,Achievable,Relevant,Time‐bound

22

allindicatorsarecurrentlyneutralmeasures(e.g.3.3awhichspecifies’increase’ratherthanchange,whichwouldallowformovementineitherdirection);

• Thestandardsandparametersforachievement

arenotdefinedandsourcesarenotspecifiedbeyond,inmanycases,‘UNIFEMAnnualReports’;

• Theabsenceofbaselinesandthelimited

numberoftargetsmaketheuseofindicatorstomeasureachievementproblematic;

• SomeMRFoutputindicatorsarecurrently

limited.Theyappeartomeasurewhatisinstrumentalandcontrollable,ratherthansetadequatechallengesforchange.Clearerdefinitionisneeded,plusareflectionofthekeysignificantissues.Exampleswouldinclude,forMRF4,whichdoesnotmeasureprogressindelegatedauthoritytoCOsandforMRF1,whichdoesnotmeasureevaluationimpactondecision‐makers.

3.1.3Summarymessages

WhiletheStrategicPlananditstheoryofchangeandresultsframeworksprovidedarelevant,appropriateandconceptuallysoundarticulationofUNIFEM’scoremandatearoundtheachievementofGEWE,ithassometechnicalweaknesseswhichwouldrequirecorrectionundertheUNWomenstrategicplanningprocess.Theseinclude:

• TheinherentresultslogicoftheStrategicPlanisnotadequatelydeveloped,lackingforexample,analysisofassumptionsandriskandadequate

specificationofinterventionlogictodemonstratetherelevanceandadequacyoftheoutcomestowardsachievingGEWEovertime;

• Resultschainslacksufficientexplicationto

showanticipatedcausalprogressionandthe

envisagedUNIFEMcontributionanddonotprovideacomprehensivepictureofthestarting

point(baselines);expectedendpoint(targets)orinterimsteps(milestones);

• Thereisnotcurrentlysufficientrecognitionofpotentiallydifferentpathwaysofchange,toallowforinnovationandflexibilityatcountryor

sub‐regionallevel,andtotakeaccountofunintended(positiveandnegative)consequences;

• DespiteimprovementstoStrategicPlan

indicators,someremainnarrowanddonot

supportcomprehensiveresultstrackingandmanyaredifficulttomeasure;

• Intermsoftheirsequencing,thecorporateStrategicPlanandtheplanningprocesshavenotadequatelyallowedfortheprimacyofcountry

strategyplanningandthesettingofspecificobjectivesandoutputsincontext.Thedevelopmentofcountrystrategieswasanew

departureforUNIFEM:thoseresulting–plusSRSs‐havethereforebenefitedfromthecorporatestrategicframeworktheStrategicPlan

provides,buthavebroadlyacceptedtheunderlyingunderstandingofchangeandoutcomes;withouttestingandvalidatingthese

incontext(beyondsomeprioritisationatoutcomelevel);

• TheroleofkeyUNIFEMstrategies(e.g.capacitydevelopment,knowledgemanagement;networkingandcoordination)inlinking

outcomesandprogressingchangeisnotsufficientlyelaboratedintheStrategicPlan,noristheStrategicPlanspecificontheroleofand

targetsforglobalandintergovernmentalprocesses;

• ThenecessarylinksbetweentheMRFandDRFarenotmadeexplicit.

KeyFindings

23

3.2 Accompanyingsystems:Whatistheirrolein

supportingStrategicPlandelivery?

EffectiveRBMrequiresafundamentalorientationofsystemsandprocessestomakeresultscentraltothefullprogrammecycle.ItwasbeyondtheremitofthisstudytoundertakeacomprehensivesystemsauditofUNIFEM.However,thestudyhasreviewedthesystemsandprocessesinplacetosupportthedeliveryoftheStrategicPlananditsobjectives,inparticularguidancematerial,theresultstrackingsystemandmonitoringandevaluationsystems.Withaviewtoinforminganyfuturesystems,ithasfocusedontheirutilityandrelevance,particularlyatCO/SROlevels.3.2.1Organisationalprogress

Guidancematerial:Asetofguidancenotes(1‐7)weredevelopedin2008toaccompanytheStrategicPlanwithfurtherguidanceonextensionofregional,sub‐regionalandthematicstrategiesprovidedin2009andoncountrystrategiesinJanuary2010.MostattentioninthismaterialisfocusedonsupportingthedevelopmentofSRSswithemphasisonalignmentwiththecorporateStrategicPlan.IntermsofaqualitysourceofsupportforStrategicPlanimplementation:

• Thereisemphasisthroughoutontheneedforcontextanalysisandjustificationsupportedbydataandreference(thoughlittlediscussiononpotentialsources);

• Theguidanceisclearthatallstrategiesshould

indicatetheirevaluationplan,supportedbyguidancefromEvaluationUnit.TheneedforresultsbasedevaluationstosupportandprovideevidenceofStrategicPlanimplementationisemphasised;

• Theearly2010CountryStrategyGuidance

recognisesthenecessityofmonitoringandreportingatthislevelforthefutureevaluationoftheStrategicPlan(i.e.tobeableto

demonstrateachievementsandUNIFEMcontribution).

Resultstrackingsystem:Theresulttrackingsystemwasrelativelyrecentlyimplemented(resultswereuploadedelectronicallyforthefirsttimein2009).Thesystemrepresentedastep‐changefortheorganisationinthewaythatitcollectedanduseddataforperformancemanagement.Thestudyfindsthattheresultstrackingsystemhasdeliveredsomehighlysignificantbenefitsfortheorganisation:

• ItsupportedaninstitutionalemphasisonresultsreportingwhichrecognizestheimportanceofresultsandthespecificobjectivesofUNIFEMundertheStrategicPlan;

• Itenabledfarmoresystematicreporting(using,

inparticular,projectandprogrammedataatfieldlevel)againstspecifiedresults,whichhasinturnenabledthecentralizedcollation,synthesisandmanagementofdataasreflectedintheAnnualReportandinreportstotheConsultativeCommitteeandExecutiveBoard;

• Itgeneratedmuchgreaterinstitutional

coherenceforresultsreporting,intermsofthematicsections,geo‐sectionsandfieldlevelreportingintoasinglesystem;

• ItprovidedmuchgreaterclarityforSROson

reportingrequirements;• Itenabledtheupwards(SRO‐HQ)aggregationof

resultstowardsanalignedsetofobjectives;• TheuseoftheATLASsystemforcoding

expenditureonoutcomeareasallowedforsomerecordingoffinancialallocationtoworkareas.

Monitoringandevaluation:Evaluationsystemsandmechanismsappeartohavemademoreprogressthanthoseformonitoring,inaligningwiththeStrategicPlanandwithresults:

• Examplesofevaluationplanningsystemalignment(atHQandfieldlevel)include:theEvaluationStrategy’s(2008)veryspecificreferencetotheDRFGoalandMRFOutputs1.1,

KeyFindings

24

1.4and1.5;theexplicitstatementofUNIFEM’stheoryofchangewithinkeydocuments(suchastheMeta‐Evaluation2009);thedevelopmentofevaluationplansandreportswithinsub‐regionalstrategiesandcountrystrategies;

• AdraftMonitoringandReportingPolicywas

developedinMay2010inresponsetotheStrategicPlan’sidentificationofresultsandrightsbasedmonitoringandreportingasachallenge(thoughthisappearstobeatquiteanearlystageofdevelopment/dissemination.)


Perhapsinevitably,however,anumberofchallengeshaveemergedaroundtheresultstrackingandotherperformancemonitoringsystems.Thisisparticularlyevidentatfieldlevel.Challengesforguidancematerial:

• Littleguidanceisavailableonindicators–relatedtotheoveralllimitedplanningguidanceprovided;thisomissionreflectsalackofemphasisonindicatorsasthecentreofeffectiveresultsmeasurement.Inparticular,thereisagapinguidanceatfieldleveltosupporttheshiftfromthegenericheadlineindicatorsoftheStrategicPlantocontextspecificindicatorsforsub‐regionalandcountrystrategies(andtosomeextentThematicStrategies),whichcanstillbeaggregated;

• Therearefewreferencestobaselinesandno

definitionsoftheircontent,functionand/orrationale,beyondanannextoGuidance4requiringatablesummarisingnationallevelinformation.Ifproduced,thiswouldhelppopulateabaselineonnationaldevelopmentplanningandframeworksonGEWE(suchasCEDAW,nationallawsandpoliciesetc.);

• Occasionalreferenceismadetohumanrights

andtoCEDAW,butoverallthereislittleornoguidanceortoolswithintheStrategicPlantosupporttheintegrationofarightbased

approach–perhapsrelyingontheuseofseparateresourcesforthis.(Thispointwasalsomadeinthe2009Meta‐Evaluation,whichhighlightedthisasagapinprogrammedesign);

• NospecificreferenceismadetotheUNIFEM

theoryofchangeinanyoftheGuidancematerials.Themostwidelyknownreference(apartfromtheStrategicPlannarrative)appearstobeintheCapacityDevelopmentStrategyMarch2010(section3.1).Stafffamiliarwiththismaterialdidemphasisetherelevanceandutilityofthevisualandaccount,butoverallfewwerefamiliarwithit(somebeingintroducedduringsitevisitsforthisstudy).

Challengesforresultsreporting:

Theresultstrackingsystemisheavilygearedtowardstheproductionofthe(centrally‐generated)AnnualReport.Consequently,thesystemwasnotbeingused–eithercentrallyorbystaffinfieldoffices‐toitsfullcapabilityforperformancemonitoring,progresstrackingortrendanalysis.Sincefeedbackloopsareheavilyupwardsatthisstage,information,whichcouldinformplanninganddecision‐makingatfieldlevelisnotbeinggeneratedorfeddownwards.Specificexamplesoftheselimitationsinclude:

• Theemphasisonreportingto(sometimesnarrow)StrategicPlanindicatorsconstrainthecapacity–particularlywithincountries‐toreportagainstwiderresultsandchanges/processesgenerated,suchascultural/political/attitudinalchange.Someveryclearexamplesofthiswereprovidedthroughconsultationsandfieldstudies.Theseinclude:thegenerationofpartnershipsanddialogueforamorecoherentapproachtoGEWEin‐country;thestimulationofmulti‐stakeholderdebateon‘models’ofgenderinnewpoliticalcontexts;theimprovementoflocalpolicyenvironmentsthroughmoreconsensus‐baseddecision‐making;attitudinalchangeandthedevelopmentoflocal‐levelGEWEpolicyandactionplans.Oftenthesechangescouldbe–andwere–reportedthroughmorenarrativedonorreports;

KeyFindings

25

• Thesystemisunabletoreflecttheoftenrapid

contextualchangeswhichoccuratcountrylevel,suchaspoliticalorgovernanceshifts.Thisconstrainsboththetypesofresultsbeingaimedforatlocallevel,aswellastheirreporting;

• Thesystemhasalackofcapabilitytogenerate

reportstailoredtotheSROandprogrammaticlevels.Consequently,somesectionsandoffices(includingAndeanSROandViolenceAgainstWomenthematicsection)aredevelopingtheirowninternalmanagementinformationandprogressreportingsystems–representingineffectaduplicationofresources;

• Whilethereissomecapabilityforupwards

trackingataggregatelevel,thesystemasitstandsdoesnotyetallowprogressovertimetobereported.COsandSROsfindtheirreportingtobefocusedonachievements,withthedynamicsofprogress(orlackof)rarelymonitored.Alongsidetheabsenceofbaselines,targetsandmilestonesorbenchmarksattheselevels,thismakesitdifficulttofullyassessperformance/distilandapplylearning;

• Thereisnoevidenceoftrendanalysis/

feedbackontrendscorporatelybeyondtheannualreportreviewandcentralreportssuchasthosetotheSecretaryGeneralandExecutiveBoard.Examplesoftheformermightincludeanalysisagainstprogressinparticularthematicareas,orinterimreportsongapsbeingidentified.Severalsectionsandofficescommentedonthelackofdownwardsfeedback,andfieldvisitsfoundnoevidenceatallofanysuchinformationbeingeithergeneratedorprovidedtosupportoperationalorstrategicplanning,ortogiveasenseofthe‘stateofplay’ofwiderprogress;

• Thereareonlyfragmentedsystemsinplaceto

supportprogresstrackingagainstindicatorsandthereisnouseoftrafficlightorothermechanismstohighlightprogression;

• Asreportedinmoredetailinsection3.3below,

thecodingoffinancialallocationsundertheATLASsystemisverydependentonindividualinterpretation;meaningthatthedataheldisunlikelytobefullyrobust.

Onfeedbackloopsspecifically:whilethereissomeevidencethatAnnualReportreviewsandevaluationfeedbackloopsarestartingtoinformprogrammatic/strategicactivity,theseareatanearlystageofdevelopment,and,inthecaseofevaluationareasyetunsystematic.Feedbacklinksbetweennormativeandoperationalactivityarenotsupportedbytheresultstrackingsystem,whichdoesnotactivelyseektomaketheseconnectionsortoprovideincentivestoaddressthese.Thereareclearinstancesofeffortstogeneratesuchlinkagesatdesignstage(evidentforexampleinthecaseoftheViolenceAgainstWomenFundactiveintheColombiaCO,wherespecificlinkshavebeenmadeintorelevantdialoguefora),buttheresultstrackingsystemdoesnotallowforcomprehensivemonitoringorevaluationoftheirprogressinbuildingtheselinks.Challengesformonitoringandevaluation:

• Whiletheintroductionofanevaluationmanagementresponsetoolrepresentsgoodprogress,thereisaneedforgreaterbuy‐intotheprocess(manyfeltthattheresponsetimewastoolong)andforaclearerandbroaderunderstandingofitspurposeandrole.Increasedcompliancewiththeevaluationpolicyisnecessarytoensurethataclearconnectionismadebetweenevaluationfindingsandstrategicdecision‐making(notonlywithinprojects).

• Thefirstbiannualsub‐regionalstrategy

evaluationplansweredevelopedbeforethecorporateEvaluationPolicy(2009,thoughtheStrategywasdevelopedin2008)andthusongoingguidanceandcapacitydevelopmentwillbeneededtoimplementresults‐focusedevaluationsandtousefindingseffectively.

• ThedraftMonitoringandReportingPolicy(May

2010)providesastart,butevidencehashighlightedthegeneralisedabsenceofamonitoring–asopposedtoreporting‐culture

KeyFindings

26

andassociatedmechanisms,capacityanddedicatedstaffwiththeremitandtimetomonitorresults.ThisisakeychallengefortheStrategicPlanandarecurringthemeinfieldvisitsandconsultations(seesection3.3).

ChallengesforRBMsystemsoverall:

Theprogramme/projectcyclemanagement(PCM)systemunderpinningtheStrategicPlanisnotfullyalignedtoeffectiveRBM.Themanagementmechanismstoensurethatresultsperformanceisbeingtrackedandinformingdecision‐makingatalllevelsarenotyetsystem‐wideoradequate.Forexample,workplansarelargelyactivitybasedandsupportcompliancewithdeliverablesratherthanprogresstoresults(apartfromaheadlineresult,workplanstypicallyfocusonactivitiessuchastraining,studytours,analysis,etc.ratherthantheresultantchangeexpectedfromtheseactions.Fieldstaffinterviewedindicatethattheytypicallyreportagainstcompletionofactivities,ratherthanprogresstochange/results);andthetime,resourcesandmechanismstoundertakethenecessaryreflectionandadjustmentsacrossacountry/sub‐regionalprogramme(asopposedtodistinctprojects)arenotsufficient.Capacitiesareoftenlimited,withseveraloftheofficesvisitedandconsultedindicatingthattheyhavehadverylittleorientationtoRBMbutexpressinganappetiteforit–anissuepreviouslyhighlightedintheGenderResponsiveBudgetingevaluation10.• QualityassurancefortheStrategicPlanand

relatedstrategiesrestsprimarilywithgeographicsections(GuidanceNote4,Annex1),whileapprovalrestswiththeProjectApprovalCommittee(PAC).GoodpracticeatHQ,suchasthereviewofSRS,ishamperedbythelengthoftimetakenforthereviewanditsfindingstobedisseminated.Arepeatedthemefrominterviewsistheneedforamoresystematicapproachtoqualityassuranceandmoresustainedon‐goingsupportintheformofguidance,trainingand

technicalassistance.ThefollowingboxprovidesashortstocktakeofRBMsystemsasviewedfromthecountryperspective.


KeyFindings

Box1:Rapidstock‐takeofsystemsatcountryleveltosupportresults

Countryofficesarehighlyvaried:ThereareawidevarietyofCOs,someverysmall(e.g.Moldova),someverylarge(e.g.Pakistan,Colombia,Afghanistan).Some,suchasSouthSudan,operateatalargelyhumanitarianlevelwhileothers,e.g.inCEEarebasedinmiddleincomecountries.CorporateStrategicPlanguidanceatpresentisnotadequatetotakeaccountofthesevariations.

Countrystrategydevelopmenthasnotbeencentral:InthepresentStrategicPlan,countrystrategiesweredevelopedafterthecorporateandsub‐regionalstrategies.Someofficesreportthattheyfollowedascomprehensiveanapproachtocountrystrategydevelopmentaspossible,whileforothersitwasnomorethan‘acutandpasteexercisetofulfillacorporaterequirement’(arespondent).TheStrategicPlanhasrecognizedthateffectingchangesinpeople’slivesislargelydependentoncountry‐levelinterventionsbutthisisnotyetreflectedinthestrategicplanningprocess.

Thestructureandprocessesoftheofficeneedstoreflectaresultsfocus:Inreality,teams,designationsandresponsibilitiesaremoreproject/thematicthanoutcome/resultsdetermined.ThisisdespiteUNIFEM’sstatedcommitmenttoaprogrammeapproach,andeffortstomovefrommultiplesmallprojectstolargerprogrammes.Officeworkplans,deliverablesandquarterly/annualreviewmechanismsarenotsufficientlyalightedtokeysub‐regional/countrystrategyresultstohelpteamsinmeasuringprogresstowardsresults–thuslimitingtheiruseforplanninganddecision‐making.

Resourcesneedtobelinkedtoresults:Thereareproblemswithhumanresources(suchasrelianceonshorttermcontractsandlackofexpertiseinkeyareassuchasmonitoringandRBM);andfinancialresources(predictabilityofcoreandnon‐coreanddurationofsupport),whichnegativelyimpactonresultseffectiveness.Alliedwiththisarebureaucraticrequirementsandtheslowpaceofdecentralization,e.g.relatedtodelegatedauthorityandapprovallimitswhichhampersaresultsfocus(anissueforallofficesinfieldstudies).

10SocialDevelopmentDirect(August2009)CorporateEvaluationoftheProgrammePortfolio:UNIFEM’sworkonGender‐ResponsiveBudgetingStage2SynthesisReportUNIFEMEvaluationUnit

27

Insummary,thesystemsimplementedrepresentamajortransformationinthewayUNIFEMsoughttobothcreatearesults‐basedculture,andtocollectandanalysedataforperformancemonitoring.Keymessagesfromtheexperienceofimplementationareasfollows:

• TheprovisionofguidancematerialtosupportsectionsandofficesintranslatingtheStrategicPlanintoarelatedsetofstrategiesconstitutesgoodpractice.However,currentlyavailableguidancedoesnotalwaysprovidesufficientlytimelyorcomprehensiveinformationtostaffforplanning,managingandreportingontheirwork;

• Whiletheresultstrackingsystemhasenabledcomprehensiveresultsreportingatgloballevel,itsgearingtowards(centralised)annualreportinghasmeantthatitsrelevanceandutilityforongoingperformancemanagementandtrendanalysishasbeenconstrained.Atpresent,thereisoverrelianceonreportingonachievementsaloneandinsufficientattentiontotrackingprogressfortrendanalysis,toenableadjustmentanddecisionmaking,e.g.throughthegenerationoflocally‐relevantreportsonperformanceagainstspecificareas;

• Currently,limitedanalysisoftrends/progress

anddownwardsfeedbackloopsisconstrainingtheuseofvaluableinformation.TosupportRBM,thereisurgentneedforsuchloopstobeformed,andforrelevantinformationtobefedthroughthemtosupportlocaldecision‐making;

• Indicatorsandevaluationplanninghavemuch

potentialtosupportresultsreporting.However,thesystemcurrentlychieflysupportsresultsreportingrelatedtofixedandsometimesnarrowglobalindicators.Whilethisisimportant,itisnotsufficient,andcurrentsystemsdonotcurrentlyadequatelyrecogniseorsupport(i)amorenarrativeformofreportingbeyondspecifiedindicators(andtextboxes)or(ii)thepossibilityofdifferentpathwaysofchangetoachievingoutcomesandthegoal(allowingforflexibilityandinnovationwithinthecontext);

• Whilesignificantprogresstowardsresults

measurementhasbeenmade,presentsystemsarenotyetgearedtosupportcomprehensiveresultsmanagement(throughthefullcycle).Thereareparticularshortcomingsinmonitoring,includinglackofdedicatedstaffandsystems(thoughevidenceexiststhatprogresscanbemadewhenthisexpertiseisavailable,e.g.AndeanSRO);andabsenceofalignedplanningandreviewprocessesthatsupportcomprehensiveperformancetracking.GoodprogresshasbeenmadetostrengthenresultsintheEvaluationPolicyandStrategybutfurtherworkremainstoembedafocusonresults.

3.3 Theexperienceofimplementation:What

hasbeentherealityoftakingupandusingtheStrategicPlananditssystems?

Thetwoprevioussections(3.1and3.2)haveshownthat,despitetheverysignificantinstitutionalimprovementsinresultsmanagementandreportingwhichtheStrategicPlananditsresultsframeworks,plustheirassociatedsystems,havegenerated,thereremainsometechnicalandprocessweaknesses.TheseweaknesseshaveconstrainedtheabilityoftheStrategicPlan(plusitssystems)toprovideabasisforinstitution‐wide,multi‐level,systematicresultsmanagement.ThissectionofthereportreflectsontherealexperienceofimplementingtheStrategicPlananditsresultsframeworksatregional,sub‐regional,countryandthematiclevel.ItlooksatthewaysthemainaspectsoftheStrategicPlananditsassociatedsystems‐thetheoryofchange,theDRF,MRFandIRF,theresultsreportingprocessandevaluationssystems–wereactuallytakenupandusedbyUNIFEMatitsdifferentlevelsofoperation,includingcountryandsub‐regionallevels.Consequently,itidentifiessomestrengths,someweaknesses,andsomethingsthatcanbetakenforwardintoUNWomenplanningprocesses,aswellassomethingsthatcouldbechangedorabandoned.

KeyFindings

28

3.3.1Organisationalprogress

BoththecorporateStrategicPlanitselfandtheunderlyingunderstandingofchangehaveclearrelevanceandutilityatthematic,regional,sub‐regionalandcountrylevels.Thereisstrongcoherenceinresultsplanningacrossthesespheresofwork.InstitutionaltractionfortheStrategicPlan–andparticularlyitsunderstandingofchange–wasgenerallystrongwheretherewerestaffwhoknewandunderstoodit,e.g.wheretheyhadbeeninvolvedintheprocessofitsdevelopment.TheStrategicPlanwasdescribedbyoneofficeasa‘quantumleap’forgeneratinginstitutionalcoherenceandconsistencyinplanningandoperations.Specifically:

• Atfieldlevel,officesconsideredthetheoryofchangerelevantanduseful,11bothintermsofinternalconceptualisationoftheirworkandasameansofdialoguewithpartners,e.g.explainingtheUNIFEMremitandoperationstocivilsocietyorgovernmentstakeholders(AndeanRO)and/ornegotiatingthestrategicobjectivesofjointprogrammingwithdonors(SouthAsiaRO).ThisisdespitethelackofreferenceinboththeStrategicPlandocumentanditsassociatedguidancetothetheoryofchange,oranytraining/adviceonhowtoapplyit;

• TheStrategicPlanunderstandingofchange

clearlyhadstrongresonancewith(ifnotexplicitlinkageto)planningatthedifferentlevelsofUNIFEMoperations.Thisisparticularlyevidentatcountry/sub‐regionallevel,wheresomeverygoodexamplesoflocalisedtheoriesofchangeareevident(e.g.WestAfricaSROandPakistanCO),thoughnotallthesewereproducedwithreferencetotheglobaltheory.Overall,nodivergingtheoriesofchangeacrossanyareasofactivity–thematic,sub‐regionalorcountry‐havebeenidentified.ThetwoFundshavetheir

ownstrategicrationales,butthebreadthandgeneralityoftheglobaltheoryofchangeenablethemtobroadlycoherewithit;

• ExceptingthetwoFunds(theUNTrustFundis

notablynon‐aligned,thoughfieldstudiesindicatedeffortsatsub‐regionalandcountryleveltoensurelinkage;whiletheFundforGenderEqualityhasmaderecenteffortstoalignitsM&EFrameworktotheUNIFEMStrategicPlan12)thereisstronginternalandexternalcoherenceacrossstrategicplanningdocumentationatalllevelswiththeStrategicPlan.Noneofthesampleof24strategicplansreviewed(thematic,regional,sub‐regionalandcountry)showedanyaspectsofnon‐coherence(barajustifiedselectiveapplicationintheSouthSudancaseduetotheveryspecificgeo‐politicalcontext).Incontrast,therearesomegoodexamplesofmanagementactiontoensurecoherenceincludingcomprehensivealignmentexercisesatsub‐regionalandcountryleveltoensurethiscoherence,suchasthemajoreffortcarriedoutbytheAndeanSROandColombiaCO,andtheuseinPakistanCOoflaminatedcardsoftheDRFandMRFoutputstoguidestaffintheirdailywork;

• TheStrategicPlanresultedinmorealigned

outcomesandoutputsatalllevels,atleastintermsofnarrativeanddocumentation.Virtuallyall13strategicandprogrammaticdocumentationreviewed‐plusevidencefromthefieldvisits‐showedamajorinstitutionaleffortatcoherencewiththeDRFandMRF.ExamplesincludethealignmentexerciseintheAndeanregiondescribedabove,whichlastedaroundthreemonthsandinvolvedreformulatingprogrammaticandstrategicresultstofitwiththeglobalframeworks.Thesameistrueatthematiclevel,whereallprogrammaticandstrategicframeworksreviewedreflectedintheir

KeyFindings

11AllofficesconsultedforthisstudyagreedontherelevanceandutilityoftheTheoryofChange,thoughseveralstaffmembers–mainlyatcountrylevel‐wereactuallyintroducedtoitthroughtheprocessofthestudyitself.12FundforGenderEquality:MonitoringandEvaluationFramework2010‐201313NotethatSouthSudan’scountrystrategyexplicitlystatesthattheMRFoncoordinationhasnotbeenincorporatedintothecountrystrategyasaresultofSudan’sgeo‐politicalfragmentation.

29

documentationatleastcoherencewiththeDRFandMRFoutcomesandoutputs,andofthesampleevaluationsreviewed;

• TheStrategicPlanhastosomeextentinfluenced

thesettingoftargetsatsub‐regionalandcountrylevel–inmostcases,thesewerenotinplacebeforetheStrategicPlanandalignmentexerciseswereimplemented(the2009MetaEvaluationhighlightedtheproblemofjudgementofperformancefromthisinformationwithoutbaselinesonstartingpointsandclarityonexpectedtargets).Examplesincludethemonitoringframeworkssub‐regional/countrystrategiesfromtheAndeanSROandColombiaoffices,whichestablishcleartargets(thoughseebelowregardingtheactualimplementationoftheseframeworks);

• Insomeoffices(thoughveryfew,andtendingtobenewer),thereisevidenceoftheStrategicPlaninfluencingprogrammestrategies.MoldovaCOinthedesignoftheirWomen’sEconomicEmpowermentprogramme,usedtheUNIFEMunderstandingofhowchangehappenstothinkthroughseveralessentialprogrammecomponentsoni)policyandlaws;ii)institutionalcapacity;iii)theworkofadvocates;andiv)womentargetgroups;

• Intermsofhumanrightsbasedapproaches,

evaluationstrategicplanningdocumentationemphasisesthis,andmakesanexplicitlinkagetotheStrategicPlan(referencingUNIFEM’scommitmenttoanactiverolewithinUNEGtopromotegenderequalityandhumanrightssensitiveevaluations).Atfieldlevel,andparticularlywithincountrystrategies,strategicplanningdocumentationdoessystematicallyrefertohumanrightsandgenderequity(allofthesurveyeddocumentationmentionrights)–butthisappearstobeoccurringlargelyasaresponsetocontextratherthanasareactiontotheStrategicPlaninitself.

MostoftheexternalstakeholdersconsultedwereunawareoftheStrategicPlandocument,reflectingthesensedescribedbelowofitslimitationsasacommunicationstool.However,allthoseinterviewed–government,donorpartnersandcivilsociety‐duringfieldvisitsreflectedaperspectiveonUNIFEMaimsandactivitycoherentwiththeStrategicPlan’saimsandobjectives,thetheoryofchangeandUNIFEM’sinnovativeandcatalyticmandate.Therewerenoexceptions.3.3.2Challengesidentified

Notwithstandingthe‘quantumleap’describedoftheStrategicPlan’sroleingeneratingclearaimsandobjectivesforUNIFEM,andabasisofinstitutionalcoherencetoaddressthem,itwasevidentfrominterviewsandfieldvisitsconductedthatanumberofchallengeshavearisenintheStrategicPlan’sactualimplementation.Manyofthesearisefromthetechnicalandprocessissuesidentifiedin3.1and3.2above;theyareperhapsrelatedtotherecentnessoftheprocess,plusresourceconstraints.TheyprovidesomevaluablelearningforUNWomengoingforward.Coherenceofstrategicplanning:Despitethestrongnarrativecoherenceoutlinedabove,thereremainanumberofgapsaroundinterlinkages.Theseinclude:

• Limitedexplicitlinkagesfromcountryandsub‐regionalstrategiesuptoregionalstrategies,forexampleEuropeandSouthAsia;

• Limitedexplicitlaterallinkagesbetween

thematicandSRSs/CSsforexampleanumberofcountrystrategiesrefertoactionstotackleViolenceAgainstWomenbutdonotexplicitlyorimplicitlyrefertotheUNIFEMEndingViolenceAgainstWomenstrategyof2008‐201114;

• Withsomeexceptions,therewasgenerally

limitedcontextualisationofStrategicPlanoutputsandindicatorsinSRSsandCSsorasimpletransferoftheglobalonestothesub‐regionalorcountrystrategy(e.g.CaribbeanSRO

KeyFindings

14ALifeFreeofViolence:UnleashingthePowerofWomen’sEmpowermentandGenderEquality(UNIFEM2008)

30

thoughSouthAsia/Andean/WestAfricaSROsandassociatedCOswereexceptions).SeeBox2forexamples.

Box2:IllustrationsofSRScontextualisationofStrategicPlanIndicators

CorporateStrategicPlanOutputIndicator

SubregionalStrategyequivalentindicator(s)

7.1aCapacityassessmentandsurveysindicatingincreaseinknowledgeandskillsinmainstreamingGEWEinpolicy,servicedeliveryormediainstitutions

ContextualisedSubregionalStrategyIndicators(WestAfrica)7.1a):NumberofcountriesapplyingGRB7.1b)BudgetoftheMinistryofagricultureinSenegalindicatesincreasedallocationforruralwomen7.1c)Evidenceofenforcementofwomen’shumanrightsbythepolicy,customs,correctionandcourtsinpostconflictcountries7.1d)EvidenceofbroadmediacoverageongenderequalityandWR7.1.e)ExtenttowhichgenderisincludedintothetrainingcurriculaofENSEAinCôted’Ivoire

2.1aEvidenceofhowUNIFEMpartners/othersusedknowledgeproducts/tools/processesthatweremadeaccessiblebyUNIFEMinrelationtoengenderingofconstitutions/laws/legalframeworks/policies/strategies

LimitedContextualisedSubregionalStrategyindicators2.1a)Numberofknowledgeproducts/toolsthatUNIFEMhasmadeaccessible.2.1b)EvidenceofhowUNIFEMpartners/othersusedknowledgeproducts/toolsthatweremadeaccessible

However,despitethestrongnarrativecoherenceacrossprogrammaticandstrategicdocumentationwiththeStrategicPlanandthetheoryofchange,theactuallevelofinfluenceoftheStrategicPlanonoperationalplanningatfieldlevelisquestionable.Mostalignmenttakesplacearoundtheresults

frameworks,andthereissomeclearevidenceofretro‐fitting.AccordingtoNoradinitsassessmentofUNIFEMAfghanistan‘TheStrategicPlanisbasedonUNIFEM’sglobalstrategyandtheoutcomesandlogframeshavebeenadoptedwithoutmuchadjustmenttothenationalcontextoractualactivities.’15WhiletheinteractionsbetweentheStrategicPlanandprogrammaticactivityareexpectedtobeiterative–inthatthetheoryofchangebothaimstoshapeprogrammaticdesignandtobeinformedbythis–thereislittleevidenceoftheseloopsbeingtestedandenactedinpractice.TheonlyevidenceoftheStrategicPlanshapingprogrammaticengagementoccursinneweroffices‐perhapsbecauseofthebreadthandgeneralityofthetheoryofchangeandoutcomes,aswellasthehistoryandimperativeofdonorfundedprogrammes.ExceptionsincludeMoldovaCO,whereaprogrammedesign–thoughnotactualprogrammeselection–tookplaceinaccordancewiththeStrategicPlan,andWestAfricaSRO,wherethesub‐regionalstrategydrawsitsprioritycountriesinlinewiththeStrategicPlan.TheReviewofUNIFEMSub‐regionalStrategy2008‐09concludedthat,generallyspeaking,SROsinthesub‐regionsseemtobeworkingintheplannedcountriesbutthatthelackofcountry‐levelplanningandbudgetinginthesub‐regionalstrategymadeitdifficulttoassesstheextentofthis.1ItalsohighlightstheabsenceofcorporateguidanceclarifyingSROflexibilitytoreacttochangesinregionalcontextsovertime.

Institutionaltractionandownership:FullownershipoftheStrategicPlanhasbeenconstrainedbyalackofbroadinstitutionalengagementinthedevelopment/validationprocessofthedocumentitself,asfollows:

• SeveralcountriesandSROsinterviewedreferredtoaveryunclearfinalisationprocess,withsomeundertheimpressionthattheywerestillworkingtoadraftdocument.Atcountrylevel,thislackofengagementhasplayedoutinthe

KeyFindings

15Norad(2009)AssessmentofUNIFEMAfghanistanDiscussionReport21/2009page1016ReviewofUNIFEMSub‐RegionalStrategies(2008‐2009)SynthesisReport,RevisedDraftReport,AlisonKing,30September2010

31

lackofacomprehensiveanalyticalandconsultationprocessunderlyingcountrystrategydevelopment,perhapsduetothetighttimeframesetfordevelopingthese.Somecountriesreportedseeingthisasamechanisticandartificialprocess,drivenbyHQ,whichdidnotallowsufficienttimeforplannerstoengageindialoguewithlocalpartnersortoembedplanninginconcurrentitemssuchasUNDAFornationalstrategicdevelopmentprocesses;

• Whileothersdidreportamajorefforttoensurecountry‐ledstrategies(e.g.someCIScountries,someinArabStatesand(incommonUNapproaches)thosecountrieswhereaDeliveringasOneinitiativeisunderway),overallthishasresultedinamissedopportunitytotest,validateandcontextualiseUNIFEM’sunderstandingofchangeatcountrylevel;

• Linkedtothisisaclearweaknessaroundtheinstitutionalisationofthetheoryofchange,includingtheGoal.Despitearangeofcommunicationchannels,includingbi‐annualglobalandregionalstrategicplanningworkshops,veryfewoftheofficesconsulted(beyondexceptionssuchasWestAfrica,SouthAsiaandAndeanSROs)hadgoodknowledge/understandingofthetheory(includingtheGoal),itsroleandfunctionandwhy/howitcansupportoperationalplanning,beyondthoseinvolvedintheirdevelopment.Thosewhohadcomeacrossithadoftendonesoviaotherroutes,suchasthroughachancereferencebyacolleague,orviadiscussionsaroundthecapacitydevelopmentstrategy.WhilethismaybeduetothelackofexplicitreferencetothetheoryofchangeintheStrategicPlandocumentitself,plusanyassociatedguidance,thesecommunicationchannelsclearlydidnotenableasystematicdialogueacrosstheorganisationabouthowUNIFEMunderstandschange,andthetestingofthisatfieldlevel.OneSROcommentedthattherealtestwastherelevanceofthetheoryofchangewithinprojects;

• Reflectingtheconcerninsection3.1above,manystafffeltthatthecurrentGoalwasnot

sufficientlymotivationalorreflectiveoftheworktheydoandwhy.Fieldvisitsalsoindicatedacleardisconnectbetweenthenarrativealignmentseenindocuments(asreportedonaboveinsections3.1and3.2)andtheactuallevelofawarenessandbuy‐inofoperationalstaff.Manystaff(evenwheretheywereawareofthem)statedthattheysawtheStrategicPlan/theoryofchange/Goalasrelativelyremotefromtheirdaytodayprojectandprogrammeworkinveryfluidandchallengingcontexts.

ResultsTracking:TheresultstrackingsystemhaspresentedanumberofchallengesforCOsandSROs,aswellasforthematicandgeo‐sections–despitetherecognitionthatisdoesallowofficestoreporttheircontributiontoglobalaimsdirectlytoHQ.Specifically:

• Forfieldoffices,theheavyfocusonStrategicPlanindicatorsasthesingleformofresultsmeasurementhaspresentedamajorchallenge.COsandSROsarenotableunderthecurrentresultstrackingsystemtopresentabroaderpictureofthechangesbeinggenerated(orthedifferentsortsofpartnerships/interventionsrequired)which,asreportedabove,oftenconsistofvaluable(oftenprocess‐oriented)shifts(seeexamplesinsection3.2above).WhilethereisnoevidenceofCOschangingorconstrainingactivitytofitwithresultstrackingrequirements,thereiswidespreadevidenceoftheconverse:COsreportinginonlyalimitedwaycentrallyonchangesbeinggeneratedbecauseofthelimitationsofthecurrentsystem;

• Theindicatorsthemselvesareastickingpoint.

Institutionalbuy‐intothemhasnotbeenhelpedbyanongoingprocessofredefiningandrefinementwhich–thoughessentialinitself‐hasprovenconfusingforCOsandSROs.Thefactthatmuchprogresscanbe(andis)‘dicedandspliced’tofitanumberofdifferentindicatorsandoutcomeareasdoesnotservewiderresultsreportingwell.Thelackofguidanceondevelopingcontextualisedversionsofindicatorshasalsobeenaconstraint,sincethefixedandoftennarrownatureoftheindicatorsdoesnot

KeyFindings

32

reflectthesortsofrapidresponsesneededinquickly‐changingcontexts(citedbyColombia,Haiti,PakistanandPalestine).Overall,thelackofscopeforbroaderresultsreportingisconstrainingtheshifttowardsacultureofmanagingforresults;

• Thesystem’sinabilitytogeneratelocally‐

relevantreportsisasignificantconstraintforfieldoffices.Ithasbeenakeyfactorinthelackofperceivedrelevanceandutilityofthesystem(thereisastrongperceptionthatthesystemexistsprimarilyfortheuseofHeadquartersandforthepurposesoftheAnnualReport).Allofthoseofficesvisitedforfieldstudies,andallofthoseinterviewed,citedthelackofanalysisavailabletothemthroughthesystemasaconstraint–and,atthetimeofconsultation(November/December2010),allwereengagedincollating/submittinginformationfortheAnnualReport(nonewereengagedinuploadinginformationonanongoingbasisthroughouttheyear);

• Anothershortcomingofreportingistheabsence

ofaready‐accountofthematicandoutcome/outputlinkages.Intheirsub‐regionalstrategy,CISincludesamatrixhighlightingtheseconnectionsthroughtocountrylevel,butrespondentsinthisstudyfrequentlycommentedonthechallengeofallocatingachievementsacrossthemesandoutcome/outputareas;

• Therequirementforofficesaroundthe

submissionofevidencefortheachievementofresults,whilerecognisedasnecessaryinitself,hasprovenasignificantburden.Allthoseofficesconsultedfounditchallengingtoidentifythespecific,up‐loadablepieceofevidenceto‘prove’anygivenresult(andarenotalwaysclearontherequirementsfor,anduseof,thismaterial,despitesomecurrentguidance).TheReviewoftheSub‐regionalStrategyalsocommentedonthedisproportionatetimespentonthisexercise,inviewofitsuncertainvalue(lackofclarityon

extenttowhichdocumentsuploadedarebeingassessedandusedbyotherpartsoftheorganisation).17

Financialreporting:WhiletheStrategicPlanaimedtocreateaclearlinkbetweenprogrammaticoperationsandexpenditure,theevidenceisthattheseintentionshavenotdeliveredasplanned.Specifically:

• Staffinfieldofficescitedthelackofalinkbetweenprogrammaticandfinancialdataasadrawbacktoplanningandreporting.AllocationofresourcesagainstATLAScodesishighlyvariable,dependingonindividualinterpretationofeachcode,andstaffresponsibleinSROsagreedthattheallocationandentryprocessisalargelyintuitiveone.Accordingly,thedatathatisheldatcentrallevelonexpenditureagainstoutputsandoutcomesisveryunlikelytoberobust,andtheRBM‐desirablegoalofasingleperformancereport,whichmeaningfullycombinesprogrammeandfinancialinformation,hasnotyetbeenachieved;

• Thereisalsoneedtoclarify–andreflectintheIRFformat–whetherEndingViolenceAgainstWomen(EVAW)andHIV&AIDSarebeingtreatedasonecombinedortwoseparatethemesforplanningandreportingpurposes.

Informationflows:Whiletheresultsreportingsystemsaimstogeneratebothupwardsanddownwardsinformationflows,itiscurrentlynotbeingutilisedtoitsfullpotential.Informationflowsarelargelyupwards,contributingtotheperceivedlackofutilityofthesystematfieldlevel.Specifically:

• InformationflowsfromCOsthroughtoSROsarelargelyupwards,withsomelinkagetogeo‐sectionsandsomelesserconnectionstothematicdivisions.Noclearcasesofdownwardsfeedbackflowsandloopswerereported,withstaffcommentingexplicitlyontheabsenceofthese,andtherehasbeennotrendand

KeyFindings

17Opcitpage8

33

performanceanalysisconveyedbacktoCOsandSROsthusfar;

• Staffindicatedthatthecontactthattakesplace,e.g.fromcountrythroughtothematicorregionallevelislargelybasedoninteractionbetweencolleaguesasdevelopmentprofessionalssharingtechnicalexperienceandexpertise,andoftenrootedinexistingrelationships.Itisnotsystematicallylinkedtoresultsreporting.Thereisalsoevidenceofalackofalignedprogramming,withoneofficecitingtheexampleofcountrylevelprogrammesbeinginitiatedfromHQwithoutcontactbeingmadewiththeSRO;

• Staffatfieldlevelcitedinstancesofseekinginformationorclarificationaroundresultstrackingissuesfromheadquarters,withnoresponsereceived;

• Forprovidinginformationtopartners,theStrategicPlandocumentitselfisnotperceivedbystaffasausefultool–itistoodenseforeasycommunication.Theneedfora‘communications’versionwascitedonseveraloccasions.

Structuralissues:Inadditiontothetoolsandsystemsissuesmentionedabove,therealisationofthechangelogic(theoryofchange)‐andconsequentlytheStrategicPlan‐atsub‐regionalandcountrylevelhasalsobeenimpededbythreestructuralissuesinUNIFEMoperations:

• Firstly,thelegalstatusoftheUNIFEMofficeatcountrylevel.WhereUNIFEMdidnot,inseveralcases,haveadirectlegalbasisforactionatnormativelevelandconsequentaccountabilityto,e.g.nationalgovernments,thismeansthattheUNIFEMofficedidnotalwayshavethelegalorpoliticalspacetodirectlyengageinthewaythattheStrategicPlanunderstandingofchangeimplies.TwocaseswerecitedwhereUNDPisthelegalentityinthecountry;therespectiveUNIFEMcountryrepresentativewasheavilyinvolvedindiscussionsaroundthenewUNDAFbutcouldnotformallysigntheUNDAFonbehalfofUNIFEMduetothelackoflegalstanding.All

o GiventhebreadthandambitionoftheaimsoftheStrategicPlan,thelevelofresourcesrequestedintheStrategicPlanappearsinadequatetoattainthis.BeyondtheStrategicPlanandIRF,thisisevidencedinthefundingandhumanresourceframeworksofSRO/COsandtheiraimsandambitionsasevidencedinstrategicplans(sub‐regional/countrystrategy);

o Theinsufficiencyofcoreresourcesisactively

constrainingtheflexibilityofoperations,particularlyatcountrylevel,e.g.intheoptiontoengageinstrategicpartnerships–citedbyatleastoneCO(Colombia).CountriesinCEEfaceparticularchallengesasdonorschanneltheirsupportintocross‐Europeanstructures;

o DuetoUNprocurementdifficulties,theuse

ofannualcontractsforthedeliveryofcoreUNIFEMbusiness(themajorityofthestaffinmostoffices)meansahighlevelofinsecurityandturnover,withconsequentimplicationsforstakeholderrelationships/officecapacity(thiswasarepeatedthemewithinthefieldstudies);

o Thereisanextremelyhighvolumeoftime

spentonbureaucratictasks(administration,logistics,with70%upwardsofadvisorytimebeingcitedinatleasttwoCOs),whichinmanycasesstemsfromtheroleoftheSROinfinancialcontrolandUNDP’sroleaslegalentity;

KeyFindings

officeswherethiswasthecasecitedmajordelaysandbureaucracyforprocurementasaresultofthisconstraint;

• Secondly,thereisaclearmismatchbetweentheavailabilityofresources–bothhumanandfinancial–andtherequirementsfortherealisationoftheStrategicPlanandthetheoryofchange.Thereareanumberoffactorsatplayhere:

34

o IncommonwithotherUNagencies,thereisaculturewhichappearsoverlyfocusedoncomplianceratherthanresults(sustainedinsomeinstancesbyseparateannualworkplanningmilestonesandprojectandindividualworkplanswhichareactivityfocusedratherthanbeinglinkedtosubstantiveresultsframeworks);

o Therehasbeenaninsufficientinvestmentin

monitoringexpertiseandresultsbasedtrainingforstaffwhoareexpectedtoimplementtheStrategicPlan–nostaffconsultedinfieldvisitshadexperiencedsignificanttrainingineitherarea.


ExperiencetodateshowsthatwhiletheStrategicPlan,itsresultsframeworksandassociatedsystemshaveprovenbothapowerfultool,andavaluableprocessforUNIFEMonitsjourneytowardsaRBMorientation,somesignificantlimitationsremained.Keymessagesinclude:

• TheStrategicPlananditssystemsprovidedaclearorganisingframetomakeexplicitUNIFEM’sworktoitsstaffandpartners,tosetclearaimsandobjectivesandtoassessprogresstowardsthese.Perhapsmostsignificantlyithelpedtogenerateinstitutionalcoherenceandconsistencyacrosstheorganisationinitseffortstomeetthechallengesofitsmandate.ItalsoenabledmoreeffectivecommunicationofUNIFEM’sworktopartners(thoughtheStrategicPlanitselfisnotausefulcommunicationstool);

• However,thepotentialvalueofthetheoryofchangewasnotbeingmaximized.Itisacceptedasconceptuallyvalidbuthasnotbeencontextuallyvalidated‐thatis,theredoesnotexistadequateinternalisationandbuy‐inatcountryorsub‐regionallevel.TheStrategicPlanGoalisconsideredinadequateandtolackrelevancefortherangeofactivitiesthatneedtohappenonthepathtowardstherealisationoftheUNIFEMmandateincountries;

• TheStrategicPlanappearstobelargely

providingaconceptualumbrellaforthematicandstrategiccoherence,ratherthanactingasastrategicdriverforoperations.ThebreadthandgeneralityoftheUNIFEM’stheoryofchangeanditsoutcomesmeanthatmostUNIFEMactivityfitwithinthese,evidencedbythesuccessfulretro‐fittingofalignmentexercises.Whilethereispotentialinnewerofficestobeguidedbythetheoryofchange,itsbreadthmeansthatvirtuallyanyactivityaimedatcontributingtoGEWEwouldfitwithinit;

• Theresultstrackingsystemlacksadequate

institutionalbuy‐inatfieldlevel,andisconstrainingreporting(aswellasinsomeinstancesprogramming)onchangesgeneratedandprogressmade.Alackofanalysisflowingdownwardscontributestoawidespreadperceptionofinadequateflexibilityandlackofutilityatcountrylevel.Guidanceonitsusehasbeeninadequate.Financialreportingagainstoutputareasisunlikelytoberobust;

• Critically,UNIFEMresources–humanandfinancial‐appearoutofsyncwiththeaimsandambitionsoftheStrategicPlan.Thereisinadequatecoherencebetweenthetwoaspects(IRF‐DRFcentrally;sub‐regional/countrystrategyandresourceplans/staffingallocationandcontractbasisatfieldlevel);

• Broadlytodate,attentiontoplanningand

managementforresultshasbeendominatedbyanemphasisonresultsmeasurementinstead.Consequently,informationflowsarelargelyupwards;reportingagainstindicatorshastakenprecedenceoverintensiveanalysisonthetypesofchangeintendedandthemosteffectivewaysofachievinganddemonstratingthis;andthesortsofprocessshiftsrequiredtogeneratelong‐termsustainablechangeonthepathwaytowardsGEWEarenotcurrentlybeingeitheremphasisedorrecognised;

• Anumberoftheseshort‐comingsstemfromthegapbetweentheStrategicPlan’srecognitionofdevelopmentchangehappeningatthenationallevel,andthetop‐downplanningprocesses,

KeyFindings

35

wherebycountrystrategiesweredevelopedafterthecorporate,regionalandsub‐regionalstrategies,andsometimespurelyasaformality.

3.4 TheUNIFEMmandate:howhastheStrategicPlansupportedthedeliveryofUNIFEM’sremitaroundnormative/operationalactivityplusUNcoordinationprocesses?

Withintheanalysisabovearoundtechnical/systems/implementationissues,thestudywasaskedtoconsiderthespecificquestionsof:theroleoftheStrategicPlananditsresultsframeworksinhelpingUNIFEMrealisebothitsnormativeandoperationalmandate,plusitscommitmenttosupportingUNcoordinationfortherealisationofGEWE.Thesequestionshavebeenconsideredbothfromatechnicalanalysisanda‘realityofimplementation’pointofview.3.4.1Organisationalprogress

IntermsoftheStrategicPlan’sroleinsupportingUNIFEM’snormative‐operationalmandate,thestudyhasfoundsomesignificantprogress.Specifically:

• AnexplicitconnectionwithintheStrategicPlananditsunderstandingofchangebetweenthenormativeandtheoperationallevelsofwork,plusaclearemphasisonmicro‐meso‐macrolinks;

• TheGoalandalloftheoutcomesoftheDRFwith

clearpotentialtodriveanddemonstratenormative‐operationallinkages;

• Offices(whetherconsciouslyorunconsciously)

workingtogeneratenormativeandoperationallinks,andtomaximizemicro‐meso‐macroconnections,asfollows:

o Keynormativeframeworks(CEDAW,

Beijing,1325andHRBAs),aswellasnationalandregionalframeworks(e.g.EUAccessionandProgressReportsbyCEEandBelendoParaandtheBrasiliaConsensusbyLAC),explicitlycitedandusedbyofficestoinform

operationalaswellasnormativework.ReferencestoCEDAWandHRBA,inparticular,areprominentinprogrammedocumentsacrosstheregionsandwerecitedbyallstaffconsultedasthemainframeworkforguidingtheirwork;o Agrowingawarenessatfieldlevelofthe

needtobridgethegapsbetweenthenormativeandoperationallevels,plussomeprogrammaticevidenceofstrategiestoaddressthese(mostlyimplicit);

o Aclearunderstandingoftheimportance

oftheupwardslinkages,plusanincreasingdrivetoworkatnormativelevelespeciallyinfieldsites.ExamplesofUNIFEMinfluencingmorewidelyonthisincludetheMoldovaCO’sdevelopmentofapaperforUNCTonentrypointsshowinglinkstoCEDAWandMDGs.

IntermsoftheStrategicPlananditsresultsframeworkssupportingUNcoordinationforgreaterrealisationofGEWE,thefollowinggainsareevident:

• ProminentintegrationofMRFoutput2onUNcoordinationintostrategicplanningatglobal,regional,sub‐regional,countryandthematiclevels.TheSouthAsiaSROteamindicateda‘mindsetchange’whichhasbeenfacilitatedbythisoutputintermsoftheirapproachtoworkingwithUNCTs.TheInstitutionalDevelopmentTeamatHQengagessignificantlyonthisoutputandalsocoordinatestheworkdoneacrossUNIFEMonMRF2.Atcorporatelevel,theUnitedNationsDevelopmentGroup(UNDG)isthemainpillarthroughwhichUNIFEMoperatesandinfluencesandisveryengagedinanumberofworkinggroups(e.g.HRBAinEvaluation;RBM;UNDAFGuidelines,UNAccountabilityonGenderEquality);

• Evidenceofconsiderablerespectbypartnersfor

UNIFEM’stechnicalexpertiseinmainstreamingGEWEatnormativelevels,andgoodexamplesoftangibleoutcomesinachievingthis.These

KeyFindings

36

includeastrongerpresenceofGEWEacrosstheworkofthemedaidco‐ordinationgroupssuchasinLiberia;strongergenderindicatorsinUNCTmonitoringframeworksasinEcuador;andUNIFEMworkinAlbaniaindevelopmentofaGenderFrameworkfortheUNCTaspartoftheCommonCountryProgrammeDevelopment(2012‐2016).PartnersconsultedduringsitevisitswerehighlypositiveaboutthesensitiveandstrategicapproachthatUNIFEMstaffwereadoptingtoworkwithotherUNagenciesonGEWE,acknowledgingothers’comparativeadvantageandentrypointsandbeingwillingtoworkfrombehindthescenes;

• Atfieldlevel,officesplayingastrongandoften

criticalroleinmainstreaminggenderacrossUNDAFsandothercoordinationprocessessuchasDeliveringasOne(oftenviamechanismssuchasinter‐agencyworkinggroups,orstakeholderplatforms).UNIFEMcurrentlyleadson53GenderThemeGroups(outof1oocountries)andtechnicaladviceisalsoprovidedfromheadquarters;

• OpportunitiesappeargreaterwhereUN

coordinationispromotedthroughe.g.DeliveringasOnepilots.ThereisclearevidenceofUNIFEMoperatingstrategicallytomaximiseandutilisethisspacetoadvanceGEWE.Intheprocess,someofficeshavealsogainedrespectfortheirexpertiseinhumanitarian,peaceandconflictresolutionsituations,e.g.Sudan,PalestineandPakistan.


Notwithstandingthesegains,somelearninghasemergedforanyfuturestrategicplanningprocess.Overall,whileaconsiderablevolumeofworkaroundUNcoordinationandcarryingoutthenormative/operationalmandateistakingplace;thisisnotnecessarilybeingdrivenbytheStrategicPlan

(thoughtheStrategicPlandoesprovidelegitimisationifrequired).Inspecificterms:

• Thelegalstatusofthelocalofficeisamajordeterminantintheextentandnatureofengagementatnormativeleveldirectlyand,asregardscoordination.ExampleswerecitedofUNIFEMcountryprogrammesplayingamajorroleinUNDAFdevelopmentandinthemainstreamingofGEWEacrossthis,butUNIFEMrepresentativesthenbeingunabletoformallysigntheUNDAFonitsfinalisation(e.g.theColombiaCO);

• UNIFEMleadershipandroleinbothnormativeworkandUNcoordinationwasheavilydependentonthelocalofficecommunicatingandstrategicallydeployingitsmandate–whichappearstohappenbyvaryingdegrees.Inthefieldsites,acleardisconnectwasevidentbetweenrecognitionoftheUNIFEMmandateandexpertiseforGEWEandtheactualStrategicPlandocument.Severalofficesandsomenon‐UNpartnersinterviewedduringsitevisitsaswellassomedocumentationreviewed18pointoutthatacceptancewithintheUNCTofUNIFEM’sroleasadriverofGEWEisnotautomatic.IfUNIFEMstaffwereunclearabouttheroleof‘driver’‐whichisstatedbutnotexploredintheStrategicPlan‐thenotherUNagencieswillbeevenlessclear.BroaderchallengestoUNcoordination,suchasstatus/hierarchy,territorialismandactual/perceivedcompetitionforresources,alsoaffectthisrelationship.SomeCOsreportstormytimessteeringacourse,whileothersreportthattheyarenowinagoodsituationfollowingseveralyearsofeffort;

• WhilstMRF2isreportedonviatheresults

trackingsystem,thereislittleevidenceofitfunctioningasadriverfortheroleofUNIFEMwithinUNcoordination.Rather,thisappearstobehappeningprimarilyasaresponsetocontext,

18Norad(2009)AssessmentofUNIFEMAfghanistanDiscussionReport21/2009andSocialDevelopmentDirect(2010)LightTouchReviewofUNIFEMProgressagainst2008‐2011StrategicPlan(reportforDFID)

KeyFindings

37

andtobedrivenbythetechnicalandstrategiccapabilitiesplusthestatus(e.g.theirgrade/contractstatus)ofin‐countrystaff.LinksstillneedtobemadebetweenthisMRFoutputandtheDRF;

• FieldvisitsandinterviewsindicatedmuchevidenceofUNIFEMbeingactiveinUNCTs(e.g.CIS,Pakistan,Colombia,AndeanSRO)andofactivelysupportingattentiontogendermainstreaminginUNDAFs(Albania,AndeanSRO)andtheevaluationofUNDAFs(Moldova).TherearealsoseveralinstanceswhereUNIFEMwasunabletoplayanactiverole,evenwheninvited(e.g.Syria),duetoalackofpresence/resourceconstraints.TheCISRO,hassucceededinestablishingfivepostsofGenderAdvisortosupportUNCTsinanumberofcountries.Staffincountrypointoutthechallengesoftakingonalead/coordinatingrolewhenpresenceisslim–perhapsjustoneperson;whilepreviouslysucharolemighthavebeenplayedbyUNFPAwithconsiderablygreaterhumanresourcestoundertakethetask.AnActionLearningProjectisunderwayinthreecountriesovertwoyearsandshouldprovideuseful–andmuchneeded–informationongoodpractice.ThecurrentlevelofinvestmentinUNcoordinationhasnotbeenassessedfromavalueformoneybasis–whichmightbeaworthwhilestudytoundertake;

• Whilstnormative‐operationallinksarebeing

workedonprogrammatically,thereareveryfewexplicitorconcreterationales/strategiesidentifiedwithinRSs,SRSsandCSsforstrengtheningthelinkagesbetweenthese(thoughsomeRSs/SRSs,suchasthatforAfricaandtheEastandHornofAfricaSRS,doatleastrecognizetheneedformoreattentiontothegenerationoftheselinkages.)Therationaleforthedifferentlevelsofwork(micro,meso,andmacro)isalsoonlyrarelyexplicitinSRSsandCss,andwhileevidencepointstoworktakingplaceatthesedifferentlevels,thethinkingarticulationofstrategiesisunclear;

• Feedbackloopsfromthenormativetothe

operationalareconstrained‐therearesome

instancesofnormativeworkinformingoperationalplanningandactivity(e.g.throughpartnershipswithstateactorsonregional‐levelprogrammingorsupporttonetworks),butfewofoperationalworkinformingnormativeactivityotherthanthefeedinginofgoodpractice,e.g.experiencewithOutcome8operationalactivitiesinformingpolicydialogue;

• Thereappearsatfieldleveltobelittlereference

toECOSOCnormativeframeworksortoSecurityCouncilresolutions(beyond1325);

• Theemphasisoftheresultstrackingsystemon

specificindicatorsmeansthatthelonger‐termprocesses,whichleadtostrongernormative‐operationallinksarenotalwaysbeingcapturedinresultsreporting.Outcome8inparticularprovidesanopportunity,butconsultationsindicatedthatreportingonthisresultareahasbeenhamperedbydifferencesinitsinterpretation,i.e.about(i)attitudinalchange,e.g.aroundHIVandGBV,or(ii)modelsandsouth‐southexchange.Thereisdemandforsomeillustrationsofgoodpracticerelatedtothisoutcomeandhowtosupportthelinkages.


Aconsiderablevolumeofworkistakingplacetoaddressnormativeandoperationallinkages,andtosupportthemainstreamingofGEWEwithinUNcoordinationprocesses,atheadquartersandfieldlevels.TheStrategicPlan(astheframeworkforthedeliveryofUNIFEM’smandate)hasprovidedandlegitimisedaplatformfordiscoursewithpartnersontheseissues.However,ithasnotactedasadriverforstrategicoroperationalactivity:

• TheStrategicPlandoeshighlightthenormative‐operationallinkagesandtheroleofUNIFEMinUNcoordinationfortherealizationoftheGEWEmandate.Howeverstrategiesandmechanismsforoperationalisingthesecommitments;guidanceand,particularly,disseminationofexamplesofgood/promisingpracticedonotadequatelysupportthiswork;

• WhileUNIFEM’smandateappearstobewell

KeyFindings

38

understoodbypartners,includingotherUNagencies,theStrategicPlanismoreofaninternalplanningdocumentandisthereforenotgenerallywellknown.Thereiscurrentlynostrategyorguidanceoncommunicatingthis;

• UNIFEM’sroleasadriverofGEWEwithinUNCTs

isnotalwaysunderstoodoracceptedbyotheragencies.Thereisscopeforclarifyingthis(andtheconsequencesintermsofresourceallocation)aswellasreviewingvalueformoneyfromcurrentlevelsofinvestmentincoordinationfora.

3.5 Theinformationbase:TowhatextenthastheStrategicPlandeliveredaninformationbasetosupportanylaterevaluation?

ThissectionofthereportconsiderstheinformationbasegeneratedthroughtheStrategicPlanandit’sresultsframeworks,whichwouldfacilitatelaterevaluation.Itfindsthatoverall,theimplementationoftheStrategicPlananditsresultsframeworkshasdeliveredapartial–thoughfarfromcomprehensive–evidencebasetosupportanysubsequentevaluation.Thisevidencebaseiscomprisedof:uptodateperformanceinformationontheStrategicPlanresultsandprocessareas;verylimitedbaselineinformation;andsome(thoughstillfragmented)evaluationandresearchevidence.

3.5.1OrganisationalprogressThecorporateeffortdevotedtoembeddingtheresultstrackingsystemhasresultedinastep‐changefortheorganisationingeneratingacoherentevidencebasetosupportlaterevaluation.Themainelementsofthisare:

• Theavailabilityofrecentandfullyaligned

performanceinformationonthekeyresultandprocessareasoftheStrategicPlan.Throughtheonlinetrackingsystem,datafor2008and2009

arepresent,howeverthosefor2010formostsampledofficeshadnotbeenuploadedatthetimeofwriting;

• Performanceinformationwhichincludesthe

aggregationofdatafromCOstocorporatelevel

viaSROsandregionaldivisions,reflectedintheproductionoftheAnnualReportandreportstotheConsultativeCommitteeandExecutive

Board;

• Theemergenceofsomebaselinedata,particularlysupportingOutcomes1,2and4on

KeyFindings

Box3:ExperienceofEVAWUNTrustFundandGenderEqualityFundwithbaselinesForthelongerestablishedUNEVAWTrustFund,thequalityofbaselinesisfoundbystafftovaryconsiderably,asdocapacitiesoforganisationstodevelopthem.StaffnowencouragethedevelopmentofcountrybaselinesusingacademicandNGOexpertise,whichcanlargelyprovidetheinformationneededforindividualgranteeproject/programmebaselines.TheFundforGenderEqualityisatthepointofrequestingbaselinestobeproduced,havingallowedtimeforbaselinestudiesanddatacollection.Externaltechnicalexpertisewasprovidedtogranteesindevelopinglogframes,includingworkonindicatorsandbaselines.WhileneitherFundhasattemptedtocreateacentralisedbaseline,bothareinvestinginoverallassessmentofachievements.TheUNTrustFund(with83activegrantsin72countries)iscurrentlylookingatoutcomesachievedusinganoutcomemappingandverificationapproachthatincludeslookingatbaselines.TheGenderFundiscurrentlyworkingwith40granteestoagreecross‐cuttingindicatorsandaligngranteelogframeswiththeoverallFundM&Eframework.

39

3.5.2Challengesidentified:

Despitethesegains,anumberofchallengesremaintotheevidencebase,whichwouldneedtobeaddressedinpositioningUNWomencomfortablyinrelationtotheproductionofitsevidencebaseforlaterperformanceassessment.Theseinclude:

Overall,theperformanceinformationavailableunderOutcomes5‐8todateislesscomprehensive.Factorsinclude:

• Thechallengesofuploadingevidenceof

progressintheseareas,whichareoftenlesstangibleandrequireoffice‐leveldecisionsonwhatconstitutesprogressagainstindicators(see

critiqueinAnnex4);• ThedefinitionchallengesaroundOutcome3,

KeyFindings

19MOWA(2008)WomenandMeninAfghanistan:BaselineStatisticsonGender

thenormativeframe–forexampleallofthe

sub‐regionalstrategiesreviewedcontainsomenormativebaselinematerial‐andevidenceofsomestrongcountryexamplesofbaselines,such

astheColombiaCOmonitoringmatrix–thoughthishasnotbeengenerallytakenupandusedacrosstheoffice.Thereiswidespread

recognitionoftheimportanceofbaselinesandadesiretoestablishthem,thoughcoverageisgenerallymoresystematicatprojectlevel,often

inresponsetodonordemand.ThereisaquestionoverwhetheraclearrationaleexistsforbaselinesatSROlevel:Oneofficefeltthat

resourceswouldbemoreusefullydeployedintogeneratingprogrammeandcountry‐levelbaselines;

• Someinstancesofhighqualitycontextanalyses,

whichprovideanarrativeformofbaseline

information,suchastheCaribbeanSRSandtheEVAWstrategyplus,e.g.AlbaniaCO,whichhasundertakenanumberofcountrystudiesand

statisticalreviewstosupportcountryplanning(thoughnotyetpulledthesetogetherinto

baselineform).Anumberofoffices,e.g.Afghanistansupporttheproductionofnationalbaselineinformationongender,whichshould

informCSbaselines;19• AMeta‐evaluation(2009):whichcovers2004‐

07plus2008ofthecurrentStrategicPlan,wasdesignedtobeStrategicPlanalignedandtointegratehumanrightsandgenderequityinevaluationdesign.Thisdrawsoutagooddealofsynthesisfromthereportsreviewed.Sincethisreviewedworktookplacein2008orearlier,thereportcould,therefore,arguablybeconsideredasrelevanttotheStrategicPlanbaseline(forbothDRFandMRF);

• Generallymoresystematicbaselinecoverageatprojectlevel,dueinlargeparttodonordemand.Clearexampleswereprovidedfromthefieldvisits,suchasthetwoFundsoperatinginColombia(UNEVAWandtheGenderEqualityFund),whichemployclearbaselinesandtargets.InformantsfrombothFundsindicatethatbaselinesarearequirementandtheireffortsonthisscoreareillustrative–seeBox3;

• Duetothestrongalignmentoftheevaluations

systemwiththeStrategicPlan,detailedand

relevantmonitoringandevaluationplansatSRO/thematiclevel,e.g.thatproducedbyAndeanSRO,andtheemergenceofsome

evaluationswhichcoherewiththeStrategicPlanresultareas.

Additionally,thesystemsprogressmadesofarhasenabledcentral‐levelreporting–afactorwhichinitselfwouldsupportlatercorporateStrategicPlanevaluation.

40

whichisanareaofweakness(seeAnnex4);• ThelackofanalysisfeedingdowntoSROsand

COsbeyond,e.g.PACsandtheStrategicPlanningWorkshops,whichactsasadisincentiveforidentifyingareasofweaknessandintensifying

effortsaccordingly.Secondly,thecurrentsystemdoesnotallowfortheaggregationofdataovertime–thatis,whileaggregationupfromcountrytocorporatelevelispossible,reportingagainstanyparticularoutcomeover,e.g.athreeorfive‐yearperiodisnotcurrentlyfeasible.Suchreportingisessentialtodemonstrateaggregateprogressoverstrategicplanningandfundingperiods,andisparticularlyrelevantatcountryandcorporatelevels.

Thirdly,baselinedevelopmenthasbeenanareaofverylimitedprogress–thereis:

• Noclearstartingpositiononresults(with

targets)tobeachievedovertheStrategicPlanperiod(thoughtheMYFFevaluationasindicatedcouldpresentausefulstartingpoint);

• Noclearrationalefor,orstatementof,what

constitutes,a‘baseline’;afactorthathas

constrainedSROsandCOsintheirefforts.Cleardistinctionsemergedfromthefieldvisitsbetweenconceptualisations,e.g.someoffices

consideredanarrativebaselineadequatewhileothersemphasisedtheuseofstatisticaldataparticularlyagainstthenormativeframe;

• Agenerallackofcorporatedemand/

comprehensivedriveforbaselinesfromwithin

theorganisation,includingalackofcentrally‐issuedguidance–noneoftheofficessurveyedreportedrequestsforbaselineinformationfrom

HQ;• Nosystematicattempttoextractbaseline

informationtounderpintheMRFandDRFfrom,e.g.theMYFFevaluation.

Finally,thereisnocomprehensivepictureasyetofevaluationevidenceontheStrategicPlanchangeprocesses:

• Researchandevaluationsarestilllargelyproject/thematicallyfocusedandheavilydrivenbydonors.Effortsarebeingmadetoplanandmapevaluations,e.g.atsub‐regionallevel(andparticularlywhereregionally‐basedEvaluationUnitstaffareinplace,suchasinAndeanSRO),butthebodyofevaluationevidencesofarismainlyconstructedaroundtheseparameters;

• Whilethestrategiesandplansaroundevaluation

arestrongandcoherentwiththeStrategicPlanandrelevantresultsframeworks(mainlytheDRF),theimplementationoftheprocessisoftenweakerintermsoftheutilisationofevaluationfindings/theextractionoflesson‐learningtosupportstrategicandprogrammaticplanning(beyondthePACprocess).Thereisnoevidenceyetofacomprehensivelesson‐learningprocessfromevaluationswithinoffices,sectionsorglobally,andthestudyfoundinstancesinofficeswherecolleagueswerenotawareofevaluationsconductedbyprojects/programmesinareasofworkbeyondtheirown.


Overall,theprogressmadetodateinsettingparametersfortheevidencebase,andpartiallypopulatingthis,haspositionedUNIFEMsomewhatclosertowardspossessingtheinformationneededtoassessprogressinanylaterevaluation–thoughgapsremain.Specifically:

• ThesystemsandprocessessetinplacebythestrategicplanningprocesshaveenabledthegenerationofafarmorefocusedandcoherentevidencebasetosupportlaterevaluationthanwaspreviouslythecaseundertheMYFF.Attheveryleast,recentandalignedperformancedataisinplace,whichisaggregatedupfromcountrytocorporatelevel;

• However,thisbaseisnotyetcomprehensiveand,asitstands,wouldonlypartiallyallowforassessmentofprogress.Thesystemdoesnot

KeyFindings

41

allowforaggregationofinformationovertime(e.g.progressagainstaparticularoutcomeatcountryorcentrallevel).Thereisnoclearstartingpositioncentrallyonanticipatedresultsincludingcleartargetsandthetimeframewithinwhichtheywillbeachieved,andnoclearguidancecentrallyonbaselines,includingexpectations,toolsandreportingrequirements;

• BaselinesareanotableareaofweaknessforbothDRFandMRF,beyondinformationonthenormativeframesupportingOutcomes1‐4andprojectandprogrammebaselinedata(thoughthelatterisfarfromcomprehensiveandlargelydrivenbydonors);

• Anevaluationevidencebasewhichisstartingtoproduceresultsbutwhichcurrentlylacksasystematicapproachto(i)generatingevaluationinformationaroundareasofstrategicinstitutionalinterestand(ii)joiningupthefeedbackloopsbetweenevaluationreportsandstrategicplanningandoperations;

• An(asyet)absenceoflongitudinalstudiesor

clarityonspecificstudiestogenerateinformationtomeettheneedsofstrategyevaluation.

KeyFindings

42

OverallConclusionsandLessonsLearned

Thissectionbringstogetherthemainconclusionsfromthefindingsoutlinedinsection3,wheremoredetailcanbefoundinsummarymessagesattheendofeachsub‐section.ItthenidentifieslessonslearnedaroundtheStrategicPlan,thestrategicplanningprocessandsystemsforimplementation,whichhavearisenfromthefindingsandconclusionsabove.Theselessonshavebeenextrapolatedtohavebroaderrelevance,andinparticularwiththeaimofgeneratingaclearevidencebaseforUNWomenstrategicplanninggoingforward.

4.1 Overallconclusions

1. TheStrategicPlan(2008‐2011)andthe

understandingofchangeonwhichitisbasedprovidearelevant,appropriateandconceptuallysoundarticulationofUNIFEM’scoremandateonachievementofGEWE.However,therearesometechnicalweaknesses,whichconstraineditseffectiveimplementation.Theseinclude:

a. Thereisinsufficientdevelopmentofthe

understandingofchangeandresultslogic

b. Thereisinadequaterecognitionof

potentiallydifferentpathwaysofchange,e.g.thosewithinconflict/post‐conflictsituationsandinmiddleincome

contextsc. Thereisverylimitedacknowledgement

withintheStrategicPlanofthe

assumptionsandrisksthatunderlietheprocessesofchange,suchasthepoliticalfactorsthatdriveorhinderthis

d. Therolesofkeystrategies(e.g.knowledgemanagement,capacitydevelopment)inlinkingoutcomesand

progressingchangearenotsufficientlyelaborated

e. Despitetheconsiderableimprovement

fromtheMYFF,specificresultsand

neutralandmeasurableindicators/concretetargetsarenotconsistentlyappliedwithintheStrategicPlan

framework,nordoesithaveabaselinetosetastartingpoint

f. Theprimacyofcountrystrategy

planninghasbeenarelativelyrecentdeparturewithinthestrategicplanningprocess.

2. ThesystemsdevelopedforStrategicPlan

implementationrepresentamajortransformationinthewayUNIFEMhassoughttointroducearesultsbasedcultureandtocollectandanalysedataforperformancemonitoring–thoughimplementationhasnotdeliveredtheresultsorientedcultureenvisaged.Specifically:

a. StrategicPlanguidancematerialtosupportsectionsandofficestodeveloplinkedstrategiesisgoodpracticebutcanbeimprovedonintermsoftimelinessandcomprehensiveness

b. Theresultstrackingsystemenabledcomprehensiveresultsreportingatgloballevelbutitsmaingearingtowards(centralised)annualreportinghasconstraineditspotentialutilityformorelocally‐relevantperformancemanagementandtrendanalysis

c. Limitedanalysisoftrends/progressanddownwardsfeedbackloopsisconstrainingtheflowanduseofvaluableinformation

d. Thereisaneedtocomplementreportingonfixedandsometimesnarrowglobalindicatorstoenableofficesandsectionstoreportonsignificantwiderchanges

e. Considerableprogresshasbeenmadeonresultsmeasurementbutpresent

43

systemsarenotyetgearedtosupportcomprehensiveresultsmanagement(throughoutthefullprogrammecycle).Inparticularmonitoringisnotwelldeveloped,thoughsignificantprogresshasbeenmadetomovetowardsresults‐focusedevaluation

3. TheStrategicPlan,itsresultsframeworksandassociatedsystemsprovidedaclearorganisingframetomakeexplicitUNIFEM’sworkandstrategicfocustoitsstaffandpartners.Theysupportedcoherenceandconsistencyacrosstheorganisation.However,theirpotentialvaluewasnotbeingfullyrealised(especiallyatsub‐regionalandcountrylevels):

a. TheunderstandingofhowchangewillbesupportedthroughtheStrategicPlanhasnotbeensufficientlytestedandvalidatedatlocallevel

b. TheStrategicPlanisprovidingaconceptualumbrellaforthematicandstrategiccoherence,ratherthanactingasastrategicdriverforoperations

c. Theresultstrackingsystemisnotadequatelysupportinglocalreportinganddecision‐making

d. UNIFEM’sstatusintheUN,inadequatestaffingandinsufficientpredictabilityofresourcesconstraineffectiveimplementation,andareoutofsyncwiththeaimsoftheStrategicPlan

e. TheprocessofdevelopmentoftheStrategicPlanandrelatedstrategies(suchascountrystrategies)tookplacerapidly,andwithoutthetimeperiodrequiredforfullengagementanddiscussionwithstaffandpartners

4. UndertheStrategicPlanaconsiderablevolumeofworkistakingplacetoaddressnormativeandoperationallinkagesandtosupportthemainstreamingofGEWEwithinUNcoordinationprocessescentrallyandatfield

level.However,thisisalsonotyetbeingconductedtoitsfullpotential:

a. WhileUNIFEM’smandateisgenerallyunderstoodbypartners,theStrategicPlandocumentisnotgenerallywellknownorcommunicated

b. Thereisaneedforclearerdefinitionoftheroleof‘driver’withinUNCTs,afocusonimpactaswellasprocessincoordinationwork,corporatecommitmenttoagreedarrangementssecured(e.g.providinghumanandfinancialresourcestocarryoutanagreedco‐ordinationrole)andthedistillationandsharingofgood/promisingpractice

c. WhiletheStrategicPlanhasfacilitatedstafftomakeexplicittheconnectionsbetweennormativeandoperationalwork,furtherworkisneededtostrengthentheselinkages,includinggenerationofanevidencebaseonwhatisworking/notworking

5. ThemeasurestakentoprovideevidenceofStrategicPlanimpacthasenabledthegenerationofamorefocusedandcoherentevidencebaseforfutureevaluationthanwasthecaseunderMYFF.Inparticularthereisnowup‐to‐dateandalignedperformancedatawhichisaggregatedfromcountrytocorporatelevel.However:

a. Thelackofclarityonstartingposition(baseline);anabsenceofcleartargets(incountrystrategiesaswellascorporateStrategicPlan)andinadequatemonitoringsystemsandcapacity,severelylimitsrobustandcomprehensiveperformancemeasurement

b. Theevaluationevidencebaseisgraduallydevelopingbutlacks(i)asystematicapproachtogeneratingevaluationinformationaroundareasofstrategicinstitutionalinterest;(ii)feedbackloopsbetweenevaluation


44

reportsandstrategicplanningandoperations;andlongitudinalstudies

4.2 Lessonslearned

Thefollowinglessonshavebeenextrapolatedfromthefindingsandconclusionsashavingwiderrelevance(includingtheUNWomenstrategicplanningprocess):1. Anorganisation‐widestrategicplanresults

logic,wherebytheunderlyingbeliefinhowchangehappensandthecause‐effectrationaleoftheplan,needstobeclearandarticulatedinappropriatedetail;thisincludestheidentificationofalong‐termgoalthatcapturesthewholeoftheorganisation’smandate(evenifthetime‐boundstrategicplan‐specificgoalissetearlierinthechangeprocess);clarityinresultsstatementsandtheirinter‐linkages(DRFandMRF);andindicatorsthatenableaggregatetrackingbutwhichalsosupportperformancemeasurementincontext.

2. However,itisimportantforacorporateStrategicPlantoprovideastrategicframeworkthatgivesdirectiontobutdoesnotconstraincountrylevelflexibilitytorespondtocontext.Thisimpliesabroadframeworkatcorporatelevel,includinge.g.clearoutcomestatementsandtargets,butwhichallowscountriestodemonstratethroughtheirownresultsframeworkshowtheywillcontributetothesechanges.

3. Developmentandvalidationofastrategicplanthroughconsultationandacomprehensiveplanningprocesshelpsbuildtechnicalquality,ownershipandinstitutionalbuy‐in.Understandinganduseoftoolssuchasthecorporatestrategicplanandanyresultsframeworks,includingacleartheoryofchange,andalignmentwithassociatedsystems(suchasworkplanningandmonitoring)ispredicatedonthis,andarelargelydeterminedbystaffinvolvementintheprocessoftheirdevelopment.Thisimpliesthatabroad‐based

participatoryprocessinvolvingrelevantstaffacrosstheorganisationandwhichtakesaccountofbottom‐upandtop‐downplanning,isessentialforanysuccessfulstrategicplandevelopmentprocess.

4. Astrategicplanwhichincludeshigh‐levelambitionsrequiresconcomitantinvestmentinresources–humanandfinancial.DespitesignificantfinancialinvestmentandincreasedfundingovertheperiodunderreviewinUNIFEM,performancebasedbudgetingorhumanresourceplanningisnotyetfullyinplace.Thereneedtobeveryclearandexplicitlinksbetweenastrategicplanandstaffing,professionaldevelopment,andelapsedtimetoconductaparticipatorystrategicplanningprocessandtoensureassimilationacrosstheinstitution.

5. Akeymessageistheprimacyofcountry‐levelforchange.Evenwithastrategicplanwhichisclearthatthelocusofchangeisatnationallevel,successfulimplementationatcountrylevelrequiresanumberofconditionsrelatedtostructures,systems(planning,appraisal,monitoring&reporting,qualityassuranceandguidanceandevaluation),capacities,thecontextualisationofthestrategicplanandtheoryofchange,andhumanandfinancialresourcing.(ThesemessagesarenotnewtoUNWomen,buttheirimplicationsneedtobeconsideredanynewstrategicplan)

6. Theimplementationofastrategicplanneedstobeaccompaniedbyanorganisation‐wideshiftfromresultsmeasurement/trackingtoresultsplanningandmanagement.Thisrequiressignificantinvestmentoftimeandresources.Overandaboveaccountabilityneeds,thereisaneedtoensurethatanysystem’scontributiontowardsabroadercultureofresultsmanagementsupportseffectiveuseofinformationtoguideandaccelerateprogress.Elementsinclude:clearfeedbackloops;investmentinsupport/guidance,monitoringandevaluationsystems(includingatthecountrylevel);therecruitmentandcapacitydevelopmentofdedicatedstaffand/orexternal


45

expertisetosupportthesesystems.Anysuchsystemalsoneedstoallowforreportingongainsbeyondindicators,recognisingtheprocess‐basednatureofchange,thefluidityofcontextspecificity,andtheneedtoaggregateupovertime.

7. Astrategicplancanprovidestaffwithavaluabletooltopromotetheorganisation’sremittostrengthennormative‐operationalconnectionsandtobe‘adriverofgenderequality’withintheUN.Whilesuchatoolenablesstaffinmanycontextstoactstrategicallyinleveragingthismandate,spaceandperformancearedependentonanumberoffactorsincluding:coordinationmechanisms;strategiesforengagingwiththese,andguidanceprovided;thestatusandcapacitiesofstaffandofofficeswithintheUNsystem.

8. Buildinganevidencebasetosupportresults‐basedmanagementandfutureevaluationisdemandingand,realistically,needstohappenincrementallyratherthanallatonce.Keybuildingblocksincludethesettinginplaceofanevaluationpolicyandstrategy;amonitoringandreportingpolicy;lessonsfromexperiencetoguiderefinementofperformancemeasurement;andthenecessaryfeedbackloopstosupportresultsmanagement.Forbaselines,clearinstitutionaldemandandclarityofpurposeandpragmatismareneeded.


46

Recommendations

Basedontheaboveconclusionsandlessons,thefollowingrecommendationsaremadetoUNWomenwhichmayhelp(i)improveanynewplanasaresults‐basedmanagementtooland(ii)supportanynewstrategicplanningprocessinordertostrengthenboththeplanandinstitutionalbuy‐in.Therecommendationsaresupportedbyproposalsforhowtheymightbeoperationalised,withdueregardforthecurrentinternalcontext,i.e.theprocessofconfirmingtheinstitutionalarrangementsofUNWomen.Theproposalsforoperationalisationpresentedare,atthisstage,optionstobeconsideredonly.Theycannotbemoredefinitivegivenlimitedengagementbytheconsultantswiththenewentity.Indevelopingtheserecommendations,theconsultantswereclearthatthemainbenefitofthisstudywillbethatoffeedingintoUNWomen’snewinstitutionalcontext,includingitsfuturestrategicplanandtheuseofresultsbasedtoolsandsystemstoimplementitsmandate.TherecommendationshavebenefitedfromengagementwithstaffthroughtheReferenceGroupcommentsonanemergingfindingspaper,andfromcommentsinwritingandinpersononalaterdiscussionpaperwhichsupportedaworkshopinJanuary2011.

47

Recommendations

Theme/Issue Recommendation ProposalsforOperationalisation

Overarchingrecommendation

Developacorporatestrategicplanbasedaroundacentralframework,includingcorporateleveloutcomesandtargets,butwhichallowscountriesandsectionstoflexiblyidentifytheircontributionstothis.Countriesandsectionsshouldthendevelopcontextspecificresultsframeworkswithclearstartingpoints(baseline);outcomeandoutputtargets(results)andthepathway(processandinterimsteps)towardsachievingthis.

TowardsaRobustStrategicPlan

1. Buildon,makemoreexplicitandvalidatetheresultslogicofanyfuturestrategicplanthroughelaboratingfurthertheunderstandingofchangetotakeaccountofmovementovertimeandtoclarifyrelationshipsbetweencorporateandcountrylevelresultslogic(includinganyin‐betweenlevels).

a. Developalogicmodel(buildingone.g.theresultslogicinAnnex6oftheStrategicPlanoraroundtheschematicinsection2)thatisclearbutsufficientlyflexibletoallowfordifferentpathwaysofchangeforcountries,thematicsections,majorstrategies.Includedinthisprocessshouldbethearticulationofassumptionsandrisksandofexternalfactorsandthematicandoutcomeconnects.

b. Additionally,developandapplyabroad‐basedPerformanceMeasurementFramework(whichidentifiesoutcomelevelresults;coreindicators,sourcesandresponsibilitiesfortracking)toallowforcapturingcorporatelevelstrategicplanresults.

c. Developmorespecificandtailoredresultsframeworksatcountry/thematiclevel,allowingforoutputs,yearonyearprogressmilestones/resultsandhowthesefeedintohigherleveloutcomes(withexplicittargetsandcontext‐specificpluscoreindicators).

d. Agreeanddisseminateaglossaryofdefinitions(forexampletheRBMTerminologyusedinUNCountryProgrammingandcontainedintheUNDGRBMHandbook(Sept2010);

e. Haveregular,mandatoryorientationofnewstaffthatfocusesontheStrategicPlananditscorporate‐levelresults

TowardsaRobustStrategicPlan

2. Furtherstrengthenthegoal,outcomeandoutputstatementsinlinewiththeresultslogic,includingalong‐termgoalthataddressesthevisionofGEWE(toembedthelonger‐termgoalofGEWEintotheStrategicPlanresultsplanningandlogic).

a. Whendevelopingindicatorsofachievement,buildonthequalitativeaswellasquantitativeindicatorsdeveloped;specifytargetswherepossible;andidentifysourcestotestmeasurability(thePerformanceMeasurementFrameworktoolin1.bcapturesthis)

b. DevelopalimitednumberofcorecorporateStrategicPlanindicatorswithadditionaltailoredindicatorstobedecidedatotherlevelsdependingoncontext

c. Supporttheprocesswithstatisticalexpertiseandguidance;d. Acknowledgethecomplexityofsocialtransformationandthe

needtobalancelinearcauseandeffectlogicplanningthroughmoreadaptiveapproacheslesseasilymeasured

TableofRecommendations

48

Recommendations

3. Requirethedevelopmentofbaselineswithinoneyearofstrategicplandevelopment.Developmentresultbaselinesshouldbeestablishedprincipallyatcountrylevelwhileamanagementresultsbaselineneedstobedevelopedcorporatelyandatcountrylevel.

a. ProvideguidanceandtrainingonbaselinesespeciallyforthefirstyearofthenewStrategicPlan,(includingforconflict/post‐conflictsituations)andensurethedisseminationofexperienceandpromisingpracticefromanyofthefourUNWomenentitiesandthetwoTrustFunds.

b. Clarifythatabaselinedescribesthestartingpointrelativetotheplanresults(itisnotageneralisedcontextanalysis).

TheStrategicPlanningProcess

4. Recognisethattheprimacyofchangeisatthecountrylevel–somergebottom‐upwithtop‐downplanningprocesses.TimingandsequencingofplanningwillneedtotakeaccountofinstitutionaldemandsforaUNWomenstrategicplanwithinashorttimeframe(adraftinApril2011forJunesubmissiontotheExecutiveBoard)

a. Developanddisseminatearoadmapforthecorporatestrategicplanningprocesstoinformstaffandclarifyconsultationprocessesandchannelstowhichstaffandstakeholderscancontribute

b. Intheprocess,distinguishtheessentialelementsofthecorporateStrategicPlanforExecutiveBoardsubmissionandthosethatcanbeelaboratedlaterinamoreconsultativemanner;

c. Devisemechanismsforconsultationandcommunicatione.g.representativegroupsfromfieldoffices,briefingnotes,regularintranetupdates

d. Developaroadmapforthecountrystrategyplanningprocessthatallowsflexibilityforcountryconditions/linksintoUNDAFandotherlocalplanningcycles

e. Extendguidancematerialsandtraining(examplesmightinclude:guidanceonanalysingtheenablingenvironment;onselectionofindicatorsandtargets)

TheStrategicPlanningProcess

5. DevelopacommunicationsstrategytoinformstakeholdersatalllevelsontheUNWomenmandate,strategicplanningprocessandstrategicplanincludingfurtherclarificationofthenormativeandoperationallinkagesandroleasa‘driver’ofgendermainstreaminginUNCT

a. Spellouttheroleof‘driver’ofGEWEintheUN,clarifyresponsibilitiesandmodalitiesandcaptureanddisseminategoodpractice

b. SpelloutclearlythenormativeandoperationaldimensionsofUNWomen’smandate;provideevidencetodemonstratethelinkagesandhowthissupportsbetterresults/morebenefitsforwomenandmen

c. ProvideHRBAGuidanced. Elaborate,validateandidentifyexperienceofsuccessfulwork

relativetokeystrategiessuchaspartnership;knowledgemanagementandworkingwithmen

e. ConsideraspecificfocusonlearningfromUNcoordinationworkatcountrylevelandonnormative‐operationallinkagestoguideanynewstrategicplanningprocess


49

Recommendations

Systemstosupportimplementationofthestrategicplan

6. Developandstrengthenthesystemsneededtosupportresultsmanagementthroughthefullprogramme/strategycycle(includingevaluation),keepingthecountryastheprimaryunitofchange–andsetandupholdminimumstandardsforthis.

a. DevelopandrefinecurrentStrategicPlanguidanceandensuretimelydisseminationtooffices(see4.eabove)

b. Developsupportmechanismssuchascall‐downtechnicalsupport;PeerLearning;CommunityofPractice;Resultschampions

c. Developresults‐basedmanagementcapacityatcountryandsub‐regionallevel,includingdedicatedexpertiseinlargeofficesortosupportsmallerofficesfromsub‐region/region

d. Clarifyresponsibilitiesformonitoringofresults(performancemonitoring)anddevelopapracticalmonitoringmanualthatisexplicitonrolesandresponsibilitiesofdifferentlevels;includessimpletoolse.g.useof‘trafficlights’forself‐assessmentofprogress;establishmonitoringprotocolsforfieldvisits/missions

e. Developandusedownwardsandlateralfeedbackloops(aswellasupwards)

f. Investinknowledgemanagementtoolsandtechniquesandmakeavailablebothinternalandexternalknowledgemanagementspecialistsupport

g. EmbedaresultsfocusinStrategicPlanappraisaltoolsandqualityassurancemechanisms(peersupportandreviewcanbeusefulforcoherenceandquality)

h. Developresults‐basedworkplantemplates/guidanceforofficesandindividuals

i. Alignthevarioustoolsandprocessesatdifferentlevelse.g.StrategicPlan;SRS,CS,resultsframeworks,AnnualImplementationPlans,IndividualWorkPlans,AnnualPlanning,AnnualReview,Programmemeetings,ProgrammeandFinancialReporting

7. Developandrefinetheresultstrackingsystemtosupportlocallevelmonitoringaswellassystematiccentralisedresultsreporting

a. Assesstherequirementsatfieldlevelandthetechnologyrequiredtomeetthese

b. Developcountryreportingguidance/templatethatmeetslocalandcentralneeds

c. Providetrainingtorelevantstaff,collectanddisseminateexamplesofgoodpractice


50

Recommendations

8. GiventhecomplexityofsocialtransformationrequiredforGEWE,introducecomplementaryapproachestoresultstrackingthattakeaccountofnon‐linearchangeandthepossibilityofunplannedconsequences(positiveandnegative)

a. To complement linear results logic, consider also tools fromapproaches such as OutcomeMapping andMost SignificantChange to support Strategic Plan articulation andimplementation (experience of the UN EVAW Trust Fund inoutcomemappingshouldbeillustrative).

IncreasingreadinessofanyfutureStrategicPlantobeevaluated

9. RetaintherequirementforanevaluationplaninStrategicPlan,includingcountrystrategies,butbasethisonstrategicprogrammeinformationneeds,ratherthanbeingdominatedbyprojectordonorrequirementsandrequirecompliance

a. Whileretainingobjectivity,channelEvaluationUnitexpertiseintoprovisionofguidanceonaddressingevaluationneedsthroughouttheprogrammecycle

b. Promoteimprovedmonitoringasakeyinputtoevaluation,includingannualandmid‐termreviews

c. AgreeminimumstandardsforevaluationsanddevelopandembedQualityAssuranceprocesses

d. Strengthenthefeedbackloopsbetweenevaluationreportsandstrategicplanningandoperations(beyondindividualprojects/programmes)

e. Stresstheimportanceofrobustresultsframeworks(especiallyatcountrylevel)underpinnedbybaselines,andwithcleartargetsandprogressmilestones/benchmarksasessentialelementsfordemonstratingcontribution


Date post:	26-Jun-2018
Category:	Documents
Upload:	phamkhanh
View:	212 times
Download:	0 times