January 2017

Evaluation of UNICEF Girls’ Education Project Phase 3 (GEP3)

Tom Pellens, Rachel Outhred, Michele Binci, Zara Majeed, Johanna Wallin, Fatimah Kelleher, Monazza Aslam and Shefali Rai

Draft Baseline Synthesis Report prepared by EDOREN on behalf of UNICEF GEP3

September 2016

Nailatu (12) attends primary school in the town of Toro, Bauchi State, Nigeria. Nigeria continues to face challenges in ensuring quality education for children – especially girls.

Photo Credit: © UNICEF/NYHQ2014-0710/Eseibo

DisclaimerEDORENisaconsortiumofleadingorganisationsininternationaldevelopmentandeducation:OxfordPolicyManagement(OPM),andtheInstituteofDevelopmentStudies(IDS)attheUniversityofSussex,andissupportedbyUKAid.EDORENcannotbeheldresponsibleforerrorsoranyconsequencesarisingfromtheuseofinformationcontainedinthisreport.AnyviewsandopinionsexpresseddonotnecessarilyreflectthoseofOPM,IDSandEDORENoranyothercontributingorganisation.

EDOREN No2,16MafemiCrescent Tel +2348107278718EducationData,Research&EvaluationinNigeria Utako Tel +2348176678243 Abuja,Nigeria Email info@edoren.org Website www.edoren.org i

Acknowledgements

Weareverygratefultogovernmentdepartmentsandstaffinthefiveprojectstatesaswellasatnationallevel who facilitated the baseline research and shared their perspectives with the evaluation team, inparticular: the State Ministries of Education, State Universal Basic Education Boards (SUBEBs), StateAgenciesforMassEducation(SAMEs),CollegesofEducation,andtheGenderUnitoftheFederalMinistryofEducation.

WewouldliketothanktheUKDepartmentforInternationalDevelopment(DFID),whichcommissionedthisindependentevaluation,particularlyDFIDEducationAdviserMsLauraBrannellyand theDFIDEvaluationAdvisersforNigeria,MsKristenHopkinsandMrLaurieThompson,forusefuldiscussionsandadviceontheevaluationdesignandimplementation.

Thanks also go to the implementing organisation, the United Nations Children Fund (UNICEF): TerryDurnnian (Senior Education Specialist), Charles Avelino (Education Specialist), and Alassane Ouedraogo(MonitoringandEvaluationOfficer),whohavegivenmuchoftheirtimetotheevaluationteaminordertoparticipateintheevaluationdesignandshareinformationontheGirls’EducationProjectPhase3(GEP3).Special thanks are due to the GEP3 state teams and UNICEF staff in the GEP3 states for assistance inaccessinginformationandarrangingmeetingswithkeyinformants(KIs).Inaddition,wethanktheReadingand Numeracy Activity (RANA) implementation team and monitoring and evaluation support staff forcoordinating the RANA roll-out with the evaluation team, and for providing information on the RANAproject.

This evaluation is carried out in full independence by the Education Data, Research and Evaluation inNigeria (EDOREN) project. The EDOREN team responsible for the design of the evaluation and theimplementationandanalysisof thebaselinedataconsistedof:TomPellens (TeamLeaderandEvaluationSpecialist);RachelOuthred(ProjectManager,SeniorEducationSpecialist,QualitativeDesignLeadandLeadof InstrumentDevelopment); ZaraMajeed (InstrumentDevelopment andQuantitativeAnalysis Support);AndrejKveder(SurveyTechnicalLead);MicheleBinci(Econometrician);FatimahKelleher(QualitativeLead);Johanna Wallin (Qualitative Researcher); Victor Steenbergen (Qualitative Researcher); and Shefali Rai(Researcher).Thecoreteamwassupportedbyagroupofspecialistadvisers:AdrianBeavis,DavidMegill,JusteNitiema,OladeleAkogun,StuartCameron,SouroviDeandMonazzaAslam.MonazzaAslamconductedquality assurance of the Technical Report and the drafting of this Synthesis report. The key informantinterviews(KIIs)ofthebaselinetheoryofchange(ToC)assessmentwereconductedwiththeassistanceofDrHafsatLawalKontagora.ThequantitativebaselinefieldworkwasimplementedbytheOPMNigeriaofficesurvey team, under the leadership of Femi Adegoke (SurveyManager) andArogendade Ekundayo (FieldManager),withsupportbyBabatundeAkano(DataManager)andBukolaAdekolu(SurveyAdministrator).

Needless to say, during the fieldwork state education officers, local government officers, communityleaders,IntegratedQur’anicSchool(IQS)proprietors,headteachers,teachers,pupilsandparentsplayedacrucialrolebygivingtheirtimetoallowtheevaluationbaselineresearchtobeconducted;wewouldliketoextendabigthankyoutothem.

DisclaimerEDORENisaconsortiumofleadingorganisationsininternationaldevelopmentandeducation:OxfordPolicyManagement(OPM),andtheInstituteofDevelopmentStudies(IDS)attheUniversityofSussex,andissupportedbyUKAid.EDORENcannotbeheldresponsibleforerrorsoranyconsequencesarisingfromtheuseofinformationcontainedinthisreport.AnyviewsandopinionsexpresseddonotnecessarilyreflectthoseofOPM,IDSandEDORENoranyothercontributingorganisation.

EDOREN No2,16MafemiCrescent Tel +2348107278718EducationData,Research&EvaluationinNigeria Utako Tel +2348176678243 Abuja,Nigeria Email info@edoren.org Website www.edoren.org ii

For further information contact Rachel Outhred (rachel.outhred@edored.org) or the EDOREN countryoffice(info@edoren.org).ThecontactpersonatUNICEFisTerryDurnnian(tdurnnian@unicef.org),andtheresponsibleDFIDadviserisLauraBrannelly(L-Brannelly@dfid.gov.uk).

©EDOREN iii

Executivesummary

Thisreportpresentsthefindingsofquantitativeandqualitativebaselinedatacollectionundertakenby

EducationData,ResearchandEvaluation inNigeria (EDOREN)aspartofamulti-yearevaluationof the

GirlsEducationProjectPhase3(GEP3).Itcomplementsthefulltechnicalreportofthebaselineevaluation,whichdescribesthemethodologyfortheevaluationandthefullsetofquantitativeandqualitativefindingsindetail.GEP3 is an eight-year project (2012–20) that seeks to improve school access, retention and learning

outcomesforgirlsinfivenorthernNigerianstates.ItismanagedbytheUNChildren’sFund(UNICEF)andfundedbytheUKDepartmentforInternationalDevelopment(DFID).Between2014and2017theprojectispiloting a series of interventions in primary schools and Integrated Qur’anic schools (IQS). Those mosteffective in improvingeducationoutcomes for girlswill be scaledup.Akey focusof theevaluation is toinformdecisionsrelatedtothescale-upofGEP3’sinterventions.Thescopeoftheevaluationinvolves:1. ahigh-levelexplicationandexaminationofGEP3’stheoryofchange(ToC);2. animpactevaluationofGEP3’searlylearningintervention;and3. aperformanceevaluationofGEP3’ssupporttoIQSs.

Thebaselineevaluationhasthreecoregoals:first,tocapturetheeducationsituation,andperceptionsofthissituation,atthestartofthere-designedGEP3,sothatchangescanbemeasuredduringfollow-updatacollection rounds, andproject attributionor contribution canbe assessed; second, to answer evaluationquestions about the relevance of the project; and third, to provide evidence to informGEP3’s ToC andprojectimplementation.

ExaminationofGEP3’sToC

Methodology

ThebaselineassessmentofGEP3’sToCfocusesontwoevaluationquestions:

• HowplausibleisGEP3’sToCinthecontextoftheGEP3states?

• HowappropriateareGEP3’sinterventionsintermsoftheimplementationstrategy?

Theplausibility of the ToC is examinedby reviewing stakeholders’ understandingof intendedoutcomes,interrogating the logic of the outcome chain, and identifying factors that have a key bearing on theachievement of the stated outcomes. GEP3’s implementation strategy is assessed on three dimensions:stakeholderinvolvement, implementationcapacityandtheextenttowhichtheprojectfocusesonunder-served groups. This assessment is based on key informant interviews (KIIs) conducted largelywithGEP3state-levelstakeholdersinAugust2015.KIswerepurposivelyselected,withthesupportoftheGEP3stateteams,basedontheirknowledgeoftheproject.

Keyfindings

The plausibility of GEP3’s ToC is supported by its coherent logic, synergies across interventions, and

stakeholders’commonunderstandingofitsmainobjectives–althoughtherearecaveatshere.Synergiesbetween the project’s interventions could enhance its impact, but also present risks, in thatimplementation difficulties on one dimension of the project could undermine its performance on otherfronts. A second key caveat is thatwhile stakeholders understandGEP3’smain objectives, the project’soperational design is not always understood. For instance, at the time of the interviews government

©EDOREN iv

stakeholdersinvolvedinteachertrainingdidnotyetunderstandwhatcapacitiesneedtobedeveloped,inwhatways, andwhen. This couldmake it difficult for them to assessGEP3’s performanceor be held toaccountfortheresultsachieved.

Stakeholdersnoted that theproject’sobjectiveof increasinggirls’enrolment is feasible,particularly in

primaryeducation,butthatimprovementsinretentionandlearningoutcomeswillbehardertoachieve.

Stakeholderswereoftheviewthatenrolmentdrivesandcashtransfersarebotheffectivestrategiesthataddressimportantbarrierstogirls’accesstoeducation,andthattheycomplementoneanother.Retentionis held to bemore challenging as it is shaped by supply-side factors, such as the presence of sufficientteachers,thequalityofteaching,andthequalityoftheschoolenvironment.Stakeholderswerelesslikelyto recognise learning as a key expected project outcome. When they did, they noted that learningoutcomesaremoredifficult to improve thanaccess,owing to the lowqualityof teaching,particularly inrural areas, and insufficient investment in the supply of quality education. There was wide consensusamongstKIs that tackling thequalityof teachingandteachers iscentral to improving learningoutcomes.ThisisconsistentwiththefocusonthisvariableinGEP3’sToC.

Awiderangeofstakeholdersendorsedthe importanceof theearly learning intervention.Stakeholderssupporttheemphasisonliteracyduringearlygradesasafoundationalskill.However,thereisnoconsensusamong stakeholders about the extent towhich the use of amother tongue is a necessary condition toimprovelearningoutcomes.ThebaselinefindingsoftheearlylearningevaluationconfirmthatHausaisthelanguage of the immediate environment in Katsina and Zamfara, but they also highlight that Hausaknowledgeamongteachersis low.Thisneedstobetakenintoaccountinthedesignoftheearlylearningintervention.

GEP3’s interventions to support school governance are considered to be largely promising, although

thereare some risks related to the targetedempowermentof School-BasedManagementCommittees

(SBMCs).SBMCempowermentisbothapivotalintermediaryoutcomeinGEP3’sToC,aswellasoneofitsmostprecariouslinks.TheexpectationsplacedonSBMCsintermsofrolesandresponsibilitiesareveryhigheventhoughtheseorganisationsareoftenstartingfromaverylowbase,particularlyinIQSs.Atthestatelevel, High-Level Women Advocate (HiLWA) members engage with decision-makers and schoolcommunitiesandhavethepotentialforindirectinfluence.TheGirlsEducationSteeringCommittee(GESC)alsohasthepotentialtosupportgovernance,totheextentthatitisactive,sufficientlylocallyownedandaddresses key challenges. GEP3’s support to the educationmanagement information system (EMIS) andAnnualSchoolCensus(ASC)willplausiblyimprovedatareliability,althoughthiswillnotnecessarilyleadtodatabeingusedinthepolicy-makingprocess.

The findings point to some specific risks to the scale-up process. The scale-up process is reliant ongovernmentfunding.However,thereissignificantuncertaintyabouttheextenttowhichthisfundingwillbeforthcoming.ThisisdespitecertainsupportivemeasuresbeingintroducedaspartoftherecentdesignofGEP3,notablyanincreasedemphasisonhigh-leveladvocacyandafocusonamoremanageablesetofoutcomes. The scale-up of GEP3’s support to IQSs is particularly uncertain because the institutionalmandate over IQSs is unclear, the number ofwell-established IQSs available for scale-up is limited, andthere are supply-side constraints, particularly with regards to facilitators. One further constraint is thatstakeholdersdonothaveaclearunderstandingofhowscale-upisexpectedtotakeplace.

Overall,thebaselinefindingspointtoanumberofriskstothecausallinksintheToC.Criticalassumptionsrelatedtothemanagementandresourcingoftheeducationsystemarehighlyuncertain,inparticularthereleaseofgovernmentfunding,school investmentandhumanresourcecapacityremainingonaparwithincreasedenrolment,effectivemonitoringattheschoolandinterventionlevels,andSBMCs’abilitytoplaythewide-rangingrolethatisbeingsoughtaspartoftheproject.

Baseline findings on implementation capacity are mixed. The project makes an appreciable effort toinvolve a variety of stakeholders. However, their involvement in operational planning is uneven across

©EDOREN v

interventions. Government capacity building is embedded in project design, but an effective system isneeded to identify and address themost important capacity development needs.Monitoring capacity isrightly emphasised in GEP3’s re-design but its operationalisation has yet to advance. With regards toequity,severalGEP3interventionsaredesignedtobeequityenhancing,butitisunclearwhetherthemostvulnerablehouseholdsandgroupsareabletobenefitfromandactivelyparticipateinGEP3interventions.Notably,GEP3isnolongeroperatinginLocalGovernmentAreas(LGAs)withthehighestgendergapinallstates.

ImpactevaluationofGEP3’searlylearningintervention

GEP3’searlylearninginterventionaimstoimprovetheearlylearningskillsofchildreninprimaryGrades

1to3(Primary1(P1)–Primary3(P3))inthemothertongue,whilealsopreparingchildrentolearnwithEnglish as a language of instruction by the time they transition to Grade 4. A key measure of theintervention’ssuccesswillbe improved literacyskills.The interventionwillbe implementedoverathree-yearperiod (2016–2018) insixLGAs inZamfaraandKatsina (threeperstate).The interventionhas threekeycomponentsatschoolandcommunitylevel:theprovisionofapackageofHausa-mediumteachingandlearning materials to schools; early grade professional development for teachers and head teachers(includingmonthlyschoolvisits);andasetofcommunityawarenessandengagementactivitiestosupportearlygradeliteracy.

Methodology

The evaluation of the early learning intervention uses a theory-based approach and is designed as a

clusteredrandomisedcontrolledtrial(RCT),stratifiedbyLGAandtypeofschool(primaryschoolvs.IQS),and randomised at the school level. The intervention’s ToC was used as a framework to formulate theevaluationquestions.TheRCTdesignallowstheevaluationteamtomeasuretheattributableimpactoftheearly learninginterventiononlearningoutcomesbycomparingoutcomechangesinatreatmentgroupofschoolswith those ina control group that is statistically similaronaverage.TheRCTdesign is combinedwith the overarching theory-based evaluation approach to measure not only changes in final outcomevariablesbutalsonetchangesinintermediaryoutcomevariablesalongtheassumedcausalchain.Thiswillallowtheevaluationteamtounpackhowchangetakesplace.

Inorder tomeasure changes inoutcomesapanel surveywasdesigned fordata collectionatbaseline,

midlineandendline.Datawillbecollectedateachofthesethreestagesinasampleof120publicprimaryschoolsand120IQSsdrawnfromthesixinterventionLGAsacrossKatsinaandZamfara.Halfofthepublicprimary schools and IQSs form part of the treatment group, while the other half serves as the controlgroup. Baseline data collection was conducted in October–November 2015 during the first term of the2015–2016 school yearbefore the startof implementationof theearly learning intervention.Within thesample schools, male and female pupils and teachers were randomly sampled to form part of a panelsurvey.Sevendatacollection instrumentswereadministeredwithineachschool:pupilEnglishandHausaliteracy assessments, pupil and teacher questionnaires, a teacher knowledge and skills assessment, ateacherclassroomobservationandaheadteacherquestionnaire.Theliteracyassessmentswerecarefullydesignedandpilotedtoensurethatitemdifficultymatchedpupils’ability.

Analysisofthebaselinedataindicatedthatrandomisationhadworkedtocreatecomparabletreatment

and control groups. We assessed whether the randomisation had achieved its intended purpose bychecking whether key outcome variables and school-, teacher- and pupil-level characteristics differedbetweenthetreatmentandcontrolgroupsatbaseline.Thegreatmajorityofvariablesinvestigateddidnotshowanystatisticallysignificantdifferencebetweenthetwogroups.

©EDOREN vi

Keyfindings

Thebaselinefindingshighlightthattheearlylearninginterventionwillbeimplementedinachallenging

context.Thevastmajorityofschoolsarelocatedinruralareasandhavepoorinfrastructure.Whilealargershare of IQSs than public primary schools have access to drinking water and electricity, they have lessrooms on average and have less access to books and functional toilets for girls. Teachers in IQSs arepredominantlymale. The baseline also found that 40% of IQSs only have one teacher who teaches theintegrated curriculum subjects, which raises questions about the applicability of the school-level peermentoringapproachintheseIQSs.

Head teachers make some effort to address teacher attendance, but are less likely to take action to

improvethequalityofinstruction.Almost90%ofheadteachersinpublicprimaryschoolsreportedtakingactiononteacherattendance.ThecorrespondingfigureforIQSswasjustover50%.Overhalfoftheheadteachers interviewedhadnotobserveda single lessonduring theprevious school term.Similarly,halfofheadteachersdonothavemeetingswithteachersormeetthemlessthanonceamonth.

Anotableshareofearlygrade teachersdonot teach languages.Roughlyhalfof teachers reported thattheyteachonlyonesubject,ofwhichjustover40%teachasubjectotherthanHausaorEnglish.GiventhattheReadingandNumeracyActivity (RANA)willemphasiseearlygradereading, thispoolof teachersmaynot be suitable candidates for the intervention. This needs to be taken into account when selectingteachersfortraining.

Theteacherssurveyeddemonstratedverylimitedknowledgeandskillsinrespectofmostofthedomains

covered by the teacher assessments. Less than 3% of teachers were able to display competence inidentifying lowperformers,evidencing judgementsanddiagnosingpupilperformance, interpretingwordsandphrases,andwritingskills.AlthoughallteachersreportedthattheyspeakHausa, lessthan40%wereabletodisplaycompetenceinprimaryGrade1and2-levelHausa.ThisissignificantforaninterventionthatfocusesonteachinginHausaasteachers’subjectknowledgehasimportantimplicationsforthequalityofteaching.Teachers’verypoorskills in identifying lowperformersanddiagnosingpupils’performancearelikely to present key hurdles to improving teaching quality given the wide recognition in the educationliteraturethatchildrenlearnbestwhenteachingistargetedtowhatthechildisreadytolearn.Therearesomeminorvariationsinperformanceacrossdifferentgroupsofteachers,butknowledgeandskillslevelsareconsistentlylowwithinallgroups.

Thesefindingsregardingteachers’knowledgeandskillshaveafewkeyimplicationsfortheearlylearning

intervention.Theverylowlevelsofknowledgeandskillsamongstteachersatbaselineindicatesthatthereis substantial scope for improvement in this area. However, it also presents challenges, in that there isoftenalackofbasicfoundationalskillsthattheprojectcanbuildon.Thefindingsshouldserveasausefulinput to the RANA implementation team as they highlight some of the key areas of weakness that theintervention would need to address. They also provide an indication of the scale of the challengesconfrontingtheproject,whichwillhaveimplicationsforimplementationdecisions–forinstancerelatedtothefrequencyandcontentoftraining,andthefocusoftheschoolvisits.

Classroom observations indicate that the quality of early grade instruction is low across all groups of

teachers.Theextentofpupil-centred learningobservedatbaselinewas low,aswas theextent towhichteacherslinkthelessontopreviouslearningandlearningobjectives.Incontrast,timeontaskwashigh,atanaverageof96%ofatotallesson.However,thisfindingshouldbeinterpretedwithcautionasitislikelythatthepresenceofobserversintheclassroomledtoanincreaseintimeontask.Thebaselinealsosoughttomeasuregender-sensitiveteachingpracticesusingclassroomobservationdata,butextremecomplianceeffectswereobservedacrossall items,raisingquestionsaboutthevalidityandreliabilityofthemeasure.Teachers’ responses to the assessments indicate that teachers are aware of objectives to target girls inclass,astheystatethatitisimportanttofocusongirls,buttheirresponsesalsopointtothepersistenceofdeeplyingrainedgenderbiases.

©EDOREN vii

Theavailabilityanduseof teachingand learningmaterialswasobserved tobevery low. This suggeststhatthedistributionoflearningmaterialsbytheRANAinterventioncouldfillanimportantgap.However,itwillbevital formaterialstobepitchedatanappropriate level, taking intoaccountthevery lowlevelsofknowledgeamongstbothteachersandpupils.

Hausa-basedteachingisprevalentintheearlygrades.Hausawasusedinallclassesobservedatbaseline.In roughly half of classes, a second languagewas used in addition to Hausa. However, as noted above,manyteacherslackcompetenceinbasicHausa.

The baseline sought to assess teacher motivation and attendance as these variables are likely to

influence the extent to which improved knowledge and skills amongst teachers translate into better

teaching.Teachermotivationwas relativelyhighon threedimensions: theeffortput into,andperceivedimportance, of teachers’ work; their enjoyment of teaching; and interaction between teachers. Scoresrelated to pressure andwork-related tension and teachers’ perceived self-efficacywere low. Therewaslimited variation across teachers. Absenteeism was higher in IQSs than in public primary schools, withfacilitators reporting that theywere absent for an averageof eight days over theprevious 60days. Thecorrespondingfigureforpublicschoolswasthreedays.

ThebaselinefindingsindicatethatinbothEnglishandHausa,veryfewpupilshavelevelsofknowledge

that are appropriate for their grade. In both subjects, the vast majority of P2 pupils assessed haveknowledgeand skills expectedofpre-school children (pre-literacy skills). Furthermore, inboth cases, thepeak of the distribution of performance falls well below the cut-off point between pre-literacy andemergingliteracy.Thissuggeststhatsubstantialeffortwouldbeneededtoachieveasignificantincreaseintheshareofpupilsmovingfrompre-literacytoemergingliteracy.

Pupils’knowledgeofphonic isparticularly low.Thepsychometricanalysis foundthat itemsthat requireknowledgeofphonicsrankasthemostdifficultitemsinboththeHausaandEnglishassessments.Correctlysoundingoutlettersandidentifyingsimilarsoundswasmoredifficultforpupilsthanwritingorreadingfullpassages.

Learningoutcomesdiffer by age and gender.Older pupils in P2 performbetter than younger ones andboys perform better than girls. Gender differences in performance are small in the younger years, butincreaseoncegirlsreachpuberty(around12yearsofage).

Regressionanalysis indicatesthatpupilcharacteristics,socio-economicbackgroundandschooltypeare

associatedwithlearningoutcomes,butteachercharacteristicsarenot.Thelatterfindingmayreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetected if there were greater variation in competency levels across teachers, and if their ability toinfluencepupils’learningoutcomeswashigher.Thisalsoimpliesthatiftheearlylearninginterventionleadsto substantial improvements in teachers’ knowledge and skills we should see a more significant linkbetweenteachercharacteristicsandlearningoutcomesatendline.

EvaluationofGEP3’ssupporttoIQSs

GEP3’s support to IQSs (IQSS) seeks to improve education outcomes at these schools, with the aim ofprovidinganacceptablealternativeformofqualitybasiceducationforgirls.Theinterventionseekstoraiselearning outcomes in basic literacy and numeracy for pupils at IQSs (especially girls), improve retentionamong girls, and (to a lesser extent) increase girls’ enrolment. The project targets registered Islamiyya,Qur’anicTsangayaEducation(IQTE)centresthatofferanintegratedcurriculumandinwhichatleast40%ofpupilsaregirls.TheseIQSslargelyoperateascommunity-basedinitiatives,butarewillingtobuildlinkswithgovernmentforthepurposesofmonitoringandtechnicalsupport.GEP3’sIQSSinvolvestheprovisionof training and mentoring of IQS facilitators, training for head teachers, the distribution of classroom

©EDOREN viii

teachingandlearningmaterials,capacitybuildingforCentre-BasedManagementCommittees(CBMCs)andtheprovisionofmini-grants.

Methodology

Theevaluationapproachdrawsontheprinciplesofcontributionanalysisandreliesonastrongmixof

quantitative and qualitative methods. It does not make use of a comparison group to make causalinferences but rather seeks to make credible causal claims about the intervention’s contribution toeducationoutcomesbyverifyingthechainofexpectedresultsandassumptionsasperacredibleToC,aswell as assessing alternative explanations for the outcomes observed. The evaluation will assess threecontributionclaims:

• GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs;

• GEP3’sIQSScontributestoanimproved,girl-friendlyenvironmentwithintheschools;and

• More effective teaching of formal subjects and an improved, girl-friendly environment contribute toimprovedlearninglevels,particularlyamonggirls.

ThequantitativecomponentoftheevaluationconsistsofrepresentativesamplesurveysamongGEP3IQSsinthe12GEP3LGAsinBauchiandNiger.Thebaselinesurveytookplaceinasampleof60IQSs,stratifiedbyLGA.Itinvolvedtheadministrationofthesameseveninstrumentsusedfortheearlylearninginterventionevaluation, as well as a pupil numeracy assessment and CBMC questionnaire. The baseline qualitativeresearchtookplaceinsixIQSsthatwerepurposivelysampledusingtypicalandextremecasesamplingandthat were also included in the quantitative survey. The same set of IQSs will be visited at midline andendline.

Keyfindings

ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs

The landscape of IQSs is complex. Variations in school structures, pupil–teacher ratios, facilitators’qualifications,schoolleadershipandmanagement,andthenumberandhoursforwhichintegratedsubjectsaretaughtperschoolsuggestthattheoutcomesofinterveningintheseschoolsarelikelytodifferfromoneanother. Some IQSsmaybemobile,whichwouldhave implications for continued teachingand learning,particularly for girlswho are unlikely tomovewith theMallam.1 The extent of integration varies acrossIQSs,andisloweronaverageinBauchithaninNiger.Acrossthetwostates,headteachersreportedthatanaverageof threehoursperweek is spentonteaching the integratedcurriculum,which iswellbelowtheeight hours recommended by official guidelines. A quarter of IQSs only have one facilitator, which willconstrainamentoringprocessbasedonschool-basedpeerinteraction.Onecrucialsupportivefactoristhatintegrationdoesappeartohavegainedacceptanceamongparentsandthecommunity.

Facilitators’knowledgeandskillsinkeydomainsassociatedwitheffectiveteachingareverylow.Only2%of facilitators were able to display competence in identifying low performers. Less than 1% hadcompetence in writing skills and none of the facilitators assessed were competent in evidencingjudgementsanddiagnosingpupilperformance.Only34%offacilitatorswereabletodisplaycompetenceinGrade1and2-levelHausa.Mostfacilitatorsdisplayalackofunderstandingofwhatstepstheycantaketoimprovepupils’performance.Aswiththeearly learning intervention,thesefindings indicatethatthere istremendous scope for improvement in teachers’ performance, but also that it will be crucial for theinterventiontobecarefullytargetedtotheverylowlevelsofskillsandknowledgeamongstfacilitators.

1ThetermMallamcanhavedifferentinterpretations.WeconsidertheMallamtobetheheadofthereligiousschool,responsibleforreligiouseducation.

©EDOREN ix

Facilitators scoredverypoorlyona composite indexof effective classroompractices. Inparticular, thedepthofpupil-centredlearningwasfoundtobelowandtherewaslittleeffortamongfacilitatorstolinkthelesson to previous learning and learning objectives. Time on task was generally high: in 65% of lessonsobservedpupilsspent100%ofthelessonontask.However,thismayhavebeeninfluencedbythepresenceofobservers.

Thereismixedevidenceonfacilitators’attitudesandclassroompracticestowardsgirls.Thefacilitatorsinthesixcasestudyschoolsexpressedpositiveattitudestowardsgirls’education,althoughtheseresponsesarepronetosocialdesirabilitybias.Indeed,bothqualitativeandquantitativedatapointtothepersistenceofgenderbiases–forinstancerelatedtogirls’abilities.Thefindingsindicatethatfacilitatorsareawareofand sometimes practice gender-sensitive techniques, such as actively engaging both girls and boys.However, thesepractices seem tobeperformed in a tokenisticway—in linewithdevelopmentprojects’sensitisationefforts—andgender-biasedclassroompracticescontinuetotakeplaceinIQSs.

Aswiththeearlylearningintervention,thebaselineevaluationsoughttoassessfacilitators’motivation

and attendance. On average, facilitators think that their role is important and they enjoy working asteachers, but they have poor perceptions of their teaching efficacy. The qualitative research found thatfacilitatorsfeelintrinsicallymotivatedbywhattheydo,andthatschoolstakeholders,includingpupils,thinkthatfacilitatorsasgenerallydedicated.However,italsoindicatesthatthefactthatmostfacilitatorsarenotpaidmakesitdifficulttoattractqualifiedfacilitatorsandholdthemaccountable.Thequantitativefindingsconfirmthataverysmallshareoffacilitatorsarepaidasalaryorastipend(3%inBauchi,33%inNiger)andpointtoapositivelinkbetweenreceivingremunerationandteachermotivation.Roughly75%offacilitatorsreported that they had been absent at least once in the previous threemonths. Furthermore, the casestudy findings indicate that few IQSshave set timetables, and formal subjects are taughtasandwhenafacilitatorisavailable.

Theavailabilityanduseofteachingandlearningmaterialsisverylimited.AlmostnoHausamaterialsareavailable,althoughitisimportanttonotethatHausaisnotthemothertongueofallpupils.InBauchi,93%of sampledchildren reported speakingHausaathomebut the corresponding figure inNiger is just54%,with43%speakingNupe.

SchoolleadershipinIQSsiscomplex,whichraisessomechallengingquestionsaboutwhopreciselyshouldbe targeted by the pedagogical leadership component of the IQS intervention. IQSs have a variety ofdifferent leadership roles (Mallam, head teachers, proprietor). These may be held by one person ordifferentpeopleandthewayinwhichresponsibilitiesaresplitacrosstheseindividualsvariesacrossIQSs.Theprofessionalandacademicqualificationsofheadteachers inIQSsaregenerally low,and,onaverage,below those of facilitators. This raises questions about head teachers’ technical capacity to exercisepedagogical leadership over facilitators. The appointment of a head teacher is not necessarily based onability and qualifications but has to dowith social statuswithin the community and perceptions aroundleadershipmoregenerally.

ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs

CBMCs have some potential to bring about improvements in the learning environment in IQSs. Themanagement of IQSs is seen as collective, without clearly defined and delegated roles, which providesopportunitiesforcommunity-basedmanagementstructureslikeCBMCs.Thebaselinefindingsalsoindicatethatmost CBMCs are established and active, as indicated by regularmeetings andmonitoring activities.CBMCs have considerable community representation, although women are under-represented and veryfewmembersarechildren.This indicatesthatatpresentCBMCsareunlikelytoprovideaforuminwhichgirls’voicesandneedscanbeheard. Inall IQSs included in thequalitativestudytheMallamholdsakeypositionintheCBMC,suchaschairperson.ThisraisesquestionsabouttheextenttowhichCBMCscanplayaneffectiveroleinholdingschoolleaderstoaccount.

©EDOREN x

CBMCsdomakeanefforttoraiseresourcesfromthecommunity,butthisappearstoofferlimitedscope

inregardtobridgingresourcegapsinIQSs.ThequalitativeresearchindicatesthatCBMCsfeelthatalackof fundshinders theirability to ‘solve’problemsat IQSs.Theyalsonote thatwhileparentsarewilling tocontributeresources,theyoftenhavelimitedcapacitytodoso.ThesefindingshighlightthecaseforCBMCtrainingtogobeyondanarrowfocusoncommunityresourcemobilisationtoencompassthemobilisationofresourcesfromalternativesources.

CBMCmembersseemtolargelyunderstandtheirrolesandresponsibilitiesbutoftenlackthecapacityto

enact them. Key areas in which CBMCs are under-performing are school planning and financialmanagement.CBMCmembersdomakeaneffort tomonitor thequalityof the teachingand learning, inparticular pupils’ and facilitators’ attendance. CBMC members assert that the main constraint thatunderminestheirperformanceisalackoffunds,particularlytoimproveinfrastructureandpayfacilitators.

Most IQSs do not offer a girl-friendly school environment at present. The vast majority face majordeficiencies inphysical infrastructure.Only3%of the schoolshaveaccess toawater source,20%of theIQSsdonothaveaphysicalclassroomstructureforthestudents,andlessthan25%havefunctioningtoiletsforpupils,withonly8%havingfunctioningtoiletsforgirls.Only3%ofIQSshaveamothers’group,teacher–student association or pupil groupwhere students can discuss their concerns. Gender-biased classroompracticesandattitudesprevail,whichcanaffectgirls’learningopportunitiesandself-confidence.However,there appears to be an openness among school leaders to try and address these issues. ResourcesmobilisedbyCBMCsareinvestedinschoolsandgirl-friendlyinvestmentsareconsidered,buttheamountsraisedappearinsufficientgiventhepoorconditionoftheschoolenvironment.

Theextentofgovernmentengagementwith IQSs is limited.Themajorityof IQSs in thesamplehadnotbeen visited by a government official during the previous term. Government officials say that they facechallenges in reaching all the schools on a regular basis. Communities generally view support from thegovernmentnegatively,consideringgovernmentactorstobeunreliable.

ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschool

environmentcontributetoimprovedlearninglevels,particularlyamonggirls

Thebaselineevaluationidentifiessomekeyfindingsthatarerelevanttothiscontributionclaim.Girlsandboys are both equally likely to attend IQSs. However, both face challenges in regard to attending andremaining fully engaged with the learning process because of out-of-school responsibilities, which arelinked to household poverty. A large share of childrenwho study in IQSs attend another school aswell.There were significant cross-state differences, with 72% of pupils in Bauchi reporting attending otherschools,ascomparedtoonly13%pupilsinNiger.InBauchi,mostofthesechildrenwereattendingapublicprimary school (94%), as compared to 55% in Niger. This indicates that interventions at local primaryschoolsmayconstituteanalternativeexplanationforanyimprovementsinlearningoutcomesatIQSs.

Pupils’ learningoutcomesarevery low. InbothHausaandEnglishover90%ofGrade2pupilsassesseddisplayed literacy levels associatedwith pre-school pupils. Furthermore,most of these pupils are a longway from the cut-off for emerging literacy skills (those associated with the P1 curriculum). Numeracyoutcomeswerebetter,with69%ofpupilsdisplayingemergingnumeracyskills,comparedto21%withpre-numeracyskills.However,only11%ofpupilshadnumeracyskillsassociatedwiththeP2curriculum.Boysandgirlsbothperformpoorly,althoughgirls’performancedeclines,comparedtothatofboys,ataroundpuberty(roughly12yearsofage).

Continuedbarrierstogirls’accessto,andretentionin,schoolingexist.Attitudestowardsgirls’educationarenotalwayssupportive,althoughthisappearstobechanging.Changingattitudesalone,however,maynot be sufficient to bring about change in behaviour as poverty is often cited as a critical reason whyparentsdonotsend theirchild toanyschool,publicprimaryor IQS.Bothboysandgirlsoftenengage insomeformofincome-generatingactivity,whichparentstendtoviewasafinancialandpracticalnecessity.

©EDOREN xi

Thisadverselyaffectsschoolattendance.Earlymarriageisalsocitedasakeyreasonforlowenrolmentandretentionamonggirls.

Recommendations

ThebaselinefindingspointtocertainkeyrecommendationsforGEP3.ThebaselineassessmentofGEP3’sToC highlights that continued political engagement is required to ensure that government funds aremobilisedforprojectscale-up.Theprojectshouldspecifyandcommunicateitsoperationalobjectivesandstrategies to government and other implementing partners, strengthen itsmonitoring of assumptions inthe ToC, and ensure that monitoring information is used to facilitate learning and accountability. Inparticular,emphasisshouldbeplacedonmonitoringGEP3’ssupporttoSBMCs/CBMCs,giventheirpivotalroleintheproject’sToC.

Baseline findingson theearly learning interventionhighlight that theRANA interventionwillneed tobecarefully tailored to thevery low levelsofknowledgeandskillsamongst teachers toensure that trainingcontentandmaterialsarepitchedattherightlevel,andthattrainingissufficientlyintensivetofillthelargegapsinteachers’subject,pedagogicalandcurriculumknowledge.Teachercapacitydevelopmentalsoneedsto incorporate actions to change teachers’ awareness of their own potential for influencing learningoutcomes,andtheirunderstandingofhowbesttosupportpupils’learning.Thepeermentoringapproachneedstobeadaptedfor the largenumberof IQSsthatonlyhaveoneteacherwho iseligible forsupportunder the intervention. RANA also needs to sufficiently strengthen the capacity of IQS governmentstakeholders to ensure that they effectively monitor and provide support supervision. Finally, thecomplexityoftheIQSleadershipmodelindicatesthatspecialattentionneedstobegiventoensuringthatappropriateindividualsatIQSsaretargetedfortheprogramme’spedagogicalleadershiptraining.

ThefindingsontheIQSSinterventionhighlighttheneedfortheprojecttostrengthenitsmonitoringoftheintervention. The IQS context is diverse, flexible and evolving. In order to adapt the intervention to thiscontext,quicklearningandfeedbackbasedonmonitoringdataisneeded.Theselectionoffacilitators,headteachersandCBMCmembersfortrainingrequirescloseattention,verificationandmonitoring.FacilitatortrainingandmentoringneedstobecarefullyadaptedtotheverylowlevelsoffacilitatorcompetencyandtheIQScontext.Similarly,learningandteachingmaterialsneedtobetailoredtotheverylowlevelsofskillsandknowledgeamongstbothfacilitatorsandpupils,andtothelanguageoftheusers(particularlyinNiger,where Nupe is widely spoken). Training and mentoring on gender-sensitive class practices needs to gobeyond a focus on gender-sensitive teaching practices to try and tackle entrenched gender biases, forinstancerelatedtogirls’abilitytolearn.Womenandgirls’participationinshapingeducationandtheschoolenvironmentneedsfurther investigation:atpresent,bothareunder-representedonCBMCs,which limitsthe scope for thesebodies toact as forums forwomenandgirls’ participation in the sector. Finally, thebaseline findings highlight the need for GEP3 to promote the mobilisation of resources beyond thecommunityandtoadvocateforfacilitatorstobeadequatelyremuneratedbythegovernment.

©EDOREN xii

Tableofcontents

Acknowledgements Error!Bookmarknotdefined.

Executivesummary iii

Listoffigures,tablesandboxes xiv

Listofabbreviations xvi

1 Introduction 1

2 Projectandevaluationbackground 2

2.1 Objectives,scopeandinterventionstrategy 2

2.2 ObjectivesandscopeofGEP3evaluation 3

2.3 Evaluationdesignandmethods 3

2.3.1 Evaluationquestions 32.3.2 Evaluationdesignandmethods 42.3.3 Inclusionandethics 42.3.4 Externalvalidityofthefindings 4

3 BaselineToCassessment 5

3.1 Methodology 5

3.2 PlausibilityofGEP3’sToC 5

3.2.1 Generalassessment 63.2.2 Output1:Increasedenrolmentandretentionofgirlsinbasiceducation 73.2.3 Output2:Improvedcapacityofteacherstodelivereffectivelearningforgirls 73.2.4 Output3:Improvedgovernancetostrengthengirls’education 83.2.5 GEP3’spilot-to-scale-upapproach 9

3.3 AppropriatenessofGEP3’simplementationstrategy 10

3.3.1 Stakeholderinvolvement 103.3.2 Governmentcapacity 103.3.3 Equity 11

4 Baselineoftheearlylearninginterventionevaluation 12

4.1 GEP3’searlylearningintervention 12

4.2 Baselinemethodology 12

4.3 Comparisonofbaselinecharacteristicsbetweentreatmentandcontrolschools 13

4.4 Baselinefindings 14

4.4.1 Teachingintheearlygrades 144.4.2 Pupils’literacyinearlygrades 234.4.3 Analysisofrelationships 26

5 BaselineoftheIQSSevaluation 28

5.1 GEP3’sIQSS 28

5.2 Baselinemethodology 29

5.3 Baselinefindings 30

5.3.1 ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs 30

5.3.2 ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs 42

©EDOREN xiii

5.3.3 ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls 48

6 Conclusionsandrecommendations 55

6.1 Conclusions 55

6.2 Recommendations 60

Bibliography 64

AnnexA ToCoftheearlylearningintervention 65

AnnexB Evaluationquestions 68

B.1 Evaluationquestionsfortheearlylearninginterventionevaluation 68B.2 EvaluationquestionsfortheIQSSevaluation 68

AnnexC Samplesizeofbaselinesurvey 70

C.1 Samplesizeofearlylearninginterventionbaseline 70C.2 SamplesizeofIQSSbaseline 70

AnnexD Randomisationchecksfortreatmentandcontrolgroup 72

AnnexE Regressionoflearningoutcomesonkeyinfluencingfactors 75

AnnexF UpdatedToCofIQSSintervention 78

©EDOREN xiv

Listoffigures,tablesandboxes

Figure1: SchoolmanagementactionsinPSandIQS............................................................................17Figure2: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetence

acrosstheteacherknowledgeandskillssubscales...............................................................19Figure3: FrequencyofHausauseintheclass......................................................................................21Figure4: Comparisonofmotivationsubscales.....................................................................................22Figure5: DistributionofHausaliteracyproficiency.............................................................................25Figure6: MeanHausascalescorebyageandgender(95%confidenceinterval)...............................26Figure7: CoresubjectstaughtinIQSs..................................................................................................33Figure8: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetence

acrosstheteacherknowledgeandskillssubscales...............................................................36Figure9: Comparisonofmotivationsubscales,IQSSsample...............................................................37Figure10: Facilitatorlanguageuse:Shareoffacilitatorswhousedthelanguageatleastonceduring

thelesson...............................................................................................................................39Figure11: Availabilityanduseofteachingandlearningmaterials........................................................40Figure12: HeadteacherpedagogicalleadershipandmanagementactionsinNigerandBauchi..........42Figure13: Languagespokenatpupils’homes,bystate.........................................................................50Figure14: DistributionofHausaliteracyproficiency.............................................................................52Figure15: Distributionofnumeracyproficiency....................................................................................53Figure16: DiagramdepictingToCoftheearlylearningintervention....................................................67Figure17: DiagramdepictingToCoftheIQSS........................................................................................79Figure18: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’...........................82Figure19: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’

...............................................................................................................................................85Figure20: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’...................87

Table1: OverviewofGEP3interventionsduringthe2014–2017pilotperiod......................................2Table2: Priorityevaluationquestions...................................................................................................3Table3: SchoolcharacteristicsinpublicprimaryschoolsandIQSs....................................................16Table4: Summaryofteachercharacteristics......................................................................................18Table5: Summaryoffacilitatorcharacteristics...................................................................................34Table6: Summaryofheadteachercharacteristics.............................................................................41Table7: SummaryofCBMCcharacteristics.........................................................................................44Table8: Targetedandfinalsamplesizesoftheearlylearninginterventionbaselinesurvey.............70Table9: TargetedandfinalsamplesizesoftheIQSSbaselinesurvey................................................71Table10: School-levelbalance(allheadteachersinpublicprimaryschoolsandIQSs)........................72Table11: Teacher-levelbalance(allteachersinpublicprimaryschoolsandIQSs)..............................73Table12: Pupil-levelbalance(allpupilsinpublicprimaryschoolsandIQSs)........................................74Table13: Regressioncoefficientsandtheirstatisticalsignificanceofthemainfactorsinfluencing

Hausascaledscores...............................................................................................................75Table14: Regressioncoefficientsandtheirstatisticalsignificanceofthemainfactorsinfluencing

Englishscaledscores..............................................................................................................76Table15: Hausaschoolfixedeffectsmodel..........................................................................................77Table16: Englishschoolfixedeffectsmodel.........................................................................................77

Box1: RANAactivitiesatschoolandcommunitylevel.....................................................................12Box2: Measurementofteachermotivation....................................................................................22Box3: Learningoutcomemeasurement..........................................................................................24Box4: Explanatoryfactorsincludedinthepupilscorrelation..........................................................26Box5: IQSSactivitiesatschoolandcommunitylevel.......................................................................28Box6: EvaluationquestionsforGEP3’searlylearningintervention................................................68

©EDOREN xv

Box7: EvaluationquestionsforGEP3’sIQSS....................................................................................68

©EDOREN xvi

Listofabbreviations

ASC AnnualSchoolCensus

CAPI Computer-assistedpersonalinterviewing

CBMC Centre-BasedManagementCommittee

DAC DevelopmentAssistanceCommittee(OECD)

DEEPEN DevelopingEffectivePrivateEducationinNigeria

DFID UKDepartmentforInternationalDevelopment

EDOREN EducationData,ResearchandEvaluationinNigeria

EMIS Educationmanagementinformationsystem

ESSPIN EducationSectorSupportProgrammeinNigeria

FTTSS FemaleTeacherTraineeScholarshipScheme

G4G GirlsforGirls

GEP3 GirlsEducationProjectPhase3

GESC GirlsEducationSteeringCommittee

HiLWA High-LevelWomenAdvocates

HWI HouseholdWealthIndex

IQS IntegratedQur’anicSchool

IQSS IQSsupport

IQTE Islamiyya,Qur’anicTsangayaEducation

LGA LocalGovernmentArea

MS MothersAssociation

NCE NationalCertificateinEducation

NGN NigerianNaira

NGO Non-governmentalorganisation

OECD OrganisationforEconomicCo-operationandDevelopment

OPM OxfordPolicyManagement

P1 Primary1

RANA ReadingandNumeracyActivity

RCT Randomisedcontroltrial

SAME StateAgencyforMassEducation

SBMC School-BasedManagementCommittee

SCCE SeniorSchoolCertificateExamination

SUBEB StateUniversalBasicEducationBoard

ToC Theoryofchange

UNICEF UnitedNationsChildren’sFund

WCDP WholeCentreDevelopmentPlan

©EDOREN 17

1 Introduction

ThisreporthasbeenpreparedbyEDOREN.EDORENiscontractedbyDFIDNigeriatoevaluateGEP3,inlinewiththeGEP3evaluationframework(EDOREN,2015).GEP3isaneight-yearproject(2012–20)thatseekstoensuremoregirlscompletebasiceducationandacquireskillsforlifeandlivelihoodsinfivenorthernNigerianstates.ItismanagedbyUNICEFandfundedbyDFID.

Thisreportprovidesasynthesisofthefindingsofthebaselinedatacollectionaspartofamulti-yearevaluationofGEP3.ItisaccompaniedbyacomprehensiveBaselineTechnicalReport,whichincludesadetailedpresentationoftheanalysis,methodologyandinterventionsbeingevaluated.Theobjectiveofthesynthesisreportistoprovideasummaryofthekeybaselinefindingswithoutenteringintotechnicaldetails.

ThescopeofEDOREN’sevaluationworkconsistsofthreepiecesofanalysis:

• ahigh-levelexplicationandexaminationofGEP3’sToC;

• animpactevaluationofGEP3’searlylearningintervention;and

• aperformanceevaluationofGEP3’sIQSS.

Theobjectivesofthebaselineanalysisare:

• tocapturetheeducationsituation,andperceptionsofthissituation,atthestartofthere-designedGEP3projectsothatchangescanbemeasuredduringfollow-updatacollectionroundsandprojectattributionorcontributioncanbeassessed;

• toanswerspecificevaluationquestionsabouttherelevanceoftheproject;and

• toprovideevidencetoinformtheGEP3ToCandprojectimplementation.

InlinewiththescopeofEDOREN’sevaluationwork,thereportisstructuredasfollows:Chapter2providesasummaryofGEP3anditsevaluationframework.Chapter3presentsthemainfindingsofthebaselineassessmentoftheGEP3ToC.Chapter4startswithabriefpresentationofGEP3’searlylearninginterventionandtheimpactevaluationmethodology,followedbythemainfindingsofthebaselinedatacollection.Chapter5summarisesthebaselinefindingsoftheevaluationofGEP3’sIQSS.Chapter6concludes.

©EDOREN 18

2 Projectandevaluationbackground

2.1 Objectives,scopeandinterventionstrategy

GEP3isaneight-yearproject(2012–20)thatseekstoensuremoregirlscompletebasiceducationandacquireskillsforlifeandlivelihoodsinfivenorthernNigerianstates:Katsina,Sokoto,Bauchi,NigerandZamfara.ItismanagedbyUNICEFinpartnershipwithfederalandstategovernmentsandfundedbyDFID.Theprojectaimstoimproveaccess,retentionandlearningoutcomesforgirls,andtoreducethedisparitiesbetweengirls’andboys’educationoutcomes.OverthelongtermtheprojectseekstocontributetoimprovedsocialandeconomicopportunitiesforgirlsinnorthernNigeria.

In2014theprojectwasre-designedanditsToCrevised,resultinginareducednumberofinterventions.GEP3worksonapilot-to-scale-upapproach.Overtheperiod2014–2017interventionsarebeingpilotedin210publicprimaryschoolsand200IQSsineachstate.2Duringthispilotperiod,interventionsaremainlyfundedbyGEP3.BytheendofthisperiodGEP3aimstosecurestategovernments’buy-intoinvesttheirownresourcestoscaleuppilotedinterventionsthathavehaddemonstrableresults.

GEP3supportsinterventionsinthreeoutputareas:

• Output1:increasedenrolmentandretentionofgirlsinbasiceducation;

• Output2:improvedteachingandlearningsupportforgirlsinbasiceducation;and

• Output3:improvedgovernancetostrengthengirls’education.

Intermsoffinances,40.7%oftheGEP3budgetisallocatedtooutput2,26.6%tooutput3and12.9%tooutput1.

Table1providesanoverviewoftheGEP3interventionsthatarebeingpilotedin2014–2017inthedifferentstates.GEP3’sinterventionsareintendedtobebroadlydeliveredthroughgovernmenteducationstructuresatstateandlocalgovernmentlevels.Governmentbodiesattheselevelsthereforeconstitutetheprogramme’sprimarystakeholders.

Table1: OverviewofGEP3interventionsduringthe2014–2017pilotperiod

Output Intervention States

Output1

Unconditionalcashtransferprogrammeinpublicprimaryschools NigerandSokoto

Enrolmentdrivesincatchmentareasofpublicprimaryschools Allstates

GirlsforGirls(G4G)groupsinpublicprimaryschoolsandIQSs Bauchi,Katsina,Zamfara

Output2

TeachercapacitydevelopmentinpublicprimaryschoolsandIQSsPublic primary schools:BauchiIQSs:Bauchi,Niger,Sokoto

HeadteachertraininginpublicprimaryschoolsandIQSsPublic primary schools: allstatesIQSs:Bauchi,Niger,Sokoto

EarlylearninginterventioninpublicprimaryschoolsandIQSs KatsinaandZamfara

Output3

CapacitybuildingofSBMCs/CBMCsinprimaryschoolsandIQSs(includingschoolgrants)

Public primary schools: allstatesIQSs:Bauchi,Niger,Sokoto

SupporttotheASC,theEMISandIQSdatacollection Allstates

Enhancedpoliticalengagementforgirls’educationthroughGESC Allstates

2InKatsinaandZamfara,interventionswillonlytakeplacein40IQSsineachstate,basedonaninitiallistof200pilotschools.

©EDOREN 19

SupporttoHiLWAmembers Allstates

AdvocacyforFemaleTeacherTraineeScholarshipScheme(FTTSS) Allstates

2.2 ObjectivesandscopeofGEP3evaluation

ThemainobjectiveoftheGEP3evaluationistoassesswhetherandhowwellinterventionscontributetoGEP3’sobjectives.InlinewithGEP3’semphasisonlearningoutcomes,theevaluationwillprioritisemeasuringchangesinlearningoverchangesinaccesstoeducation.Theevaluationfocusesoninterventionsthatabsorbahighshareofthebudget,havearelativelyweakevidencebase,areinnovativeinnatureandareofstrategicinteresttoGEP3managementandDFID.Thesearetheearlylearningintervention,GEP3’sIQSSandthecashtransferprogramme.ThefirsttwowillbesubjecttoEDOREN-ledevaluationanddatacollection,withGEP3researchstudiesprovidingadditionalevidence.EvidenceforthethirdinterventionwillbegeneratedthroughresearchstudiesandmonitoringconductedunderGEP3.

2.3 Evaluationdesignandmethods

During2014–2015,EDOREN,incollaborationwithUNICEFandDFIDNigeria,developedanevaluation

frameworkinparallelwithGEP3’sre-design.Theevaluationframeworkdefinedtheobjectives,scope,questions,designandpartnerresponsibilitiesoftheevaluation.Theevaluationfocusesontheperiod2014–2017inordertoinformdecision-makingaroundscale-up.Therestofthissectionoutlinesthemainelementsoftheevaluationframework.SubsequentchapterscontainmoreinformationonthedesignofthedifferentevaluationcomponentsledbyEDOREN.

2.3.1 Evaluationquestions

EvaluationquestionswereselectedconsideringtheOrganisationforEconomicCo-operationandDevelopment(OECD)DevelopmentAssistanceCommittee’s(DAC)criteriaforevaluatingdevelopmentassistance.Table2providesanoverviewofthekeyevaluationquestions.3ThefirstthreequestionswillbeaddressedthroughEDOREN-ledevaluation;thequestionrelatedtothecashtransferprogrammewillbeaddressedbyUNICEF-commissionedindependentresearch;andtheefficiencyquestionwillbeaddressedthroughtheDFID-managedannualreviewprocess.ThisreportfocusesontheEDOREN-ledevaluation.

Table2: Priorityevaluationquestions

DACcriteria Evaluationquestion

Relevance • IsGEP3’sToCappropriatetoachieveitsplannedobjectiveswithinthecontextof

theGEP3states?

Effectiveness/

Impact• Towhatextenthastheearlylearninginterventionimprovedlearningoutcomesin

theearlygradesofschool,especiallyforgirls?

• HowwellhasGEP3’sIQSScontributedtobetterlearningoutcomes,especiallyforgirls,byimprovingteachingandschoolleadershipandmanagement?

• Howhasthecashtransferprogrammechangedattitudesandbehaviourinregard

togirls’educationamongrecipientcaregivers?

Efficiency • Howwellhaveprojectorganisationalprocessesfacilitatedthedeliveryofexpected

outputs?Howcouldsuchprocessesbeimproved?

3TheGEP3EvaluationFrameworkdocumentpresentsacomprehensiveoverviewofallevaluationquestionsidentified(EDOREN2015).

©EDOREN 20

2.3.2 Evaluationdesignandmethods

Theevaluationquestionswillbeansweredbyapplyingvariousevaluationapproachesusingmixed-methodsdatacollection.Approacheshavebeenselectedinlinewiththecontext,availableresourcesandmethodologicalconstraints.Theuseofatheory-basedapproachisparticularlyappropriatefortheevaluationofGEP3asithelpsunpickhowGEP3interventionsaremakingadifferencewithintheirspecificcontext.Thisisusefulformakingjudgementsaboutwhethertheoutputsandoutcomesobservedduringthepilotphasewouldbelikelytooccurduringscale-up,whichinturnwouldinformdecisionsrelatedtoscale-up.Furthermore,itallowscrediblejudgementstobemaderegardingthecausalrelationshipbetweenGEP3interventionsandtheiroutcomes,evenwhenavalidcounterfactualcannotbeconstructed.

EDORENwillimplementthreesetsofevaluationmethodologiesandactivitiestoanswertheevaluationquestionsthatareitsresponsibility:

• Thefirstsetofevaluationactivitiescentreonahigh-levelexplicationandexaminationofGEP3’sToCbasedonstate-levelKIIsanddocumentanddatareview.

• Second,EDORENwillconductanimpactevaluationoftheearlylearninginterventionbasedonarandomisedcounterfactualdesignusingmainlyquantitativeschoolsurveys.

• Third,GEP3’sIQSSwillbeevaluatedusingamixed-methodsapproachbasedontheprinciplesofcontributionanalysis.

ThedifferentdesignsaresummarisedintheirrespectivechaptersanddiscussedextensivelyintheBaselineTechnicalReportaccompanyingthissynthesisreport.

2.3.3 Inclusionandethics

TheevaluationadherestoDFID’sprinciplesofethicsinresearchandevaluation(DFID,2011)andfollowsUNICEF’sguidelinesonconductingresearchwithchildren(Grahametal.,2013).Stepstakentoensureethicsandinclusioninclude:ethicsreviewoftheevaluationdesignanddatacollectionprotocols;4attentiontoage,mothertongueanddisabilityduringdatacollectionwithchildren;anavoidanceofgenderbiasinmeasurement;5anevaluationdesignthatensuresdisaggregationbygenderwithsufficientstatisticalpower;analysisoflearningthatfocusesonpupilsatthelowendofthelearningdistribution;andintensivefieldresearchertraining.

2.3.4 Externalvalidityofthefindings

ThefindingsofthequantitativebaselinesurveysarerepresentativeoftheGEP3schoolsinwhichtheearlylearningandIQSSinterventionsarebeingimplemented.TheearlylearninginterventionisbeingimplementedinasubsetofGEP3LGAsthatwerepurposivelyselectedfortheintervention.ThequalitativecomponentoftheIQSSevaluationwasnotdesignedtoproduceresultsthataregeneralisable,butinsteadprovidesinsightsthatcomplementthequantitativefindings.Thetheory-basedapproachenablesustounpickhowandunderwhatconditionsimpactonlearningoccurs,whichallowsformoregeneralisablefindingsthatgobeyondtheinterventiongroups.

4TheevaluationdesignanddatacollectionprotocolswereapprovedbyOPM’sEthicalReviewCommittee.Inaddition,datacollectionprotocolsweresubmittedtotheNationalHealthResearchEthicsCommitteeofNigeria,whichprovidedexemption.5ARaschpsychometricanalysisofpupiltestitemswasundertakeninordertoidentifywhethertestitemswouldbeinterpreteddifferentlybygirlsorboys.Itwasfoundthattheitemsdidnotdiscriminateonthebasisofgender.

©EDOREN 21

3 BaselineToCassessment

3.1 Methodology

ThischapterpresentsthebaselineassessmentofGEP3’sToC.ItfocusesontwoevaluationquestionsrelatedtotherelevanceofGEP3:

• HowplausibleisGEP3’sToCinthecontextoftheGEP3states?

• HowappropriateareGEP3interventionsintermsofimplementationstrategy?

TheplausibilityoftheToCisexaminedby:(i)reviewingstakeholders’commonunderstandingofitsintendedoutcomes;(ii)interrogatingthelogicofitsoutcomechain;and(iii)makingexplicitthosefactorsthatareeithernecessaryfor,orahindranceto,achievingthestatedoutcomes.TheappropriatenessofGEP3’simplementationstrategyisassessedonthefollowingdimensions:(i)stakeholderinvolvementinGEP3implementation;(ii)implementationcapacity;and(iii)thelikelihoodthattheprogrammewillreachthetargetpopulationinanequitablemanner.

ThisassessmentisbasedonKIIs,largelywithGEP3state-levelstakeholders,conductedinAugust2015.KIswerepurposivelyselected,withsupportofGEP3stateteams,basedontheirknowledgeoftheprojectandspecificinterventions.KIsweregovernmentstaffinmanagementpositionsintheeducationministries,departmentsandagenciesthatGEP3partnerswith;representativesofgovernmentornon-governmentagenciesinvolvedinGEP3implementation(includingGEP3stateteamsandUNICEFfieldofficestaff);andnon-governmentinformantsthatcouldprovidefurtherinsightsregardingspecificinterventions.

3.2 PlausibilityofGEP3’sToC

Summaryanswerstoevaluationquestions

Q1:HowplausibleisGEP3’sToC

inthecontextoftheGEP3

states?

Theplausibilityofthere-designedGEP3ToCisbackedbyitspotentiallysynergetic,coherentlogicandstakeholders’commonunderstandingofitsmainobjectivesandoverarchingstrategy.Criticalassumptionsrelatedtothemanagementandresourcingoftheeducationsystemarehighlyuncertain–inparticular,thereleaseofgovernmentfunding,schoolinvestmentandhumanresourcecapacityremainingonaparwithincreasedenrolment,effectivemonitoringatschoolandinterventionlevel,andSBMC’sabilitytodeliveronitsmultipleprojectresponsibilities.

SBMCsandMothersAssociations(MAs)haveapivotalroleinthesuccessofmanyinterventions.SBMC/CBMC/MAfunctionalityandcapacitytoundertakethemultiplerolesassignedaspartofGEP3interventionsisacriticalassumptionofGEP3’sToCandrequiresclosemonitoring.

TheplausibilityofGEP3’sscale-upapproachisdependentonuncertaingovernmentfundingandvariesbyintervention.Thescale-upoftheIQSSisparticularlyuncertainbecausetheinstitutionalmandateoverIQSsisunclear,andduetotheSAME’slimitedanduncertainaccesstoresources,thelimitednumberofwell-establishedIQSsavailableforscale-up,andsupply-sideconstraints(particularlywithregardstofacilitators).

©EDOREN 22

3.2.1 Generalassessment

StakeholdersgenerallyhaveacommonunderstandingofGEP3’sintendedoutcomesandoverarching

strategy,butlackanunderstandingoftheproject’soperationaldesign.Stakeholderscommonlyrecogniseincreasedgirls’enrolmentasthemainintendedoutcomeofGEP3.Whilstretentionandlearningareconsideredimportant,theyarereferredtolessfrequently,particularlyamonggovernmentKIs,whichsuggestsGEP3isforemostassociatedwithimprovinggirls’accesstoschool,representedbyitstargetofgetting1millionadditionalgirlsintoschool.StakeholdershaverelativelygoodknowledgeofGEP3’soverallstrategyandacknowledgethatthere-designhasimprovedtheproject’sfocus.Atthesametime,theproject’soperationaldesignisnotalwaysunderstood.Attributesofintervention-specificoutputsandoutcomes—suchasprioritytargetgroups,thespecificationandmeaningofoutputs/outcomes,andtimelines—areoftennotcleartostakeholders.ThiscouldmakeitdifficultforthemtoassessGEP3’sperformanceorbeheldaccountablefortheresultsachieved.

GEP3’sdesignisperceivedtobecoherent,withpotentialsynergiesbetweenoutputsandintervention.

However,thesesynergiesalsoconstituteariskofinterventionproblemsspreadingacrossthe

programme.GEP3seekstoholisticallyimproveeducationaccess,qualityandgovernanceinthefivestatesinwhichitworks.Therearesynergiesacrossitsinterventions,whichhavethescopetoreinforceoneanother’soutcomes.However,thisalsomeansthatifacertaininterventionispoorlyimplemented,thiscouldunderminetheimpactofotherGEP3interventions.Forinstance,successfuleffortstobuildthecapacityofSBMCswouldsupporttheoutcomesofinterventionsseekingtoimproveaccess,suchasenrolmentdrives,whichareimplementedbySBMCs.Conversely,ifeffortstobuildSBMCs’capacitywereunsuccessful,thiswouldunderminetheimplementationandefficacyofenrolmentdrives.

Stakeholdersemphasisedthatthetimelyreleaseoffundsbystategovernmentsisacentraluncertain

factorthatcouldunderminethesuccessoftheproject.Stakeholdersacknowledgethattheprojectre-designhasmitigatedtheriskofstategovernmentsnotprovidingtheircommittedfinancialcontributions.Thishasbeenachievedthroughanemphasisonhigh-leveladvocacyandafocusonamoremanageablesetofoutcomes.The2015electionsarealsoseentohaveresultedinincreasedstatesupporttoeducation.Nonetheless,stakeholdersindicatethatthenon-releaseoffundsremainsanimportantconcern,withfundreleasedependentonthedecisiveinfluenceofthegovernor.

EffectivemonitoringwashighlightedbyseveralKIsaskeytothesuccessoftheproject.ThiscorrespondstobothmonitoringandoversightofschoolactivitiesbytheSBMC,headteacherorlocal/stategovernmentofficers,aswellasmonitoringofGEP3projectimplementation.SomeKIslinkthistoaneedforinterventionstohaveastrongerresultsorientation,withpeopleheldtoaccountforresults.KIsacknowledgethatmonitoringsystemsareweak,althoughinvestmentshavebeenmadeinthepasttostrengthenmonitoringandqualitycontrol.6Therefore,GEP3rightlyemphasisesmonitoringinitsToC.Effectivemonitoringrequiresacommonunderstandingofitsobjectives,rolesandresponsibilities,andapproach,andsufficientresourcesforimplementation.

Contextualfactorsarelikelytoinfluenceprojectoutcomes.InsecurityandconflicthavehadanimpactonschoolinginBauchi,ZamfaraandKatsina.WhilestakeholdersindicatedthatGEP3schoolswerenotstronglyaffectedatthetimeoftheinterviews,insecuritycontinuestoposeriskstotheproject.GovernmentpoliciesandstructuresmayalsoleadtovariationsinGEP3outcomesacrossstates.Forexample,somestateshaveagenciestargetinggirls’education.Theseagenciesmayintroduceinitiativesthatcouldinfluencegirls’educationoutcomes.Similarly,otherdonor-fundedprojectsoperatingintheGEP3statescouldcontribute

6InSokoto,schoolqualityassuranceofficershavebeenscreenedandtrainedwiththesupportoftheNorthernEducationInitiativeproject.InZamfara,theSUBEBrecentlysuppliedqualityassuranceofficerswithnewmotorcyclesandfuelallowances,accordingtooneKI.InBauchi,theSUBEBincreasedthenumberofitszonalofficesfromthreeto10,soastohavemorequalityassuranceunitsclosertotheschools.

©EDOREN 23

tosomeoftheproject’soutcomes.ThiswillmakeitdifficulttoattributeimprovementsineducationoutcomestoGEP3.

3.2.2 Output1:Increasedenrolmentandretentionofgirlsinbasiceducation

Achievingtheproject’sobjectiveofincreasinggirls’enrolmentisplausible,particularlyinprimary

education.Stakeholderswereoftheviewthatenrolmentdrivesandcashtransfersarebotheffectivestrategiesthataddressimportantbarrierstogirls’education,andthattheycomplementoneanother.Enrolmentdrivesareaninterventionthatissupportedbystakeholdersatdifferentlevels.Thesedrivesareabletopenetratethecommunities,withtheactiveinvolvementofthosecommunities.Similarly,thecashtransferinterventionscancountonahighlevelofinterestinthetargetedcommunities,accordingtotheprojectimplementationunits,whichhavereceivedstrongcapacitysupporttosuccessfullyimplementtheintervention.Fundingdeficienciesresultingindelayedcashtransfersconstitutearisktoachievingenrolmentobjectives.Inareaswherebothenrolmentdriveandcashtransferinterventionsareimplemented,girls’enrolmentisexpectedtoincrease.AstheGEP3re-designhasshiftedtheproject’sfocusfrombasiceducationtoprimaryeducation,favourableresultsareexpectedtobeachievedinprimaryeducation.

Retentionisuncertainduetosupply-sidechallenges.Stakeholderswereoftheviewthatitwillbemoredifficulttoretaingirlsinschool.Retentionisshaped,amongotherthings,bysupply-sidefactors,suchasthepresenceofsufficientteachers,thequalityofteaching,andthequalityoftheschoolenvironment.AkeyassumptionoftheGEP3ToCisthatgovernmentcansupplyprimaryschoolsandteacherstomeetincreasingdemand.InallstatesKIsperceivedthatincreasedenrolmentisnotsufficientlyaccommodatedintermsofkeyresourcessuchasteachersandclassrooms,althoughsomeseethisasaproblemthatispresentmainlyinurbanareas

SBMCsandMAsareconsideredcentraltothesuccessofinterventionstoimproveaccess.StakeholdershighlightthecentralrolethatSBMCsandMAsplaynotonlyinsensitisingcommunitiesabouttheimportanceofgirls’schoolingbutalsoinfacilitatinginterventionimplementationandsupply-sideresponsestoincreasedenrolment.TheirfunctionalityandcapacitytoundertakethesemultiplerolesisacriticalassumptioninGEP3’sToC,andonethatrequiresclosemonitoring.ThisisreflectedinGEP3’sSBMCeffectivenessmonitoringsystem.

3.2.3 Output2:Improvedcapacityofteacherstodelivereffectivelearningforgirls

Thequalityofteachingandteachersisconsideredcentraltoimprovinglearningoutcomes.Stakeholdersarelesslikelytorecogniselearningasakeyexpectedprojectoutcome,thanenrolmentandretention.Whentheydo,theyacknowledgethatlearningoutcomesaremoredifficulttoachievethanincreasingaccessforgirlsduetothelowqualityofteaching,particularlyinruralareas,andalackofinvestmentinthesupplyofqualityeducation.ThereisawideconsensusamongtheKIs,inlinewithGEP3’sToC,thattacklingqualityofteachingandteachersiscentraltoimprovinglearningoutcomes.Factorssuchasteachercommitmentandmotivation,theirdeploymentandtransfers,initialskillsandknowledge,theschoolenvironment,andmonitoringandsupervisionarelikelytoplayanimportantroleindeterminingthefinaloutcomeofteachercapacitydevelopmentinterventions.

Stakeholdersarenotsufficientlyawareofthemoreconcreteattributesofthetrainingapproachand

outcomes.Althoughtheteachercapacitydevelopmentcomponenthasbeenre-designed,atthetimeoftheinterviewsgovernmentstakeholdersinvolvedinteachertrainingdidnotyetunderstandwhatcapacitiesneedtobedeveloped,inwhatways,andwhen.WhileawelltargetedteachercapacitydevelopmentapproachispostulatedasanassumptionintheToC,whatthistargetingconcretelyentailsremainsunclear.TheplausibilityoftheToCwouldbeincreasedifsuchattributesweremorecommonlyunderstood,sotheycanbemonitoredandaccountedfor.

©EDOREN 24

ThewidevariationinthebackgroundofIQSfacilitatorswarrantsspecificattentionandadaptation.7ThereappearstobelargevariationinthebackgroundoftheIQSfacilitators,andamongtheIQSsinwhichtheyareteaching,intermsoffacilitators’teachingexperience,teachinghours,theavailabilityoflearningmaterials,teachingfacilities,pupilcharacteristicsandthetypeofIQS.HeadteachersupportwillvaryintermsofhowthisroleisfilledandbywhomsinceaheadteacherpositionisnotnecessarilyaseparatepositionfromtheproprietororMallamintheIQScontext.Therefore,notonlydoesthecapacitybuildingapproachneedtobeadaptedtotheIQScontext,butitalsoneedstotakeintoaccountlargevariationsacrossIQSs.RemunerationofIQSfacilitatorsappearstobeparticularlyproblematicforIQSfacilitators.ThebaselinefindingsoftheIQSSevaluationconfirmthataverysmallshareoffacilitatorsarepaid,despiteanallowancebeingpolicyinthestates.Whilefacilitatorsarenotmotivatedsolelybyremuneration,thebaselinefindingsfindapositiveassociationbetweenreceivingremunerationandteachermotivation,andindicatethatschoolstakeholdersstatethatitisdifficulttoholdnon-paidfacilitatorstoaccount.

Awiderangeofstakeholdersunderlinetheimportanceoftheearlylearningintervention.Stakeholderssupporttheemphasisonliteracyduringearlygradesasafoundationalskill.Thereisnooverallconsensusamongstakeholdersabouttheextenttowhichtheuseofamothertongueisanecessaryconditiontoimprovelearningoutcomes.ThebaselinefindingsoftheearlylearningevaluationhasconfirmedthatHausaisthelanguageoftheimmediateenvironmentinKatsinaandZamfara,whichvalidatestheToCassumptionforthisintervention.However,whileallteacherssurveyedreportedspeakingHausa,Hausaliteracylevelsamongteachersarelow,whichneedstobetakenintoaccountinthedevelopmentofHausateachingandlearningmaterials.ImplementingearlylearninginIQSswillrequirecarefuldesignbecauseofthedifferenceingradelevelscomparedtopublicprimaryschools,theuseofamothertongueinthisgradestructure,andthevarietyofwaysinwhichformaleducationisorganisedintheIQS.

3.2.4 Output3:Improvedgovernancetostrengthengirls’education

GEP3providessupportattheschoollevelandthestateleveltoimprovegovernancetostrengthengirls’education.

Attheschoollevel,SBMCempowermentisbothapivotalintermediaryoutcomeinGEP3’sToCaswellas

oneofitsmostprecariouslinks.TheexpectationsplacedonSBMCs,intermsofrolesandresponsibilities,areveryhigheventhoughthecapacitybuildingandempowermentprocesswillbegradual,andoftenstartsfromalowbase.ThecompositionoftheSBMCs,theiracceptanceintheschoolandcommunity,theiraccesstoschools,andtheinstitutionalisationofgovernmentsupporttoSBMCsarekeyfactorsthatwillinfluencetheircapacitydevelopment.TheyfaceamorechallengingsituationinIQSssincetheirempowermentisbasedonalargersetofassumptions,suchasSBMCsbeingestablishedinthefirstplace,theirbeingacceptedbytheIQSproprietorandtheacceptanceofintegrationinthecommunity.ItisalsoassumedthatIQSsarenotmobileorthatiftheyare,thisdoesnotaffecttheirfunctionality.

StakeholderssupportthesignificantroleofSBMCsintheschool-levelgovernanceofeducationbutthe

scopeoftheirgovernanceroleremainstobeclarified.StakeholdersconsiderSBMCsakeyinstrumentbywhichtopromotecommunityownershipofschoolsandaddressdemand,supplyandgovernanceproblems.TheynotethatchannellingfinancialsupporttoschoolsthroughSBMCs(forinstancethroughtheprovisionofGEP3grants)canmakeanimportantcontributiontotheirempowerment.WhileSBMCs’roleinschool-levelgovernanceisbroadlysupported,theirroleinregardtoinfluencingwidereducationprocesses—suchasholdingstateandlocalgovernmentstoaccountforservicedelivery—isconsideredpromisingbynon-governmentKIsbutremainsunaddressedbygovernmentKIs.WhileGEP3doesnotaimtosupportagovernanceroleforSBMCsbeyondtheschoollevel,thisislikelytohaveitslimitationsbecauseseveralaspectsofschools’performance(suchashumanresourcemanagement,provisionoflearningmaterials,schoolinfrastructure)dependonstateandlocalgovernmentaction.

7TeachersinIQSsarereferredtoasfacilitators.

©EDOREN 25

Atthestatelevel,HiLWAmembersengagewithdecision-makersandschoolcommunitiesandhavethe

potentialforindirectinfluence.HiLWAmembersseemtobeinapositiontoengagewithdecision-makerstopromotewomen’sandgirls’participationintheeducationsector.TheyalsohavethescopetopromoteGEP3’soutcomesattheschoollevelthroughtheirinvolvementinenrolmentdrivesandengagementwithSBMCs.HiLWAmembers’influencecomesfromthecredibilityoftheirmembers.Theircapacitytoinfluenceismainlyindirectandshouldnotbeoverstatedasmostmembersdonotholdpositionsofdirectinfluence.

TheGESCsoffersamulti-stakeholderplatformtosupportgovernancearoundgirl’seducationtothe

extentthatitisactive,sufficientlylocallyownedandaddresseskeychallenges.GEP3seekstoimprovegovernancethroughtheGESCsatthestateandnationallevels.Stakeholdersappreciatethemulti-sectorandmulti-stakeholdercharacteroftheGESCs.TheGESCsmetinfrequentlyduringthefirstpartof2015duetotheprojectre-designandelections,althoughinseveralstatesitseemstohavebeenreactivated.Itseffectivenesscanlikelybestrengthenedbymoregovernmentownership.ItwouldalsobeinabetterpositiontocontributetoGEP3’soutcomesifmonitoringandpoliticalengagementaroundGEP3fundingwereexplicitlyincorporatedinitsmandateandplannedfor.

GEP3’ssupporttoEMISandASCwillplausiblyimprovedatareliability,althoughthismaynotnecessarily

translateintodatabeingusedinpolicy-makingprocesses.ThecapacityneedsintermsofgeneratingreliabledataarecommonlyunderstoodandGEP3supportseemstobewelltargeted.Althoughstakeholdersexpectthatreliabledatawillbeusedinpolicy-makingprocesses,GEP3’sToCdoesnotconsidertheinterfacebetweendataproductionandusageandleavesthetranslationofdataproductionintouseasanimplicitassumption.ItwouldbeworthexaminingwhetherthedemandsideofEMISneedsstrengtheninginorderachieveeffectiveuseinevidence-basedplanning,resourceallocationandpolicyorientationtoimprovegirls’education.IncorporatingIQSdatainEMIS(beyondthoseIQSsthatarewellalignedwiththeformaleducationsystem)doesnotseemplausibleatthismoment,notleastbecauseofalackofreliableschoolidentificationdataandthecapacityconstraintsfacedbySAMEsinregardtoregularlyandsystematicallycapturingIQSdata.

3.2.5 GEP3’spilot-to-scale-upapproach

Thenatureofscale-upisnotcommonlyunderstoodanditsimplementationisdependentonuncertain

governmentfunding.AkeyassumptionunderlyingGEP3’sstrategyisthatstategovernmentsarewillingandabletoimplementlarge-scalechangeforgirls’education.GovernmentfundingavailabilitywascitedasthemainuncertainconditionforGEP3scale-up.Stakeholdersalsodidnothaveaclearunderstandingofhowscale-upwouldhappen.Limitedplanninghadtakenplaceatthetimeoftheinterviewsin2015.StateGEP3teamsandUNICEFstaffinterviewedrecognisethatplanningforscale-upisimportantandneedstostartwellaheadof2017.

Theplausibilityofscale-upvariesacrossinterventions.Thescale-upofteacherandSBMCcapacitybuildingisrelativelymoreplausiblebecausetheneedfortheseinterventionsiswellrecognisedandasupportinginstitutionalframeworkisinplace.However,scalingupthecashtransferschemeislessplausiblebecauseitiscapitalintensive,isnotwellinstitutionalised,andisseenasadonorpriority.Thescale-upofIQSSisparticularlyuncertainbecauseofalackofclarityaboutwhichinstitutionisresponsibleforintegration,theSAMEs’limitedanduncertainaccesstoresources,thelimitednumberofwell-establishedIQSsavailableforscale-up,andsupply-sideconstraints–particularlywithregardstofacilitators.

©EDOREN 26

3.3 AppropriatenessofGEP3’simplementationstrategy

Summaryanswerstoevaluationquestions

Q2:HowappropriateareGEP3’s

interventionsintermsof

implementationstrategy?

Theprojectismakinganappreciatedefforttoinvolveavarietyofstakeholders,althoughtheirinvolvementinoperationalplanningremainsunevenacrossinterventions.

Governmentcapacitybuildingisembeddedinprojectdesign,butaneffectivesystemisneededtoidentifyandaddressthemostimportantcapacitydevelopmentneeds.MonitoringcapacityisrightlyemphasisedinGEP3’sre-designbutitsoperationalisationhasyettoadvance.

SeveralGEP3interventionsaredesignedtobespecificallyequityenhancing,butempiricalverificationisrequiredastowhetherthemostvulnerablehouseholdsandgroupsareabletobenefitandactivelyparticipateinGEP3interventions.GEP3isnolongeroperatinginLGAswiththehighestgendergapinallstates.

3.3.1 Stakeholderinvolvement

Theprojectismakinganappreciatedefforttoinvolveavarietyofstakeholders,althoughtheir

involvementinoperationalplanningremainsunevenacrossinterventions.GEP3’soperationalplansupportsstakeholderownershipbyemphasisingimplementationwithandthroughstateeducationpartners.TheoperationofGEP3fromwithinstategovernmentscontributestogovernmentinvolvementandcoordinationbetweentheGEP3stateteamandgovernmentstaff.However,governmentinvolvementremainstosomeextentexternallydrivenandgovernmentpartnersarenotalwaysawareof,orlackclarityabout,operationalplanning.Theoperationalisationofstructuresandsystemsthatfacilitategovernmentinvolvement,liketheGESC,havebeeninfluencedbychangesinstaffingandtheelectionsin2015.Stakeholdersconsidertheinvolvementofabroadrangeofstakeholdersatdifferentlevels(community,local,stateandnationallevel)astrengthofGEP3’simplementationstrategy(inparticulartheparticipationofSBMCsandMAs).

3.3.2 Governmentcapacity

Governmentcapacitybuildingisembeddedintheprojectdesign,butaneffectivesystemisneededto

identifyandaddressthemostimportantcapacitydevelopmentneeds.GEP3providesdirectsupporttogovernmentstaffandmobilisesspecifictechnicalexpertisetoincreasegovernmentcapacity.Thisisdemonstratedmoreatthestatethanatthelocallevel,andvariesacrossinterventions.Giventhelimitedresourcesandthecapacityconstraints,particularlyatthelocallevel,identifyingandaddressingwherecapacityneedsarehighest,isessential.GEP3’simplementationislikelytobeinfluencedbyinstitutionalcapacityconstraints,particularlythelackofclarityregardingthedivisionofrolesandresponsibilitiesfortheintegrationofformaleducationinQur’anicschoolsacrosstheSAMEsandStateUniversalEducationBoards(SUBEBs).

MonitoringcapacityisrightlyemphasisedinGEP3’sre-designbutitsoperationalisationhasyetto

advance.Moststakeholdersnotedthatmonitoringandsupportivesupervisionareessentialforprojectsuccess.However,theyalsohighlightedthatthegovernmenthasinadequateorganisationalcapacitytomonitorinterventions.Therefore,GEP3hasrightlyemphasisedstrengtheningmonitoringcapacityaspartofthere-design.However,theresultsinthisregardwilldependonitsoperationalisation,whichhadnotyetadvancedatthetimeoftheinterviews.Thiswasindicated,forinstance,bythelowqualityinformationaboutGEP3-supportedIQSsthatwasprovidedforthebaselinesampling.

©EDOREN 27

3.3.3 Equity

GEP3isnolongeroperatinginLGAswiththehighestgendergapinallstates.EquityineducationisacentralconsiderationforGEP3,astheprojectintendstoreducegenderdisparitiesineducation.WhiletheprojecttargetedLGAswiththehighestgendergapineducationatthestart,atthetimeoftheGEP3re-designthetargetedLGAswerenolongeralwaysthosewiththehighestgendergap.Basedon2014–2015schoolcensusdata,GEP3ismostlyoperatinginLGAswitharelativelyhighprimarygendergapinprimaryeducationinBauchiandNiger,whileinKatsina,ZamfaraandSokotoGEP3isimplementedinLGAswithrelativelylowgenderdisparitiesinprimaryschools.

SeveralGEP3interventionsaredesignedtobespecificallyequityenhancing.Thecashtransferprogrammetargetsschoolswiththehighestproportionofout-of-schoolgirls.TheIQSSismeanttoexpandaccesstoqualitybasiceducationformarginalisedchildreninrurallocations.However,thedegreetowhichthemostvulnerablegroupsareactuallyreachedwillbeinfluencedbyhowequityconsiderationsareoperationalisedontheground.Forexample,empiricalverificationisrequiredastowhetherthemostvulnerablehouseholdsareabletoreceivethecashtransferasplanned.ActiveparticipationinSBMCs/CBMCsmaybelimitedtoonlyspecificcommunitymembers,duetotheconstraintsimposedbysocialnormsandschoolaccessibility.

©EDOREN 28

4 Baselineoftheearlylearninginterventionevaluation

4.1 GEP3’searlylearningintervention

GEP3’searlylearninginterventionaimstoimprovetheearlylearningskillsofchildreninprimaryGrades

1to3(P1–P3)inthemothertongue,whilealsopreparingchildrentolearnwithEnglishasalanguageofinstructionbythetimetheytransitiontoGrade4.Akeymeasureoftheintervention’ssuccesswillbeimprovedliteracyskills.Tothisend,UNICEFhascontractedathirdpartytoimplementtheRANAproject.

RANAwillbeimplementedoverathree-yearperiod(2016–2018)insixLGAsinZamfaraandKatsina

(threeLGAsperstate).Theinterventiontargets120publicprimaryschoolsand80IQSsthatformpartofGEP3’spilotschoollists(60publicprimaryschoolsand40IQSsperstate).ThemainactivitiesthatRANAwillimplementattheschoolandcommunitylevelarepresentedinBox1.

Box1.RANAactivitiesatschoolandcommunitylevel

• ProvisionofapackageofteachingandlearningmaterialsinHausa

• Earlygradeteacherin-serviceprofessionaldevelopment,includingcluster-leveltraining,weeklyschool-basedpeermentoring,on-sitemonthlysupervisorysupportandmonitoring,andheadteachertraining

• Communityawarenessandengagementactivities,includingcommunitysensitisationandtheappointmentofaliteracychampionamongtheSBMC/CBMCtocoordinateliteracyactivities

ThematerialsandteacherprofessionaldevelopmentwillberolledouttogradesP1andP2duringthesecondandthirdtermofthe2015/2016academicyear.P3willbecoveredfromthestartofthe2016/2017academicyear.TheinterventionpackagewillbesimilarforpublicprimaryschoolsandIQSs.TheRANApilotprojectisscheduledtoendinAugust2018.

AnnexApresentstheToCoftheearlylearningintervention.Itdescribesandvisualisesthecausalpathwaysofhowtheearlylearninginterventionisassumedtoimprovepupils’literacyinearlygrades,aswellasdiscussingunderlyingassumptions.ThemaincausalassumptionunderlyingtheToCisthatliteracylearningoutcomes,particularlyinHausaasthemothertongue,willimproveinearlygradesifteachingpracticesimprovethroughtheuseofimprovedteachingandlearningmaterialsandthepresenceofmoreknowledgeableandskilledteachers.

4.2 Baselinemethodology

TheevaluationofGEP3’searlylearninginterventiontakesatheory-basedapproach.Theintervention’sToCwasusedasaframeworktoformulatetheevaluationquestions(listedinAnnexB).ThesequestionsaddresstheimpactoftheearlylearninginterventiononarangeofintermediaryandfinaloutcomesidentifiedintheToC.Besidestheintervention’sToC,thechoiceofquestionshasbeenbasedonthefollowingconsiderations:anemphasisonmeasuringimpactonlearning;theinclusionofanequityperspectivebyseekingtounderstandtheimpactonthelowestperformingpupils;apreferenceforquantifyingattributableimpact;andafocusonthelinkagebetweenteachers’knowledgeandskills,andteachers’classroompractices,andlearning.

TheevaluationisdesignedasaclusteredRCT,stratifiedbyLGAandtypeofschool(primaryschoolvs.IQS),andrandomisedattheschoollevel.TheRCTdesignallowstheevaluationteamtomeasuretheattributableimpactoftheearlylearninginterventiononlearningoutcomesbycomparingoutcomechangesinatreatmentgroupofschoolswiththoseinacontrolgroupthatisstatisticallysimilaronaverage.Foreach

©EDOREN 29

typeofschoolanequalsizedsampleofschoolswasrandomlyselectedfromamongallGEP3schoolsineachofthesixGEP3LGAsinwhichRANAisimplemented.IneachLGA,halfofthesamplewasthenrandomlyassignedtoatreatmentgroupandtheotherhalftoacontrolgroup.TheRCTdesigniscombinedwiththeoverarchingtheory-basedevaluationapproach,whichmeasuresnetchangesinintermediaryandfinaloutcomesalongtheassumedcausalchain.Thiswillallowtheevaluationteamtounpackhowchangetakesplace.

Inordertomeasurechangesinoutcomesapanelsurveywasdesignedfordatacollectionatbaseline,

midlineandendline.Datawillbecollectedateachofthesethreestagesinasampleof120publicprimaryschoolsand120IQSsdrawnfromthesixRANALGAsacrossKatsinaandZamfara.HalfofthepublicprimaryschoolsandIQSsformpartofthetreatmentgroup;thisconsistsofschoolsthatreceivetheearlylearningintervention.Theotherhalfservesasthecontrolgroup,inwhichtheearlylearninginterventionwillnotbeimplemented.BaselinedatacollectionwasconductedbyOPMNigeriainOctober–November2015,duringthefirsttermofthe2015–2016schoolyearbeforethestartofimplementationoftheearlylearningintervention.8Thesurveywassuccessfulinachievingtheplannedschoolsamplesize9.

Withinthesampleschools,maleandfemalepupilsandteacherswererandomlysampledtoformpartof

apanelsurvey.10Thetargetpopulationswere:pupilsenrolledinP2grade(oritsequivalentinIQSs)andteachersteachinginP1–3grades(orequivalentinIQSs).Thefinalsampleconsistedof2,653pupils,ofwhich45%aregirls,11and477teachers(seeAnnexC).12Inaddition,theheadteacherineachschoolwassurveyed.Alldatahavebeenanalysedusingsamplingweights,andthereforerepresenttheentiretargetedpopulation.

Sevendatacollectioninstrumentswereadministeredwithineachsampledschool:pupilEnglishandHausaliteracyassessments,ashortpupilquestionnaire,ateacherquestionnaire,ateacherknowledgeandskillsassessment,ateacherclassroomobservationandaheadteacherquestionnaire.Exceptfortheteacherassessments,alldatawerecollectedelectronicallyusingcomputer-assistedpersonalinterviewing(CAPI).

Literacyassessmentswerecarefullydesignedandpilotedtoensureitemdifficultymatchedpupils’

ability.Raschmodelling(usingitemresponsetheory)wasusedtotesttheitems’psychometricpropertiesandtoplacepupilsanditemsonthesamemetric.LiteracyproficiencybandsweredrawnthroughabenchmarkingworkshopundertakenwithstakeholdersofseveraleducationprojectsinnorthernNigeria.13Teacherassessmentswereanalysedusingsubscalesforsubject,curriculumandpedagogicalknowledge.

4.3 Comparisonofbaselinecharacteristicsbetweentreatmentandcontrol

schools

Theevaluationdesignusesrandomisationtoensurethatnosystematicdifferencesexistbetweentreatmentandcontrolschools.Itthusaddressestheissueofselectionbiasinimpactmeasurement.Weassessedwhethertherandomisationhadachieveditsintendedpurposebycheckingwhetherkeyoutcomevariablesandschool,teacherandpupil-levelcharacteristicsdifferedbetweenthetreatmentandcontrol

8Midline data collection is scheduled forMay–June 2017,whichwill be the fifth school term of RANA implementation. This isbeforetheinterventionisscheduledtofinish.ThetimingofthemidlinehasbeenchosentoallowforevaluationfindingstoinformGEP3scale-updecision-makingin2017.Theexacttimingoftheendlinedatacollectionhasyettobedecided.922%oftheoriginallysampledIQSshadtobereplacedatthestartofthebaselineduetotheIQSsnotbeingconsideredeligibleforthesurvey(seeBaselineTechnicalReport).10Thesamepupilsandteacherssurveyedatbaselinewillberesurveyedatmidline.11Anequalnumberofgirlsandboysweretargetedforsamplingwithineachschool–thatis,sixgirlsandsixboys.12Thefinalpupilsampleequals92%ofthetargetedsamplesize(96%inpublicprimaryschoolsand88%inIQSs),whilethefinalteachersamplerepresents80%ofthetargetedsample(83%inpublicprimaryschoolsand74%inIQSs).13AbenchmarkingworkshopwiththeTeacherDevelopmentProgramme(TDP),GEP3,DevelopingEffectivePrivateEducationNigeria(DEEPEN)andEducationSectorSupportProgrammeinNigeria(ESSPIN)stakeholderstookplaceinNovember2015.

©EDOREN 30

groups.Thegreatmajorityofvariablesinvestigateddidnotshowanystatisticallysignificantdifferencebetweenthetwogroups.Thisindicatesthattherandomisationhadworkedtocreatecomparablegroups.Forthesmallminorityofvariablesthatshowedadifference,themagnitudeandthedegreeofstatisticalsignificancearelowanddonotrepresentaconcern.AnoverviewofthemainrandomisationchecksareincludedinAnnexD.

4.4 Baselinefindings

Thissubsectionpresentsthemainfindingsofthebaselineoftheearlylearningevaluation.ThefindingsarestructuredaccordingtotheToC.Throughoutthetextwewillmakereferencetotheevaluationquestionsinsummaryboxes.Sincetheevaluationquestionsarecausalquestionsrelatedtotheearlylearningintervention,theycanonlybeansweredatmidline/endline.Nonetheless,wesynthesiseevidencethatprovideabaselineassessmentoftheoutcomesexpectedandassumptionsunderlyingthecausalchain.

4.4.1 Teachingintheearlygrades

TheearlylearninginterventionaimstoimproveHausa-basedliteracyteachinginearlygradesasanintermediaryoutcomethatmayincreasepupilliteracylevels.Inthissectionwefirstdiscusstheteachingcontextandsomecharacteristicsoftheteachersinearlylearningschools.Next,wepresentthebaselinedataonteacherknowledgeandskills.Wefollowthiswithadiscussionofteacherinstruction,theuseofteachingandlearningmaterials,andtheuseofHausaintheclassroom.Weendthesectionwithfindingsonteachermotivationandattendance,whichareconsideredtobeimportantassumptionsintheToC.

Summaryofevidencerelatedtoevaluationquestions

Q2.Towhatextentdoes

teachers’knowledgeinliteracy

andlanguageacquisitioninearly

gradesimproveasaresultofthe

intervention?

Teachersdemonstrateverylimitedknowledgeinthemajorityoftheareasrequiredtofunctioneffectivelyasateacher.WhileallteachersreportthattheyspeakHausa,lessthan40%areproficientinprimaryGrade1and2-levelHausa.

Theteacherknowledgeandskillstestcarriedoutatbaselineshowsthatthegreatmajorityofteachersdonothavesufficientcurriculum,pedagogicalorsubjectknowledgetofunctioneffectivelyasteachers.Inparticular,whileallteachersreporttospeakHausa,lessthan40%ofteacherswereabletodisplaycompetenceinGrade1and2-levelHausa.Thebaselineevaluationalsofoundthatthevastmajorityofteachersarenotabletodisplaycompetenceinevidencingjudgementsanddiagnosingpupils’workorwriting.Theseskillsareessentialforteacherstotargettheirteachingtopupils’learninglevels,whichinturniswidelyrecognisedasanimportantcomponentofeffectiveteaching.

Theverylowlevelsofknowledgeandskillsamongstteachersatbaselineindicatesthatthereissubstantialscopeforimprovementinthisarea.However,italsopresentschallenges,inthatthereisoftenalackofbasicfoundationalskillsonwhichtheprojectcanbuild.ThefindingsshouldserveasausefulinputtotheRANAimplementationteamwithregardstoensuringthattheinterventiondesignistailoredtoteachers’existingknowledgeandskills.

Thebaselineevaluationalsofoundthatmanyteachersintheearlygradearesubject-specificteachers.ThishighlightsthattrainingwillneedtobeproperlytargetedsothatteacherswhoteachHausaliteracyaretrained.

Q3.Towhatextentdoteacher

skillsinearlygrade,gender-

sensitiveinstructionimproveasa

resultoftheintervention?

Teachers’instructionalskillsareweak.

Teachersperformpoorlyonacompositeindexthatwasconstructedtoassesstheirskillsinearlygradeteaching.Inparticular,theextentofpupil-centredlearningislowandmostteachersmakelimitedeffortstolinkthelessonto

©EDOREN 31

previouslearningortolearningobjectives.Incontrast,timeontaskwashighintheclassesobserved,althoughthisneedstobeinterpretedcarefullyasthepresenceofanobservermayhaveledtoanincreaseintimeontask.

Theearlylearninginterventionreliespartlyonapeermentoringapproachtoimproveteacherskills.Thisrestsontheassumptionthatschoolshavemorethanoneteacherengagedinearlygradeliteracyteaching.However,thebaselinefoundthataconsiderableshareofIQSsonlyhaveonefacilitatorteachingtheintegratedcurriculum.

TheToCalsoassumesthatimprovementsinteachers’skillswillbesupportedbyheadteachers’pedagogicalleadership.Thebaselineevaluationfoundthatlessonobservationandfrequentsupportiveteachermeetingsarepractisedbyonlyhalfofheadteachersinthesample.Thisunderlinesthescopeforimprovementonthisfront.

Thebaselinesurveyattemptedtomeasuregender-sensitiveteachingpractices,butacrossallitemsextremecomplianceeffectswereobserved,significantlycallingintoquestionthevalidityandreliabilityofthemeasure.Therefore,gender-sensitiveteachingcouldnotbemeasuredreliablythroughclassroomobservations.Theteacherassessmentsdoindicatethatteachersareawareofobjectivestotargetgirlsinclass,astheystatethatitisimportanttofocusongirls,buttheirresponsesalsopointtothepersistenceofdeeplyingrainedgenderbias.

Q4.Towhatextentandhowdo

teachersadjustandchangetheir

classroompracticesasaresultof

theintervention?

Hausainstructioniscommon.However,pedagogicalcompetencetoenablepupilstolearnislow.Teacherattendanceandsupportivemonitoring—conditionsthatsupportclassroompractice—requireattentioninIQSs.

Thevastmajorityofteacherstestedatbaselinewereunabletodemonstrateproficiencyinknowingthelearnersintheirsetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstandorproficiencyintheselectionofappropriatelearningandteachingmaterials.RegardingHausainstruction,Hausawasusedinallclassesobserved.Inhalfoftheclassesanotherlanguagewasusedaswell.

Teachermotivationandattendancemayinfluencethetranslationofimprovedknowledgeandskillsintoteachingpractices.Overall,teachersconsidertheirroletobeimportantandenjoyworkingasteachers,whilstbeingconsciousofthefactthattheyarelimitedintheirabilitytocontributetopupils’learning.Atbaseline,60%ofteachersself-reportedthattheyhadbeenabsentatleastonceinthepreviousthreemonths.AbsenceratesarehigherinIQSsthaninpublicprimaryschools.Mostheadteachersinpublicprimaryschoolsreportthattheytakeactiontoimproveteachers’attendance,whilejustoverhalfdoinIQSs.ThefindingsindicatethatteacherattendanceinIQSsmeritsattention.Lessonobservationsindicatedthatstudentswereengagedduringmostoftheclass.

Thelargemajorityofschools(80%)reportthattheyreceivevisitsbygovernmentorotherorganisations,andthevisitsareregular.ThisismuchlessthecaseinIQSs.ItwillbeimportanttopayparticularattentiontomonitoringpatternsinIQSssoastoensurethatvisitsbyGEP3trainersandotherstakeholdersoccurasfrequentlyasforpublicprimaryschools.

Q5.Aremorereadingand

learningmaterialsinHausaused

intheclassroomduetothe

intervention?Dotheycontribute

tomoreeffectiveteachingand

learning?

AccesstoanduseofHausamaterialiscompletelyinadequate.

ThedistributionofapackageofteachingandlearningmaterialsaspartofRANAisthereforehighlyrelevantintermsofprovidingaccesstosuchmaterials.ItwillbeimportantforthematerialstobecarefullyadaptedtothelowHausaliteracylevelsamongstteachers,andthatpeer/supervisorysupportisregularlyavailabletofacilitatetheiruse.

©EDOREN 32

Teachingcontextandteachercharacteristics

Schoolsaremostlyruralandhavepoorinfrastructure.Thegreatmajorityofearlylearningschoolsarelocatedinruralareas,withonly14%locatedinurbanareas.Theaverageschoolhasslightlymorethanfourteachersteachinginearlygrades.Thisnumberissubstantiallyhigherinurbanareas(seventeachersinurbanareas,versusonaverageafewlessthanfourteachersinruralareas).Intermsofinfrastructure,almostallschoolsarereportedlyinneedofmajorrepairs.Onlyaminorityhaveaccesstoasourceofdrinkingwater.Withregardstoschoolcharacteristicsoftenassociatedwithagirl-friendlyenvironment,only26%ofschoolshaveseparatefunctionaltoiletsforgirlsand9%ofteachersarefemale.

TherearenotabledifferencesbetweenIQSsandpublicprimaryschools,withimplicationsforproject

implementation.IQSsaresubstantiallysmallerthanpublicprimaryschoolsintermsofnumberofteachers.Importantly,40%ofIQSsonlyhaveasingleteachertoteachtheintegratedcurriculumsubjects(thecorrespondingfigureforpublicprimaryschoolsis3%).Thisfindingsuggeststhatschool-levelpeermentoringisnotapplicabletoalargeproportionofIQSs.Sincepeer-to-peermentoringisanimportantassumptionunderpinningtheeffectivenessofthetrainingsenvisaged,itisrecommendedthattheprojectidentifiesalternativewaysofensuringthattrainedteacherscanprovidepeersupport,possiblybyengagingwithteachersfromdifferentbutnearbyschools.WhilealargershareofIQSshaveaccesstoasourceofdrinkingwaterandelectricity,theyhavelessroomsonaverageandfewerhaveaccesstobooksandplaygrounds.Theyalsohavelessaccesstoseparatefunctionaltoiletsforgirls,andalmostallteachersaremale.

Table3: SchoolcharacteristicsinpublicprimaryschoolsandIQSs

Publicprimary

schoolsIQSs Allschools

%schoolslocatedinruralarea 86%

Averagenumberofrooms 9 2 6

Averagenumberofteachersinearlygrades* 6 2 4

%femaleteachersinschoolteacherroster 14% 4% 9%

%schoolsinneedofmajorrepairs 92% 97% 95%

%schoolswithsourceofdrinkingwater 41% 47% 44%

%schoolswithaseparatefunctionaltoiletforgirls 46% 6% 26%

*Thisonlyincludesteachersteachingnon-religioussubjects.Source:GEP3Baselinesurvey2015

Headteachersmakeanefforttoaddressteacherattendancebutthereislessattentiontoteacher

instruction.TherearedifferencesbetweenpublicprimaryschoolsandIQSsinthisregard.Acrossallschools,71%ofheadteachersreporthavingtakenactiontoimproveteacherattendanceduringthepreviousterm.ThisdiffersstarklybetweenpublicprimaryschoolsandIQSs,asisshowninFigure1Error!Referencesourcenotfound..ThisdifferencemaybeduetothedifferenttypeofleadershipstructureinIQSs,whereheadteachersarenotalwaystheschoolprivateproprietorandmaynotbe,ormaynotperceivethemselvestobe,responsibleforteacherattendance.Itwillbeimportantfortheearlylearninginterventiontoensurethattherelevantchainofteachingresponsibilitywithineachschoolisclearlyunderstoodanditsinfluencemaximisedwhenrunningthetraining.Inbothtypesofschool,headteachersarelessactiveinmonitoringactualteaching(seeFigure1).Lessthan50%ofheadteachersobservedalessonduringthepreviousschooltermandaround50%neverhavemeetingswithteachers,ormeetthemlessthanonceamonth.Thisisacauseforconcern.Headteacherpedagogicalleadershipshouldtherefore

©EDOREN 33

improveaspartoftheearlylearninginterventiontofacilitateimprovementsinteaching.Teacheradministration,intermsofkeepingupdatedteacherattendancerecords,appearstobewellorganisedinpublicprimaryschools,butisalmostnon-existentinIQSs.Withregardstoexternalmonitoringbygovernmentorotherorganisations,almostallpublicprimaryschoolsreportedreceivingavisitoverthepreviousschoolyear.ThecorrespondingfigureforIQSswasonly58%.ItwillthereforebeimportanttopayparticularattentiontomonitoringpatternsinIQSs,soastoensurethatvisitsbymentorsandgovernmentstaffoccurasfrequentlyasforpublicprimaryschools.

Figure1:SchoolmanagementactionsinPSandIQS

TeachersspeakHausa,theyoftenteachonesubjectandtheyhavevaryingdegreesofqualificationsand

experience.AllteacherssurveyedreportbeingabletospeakHausa.ThispartlyvalidatestheassumptionthatteacherswillbeabletouseHausaduringinstruction.However,asdiscussedbelow,thisdoesnotmeanthattheyareliterateinHausa.MostteachersalsoreportthattheyspeakEnglish,withdifferencesbetweenpublicprimaryschoolteachersandIQSfacilitators.OtherteachercharacteristicsaresummarisedinTable4.Itisworthnotingthataroundhalfoftheteacherssaytheyteachonlyonesubject,ofwhich25%teachonlymathematicsandanother18%teachonlynon-languagesubjects.GiventhatRANAwillemphasisetheinstructionofearlygradereading,therelevanceofthetrainingforthisproportionofteacherswhoteachmathematicsorother,non-languagesubjectsisworthyofattentionwhenselectingteachersfortraining.PublicprimaryschoolteachersandIQSfacilitatorsalsodifferintermsofprofessionalandacademicqualificationsandyearsofteachingexperience.Theteachertrainingwillneedtobeadaptedtothiskindofvariationinteacherbackground.

0 10 20 30 40 50 60 70 80 90 100

HTobservedalesson

HTtookactiontoimproveteacherattendance

Updatedteacherattendancerecordsexist

Percentageofschools

PS IQS

Source:GEP3Baselinesurvey2015

©EDOREN 34

Table4: Summaryofteachercharacteristics

Forsurveyedteacherstheaverage… Ofteachers…

• …ageis36years.

• …teachingexperienceis10years(11yearsinpublicprimaryschool,eightyearsinIQS);89%ofteachershaveatleasttwoyearsofteachingexperienceand71%havehadthatexperienceinthecurrentschool(79%inpublicprimaryschools–52%inIQSs).

• …teachersarelikelytoteachone(non-religious)

subject;thisbeingmostlymaths,HausaorEnglish.

• …all(100%)speakHausa;85%speakEnglish(91%inpublicprimaryschools,73%inIQSs).

• …only15%arefemale(21%inpublicprimaryschools,3%inIQSs).*

• …themajority(54%)holdanNationalCertificateinEducation(NCE)qualification(70%inpublicprimaryschools,22%inIQSs)and72%havepassedtheSeniorSchoolCertificateExamination(SCCE)(74%inpublicprimaryschools,67%inIQSs).

• …40%attendedtrainingduringthelasttwoyears(41%inpublicprimaryschools,46%inIQSs).

*ThepercentageoffemaleteachersslightlydiffersfromthedatapresentedinTable3becausethedatasourcesaredifferent.DatainTable3arebasedontheteacherschoolroster,whiledatainthistablearebasedontheteachersurvey.

Source:GEP3Baselinesurvey2015

Teachers’knowledgeandskills

TheGEP3earlylearninginterventionseekstoimproveteachers’knowledgeinliteracyandlanguageacquisitioninearlygrades.Teachersdrawonthreetypesofknowledgewithinclassroompractice:subjectknowledge,pedagogicalknowledgeandcurriculumknowledge.14

Theteacherssurveyeddemonstratedverylimitedknowledgeandskillsinthemajorityoftheareas

requiredtofunctioneffectivelyasateacher.Rudimentarylevelsofskillswereobservedamongasmallshareoftheassessedteachersinidentifyinglowperformers,evidencingjudgementsanddiagnosingpupils’performance,teacherwritingskillsandinterpretingwordsandphrases(seeFigure2).AgreatershareofteacherswasabletodisplayknowledgeandskillsinGrade1and2-levelHausaandincomprehension.Nevertheless,itisstrikingthat60%ofteacherswereunabletodisplaycompetenceinGrade1and2-levelHausa.15ThisfindingissignificantforaninterventionthatfocusesonteachinginHausabecauseteachers’lowHausaknowledgeislikelytonegativelyaffectteachers’abilitytopasstheirknowledgeontopupils.Also,teachers’inabilitytoidentifylowperformers,evidenceteacherjudgementsconcerningpupilperformance,anddiagnosethenextstepsofteachingpresentchallengestoimprovingpupillearningasastudentlearnsbestwhenteachingistargetedtowhats/heisreadytolearn.Regardingcurriculumknowledge,alargeminorityofteachers(about43%)wereunabletodemonstrateproficiencyinknowingwhatshouldbetaughttoagroupofstudents.

14Subjectknowledgereferstoknowingtheessentialquestionsofthesubject,thenetworksofconcepts,theoreticalframeworkandmethodsof inquiry.Pedagogicalknowledgereferstoknowledgeofthe learners inthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstand,andtheselectionoflearningandassessmentmaterials.Curriculumknowledgereferstoknowingwhatshouldbetaughttoagroupofstudents,knowledgeofthenationalsyllabus,understandingtheschool-andgrade-levelplanningdocumentsandknowledgeofthecontentofexaminations.15Itemstestedincludedbasicgrammarandtheinitiallettersofeverydayobjectsandanimals.

©EDOREN 35

Figure2: Percentageofteachersachievingwithinthelower,middleandupperbandsof

competenceacrosstheteacherknowledgeandskillssubscales

Therearefewdifferencesintheknowledgelevelsdemonstratedbydifferentgroupsofteachers.Althoughteacherswithaprofessionaleducationalqualificationdemonstrateslightlyhigherlevelsofknowledgeandskills,levelstendtobelowacrossallgroupsofteachers.Forexample,weobservenodifferencesbetweenteachersinpublicprimaryschoolsandIQSs,orbetweenteachersthatdidordidnotattendpreviousGEP/UNICEFtraining.

Thefindingssuggestthatteachersdonothavetheknowledgeorskillstodriveimprovementsinpupils’

learning.Aqualitativereviewofincorrectteacherresponsestoitemsintheteacherknowledgeandskillsassessmentindicatesthatteachersareunabletoascertainwhichactivitiesarebestsuitedtoimprovepupilperformanceinaspecificsubject.Teachersgenerallyperceivedpupilefforttobethecauseoflowperformanceandthereforeperceivedincreasingpupil(andtosomeextentteacher)effortasthebestresponsetolowperformance.Thefocusonpupileffortasacauseoflowperformancewithinthiscontextislikelytobeareflectionofthelowlevelsofcompetenceamongstteachers–teacherslackboththeknowledgeandskillstoimprovepupils’learning,whichleavesfewavailableoptionsforimprovingpupillearning,beyondpupils‘tryingharder’.Whileteacherssaidthattheyshouldfocusongirls’learningintheclass,theywerenotabletoexplainwhysuchafocusisnecessary.Inaddition,teachersdidnothaveaclearunderstandingofgirls’learningneeds,andinsomecaseshelddeeplyingrainedbiasesregardingthelimitedcapacityofgirlsortheirroleinsociety.

Thesefindingshaveafewkeyimplicationsfortheearlylearningintervention.First,theyhighlightsomeofthekeyissuesthattheinterventionneedstoaddressthroughanappropriatecombinationofteachertrainingandcarefullytailoredteachingandlearningmaterials.Second,theyprovideanindicationofthescaleofthechallengeconfrontingtheproject,whichcouldfeedintodesigndecisionsrelatedtothefrequencyoftrainingandthelevelatwhichitispitched.Third,theysuggestthatthereistremendousscopetoimproveknowledgeandskillsamongstteachers,although,equally,teachers’verylowlevelsofskillsandknowledgeatbaselinemayalsopresentchallengesforaninterventionthatworksprimarilythroughin-servicetrainingandsupport.

Abilitytoidentifylowperformers

Abilitytoevidence

judgementsanddiagnose

Teacherwritingskills

TeacherGrade2Hausa

Knowledge

Teachercomprehension

skills

Interpretingwordsandphrases

UpperBand 2.3 0 0 39.9 33.6 2.0

MiddleBand 6.2 .2 2.1 23.1 15.3 2.9

LowerBand 91.5 99.8 97.9 37.0 51.0 95.2

0

10

20

30

40

50

60

70

80

90

100

Percen

t

Note:Twocut-pointsweredefinedforeachsubscaletocreatethreeproficiencylevelsperscale.Theseproficiencylevelsare:LowBand–noevidenceofskill;MiddleBand–evidenceofrudimentaryskill;andUpperBand–evidenceofcompetenceSource:GEP3Baselinesurvey2015

©EDOREN 36

Teachers’instructionalskillsandpractices

Theclassroomobservationscarriedoutatbaselineindicatethatteachers’instructionskillsinearlygrade

teachingarelowacrossallgroupsofteachers.Acompositeindexwasconstructedtomeasurethisoutcomevariable.Thisconsistedofmeasuresoftheextentofpupil-centredlearningactivitiesobservedintheclassroom,observationsoftheteacherlinkingthelessontopreviouslearningandlearningobjectives,andtimeontaskinclass.Overall,teachersscoredpoorlyontheindex.Thiswasdrivenbytheirperformanceonthefirsttwocomponentsoftheindex.Incontrast,timeontaskwashigh:pupilswereobservedtobeengagedinanactivityrelatedtolearningfor96%oflessontime.Roughly85%ofclassroomswereobservedtobe‘ontask’forover90%ofthelesson.However,thesefindingsshouldbeinterpretedcarefullyasitislikelythatthepresenceofobserversintheclassroomincreasedthepercentageofontasktimeinlessons.

Therewaslimitedvariationinscoresontheindexacrossdifferentgroupsofteachers.PublicprimaryschoolteachersscoredslightlyhigherthanthoseinIQSs,whiletherewerenocleardifferencesbystate.Therewerealsonoobservablepatternsbetweenteacherpracticesandteachermotivationorperceivedteacherefficacy.Thisindicatesthatteachers’perceivedself-efficacyiseithernotrepresentativeoftheiractualcompetenciesorthelevelsofcompetencyinteacherpracticesaresolowthatdifferencescannotbedetected.

Thebaselinesurveyattemptedtomeasuregender-sensitiveteachingpractices,butacrossallitemsextremecomplianceeffectswereobserved,significantlycallingintoquestionthevalidityandreliabilityofthemeasure.Therefore,gender-sensitiveteachingcouldnotbemeasuredthroughclassroomobservations.Theteacherassessmentsindicatethatteachersareawareofobjectivestotargetgirlsinclass,butthatdeeplyingrainedgenderbiasesremain.Teacherassessmentresponsesstatingthatteachersshouldfocusongirlsintheclasswerenotsupportedbyjustificationsforsuchafocus,didnotreflecttheneedsofgirlsintheclassroomandinsomeinstancesmadeanargumentregardingthelimitedcapacityorsocietalrolesofgirls.

Useofteachingandlearningmaterials

AspartoftheearlylearninginterventionHausateachingandlearningmaterialswillbedistributed.ThisisexpectedtocontributetoanimprovementinHausa-basedteaching,insofarasthematerialsareusedandalignedwiththecompetencylevelsoftheteachers.Wethereforeexaminedtheuseofteachingandlearningmaterialsatbaseline.

Teachingandlearningmaterialsarerarelyused.Duringtheclassroomobservationsatbaseline,veryfewteachersusedatextbookorothermaterials:17%ofobservedteachersusedatextbook,while19%usedothermaterials.

TheuseofHausamaterialsisespeciallyrare.MaterialsinHausawereobservedbeingusedin2.4%ofobservedlessons.MaterialsinHausawereavailableforthesubjectobserved,butwerenotusedinafurther1%ofobservedlessons.Interestingly,almost18%ofteachersindicatedthattheyhavetheHausamaterialstheyneedtodotheirjobs.Thiswouldsuggestthatroughly14%ofteacherseitherdonotseeaneedforHausamaterials,orhaveaccesstomaterialsthattheydonotroutinelyuseduringlessons.

Hausa-basedteachinginearlygrades

TheearlylearninginterventionseekstoimproveHausa-basedliteracyteachinginearlygrades.AcompositeindexwasdevelopedinordertomeasurechangesinHausa-basedteachingintheearlygrades.16Typically,

16ThemeasuresincludedinthecompositeindexweredevisedusingthepercentageoftimetheteacherspokeHausainclassandteachers’Hausaskills.

©EDOREN 37

teachersinIQSsscoredhigherinHausa-basedteachingintheearlygradesthanteacherswithinpublicprimaryschools,asdidteacherswhodonotspeakEnglishorArabic.

Hausawasusedinallclassesobserved.InallclassroomstheuseofHausawasobservedonatleastoneoccasion.InapproximatelyhalfoftheclassesasecondlanguagewasusedinadditiontoHausa.ThefrequencyofHausausewasdocumentedduringclassroomobservationsandameasureofthepercentageoftimeforwhichtheteacherspokeHausawascalculated.Injustunder30%ofobservedlessons,Hausawasspokenbytheteacherfor50%–60%ofthelesson(asdepictedbythegreenbarinFigure3).Inalmosthalfoftheobservedlessons,Hausawasspokenbytheteacherfor30%–50%ofthelesson(depictedbytheblueandpurplebarsinFigure3).

Figure3: FrequencyofHausauseintheclass

Teachermotivationandattendance

Teachers’motivationandattendanceareassumptionscontainedintheToCthatareconsideredtoinfluencetheintermediaryoutcomesregardingteaching.LowteachermotivationandattendancemayleadtoHausa-basedliteracyteachingnotimproving,despitetheearlylearninginterventiontakingplace.Wemeasuremotivationaccordingtofivedimensionsthataretranslatedintomeasurementsubscales(seeError!Referencesourcenotfound.).17

17TheteachermotivationscalesweredrawnfromapreviousstudyundertakenbyEDORENfortheTDP.Inthisstudy,‘motivation’wasunderstoodas:amixtureofperceivedefficacyandeffort/importanceattachedtoteachingwork;butisalsomeasuredlessdirectlythroughinterest/enjoymentandpressure/tensionitems.Basedonthisdefinition,amotivatedteacherisonewho:seesthemselvesaseffectiveandasmakinganeffort;seestheirworkasimportant;isinterestedinandenjoystheirwork;andmanagesworkpressureandtension.DetailsofthetheoryunderpinningtheteachermotivationscalescanbefoundinSection3.2.9oftheBaselineTechnicalReport.

.0

5.0

10.0

15.0

20.0

25.0

30.0

ShareoftimethatHausawasusedduringlesson

Shareoflesson

sob

served

Lessthan10

10-19.99

20-29.99

30-39.99

40-49.99

50-59.99

Source:GEP3Baselinesurvey2015

©EDOREN 38

Box2.Measurementofteachermotivation

Teachermotivationisinvestigatedthroughtheuseofarangeofmotivationscaledscores,whichformthebasisfortheconstructionofacompositemotivationindex.Thesubscalescoverthefollowingdimensions:

• perceivedteacherefficacy;

• interestin,andenjoymentof,teaching;

• effortputinto,andperceivedimportanceof,teaching;

• pressureandwork-relatedtension;and

• teacher-to-teacherinteraction

Overall,teachersconsidertheirroletobeimportantandenjoyworkingasteachers,whilstbeing

consciousofthefactthattheyarelimitedintheirabilitytocontributetopupils’learning.Ananalysisofthedifferentmotivationsubscalesshowsthat,onaverage,thesurveyedteachersscorehighestoneffortandlowestonperceivedteacherefficacy(seeFigure4).Interestinandenjoymentofteachingisrelativelyhighamongteacherswhencomparedtopressureandwork-relatedtension.Itisalsonotablethattheteacher-to-teacherinteractionscoreseemstoindicatearelativelyhighlevelofcollaborationamongstteachers,whichcouldhelpthedevelopmentofspillovereffectswithinschoolsofanyteacher-specificintervention,includingteachertraining.

Figure4: Comparisonofmotivationsubscales

Overallmotivationappearsrelativelyhomogeneousacrossteachers.Acompositemotivationindexthatcomprisesallmotivationsubscalesshowsarelativelyhomogeneoussituationacrossteachers.Motivationamongurbanandruralteachersisverysimilar,andnodifferenceatallisdetectablebetweenteachersacrossstatesandtypesofschools(publicprimaryschoolsandIQSs).Whilereceivingasalaryseemstobesomewhatpositivelyassociatedwithmotivation,thedifferencebetweenteacherswhoreceiveanddonotreceiveanyremunerationissmall.

AbsenteeismisfoundtobemoreprevalentinIQSsthaninpublicprimaryschools.Thebaselinedataonabsenteeismshowsthat60%ofteachersself-reportedhavingbeenabsentatleastonceinthelastthreemonthsacrossallschooltypes.Theestimatedaveragenumberofdaysthattheyreportedhavingbeenabsentoverthisperiodwasjustunderfivedays.AbsenteeismismoreprevalentinIQSsthaninpublicprimaryschools.IQSfacilitatorsreportedhavingbeenabsentforroughlyeightdaysonaverage,compared

1.99

3.57

3.73

2.19

3.47

0 0.5 1 1.5 2 2.5 3 3.5 4

Perceivedteacherefficacy

Interestandenjoymentofteaching

Effortandimportance

Pressureandworkrelatedtension

Teacher-to-teacherinteraction

Source:GEP3Baselinesurvey2015

©EDOREN 39

tojustoverthreedaysamongpublicprimaryschoolteachers.However,attendancedataonIQSsneedstobeinterpretedwithcautionastheIQSteachingscheduleisnotasstructuredasinpublicprimaryschools.Thiscouldpartlyexplainthedifferenceinabsenteeismrates.Teacherswhodonotreceivesomeformofremunerationaremorelikelytobeabsent,andtobeabsentforlongerperiods.Itwillbeimportanttoincentivisetrainedteachersandleadteacherstoregularlyattendtheirclassessoastoensurethattheirimprovedknowledgeandteachingpracticesarepassedontopupilsinschool.

4.4.2 Pupils’literacyinearlygrades

Inthissectionwediscusspupils’literacyintheearlygradesample.First,weprovidebackgroundinformationonkeypupilcharacteristicsthatareimportantfortheanalysisandtheearlylearningintervention.WethendescribethelearninglevelsofpupilsinHausaandEnglishliteracy,whichisfollowedbyaninvestigationofthefactorsassociatedwiththeselevels.Section4.4.3theninvestigatestherobustnessofsomeofthedescriptiveassociationsthroughregressionmodelling.

Summaryofevidencerelatedtoevaluationquestions

Q1.Towhatextentdoesthe

earlylearningintervention

improveHausaliteracyand

Englishlanguagelearning

outcomesamonggirlsandboys

intheearlygradesinpublic

primaryschoolsandIQSs?

Towhatextentdoestheearly

learninginterventionreducethe

gapbetweenthelearning

outcomesofthelowest

performingpupilsandthe

expectedlearningoutcomes,as

expressedinthecurriculum?

Pupilliteracyperformanceisverylowandiswellbelowtheexpectedlearningoutcomesasexpressedinthecurriculum.Whileteachercompetenciesarenotfoundtobeassociatedwithlearningoutcomes,thismaybeduetoverylowteachercompetencylevelsandineffectiveteaching.Pupil-levelfactorsaresignificantlycorrelatedwithlearningoutcomes.

Baselinefindingsonpupils’characteristicsshowthatHausaisthemothertongueofeverypupil,whichisanassumptionoftheprogramme’sToC.However,thefindingsoftheHausaliteracyassessmentindicatethatveryfewpupilshaveacquiredtheknowledgeandskillsappropriatefortheirgrade.Furthermore,thepeakofthedistributionofperformanceinHausaliteracyfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy.WhenitcomestoEnglish,96%ofpupilscouldnotdemonstrateliteracyskillsinEnglishbeyondpre-literacyskills.AswithHausa,thepeakofthedistributionofperformancefallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy.Thissuggeststhatsubstantialeffortwouldberequiredtoachievesignificantgainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracyinbothHausaandEnglish.

Theregressionanalysisshowsthatpupils’individualcharacteristicsandsocioeconomicbackgroundareassociatedwithpupils’achievement,whilemostoftheschool-levelvariables,suchaspresenceofasourceofdrinkingwaterorseparatetoiletsforgirls,donotshowanysignificantcorrelationwithHausaliteracy.

Q6.Towhatextentdoesthe

earlylearningintervention

improvepupilretention,

especiallyretentionofgirls?

Noevidenceaboutretentionisavailableatbaseline.

Pupilcharacteristics

Hausa is the language of the immediate environment. All pupils report speaking Hausa at home (withalmostnostudentsreportingspeakingEnglishoranyotherlocallanguageathome).ThisvalidatestheToCassumptionthatHausaisthepupils’mothertongue.

©EDOREN 40

PupilcharacteristicsvarybetweenPSandIQS.IQSpupilsareolderonaveragethanPSpupils.Asubstantialproportionofpupilsareaged11yearsandover in IQS (41%) compared tohardlyany inPS (3%).18 IQSpupils in the sample are also frommoredisadvantagedbackgrounds in termsof householdwealth thanpupilsinPS.19

Pupils’learningoutcomes

In this subsectionwepresent the findings aboutpupils’ learningoutcomes.Error!Reference sourcenotfound.summariseshowlearningoutcomeshavebeenmeasured.

Box3.Learningoutcomesmeasurement

Hausa and English literacy outcomes aremeasuredbasedon two assessment tools. TheHausa andEnglish literacy assessments are each designed to test the same literacy knowledge and skills. TheEnglish literacy assessment contains 13 items,with each item beingmade up of several sub-items.RegardingtheHausaliteracyassessment,itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptsasappliedtotheHausalanguage.

The assessments test a rangeof literacy knowledge and skills across thepre-literacy, emerging andbasicliteracyranges.Knowledgeandskillstestedincludeletterrecognition,phonologicalknowledge,printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords, verbal and written grammar, writing high frequency words, reading fluency, copying andspellinghighfrequencywords.

English literacy and Hausa literacy assessments were constructed following five steps: clarifyingconstructs,testtargeting,administration,psychometricanalysis,drawingbenchmarksandsecondarydataanalysis.

Theoverwhelmingmajorityofpupilshavenotyetacquiredtheknowledgeandskillsexpectedbythe

NigeriancurriculuminHausaliteracy.ItisexpectedthatpupilswhoarebeginningP2willhavemasteredtheP1curriculumandwillbereadytoreceivetheP2curriculum.However,thelearningassessmentresultsindicatethatonly5.3%ofpupilsareperformingwithintheexpectedrange.Afurther2.6%ofpupilsareabletodemonstrateemergingliteracyskills,while92%ofthepupilscouldnotdemonstrateliteracyskillsinHausabeyondpre-literacyskills.AscanbeseeninFigure5,mostpupilsscorewellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthatachievinglargegainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracywillrequiresubstantialeffort.

ThemajorityofpupilshavenotyetacquiredeitheremergingorbasicEnglishliteracyskills.Lessthan1%ofpupilsdemonstratedatleastsomeskillsthatfallwithinthebasicEnglishproficiencyrangeand3.3%ofpupilsdemonstratedatleastsomeoftheskillsthatfallwithintheemergingliteracyrange.However,96%ofpupilscouldnotdemonstrateliteracyskillsinEnglishbeyondpre-literacyskills.TheresultsoftheEnglishliteracyassessmentsuggestthatpupilproficiencyinHausaaftermorethanayearofschoolingisnotsignificantlyhigherthanpupilproficiencyinEnglish.SimilartothedistributionofHausaliteracyscores,thecuspofthedistributionofEnglishliteracyfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy.

Knowledgeofphonicsisverylow.ThepsychometricanalysisfindsthatitemsthatrequireknowledgeofphonicsrankasthemostdifficultitemsinboththeHausaandEnglishassessments.Correctlysoundingoutlettersandidentifyingsimilarsoundswasmoredifficultforpupilsthanwritingorreadingfullpassages.Thisindicatesthatcurrentlypupilswithinthecontextwhohaveachievedbasicliteracyaredoingsowithoutsignificantexposuretophonicsknowledge.Evidencesuggeststhatsystematicphonicsteachingis

18Theactualpercentagenumbersneedbehandledwithcautionasalargenumberofpupils(35%)inthesampledidnotreportage.ThismissingdatawereevenlyspreadacrosspublicprimaryschoolsandIQSs.19PovertydifferentiationisbasedonaHouseholdWealthIndex(HWI)usingaseriesofhouseholdassetsasanindicationofwealth.

©EDOREN 41

associatedwithbetterprogressinreadingaccuracythoughnotalwaysinreadingcomprehension.20Whilsttheuseofphonicsinstructionhasadvantages,ifliteracyisunderstoodasbeingabletounderstandandinteractwithwrittentext,readingfluencyisabridgetoliteracy,ratherthanameasureofliteracyitself.Furthermore,theevidencebaseonthelinksbetweenphonologicalawarenessandreadingaccuracyandcomprehensionmainlycomesfromstudiesincontextswithhighlyliterateandqualifiedteachers.Asdiscussedabove,theseconditionsdonotholdinthecontextinwhichGEP3isbeingimplemented.Theearlylearninginterventionwillneedtotakethesecaveatsintoaccountwhenpromotingaphonicsapproachtoteaching.

Figure5: DistributionofHausaliteracyproficiency

Learningoutcomesdifferbyageandgender.Malepupilsachievedhigheraveragescoresthanfemalepupils,althoughthiswasmainlydrivenbydifferencesinIQSs.Thiscanbepartlyexplainedbytheassociationbetweenage,learningoutcomesandthegendergap.Thebaselinedatahighlightthatolderpupilsperformbetterthanyoungerones(seeFigure6).Italsosuggeststhatlearningoutcomesaremorecloselyassociatedwithcognitivedevelopmentthatwithyearsofschooling–asaverageyearsofschoolingdonotdiffersubstantiallybetweenolderandyoungerchildreninthesample.WhendisaggregatingliteracyachievementbyageandgenderatrendemergesacrossbothHausaandEnglish(seeFigure6).Genderdifferencesinperformancearesmallintheyoungeryears,butincreaseoncegirlsreachpuberty(around12

20Torgeson,Brooks,andHall,2006.

©EDOREN 42

yearsofage).Inlinewiththis,thelargergendergapinIQSscouldpartlyreflectthefactthatpupilsatIQSsalsotendtobeolder.

Figure6: MeanHausascalescorebyageandgender(95%confidenceinterval)

4.4.3 Analysisofrelationships

Inadditiontothedescriptiveanalysispresentedabove,theevaluationteamconductedregressionanalysistomakemorestatisticallyrobustclaimsabouttherelationshipbetweenlearningoutcomes(HausaliteracyandEnglishliteracy)andawidersetofschool-,teacher-andschool-levelexplanatoryfactors.InError!Referencesourcenotfound.wepresentthecategoriesofexplanatoryvariablesthatareincludedinthepupilregressionmodel.TableswithregressioncoefficientsareincludedinAnnexE.

Box4.Explanatoryfactorsincludedinthepupilregressionmodel

• Pupils’personalcharacteristics,includinggenderandagegroup• Pupils’socioeconomicstatus,asdefinedbytheHWItertilecategorisation• Informationaboutpupils’schools,includingwhethertheyattendpublicprimaryschoolsorIQSs,

whethertheyattendotherschoolsinparallelandwhethertheycanwrite• Locationinformation,includingwhethertheschoolisinKatsinaorZamfaraandwhetheritislocated

inanurbanorruralarea• Schoolinfrastructure21,includingwhethertheschoolhasseparatetoiletsforgirlsandwhetherithas

accesstoasourceofdrinkingwater• Teachers’averagemotivationlevel,asdefinedinouroverallmotivationindexillustratedabove,in

theschoolattendedbythepupil• Teachers’averagesubjectandpedagogicalknowledge,asdefinedintherelevantindexesillustrated

inthesectionabove,intheschoolattendedbythepupil• Teacherknowledge1:Teachers’subjectknowledge

• Teacherknowledge2:Teachers’syllabusandcurriculumknowledge• Teacherpedagogy1:Teachers’pedagogicalknowledge

21Othervariablesonschoolcontextinformation,includingforinstanceelectricityorpupil/teacher,teacher/classroom,pupil/classroomandpupilandteachergenderratioswerenotincludedastheywouldhavereducedtheregressionsamplesize.Dataonthefactorsaboveandotherschoolaspectswereinfactcollectedonlyfromalimitednumberofschools.

300

400

500

600

700

800

900

4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Scalescore

Pupils'ageBoy Girl

Source:GEP3Baselinesurvey2015

Cutpointbetweenpre-literacyandemergingliteracy

©EDOREN 43

• Teacherpedagogy2:Mothertongueteachinginearlygrades.

Teacher-levelfactorsdonotappeartobesignificantlycorrelatedwithliteracylearningoutcomes.Inparticular,noneoftheteachermotivation,knowledgeorpedagogyindexesarefoundtobesignificantlyassociatedwitheitherHausaorEnglishlearningoutcomes.Thismayreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetectediftherewasgreatervariationincompetencylevelsacrossteachersandthattheirabilitytoinfluencepupils’learningoutcomeswouldbehigher.Thisalsoimpliesthatiftheearlylearninginterventiondoesleadtosubstantialimprovementsinteachers’knowledgeandskills,weshouldseeamoresignificantlinkbetweenteachercharacteristicsandlearningoutcomesatendline.

Thelocation(urban/rural)andtypeofschool(publicprimaryschool/IQS)areassociatedwithlearning

outcomes.Mostoftheschool-levelvariables,suchaspresenceofasourceofdrinkingwaterorseparatetoiletsforgirls,donotshowanysignificantcorrelationwithHausaliteracy.Similarly,whilstbeinginKatsinaorZamfarastatedoesnothaveanyassociationwithliteracyoutcomes,rural/urbanlocationdoes,withpupilsattendingschoolsinruralareasachievinglowerHausaandEnglishscoresthanthoseinurbanareas.Finally,pupilsinpublicprimaryschoolsappeartoachievelowerliteracyscoresinbothlanguagestestedthanpupilsinIQSs.TheregressionanalysisindicatesthatthestatisticallysignificantbetterperformanceofIQSpupilsremainsevenaftercontrollingforthepotentiallyconfoundingeffectofpupils’age.

Pupil-levelexplanatoryfactorsaresignificantlycorrelatedwithlearningoutcomes.Age,unsurprisingly,issignificantlycorrelatedwithlearning,witholderpupilsdoingbetterthanyoungerones.Householdsocioeconomicstatusisalsoassociatedwithliteracyoutcomes,withrelativelywealthierpupilsfoundtoperformbetterthantheirpoorercounterparts,althoughaneconomicthresholdexistsbelowwhichimprovementsinsocioeconomicstatusdonotappeartoaffectlearning.Atthesametime,giventhegenerallylowlevelofliteracyamongstthesurveyedpupils,itseemsreasonabletopresumethatwealthisnotsufficienttoachievegoodlearningoutcomes.Genderiscorrelatedwithlearning:malepupilsperformbetterthantheirfemalecounterparts.However,thecorrelationonlyemergeswhencontrollingforallpossibleschool-levelcharacteristicsandthestatisticalsignificanceofthecorrelationislow.RANA’scommunityengagementactivitiescouldhelptocountersomeoftheeffectsofsocioeconomicbackgroundonlearningoutcomesbypromotingasupportivehomelearningenvironmentforallchildren.

©EDOREN 44

5 BaselineoftheIQSSevaluation

5.1 GEP3’sIQSS

GEP3’sstrategyforintegratedQur’aniceducationfocusesontheimprovementofeducationinIQSs,with

theaimofprovidinganacceptablealternativeformofqualitybasiceducationforgirls.Thefinaloutcomesexpectedare:improvedlearningoutcomesinbasicliteracyandnumeracy(especiallyforgirls),improvedretentionofgirls,and,toalesserextent,increasedenrolmentofgirls.

GEP3targetsregisteredIQTEcentresthatimplementanintegratedcurriculum(becomingIQSs)andthatlargelyoperateascommunity-basedinitiatives,butthatarewillingtobuildlinkswithgovernmentforthepurposesofmonitoringandtechnicalsupport.Atleast40%ofthestudentsenrolledshouldbegirlsfortheIQStobesupportedbyGEP3.Duringthe2015–2017period200IQSsperstateacrossthesixGEP3LGAsinNiger,BauchiandSokotowillreceiveafullschool-levelsupportpackage.22Box5summarisestheIQSSpackagethatwillbeimplementedatschoolcommunitylevel(amoredetaileddescriptionispresentedinChapter4oftheBaselineTechnicalReport).

Box5.IQSSactivitiesatschoolandcommunitylevel

• TrainingandmentoringofIQSfacilitators,with,atitscore,a1.5-yearcycleofmonthlycluster-leveltraining/mentoringmeetingsinitiatedbyfive-dayinductionworkshop

• Provisionofapackageofclassroomteachingandlearningmaterials

• Trainingofheadteachers,consistingofthree-daytrainingsessionspertermoveratwo-yearperiod

• CapacitybuildingforCBMCs,consistingofaninitialmulti-daycluster-leveltrainingwithfollow-upmentoringvisits,atleastonceaterm,overaperiodofnineto12months

• Provisionofmini-grants,atmosttwiceoveratwo-yearperiod,andconditionalonthedevelopmentofaWholeCentreDevelopmentPlan(WCDP)andattendanceatCBMCtraining

GEP’sIQSSbeganinAugust2015andinvolvesa2–2.5-yearperiodofmentoringandfollow-uptraining.AfirstcohortofonefacilitatorperIQSattendedtheinductionworkshopinAugust/September2015.AsecondfacilitatorfromeachIQSwillstartthetrainingandmentoringcycleinJuly–September2016.TheheadteachertrainingwasoriginallyplannedtostartbetweenAprilandJune2016butthishasyettobeconfirmed.ThematerialsareplannedtobedeliveredtotheIQSsbyApril–May2016.TheinitialCBMCcluster-leveltrainingisscheduledtotakeplaceinJuly–September2016.

AnnexFpresentstheToCoftheIQSSintervention.TheToCisdiscussedindetailasitformsanintegralpartofthecontributionanalysisapproachthatisusedfortheIQSSevaluation.ThemainlogicofGEP3’sIQSSisthatgirls’learningcanimproveinanIQSwhentheIQScanprovidequalitybasiceducation,whichrequiresthatfacilitatorsteachmoreeffectivelyandthattheschoolenvironmentimprovesandbecomesmoregirl-friendly.EffectiveteachingisexpectedtoimprovewithGEP3’ssupportforfacilitatortrainingandmentoring,headteachertrainingandthedistributionofteachingandlearningmaterials.CBMCtrainingandtheprovisionofmini-grantsareexpectedtocontributetoanimprovementintheschoolenvironment.Atthegovernmentlevel,GEP3’sstateadvocacyandLG(E)Acapacitysupportareexpectedtocontributetotheimprovementofmonitoringandsupportsupervisionbygovernmentstaff,aswellastosustainedgovernmentfinancialsupport.Thisinturnisexpectedtocontributetomoreeffectiveteachingandanimprovementintheschoolenvironment.

22InNigerandSokotothetargetedIQSsareregisteredwiththeSAME,whichistheleadagencycoordinatingtheimplementationofthesupport.InBauchi,191GEP3IQSsaremanagedbytheSUBEB.TheremainingnineIQSsfallunderBauchiSAME.

©EDOREN 45

5.2 Baselinemethodology

TheToCofGEP3’sIQSSwasusedasaframeworktoformulatetheevaluationquestions.Theevaluationquestionsinterrogateawiderangeofthecause–effectassumptionsunderlyingdifferentstepsintheToC,inordertobetterunderstandhowchangemaycomeabout.Error!Referencesourcenotfound.AnnexBpresentsalistoftheevaluationquestions.Wewillalsosynthesisebaselineevidenceperevaluationquestionaspartofthepresentationofthebaselinefindings.TheevaluationprioritisesthequestionofhowGEP3’ssupportiscontributingtochangesinlearningoutcomes(ratherthanretentionor

enrolment),particularlybylookingatchangesinteachingandtheschoolenvironment.Similartotheearlylearningevaluation,weapplyanequityperspectiveinmeasuringlearningoutcomes.BecauseofthemultifacetednatureofIQSSandthecomplexIQScontexttheevaluationquestionsaimtoobtainanunderstandingofhowIQSScontributestointendedchangeswithintheIQScontext,ratherthantoquantifytheattributableeffectoftheintervention.Furthermore,theevaluationwillpayattentiontounintendedconsequencesoftheIQSS.Atmidlinein2017evaluationquestionsrelatedtoGEP3’scontributiontoteachingandtheschoolenvironmentwillbeanswered,whiletheeffectonlearningoutcomeswillbethesubjectoftheendlineanalysis.

ToevaluateIQSSweapplyanevaluationapproachthatdrawsontheprinciplesofcontributionanalysis

andreliesonastrongmixofquantitativeandqualitativemethods.Contributionanalysisisapragmatictheory-basedevaluationapproachthatassessescausalpathwayswithintheToC,reportsonwhethertheintendedchangesoccurredornot,andidentifiesthemaincontributionstosuchchanges.TheIQSSevaluationdoesnotmakeuseofacomparisongrouptomakecausalinferencebutseekstomakecrediblecausalclaimsaboutthecontributionIQSSismakingtoobservedoutcomesbyverifyingthechainofexpectedresultsandassumptionsofacredibleToC,aswellasassessingthecontributionofalternativeexplanationstoobservedoutcomes(Mayne,2012;DelahaisandToulemonde,2012).23BasedontheIQSSToCandinlinewiththeevaluationquestionsthreecontributionclaimswereidentified,onwhichevidence

willbecollectedduringseveralroundsofdatacollection:

1. ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs.

2. ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs.

3. ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls.

Datacollectiontakesamixed-methodsapproach,combiningquantitativeandqualitativemethods.QuantitativedatacollectionconsistsofrepresentativesamplesurveysamongGEP3IQSsinthe12GEP3LGAsacrossBauchiandNiger.Baseline,midlineandendlinesurveyswillbeconductedinacohortofIQSssampledatbaseline.ThequalitativeresearchtakesplaceinpurposivelysampledcasestudyIQSsthatarealsoincludedinthequantitativesurveysample.ThesamecasestudyIQSswillbevisitedatbaseline,midlineandendline.BothquantitativeandqualitativebaselinedatacollectiontookplaceduringOctober–November2015.2425

23Wedecidednottouseacomparisongroupforseveralreasons.First,theevaluationquestionsprioritisedunderstandingofhowIQSScontributes to intendedchangeswithin the IQScontext, rather than toquantify theattributableeffectof the intervention.Second, itwasdifficulttoconstructacomparisongroupasnosampleframeof IQSssimilartotheGEP3pilot IQSswasavailable.Third,resourceswerenotavailabletoconstructcomparisongroupsforalloftheinterventionsunderevaluation.Prioritywasgiventotheearlylearningintervention.24AtthattimetheinductionworkshopofthefirstIQSfacilitatorcohorthadalreadytakenplace.Becauseoflogisticalreasonsthebaselinecouldnottakeplaceearlier.Wedonotexpectthistosignificantlyaffectthevalidityofthebaselinedatasincethecluster-levelmonthlymeetingsareconsideredtobethecoreofthecapacitydevelopmentinterventionandhadnotyetstartedatthetimeofthebaseline.Furthermore,whileteachingknowledgeandskillsmayhavebeenaffectedtolimitedextent,itishighlyunlikelythatpupillearningoutcomeswillalreadyhavebeeninfluenced.25MidlinedatacollectionisscheduledforMay–June2017,togetherwiththeearlylearninginterventiondatacollection.

©EDOREN 46

TheIQSsurveytookplaceinasampleof60IQSs,stratifiedaccordingtothe12GEP3LGAsinBauchiandNigerKatsina(fiveIQSsperLGA,30IQSsperstate).BaselinedatacollectionwasconductedbyOPMNigeria.Thesurveywassuccessfulinachievingtheplannedschoolsamplesize.26ItisimportanttonotethatahigherproportionofthesampledIQSsarelocatedinurbanareasinBauchicomparedtoNiger(27%inBauchi,versus10%inNiger).Withintheschoolsmaleandfemalepupilsaswellasteacherswererandomlysampledaspartofthesurvey.27Similartotheearlylearninginterventionbaseline,thetargetpopulationswere:pupilsenrolledinaP2equivalentgradeandteachersteachinginP1–3equivalentgrades.Thefinalsampleincludes576pupils(ofwhich48%aregirls)and96teachers(seeAnnexC).Inaddition,theheadteacherineachschoolwassurveyed,aswellasCBMCrepresentativesin52IQSs.Alldatahavebeenanalysedusingsamplingweights,andthereforerepresenttheentiretargetedpopulation.ThesamesevendatacollectioninstrumentsaswereusedintheearlylearningsurveywereadministeredusingCAPI.In

addition,apupilnumeracyandCBMCquestionnairewereadministeredtopupilsandCBMCmembers

respectively.

ThequalitativecasestudiestookplaceinsixIQSsusingpurposive‘typicalcasesampling’and‘extreme

casesampling’.CaseswereselectedfromthreedifferentLGAsperstatethatwerethemselvesselectedafteracategorisationaccordingtoaveragepublicprimaryschoolperformanceongirls’educationindicators.28Thequalitativeresearchmadeuseoffourresearchtechniquesorinstruments:qualitativeclassroomobservationandunstructuredteacherpracticediscussions;KIIswiththeIQSproprietors/Mallams,headteachers,communityleadersandlocalgovernmentIQSofficers;andfocusgroupdiscussionswithparents,girlpupilsandboypupils.AppliedthematicanalysiswasusedtointerpretthedataaccordingtothemesthatwerebasedontheIQSSToC.

5.3 Baselinefindings

Inthissectionwepresentasynthesisofthebaselinefindingspercontributionclaimsetoutinthemethodology.Inlinewiththemixed-methodsapproachwecombinequantitativeandqualitativedatainthesynthesis.Amoredetailedpresentationofthedataisincludedintheevaluationtechnicalreport.Throughoutthetextwewillmakereferencetotheevaluationquestionsinsummaryboxes.SincetheevaluationquestionsarecausalquestionsrelatedtotheIQSSintervention,theycanonlybeansweredatmid/endline.Nonetheless,wesynthesiseevidencethatprovideabaselineassessmentoftheoutcomesexpectedandtheassumptionsunderlyingthecausalchain.

5.3.1 ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformal

subjectsinIQSs

GEP3’sstrategyforintegratedQur’aniceducationaimstoprovideaccesstoqualityeducationinIQSs,particularlyforgirls,byimprovingeffective,gender-sensitiveteachingoftheformalsubjectsincludedintheharmonisedintegratedcurriculum.GEP3’scontributiontomoreeffectiveteachingassumesthatIQSfacilitators’knowledgeandskillsandgender-sensitiveclasspracticescanbeimprovedthroughtrainingandmentoring.Mentoringisalsoassumedtoincreaseteachermotivation,whichcanagaincontributetomoreeffectiveteaching.Inaddition,thedistributionofHausateachingandlearningmaterialsisexpectedtosupporteffectiveteachingprocesses,asisimprovedpedagogicalleadershipandschoolmanagementbytheheadteacher.Inthissectionwesummarisethebaselinefindingsonthesedifferentintermediaryoutcomes.Wefirstdescribethecontextinwhichtheteachingtakesplacesandsomeofthefacilitators’

2635%oftheoriginallysampledIQSshadtobereplaced,mainlyduetotheIQSnotbeingconsideredeligibleforthesurveygiventhestudyuniverse(seeBaselineTechnicalReport).27Unliketheearlylearninginterventionpanelsurvey,acrosssectionalsurveyisusedforpupil-leveldatacollection(seeBaselineTechnicalReport).28Thefollowingindicatorswereused:thegenderparityindex,girls’transitionrateinprimaryschoolandacompositeindexofthepupil-to-qualified-teacherrate,theshareofqualifiedfemaleteachersandthepupil–classroomratio.

©EDOREN 47

characteristics.Amorein-depthdiscussionoftheschoolenvironmentisprovidedinthediscussionofContributionClaim2.

Summaryofevidencerelatedtoevaluationquestions

Q1:HowwellhasGEP3teacher

capacitydevelopment(training

andmentoring)contributedto

improvedteacherknowledgeand

skills,andmoreeffective

teachingintheclassroom?

Levelsoffacilitatorknowledgeof,andcompetencein,effectiveteachingareverylow.Additionalcapacitydevelopmentisthereforehighlyrelevantbutneedstobewelltargetedtotheneedsofthefacilitators,toreachtheappropriatefacilitators,andtotakeintoaccountthediverseandchallengingteachingcontextinIQSs.Facilitators’motivationisshapedbyissuesofremuneration,socialstatusandcommunityrolesandrelations.

Facilitatorsdemonstratedverylowlevelsofpedagogical,curriculumandsubjectknowledge.Forinstance,lessthan3%offacilitatorscoulddemonstratecompetenceinidentifyinglowperformers,interpretingwordsandphrases,anddisplayingwritingskills.Theuseofpupil-centredapproacheswasalsolimited.Thiswasdespite43%offacilitatorsreportinghavingreceivedtraininginthepasttwoyears.Theappropriatefacilitatorsarenotalwaystrainedandonlyalimitedshareoffacilitatorshaveprofessionalteachingqualifications.Facilitatorsshowinterestinbeingtrained,particularlyinsubjectknowledge.

Facilitatorsconsidertheirroletobeimportantandenjoyworkingasteachers,whilsthavingamorenegativeperceptionoftheirteachingefficacy.Facilitatorsthemselves,aswellascommunitymembers,donotperceivetheirlackofcompetencetobeacauseoflowpupilperformance.WhileremunerationdoesnotappeartobethemainmotivationforfacilitatorstoteachinIQSs,thelackofadequateremunerationmakesitdifficulttoattractqualifiedfacilitatorsandholdthemaccountable.

AquarterofIQSsonlyhaveonefacilitator,whichwillconstrainamentoringprocessthatisbasedonschool-basedpeerinteraction.90%ormoreofthefacilitatorsreportspeakingHausaandEnglish,althoughthisdoesnotmeanthattheyareliterateinHausaorEnglish.Notably,only34%offacilitatorswerefoundtobefullycompetentinGrade1and2-levelHausa.

TheteachingoftheintegratedcurriculuminIQSsappearstohavegainedacceptancebutisimplementedtovaryingdegrees.Thisislikelytoinfluencethescopeforgainsineffectiveteaching.Facilitatorattendanceseemstobeflexible,withnon-religioussubjectsbeingtaughtasandwhenafacilitatorisavailable.Amorestructured,intensifiedteachingprogrammewillbehardtoenforcebecausebaselineresultssuggesttheneedforintegrationtobegradualandfortheschooltoretainitsQur’aniccharacter.

Q2:HowwellhasGEP3teacher

capacitydevelopment

contributedtoanimprovement

ingender-sensitiveteaching?

Whilefacilitatorsdisplaysomegender-sensitivetechniques,gender-biasedclassroompracticesprevail.

Facilitators,whoaremostlymale,expresspositiveattitudestowardsgirls’education.Theyarealsoareawareof,andsometimesdisplay,gender-sensitivetechniques.However,genderbiasescontinuetoaffectclassroomorganisationandpracticesduringtheteachingoftheintegratedcurriculum.Thisreflectsentrenchedgenderedattitudesandexpectationsaboutthebenefitsofeducationforgirls,andculturalnormsabouthowgirlsandboysrelate.

Q3:HowwellhasGEP3head

teachercapacitydevelopment

(trainingandmentoring)

contributedtoimproved

pedagogicalleadershipand

SchoolleadershiprolesinIQSsarenotclearlydefined,arepossiblyshared,andarenotalwaysassignedbasedonabilityandqualifications.Thisislikelytocomplicatetheprocessofidentifyingwhopreciselyshouldbetargetedfortraining.

SchoolleadershipinIQSsiscomplexandissharedamongstmany

©EDOREN 48

schoolmanagement? stakeholders.IQSsattimeschooseheadteachersbasedonsocialstatusandperceptionsregardingleadership,ratherthanbasedonteachingknowledge.Thelackofclarityaboutleadershiprolescanmakeitdifficulttoidentifywhoseleadershipcapacitiestobuild.GEP3needstoconsiderwhetherthoseattendingtrainingarethemostqualifiedtocarryoutaleadershiprole,particularlywithregardstopedagogicalleadership.

Headteachersdemonstratesomedegreeofpedagogicalleadership(particularlyinNiger)andtakeactionstoimprovepupilandteacherattendance.However,record-keepingispoor.43%hadattendedtrainingduringthelasttwoyears.Headteachers’professionalandacademicqualificationsaregenerallylowand,onaverage,belowthoseoffacilitators.

Q4:Towhatdegreehave

teachingandlearningmaterials

suppliedthroughthe

interventionbeenperceivedby

teachersandheadteachersas

appropriateandwelltargeted?

Havetheybeenusedformore

effectiveteaching?

TheavailabilityofteachingandlearningmaterialsinIQSsisverylimited.GEP3’sdistributionofsuchmaterialscanfillanimportantgap.However,materialsneedtobeappropriateforthelanguagesspokenintheIQS,thesubjectstaughtandtheskilllevelsofthefacilitators.

Theavailabilityofteachingandlearningresourcesisverylimited.AlmostnoHausamaterialsareavailable,althoughHausaisnotnecessarilythemothertongueinallcases.InNiger43%ofpupilsreportedlyspeakNupeathome.IQSsmostlyteachmathematicsandlanguagesascoreintegratedsubjects,whichmakesthedistributionofnumeracyandliteracymaterialsbyGEP3wellalignedwiththesubjectstaughtintheIQSs.GiventhelowHausaliteracylevelsoffacilitatorsHausamaterialswillneedtobefit-for-purpose,notonlyinrelationtowhatneedstobetaught,butalsowithrespecttotheskilllevelsoffacilitators.

Teachingcontextandteachercharacteristics

IQSsprovideadiverseandchallengingteachingcontext.20%oftheIQSsampledidnothaveclassrooms,and30%hadonlyoneclassroom.Asshowninthequalitativecasestudies,someIQSsprovideclassesintheopenair,withablackboardhangingontheouterwalloftheMallam’shouse.Gradeprogressionisorganisedinvariousways.Aroundtwo-thirdsoftheIQSsinthesurveysampleseemtobeorganisedinsixlevels,likepublicprimaryschools;theremainderorganisetheteachingofthecurriculumintooneortwostages.Classroomobservationdataindicatethatthepupil–teacherratioisquitevaried,withfromfourto183childrenbeingtaughtbyoneteacher,withameanof45.Therefore,thefacilitatortrainingwillneedtotakeintoaccountthediverseteachingcontextofthefacilitators.

ThepotentialmobilityofthetraditionalTsangayaQur’anic29schoolsislikelytoposesomechallenges.

Thiswouldlikelyinterruptteachingoftheintegratedcurriculumandmayaffectthegenderinclusivenessoftheschool,asthefacilitatormaynotmovewiththereligiousteacher(theMallam),theMallamwouldmoveonlywiththeboys,andthefacilitatormaynotcontinueteachingtheintegratedcurriculumintheabsenceoftheMallamandthemalepupils.Atbaselinewedonothavequantitativedataonthemobilityoftheschoolsample.TheheadteachersofthecasestudyIQSsallstatedthattheirschoolsaresedentary,butinoneschoolinBauchipupilsindicatedthatmovingwasplannedforthefuture.GEP3needstocloselymonitorthemobilityoftheIQSs.Schoolmobilityanditsconsequencesforeffectiveteachingoftheintegratedsubjectscouldbediscussedaspartoftheheadteachertraining,inordertosensitiseschoolleadership.

IntegrationhasbeenimplementedacrossBauchiandNiger,buttovaryingdegrees.AconditionforeffectiveteachingofformalsubjectsisfortheQur’anicschooltobeactuallyintegrated–thatis,providingtheintegratedcurriculum.Thisisnotnecessarilythecase:22%oftheGEP3IQSscontactedforthebaseline

29TheTsangayaelementmeansthattheinstitutionhasamobileelement,inwhichtheMallammoveswithhispupils‘inthebeliefthatanitinerantlifeisessentialforthemtofullyconcentrateontheirstudy’(Antoninis,2012).

©EDOREN 49

surveywerenon-integratedand14%hadbeenintegratedforlessthanoneyear.Amongthesurveyedschools,IQSswereintegratedonaveragefortwoyearsinBauchiandthreeyearsinNiger.FewIQSsprovideallfivecoresubjectsoftheintegratedcurriculumintheearlygrades.30Interviewswithheadteachersindicatethatonly12%ofIQSsteachmathematics,Hausa31,English,basicsciencesandsocialsciences,whileEnglishandmathematicsareprovidedinaround90%ofIQSs(seeError!Referencesourcenotfound.).Accordingtotheheadteachersinterviewed,Hausaisalsofrequentlytaught,althoughonly32%offacilitatorsinterviewedindicatedthattheytaughtHausa.32Thefactthatnotallcoresubjectsaretaughtmayberelatedtothelimitednumberofteachinghoursfornon-religioussubjects.Headteachersreportedthatonaveragethreehoursperweekarededicatedtotheintegratedcurriculum,whichissubstantially

belowtheeighthoursrecommendedintheNationalBenchmarkforIntegratedBasicEducation.Furthermore,thequalitativecasestudiessuggestthatactualinstructionaltimeismorelimitedthanthetimedeclaredbytheheadteachers.

Figure7: CoresubjectstaughtinIQSs

ThedegreeofintegrationissomewhatdifferentinNigerandinBauchi.Intheformerstate,20%ofIQSsprovidethefivecoresubjects,whilethisisonlythecasefor3%ofIQSsinBauchi.ReportedinstructionaltimeisfourhoursonaverageinNiger,comparedtotwohoursinBauchi.Hence,IQSsinNigerseemtobemoreintegratedthaninBauchi,whichisconfirmedbythedifferenceinthenumberofyearsofintegrationandinthefactthatmoreIQSsinNigerareorganisedintosixlevels,likepublicprimaryschools.Thevariationinthedegreeofintegrationislikelytoaffectthegainsineffectiveteachingthatcantakeplace.Also,thevariedadoptionoftheintegratedcurriculumdeservesattentionduringimplementation,totheextentthatGEP3seekstoimproveeffectiveteachingofallcoresubjects.ThefactthatmostsurveyedIQSsareteachingmathematicsandlanguagessupportstheplausibilityofliteracyandnumeracybeingthecoreGEP3learningoutcomestoimprove.

Integratededucationappearstohavegainedacceptance.Inorderforeffectiveteachingoftheintegratedcurriculumtotakeplacefacilitators,proprietors,parentsandtheschoolcommunityneedtobesupportive

30AccordingtotheNationalBenchmarkforIntegratedBasicEducationinQur’anicschools(2013)thefirststagecurriculum(P1–P3)ismadeupofmathematics,thelanguageoftheimmediateenvironment,English,socialstudiesandbasicsciences.31Hausaisnotnecessarilythelanguageoftheimmediateenvironmentinallschoolcommunities–particularlyinNiger,whereFulfuldecanbethemothertongue.WefocusonHausasinceGEP3willbeprovidingteachingandlearningmaterialsinHausa.32ThiscouldbeduetothefactthatanotherfacilitatorteachesHausa,oritmaybethatheadteachersareover-reportingtheteachingofHausa.

0

10

20

30

40

50

60

70

80

90

100

English Mathematics Hausa Basicscience Socialstudies

PercentageofIQS

Source:GEP3Baselinesurvey2015

©EDOREN 50

ofintegratedQur’aniceducation.Thequalitativecasestudiesindicatethatthisisthecase.Thisisunderpinnedbyarecognitionthatbothreligiousandnon-religiousformsofeducationareimportant,andthatneithertypeofeducationissufficientinisolation.However,interviewedstakeholdersindicatedthatintegrationneedstobegradualandthattheschoolsshouldnotlosetheirQur’aniccharacterandbeturnedintoformalschools.

MostIQSshaveoneortwofacilitators.Theaveragenumberofteachersofnon-religioussubjectsis2.4inBauchiand3.2inNiger.Thisdifferencesuggests,again,thatinNigerIQSsaremoreintegrated.25%ofIQSshaveonlyoneteacherofnon-religioussubjects,while37%havetwo.Facilitatorsmostlyteachoneortwosubjects.MorefacilitatorsteachmultiplesubjectsinNigerthandosoinBauchi.InIQSswithonlyonefacilitatorschool-basedpeerinteractionasamechanismtostrengthenknowledgeandskillswillnotbefeasible,whichmaylowertheimpactoftheIQSSintervention.InIQSswithmorethantwofacilitatorsGEP3needstoconsiderhowknowledgecanbedisseminatedandembeddedbeyondthetwofacilitatorstargetedforGEP3training.Thisishighlightedbythequalitativecasestudies,whichindicatethatknowledgesharingaftertrainingdoesnotnecessarilyhappen.Sincefacilitatorsmostlyteachalimitednumberofsubjects,itwillbeimportantforGEP3totargetthosefacilitatorsfortrainingwhoactuallyteachthesubjectstowhichthetrainingrelates.

ThefacilitatorprofilesdifferslightlybetweenBauchiandNiger.Only9%ofthesurveyedfacilitatorswerefemale,althoughthiswashigherinNigercomparedtoBauchi,despitemoreIQSsinBauchibeinglocatedinurbanareas(seeTable5).FacilitatorsinBauchiaremoreexperiencedandprofessionallyqualified.ThismayberelatedtothefactthatinBauchitheSUBEBismanagingmostoftheGEP3schools,whileinNigertheSAMEisresponsible.Acrossthetwostates,lessthanhalfofthefacilitatorshavesomedegreeofprofessionaleducationalqualification(NCEorGrade2certificate).Thefacilitatortrainingneedstobeadaptedtothisfact.Thequalitativecasestudiesconfirmthattheleveloffacilitatorqualificationvariessignificantlyacrossthecases.Amongthecases,themoreprofessionallyqualifiedfacilitatorsarealsoteachinginpublicprimaryschools.Thefacilitatorswithahigherlevelofqualificationfeltmoreconfidentteachingtheintegratedcurriculum.Withregardstolanguage,90%ormoreofthefacilitatorsreportedspeakHausaandEnglish.ItisimportanttoacknowledgethisfactsincetheIQSSinterventionincludesthedistributionofHausateachingmaterials.However,aswillbediscussedbelow,thisdoesnotmeanthatthefacilitatorsareliterateinHausa,whichmeansthatcarefulattentionwillneedtobegiventothefacilitatortrainingmethodologyused.InNiger39%ofthefacilitatorsindicatedthattheyspeakNupe,whichisnoteworthysincealmosthalfofthepupilsinNigerindicatedthattheyspokeNupeathome.

Table5: Summaryoffacilitatorcharacteristics

Forsurveyedfacilitatorstheaverage… Ofteachers…

• …ageis32years;

• …teachingexperienceissevenyears(nineyearsinBauchiandfiveyearsinNiger);82%ofteachershaveatleasttwoyearsofteachingexperience;and

• …teachersarelikelytoteachoneortwonon-religioussubjects–thesebeingmostlymathematicsorEnglish.

• …96%speakHausa;90%speakEnglish;17%speakArabic;inNiger39%speakNupe,while13%inBauchispeakFulfulde.

• …only9%arefemale(2%inBauchi,16%inNiger));

• …14%alsoteachreligioussubjects;

• …35%holdanNCEqualification(51%inBauchi,20%inNiger)and66%havepassedtheSCCE(61%inBauchi,70%inNiger);33%inBauchihaveareligiouseducationqualification,comparedto11%inNiger;and

• …43%attendedtrainingduringthelasttwoyears(36%inBauchi,49%inNiger).

Source:GEP3Baselinesurvey2015

©EDOREN 51

Thereisroomforfurtherexpansionanddeepeningoffacilitatortraining.43%offacilitatorsreportedthattheyhavereceivedtraininginthepasttwoyears.Teacherswithaprofessionalqualificationaremorelikelytohaveattendedtraining.Amongstthosewhoreceivedsomeformoftraining,around80%reportedbeingtrainedonteachingmethodsandaround70%indicatedthatthetrainingwasimplementedbyGEP/UNICEF.Thesurveyalsoidentifiedthattheappropriatefacilitatorsarenotalwaystrained–notalltraineeswhoattendedGEP3’sfacilitatortraininginAugust–September2015wereinfactfacilitatorsteachingatanIQS.WithinthesampledIQSsforwhichthesurveyteamhadlistsoftrainedfacilitators22%offacilitatorswereunknownornotteachingattheIQS.Accordingtothefindingsofthequalitativecasestudiesfacilitatorsareinterestedinreceivingtraining:theyexpressaneedformoretraininginsubjectknowledgeinordertoeffectivelyusethetoolsthatseemtobethefocusofthetraining,togetherwithsensitisationaboutthevalueofinclusiveinstruction.ThecasestudiesalsosuggestthatfacilitatorswithinthesameIQSarenotnecessarilyawareofeachother’sparticipationintraining,whichmayhinderknowledgespillover.Hence,knowledgedisseminationstrategiesneedtobeconsideredtoreinforceimpact.33

Facilitators’knowledgeandskills

Facilitatorsdisplayedverylowlevelsofcompetenceacrossthesixdomainscoveredbytheteacher

assessments.Levelsofcompetencewereparticularlylowonthetwodomainsassociatedwithpedagogicalknowledge:only2%offacilitatorswerecompetentinidentifyinglowperformersandnoneofthefacilitatorswerecompetentinevidencingjudgementsanddiagnosingpupilperformance(seeFigure8).Thereisaverystrongtraditionineducationalresearchthatacknowledgesthatastudentlearnsbestwhenteachingistargetedtowhats/heisreadytolearn.Facilitators’weakskillsonthesefrontsthereforepresentmajorchallengestoimprovingpupillearning.

Facilitatorsalsodisplayalmostnoevidenceofcompetenceinwritingandinterpretingwordsandphrases.KnowledgeofGrade1and2-levelHausaappearsmoreprevalent,althoughalargeshare(66%)offacilitatorswerenotabletodisplaycompetenceinthis,whichraisesquestionsaboutfacilitators’abilitytoraisepupils’learninglevelsinthisarea.Similartothefindingsoftheearlylearningbaseline,writingskillsappeartobecorrelatedwithotherskills,whichsuggeststhattheliteracylevelsofthefacilitatorsmaybeakeyissue–ifthoselevelsarelowthislimitsperformanceacrossarangeoftheareasfacilitatorsneedtobecompetentininordertoimprovepupillearning.

Thequalitativecasestudyfindingsreinforcethispicture.Facilitatorswereoftenunabletoexplainwhichsubjectmattershouldbetaught,orwhytheyteachwhattheyteach.Manyfacilitatorswereteachingatthelevelthattheywerethemselvestaughtto.Whilstfacilitatorsdisplayedawarenessofteachingandpedagogicalmethods,inmostcasesfacilitatorsthemselvesdidnotseemtohavesufficientsubjectknowledgetoteacheffectively.

33Thefindingsofthemotivationcompositeindex(seesectiononfacilitatormotivationandattendance)indicatethatfacilitatorshavearelativelypositiveperceptionaboutteacher-to-teacherinteraction.Thereisthereforeopportunitytopromoteknowledgeexchange.

©EDOREN 52

Figure8: Percentageofteachersachievingwithinthelower,middleandupperbandsof

competenceacrosstheteacherknowledgeandskillssubscales

Facilitatorsdonotseemtoperceivetheirownlackofcompetencetobeacauseoflowpupil

performance.Facilitatorsgenerallyperceivepupilefforttobeboththecauseoflowpupilperformanceaswellasthebestresponsetolowperformance.Whenfacilitatorswereaskedwhatstepstheycouldtaketoimprovepupils’performance,theirresponsesfocusedonencouragingpupilstoincreasetheireffortandtopayattentioninclass.Therewaslittlediscussionoftheerrorsinpupils’work,andwhatfacilitatorscoulddotoimprovethis.Thefocusonpupileffortwithinthiscontextislikelytobeareflectionofthelowlevelsofcompetenceamongstfacilitatorsthemselves,andtheirlackofunderstandingofwhattheycoulddoto

improvepupils’performance.Arelatedfindingofnotefromthequalitativecasestudiesisthatcommunitymembersoftenperceivefacilitators’subjectknowledgetobehigh.Thisisinstarkcontrasttothefindingsoftheteacherassessments,andcanbeexplainedbyparents’ownlimitedknowledge.Thissuggeststhatparentsareunlikelytoholdfacilitatorstoaccountforlowqualityteaching.

Gender-sensitiveclasspractices

Facilitatorsstatedhavingpositiveattitudestowardsgirls’education,butgenderbiasespersist.Thefacilitatorsinthesixcasestudyschoolsexpressedpositiveattitudestowardsgirls’education.However,itisunclearwhetherthisaccuratelyrepresentsfacilitators’attitudesaswecannotruleoutsomedegreeofsocialdesirabilitybiasintheseresponses.34Indeed,bothqualitativeandquantitativedatapointtosomedeeplyingrainedgenderbiases.Somecasestudyfacilitators,whileperceivinggirls’educationtobeimportant,thinkthatgirlsarelessablethanboys,andthatgirlsdonotrequireasmucheducationasboysdo.Inallbutonecase,facilitatorsperceivedboystobemoreintelligentthangirls.Facilitators’attitudesseemtobeinfluencedbytheperceivedimportanceattachedtogirls’education,giventhegenderedroles

34Socialdesirabilitybiasreferstothetendencyofrespondentstoanswerquestionsinamannerthatwillbeviewedfavourablybyothers.Thisislikelytohavebeenthecaseinthequantitativesurvey,inwhichweobservedextremecomplianceeffectsacrossallitemsdevelopedtomeasureattitudestowardsgirls–thispointstosocialdesirabilitybiasandcallsintoquestionthevalidityandreliabilityofthemeasure.Therefore,changesinthegendersensitivityoffacilitators’classroompracticesisbestdetectedthroughthequalitativeresearch,althoughsomedesirabilitybiasmayalsobeobservedheretoo.

0%

20%

40%

60%

80%

100%

Abilitytoidentifylowperformers

Abilitytoevidence

judgementsanddiagnose

Teacherwritingskills

TeacherGrade2Hausa

knowledge

Teachercomprehensio

nskills

Interpretingwordsandphrases

Upperband 1.9 0 .9 34.0 25.4 2.8

Middleband 6.5 .9 .9 27.2 18.1 4.2

Lowerband 91.6 99.1 98.1 38.8 56.5 92.9

Note:Twocut-pointsweredefinedforeachsubscaletocreatethreeproficiencylevelsperscale.Theseproficiencylevelsare:LowBand–noevidenceofskill;MiddleBand–evidenceofrudimentaryskill;andUpperBand–evidenceofcompetenceSource:GEP3Baselinesurvey2015

©EDOREN 53

girlsareexpectedtotakeupinfuturelife.Thisisconfirmedbytheresponsesaspartoftheteacherassessments,whichindicatedthatgirlsareperceivedtobenaturallylesssuitedtothedemandsofeducationthanboys,ormoresuitedtootherroleswithinsociety.

Whilefacilitatorsdisplaygender-sensitivetechniques,gender-biasedclassroompracticesprevail.Bothqualitativeaswellasquantitativefindingsindicatethatfacilitatorsareawareof,andsometimespractice,gender-sensitivetechniques,suchasactivelyengagingbothgirlsandboys,andevenfocusingmoreongirls.However,thesepracticesseemtobeperformedinatokenisticway—inlinewithdevelopmentprojects’sensitisationefforts—andareunlikelytobeabletosurmountthegender-biasedclassroomarrangementswithintraditionalQur’anicschools,suchasthepracticeofsegregatingboysandgirlsintheclassroom.Furthermore,teachingpracticesareinfluencedbyfacilitators’andpupils’gender-biasedattitudesaboutgirls’abilitytolearnandtheimportanceofeducationintheirfuturelife.Itis,therefore,importantthatfacilitatortrainingnotonlyencouragestheuseofgender-sensitivetechniquesintheclassroom,butthatitalsoseekstocontributetothetransformationofthefacilitators’attitudestowardsgirls’abilitiesandtheimportanceofgirls’education.

Facilitators’motivationandattendance

WithintheIQSSToCfacilitatormotivationandattendanceareconsideredbothtobeintermediaryresultsofmentoringandbetterschoolmanagementaswellassupportingfactorstoachievemoreeffectiveteaching.Wemeasuremotivationthroughafive-scalemotivationindex(seeError!Referencesourcenotfound.intheearlylearningbaselinesection).

Overall,facilitatorsconsidertheirroletobeimportantandenjoyworkingasteachers,buttheyhavea

negativeperceptionoftheirteachingefficacy.Thequantitativefindingsofthemotivationscalescoresareverysimilartothoseoftheearlylearningsample.Onaverage,thesurveyedfacilitatorsscorehighestoneffortandlowestonperceivedteacherefficacy(seeFigure9).ThefacilitatorsinthecasestudyIQSsreportfeelingintrinsicallymotivatedbywhattheydo,andschoolstakeholders,includingpupils,considerfacilitatorsgenerallytobededicated.

Figure9: Comparisonofmotivationsubscales,IQSSsample

1.99

3.57

3.73

2.19

3.47

0 0.5 1 1.5 2 2.5 3 3.5 4

Perceivedteacherefficacy

Interestandenjoymentofteaching

Effortandimportance

Pressureandworkrelatedtension

Teacher-to-teacherinteraction

Source:GEP3Baselinesurvey2015

©EDOREN 54

Facilitators’motivationisshapedbyissuesofremuneration,socialstatusandcommunityrolesand

relations.Thequalitativefindingsindicatethatthefactthatmostfacilitatorsarenotpaidmakesitdifficulttoattractqualifiedfacilitators.Thequantitativefindingsconfirmthataverysmallshareoffacilitatorsarepaideitherasalaryorastipend(3%ofthesurveysampleinBauchiand33%inNiger)andpointtoapositiveassociationbetweenreceivingremunerationandteachermotivation.FacilitatorsinthecasestudyIQSsreportnotfeelingvaluedduetotheirlowpay,asitmakesthemstruggleto‘moveupalevel’inlifeascomparedtotheirpeers.Lackofremunerationislikelytoparticularlyinfluencemenduetothegenderedexpectationsofmenasbreadwinnerswithinsociety.StakeholdersofthecasestudyIQSsacknowledgethatthereisaninherentinconsistencyhere,withfacilitatorsexpectedtodeliveratthestandardofateacher,whilefendingforthemselvesfinancially.Asisevidentfromthequalitativefindings,bynotpayingfacilitators,theexpectationsplacedonafacilitator,andthedegreetowhichthecommunitycanholdfacilitatorstoaccount,decrease.Yet,remunerationclearlyisnotthemainmotivationforfacilitatorstoteachatanIQS.InallsixcasestudyIQSs,facilitatorsaremembersofthelocalcommunityandseveralhavepreviouslybeenpupilsundertheQur’anictutelageoftheMallamorhispredecessor.Stakeholdersperceivedthistobethemainreasonwhyfacilitatorsagreetoteachvoluntarily.Thequantitativefindingssuggestthatpedagogicalleadershipmayhaveaninfluenceonfacilitatormotivation,aspositedintheIQSSToC.Facilitatorswhohaveindividualmeetingswiththeheadteachersaremoremotivatedthanthosewhodonot.

Teacherattendanceisflexible.Around75%ofthefacilitatorsreportedhavingbeenabsentatleastonceduringthepreviousthreemonths,withfacilitatorsinBauchibeingmorelikelytobeabsent,andforlonger,thanfacilitatorsinNiger.Thereisnomarkeddifferenceintheabsenteeismratesforteachersbyremunerationstatus.ThecasestudyfindingshighlightthatfewofthecasestudyIQSsseemtohavesettimetables,andformalsubjectsaretaughtasandwhenafacilitatorisavailable.TeacherattendanceinthecontextofIQSsisthereforeaflexibleconcept.Furthermore,teacherattendancedoesnotnecessarilymeanthattheteacherisprovidinginstruction.Forexample,inseveralofthecasestudyIQSsitwascommonforpupilstorun(partof)theclasswiththehelpofaclassmonitor–thatis,apupilwhoteachesyoungerstudentsifthefacilitatorisnotpresent.Ingeneral,theinterviewedstakeholdersperceivethefacilitatorstobepunctualanddonotseefacilitatorabsenteeismasakeyreasonforlossofinstructionaltime–rather,theyattributethistoanIQSnothavingenoughfacilitators.Instructionaltimeisalsoinfluencedbyfacilitatorsandpupilshavingtoengageinincome-generatingactivities.Additionally,manypupilsinthecasestudyIQSsalsoattendpublicprimaryschool,thusrestrictingIQSinstructionaltimetotheafternoonandevening,whenQur’anicteachingalsooccurs.

Facilitators’teaching

Overall,levelsofcompetenceineffectiveteacherpracticeswerelowacrossallgroupsoffacilitators.ThefindingsofthebaselinequantitativesurveywereusedtodevelopacompositeindextomeasuretheToCintermediaryoutcome:toimproveteacherpracticesandgendersensitivity.Theindexisbasedontheextentofpupil-centredlearningactivitiesobservedintheclassroom35,observationsoftheteacherlinkingthelessontopreviouslearningandlearningobjectives36,andtimeontaskinclass37.Scoresontheindexwerelowforallgroupsoffacilitators,althoughwithsomevariations.Facilitatorsmeetingheadteachersregularly(onceaweekormore),olderfacilitators(aged>50)andthoseinBauchidemonstratedsomewhathigherlevelsofeffectiveteachingpractice.Thequalitativecasestudiesfurtherindicatethatfacilitatorshavesomeawarenessofpedagogicalpractices(forexample,theimportanceoflessonplanningandactivelyinvolvingpupils)butlackadeeperunderstandingofthevalueofcertainmethodsorwhentomakeuseofwhichmethodorpractice.

35Whichinvolved,forexample,observationsofteachersassistingingroupdiscussion,usingachild’sname,askingorrespondingtoanopenquestionetc.36Thisincluded,forexample,observationsoftheteachertalkingaboutthepreviouslesson,outliningtheobjectivesoftheobservedlessonetc.37Measuredaspercentageoftimethepupilswereengagedduringlessonobservations.

©EDOREN 55

Timeontaskwasgenerallygoodinlessonsobservedaspartofthequantitativesurvey—withinthemajorityoflessonobserved(64.5%),childrenspent100%ofthelessonontask—althoughthismaybeinfluencedbyobserversbeinginthelessons.Inabout10%ofclassroomsfacilitatorswerenotpresentatsomepointduringclass.Asmentionedabove,thequalitativecasestudieshighlightthatinstructionaltimeinIQSsisorganisedinaflexible,adhoc,wayandvariesacrossIQSs.Pupils,facilitatorsandotherstakeholdersreportedthatlessonsnormallylastaround20minutes,andthatabouttwosubjectsaretaughtperday,butwithstrongvariationacrossIQSs.

Hausaisthemostcommonlanguageofinstructionbutoftenvariouslanguagesareusedinteaching

duringthesameclass.InBauchi,themajorityoffacilitators(72%)usedjustonelanguageduringthelessonobservation.Afurther24%wereobservedusingtwolanguages.InNiger,25%offacilitatorsusedjustonelanguageduringthelesson,while60%usedtwolanguages.Asmallminorityoffacilitatorsinbothstatesusedthreeormorelanguagesduringalesson.ThemajorityoffacilitatorsinbothstateswereobservedusingHausaatleastonceduringthelesson(100%inBauchiand72%inNiger,seeFigure10).Englishwasalsowidelyused,particularlyinNiger.48%ofteachersobservedinNigeralsomadeuseofalanguageotherthanHausa,EnglishandArabic(theclassroomobservationtooldidnotrecordwhattheseotherlanguageswere,althoughitislikelythatNupefeaturesprominentlyheregiventhatitisspokenbyanotableshareoffacilitatorsandpupils).

Figure10: Facilitatorlanguageuse:Shareoffacilitatorswhousedthelanguageatleastonce

duringthelesson

Availabilityanduseofteachingandlearningresources

Theavailabilityofteachingandlearningresourcesisverylimited,ashighlightedbyFigure11.Themostcommonlyusedresourcesarethosemadebyhandbyfacilitators.Atextbookwasavailableandusedinonly9%ofclassesobserved.Whenthematerialswereavailable,theyweremostlyused.ThequalitativecasestudiespointagaintostrongdiversityamongIQSs:onecasestudyIQShadaccesstoawiderangeofresourcesincludingcomputers,whileothersdidnothaveanyformalmaterialsbutusedstones,sticksandbottletopstofacilitatemathematicsclasses.SomecasestudyIQSshadreceivedacurriculumforteachingHausa,andsomeflashcardsandotherteachingaids,whichfacilitatorsperceivedtoappropriate,althoughthereliabilityofthisfindingneedstobeinterpretedwithcautionasfacilitatorsseemedtoperceiveaccesstoanymaterialstobepositive.Thecasestudiesalsoindicatethatmanychildrenlackbasicmaterials,such

0

10

20

30

40

50

60

70

80

90

100

Hausa English Arabic Other

ShareofFacilitators(%

)

Bauchi Niger

Source:GEP3Baselinesurvey2015y

©EDOREN 56

asnotebooksandpens,theprocurementofwhichistheresponsibilityoftheparents.Severalrespondentsconsideredtheinabilityofparentstopurchaselearningmaterialstobeareasonwhychildrenwillnotattendschool.

Figure11: Availabilityanduseofteachingandlearningmaterials

AlmostnoHausamaterialsareavailablebutitshouldbenotedthatHausaisnotnecessarilyalwaysthe

mothertongue.Figure11showsthatmaterialsinHausawereavailableinalmostnoneoftheobservedlessons.ThisprovidesaclearbaselinefortheIQSSintervention,withmuchopportunityforimprovement.However,whilealmostallfacilitatorssurveyedreportedspeakingHausa,correspondinglyonly73%ofthesampledchildrenreportedspeakingHausaathome–withastrongdifferencebetweenBauchiandNiger:93%inBauchi,versus54%inNiger.InNiger43%ofpupilsreportedlyspeakNupeathome.Furthermore,theteacherassessmentdemonstratesthatfacilitatorshavelowliteracylevelsinHausa,henceHausamaterialswillneedtobefit-for-purpose,notonlyinrelationtowhatneedstobetaught,butalsowithrespecttotheskilllevelsoffacilitators.

Pedagogicalleadership

SchoolleadershipinIQSsiscomplex.IQSsareprivateinstitutionsthatcanbeownedbyindividualproprietorsorownedinamorecollectivewaybythecommunitywhilststillhavinganindividualincharge.ThereligiousteachingisgenerallyledbyaMallam,whointhemoretraditionalQur’anicschoolsisgenerallyalsotheproprietoroftheschool.38Withintegration,theheadteacherfunctionsofthenon-religiouseducationareaddedtothisexistingQur’anicschoolleadershiparrangement.Thecasestudiesshowthattheschoolleadershippositions(proprietor,Mallam,headteacherandCBMCmembers)arenotnecessarilyseparateroleswithintheIQScontext.InthecasestudyIQSs,themostcommonscenarioseemstobeonepersontakingonthethreerolesofproprietor,Mallamandheadteacher.Thequantitativedatawerenotabletoconfirmthis.Inone-thirdoftheIQSsurveysample,theheadteacherwasalsoconsideredtobetheproprietor,andthiswasmorecommonforschoolsinBauchi.However,thequantitativedataneedtobeinterpretedwithcautionbecauseofthevariouswaysinwhichthedifferentpositionsarereferredto.Ingeneral,thereisalackofclaritywithinmostcommunitiesregardingwhohaswhatrole,andwhatthatroleentails.Thelackofclarityabouttheserolescanmakeitdifficulttoidentifywhoseleadership

38AsAntoninis(2014)outlines,therearebroadlytwotypesofreligiouseducationinstitutioninnorthernNigeria:Qur’anicreligiouseducationinstitutionsandIslamiyyareligiouseducationinstitutions.Qur’anicinstitutionsencompassbothmakarantaallo(‘schooloftheslate’)andTsangaya.

0.0

5.0

10.0

15.0

20.0

25.0

Video Audio

Toolsorobjectsfromthelocal

environment

HandmadeMaterialsin

HausaPoster,chartorpictures

Textbook

Resourcesused 0.0 0 3.8 19.9 1.0 1.9 8.8

Presentbutnotused 0.0 0 4.8 4.7 .9 1.9 1.9

Percen

tofo

bservedlesson

s

Source:GEP3Baselinesurvey2015

©EDOREN 57

capacitiestobuild.Differentactorsmaybeinvolvedintheimplementationofsimilarrolesandthemostqualifiedpersonmaynotleadontherole,forexample,ofpedagogicalleadership.

Headteachersarealmostalwaysmaleandhavelowprofessionalandacademicqualificationswith

differencesbetweenBauchiandNiger.39AlloftheheadteachersinBauchiaremaleandonly7%ofheadsinNigerwerereportedtobefemale.TheheadteachersinBauchiareabout10yearsolder,onaverage,thanheadteachersinNiger,have10moreyearsofworkexperienceatthecurrentschool,andhavemoreyearsofteachingexperience(seeTable6).GiventhatintegrationisrelativelyrecentinBauchi,thissuggeststhat,particularlyinBauchi,theheadteachermayalsobetheMallam.Theprofessionalandacademicqualificationsofheadteachersaregenerallylow,and,onaverage,belowthoseoffacilitators,whichmaychallengetheheadteachers’technicalcapacitytoexercisepedagogicalleadershipoverfacilitators.HeadteachersinNigerhavehigheracademicqualifications:57%havepassedtheSSCE,versusonly23%havingdonesoinBauchi.Intermsofprofessionalqualifications,inbothstatesonlyaminorityofheadteachershaveanNCE,butamajorityofheadteachersinBauchihaveareligiouseducationqualification,indicatingagainthattheheadteacherroleisquiteoftentakenupbytheMallaminBauchi.Thequalitativecasestudiesfurtherhighlightthatthequalificationsoftheheadteacherdifferconsiderablyacrosscases,withsomehavingnoformaleducationatall.However,mostheadteachersappearedconfidentintheirabilitiestoinstructtheirfacilitators.However,pedagogicalleadershipseemstobesharedamongstmanystakeholders,withCBMCmembersguidingheadteachers,headteachersadvisingfacilitators,andproprietorsandcommunityleadersalsoofferingtheirperspectives.

Table6: Summaryofheadteachercharacteristics

Forsurveyedheadteacherstheaverage… Ofheadteachers…

• …ageis40years(44yearsinBauchi,36yearsinNiger);

• …yearsofworkexperienceatcurrentschoolis10years(15yearsinBauchiandfiveyearsinNiger);and

• …yearsofworkexperienceasateacheris14years(17yearsinBauchi,10yearsinNiger).

• …only3%arefemale(0%inBauchi,7%inNiger);

• …23%holdanNCEqualification(27%inBauchi,20%inNiger)and40%passedtheSSCE(23%inBauchi,57%inNiger);56%inBauchihaveareligiouseducationqualification,comparedto20%inNiger;and

• …43%attendedtrainingduringthelasttwoyears(37%inBauchi,50%inNiger).

Source:GEP3Baselinesurvey2015

Theappointmentofaheadteacherisnotnecessarilybasedonabilityandqualificationsbuthastodo

withsocialstatuswithinthecommunityandperceptionsaroundleadershipmoregenerally.InthecasestudyIQSs,theheadteacherroleismainlyconsidereda‘schoolleader’role–havingthesocialstatustosuperviseteachersandprovideadvice,andinfluencethecommunitywithregardstointegrationandgirls’education.Assuch,Mallamsmaybeassignedthepositionbasedonperceivedpoliticalandsocialstatus,ratherthanpedagogicalteachingknowledge.However,giventheiroftenlowacademicandprofessionalqualificationsitisquestionablewhethertheheadteachershavetheabilitytoprovidepedagogicalsupervisionandmentoring.Itwilllikelybeachallengetopinpointwhototrain,andforwhatleadershiprole,sincetheassignedheadteachersmaynotbethemostqualified,intermsofformaleducation,totakeupapedagogicalleadershiprole,buttheymayhavethesocialstatustoprovidemoralleadership.

MoreheadteachershavebeentrainedinNiger,whilemoreexternalmonitoringishappeninginBauchi.AlargershareofheadteachersinNiger(50%)havereceivedsomesortoftraininginthelasttwoyearsthaninBauchi(37%).Headteacherswithhigherprofessionaloracademicqualificationsweremorelikelytobe

39Wedefineheadteachersasthepersonswhoareresponsibleforday-to-daymanagementandsupervisionofthenon-religiousteaching.

©EDOREN 58

trained,andthisdifferenceislargerthantheBauchi/Nigerdifference.ThoughbothstatesreportedtrainingbeingmostlyorganisedbyGEPandUNICEF,moreheadteachersinBauchireportedtrainingbySUBEBortheLGEA,whichsuggestsgreaterinteractionwithgovernmentbodies.ThisisalsosupportedbythefindingthatagreatershareofschoolsinBauchireportedreceivinganymonitoringvisitsfromagovernmentbodyduringthelastterm.ThismaybepartlyexplainedbythefactthatmoreIQSsintheBauchisamplearelocatedinurbanareas.Ingeneral,though,only44%and33%ofIQSsinBauchiandNiger,respectively,reporthavingreceivedanymonitoringvisitfromgovernmentofficials,non-governmentalorganisations(NGOs)orexternalagenciesduringthelastschoolyear.

Headteachersdemonstratesomepedagogicalleadership,particularlyinNiger,andtheytakeactionsto

improvepupilandteacherattendance,butrecord-keepingispoor.ThereisamarkeddifferencebetweenNigerandBauchiintermsofheadteachersobservinglessonsoffacilitators.InBauchifewheadteachersreportedhavingsatinonanylessonstaughtbyfacilitatorsfortheentiredurationofthelesson,whilemorethanhalfofheadteachersinNigerreportedhavingdonesoduringthelastterm(seeFigure12).TherewerefewerdifferencesbetweenBauchiandNigerforotherpedagogicalleadershipandmanagementactions,suchasmeetingfacilitatorstodiscusstheirteachingoractionstoimproveteacherandpupilattendance,althoughheadteachersinNigerseemtobemeetingmorefrequentlywiththefacilitators,individuallyoringroups.Interestingly,trainedheadteachersweremorelikelytoobservelessons,haveobservationrecordsormeetteachers.Record-keepingbyheadteacherswasgenerallyfoundtobepoor,withonly30%ofheadsabletopresentP2pupilenrolmentorattendancerecords,andfarfewerinstancesofupdatedattendancerecords(10%).P2pupilattendancerecord-keepingwasmarkedlybetterinBauchi.

Figure12: HeadteacherpedagogicalleadershipandmanagementactionsinNigerand

Bauchi

5.3.2 ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschool

environmentinIQSs

GEP3’sIQSSseekstocontributetoanimproved,girl-friendlyschoolenvironmentbystrengtheningschoolmanagement,promotingthemobilisationofmoreresourcesandincreasingtheinvolvementofthecommunityintheIQS.ThemaingatewaytoachievetheseintermediaryoutcomesistheempowermentofCBMCs,bytrainingthemandprovidingthemwithmini-grants.Inaddition,headteachertrainingisassumedtocontributetobetterschoolmanagementaswellasimprovedgovernmentmonitoringand

0 10 20 30 40 50 60 70 80 90 100

HTobservedalesson

HTtookactiontoimproveteacherattendance

HTtookactiontoimprovepupilattendance

HTwhomeetteachersindividually

HTwhomeetteachersingroup

PercentageofheadteachersNiger Bauchi

Source:GEP3Baselinesurvey

©EDOREN 59

supportsupervision.Inthissectionwefirstdiscussthebaselinefindingsinrespectofschoolmanagement,withspecificattentiongiventotheroleoftheCBMC,followedbyfindingsonresourcemobilisation,communityinvolvement,theschoolenvironmentandgovernmentmonitoringandsupport,drawingfrombothquantitativeandqualitativedata.

Summaryofevidencerelatedtoevaluationquestions

Q4:HowwellhasGEP3’sCBMC

capacitydevelopment

contributedtoimprovedschool

managementandincreased

mobilisationofresourcesfor

schoolinvestment?

CBMCsareactiveandfunctionaltovaryingdegrees.Thereisscopeforimprovementintheirrepresentativeness,theirschooldevelopmentplanningandtheirfinancialmanagement.CBMCsareabletomobiliseresourcesbutsignificantresourcegapsremain.

CBMCsexistin87%oftheIQSSschools.MostoftheestablishedCBMCsappeartobeactive,asindicatedbyregularmeetingsandmonitoringactivities.63%ofCBMCshadalreadyreceivedtrainingduringthelasttwoyears,althoughclearareasforimprovementinCBMCfunctionalityremain,including:childrepresentation,attendanceratesatmeetings(particularlyfemale),schooldevelopmentplanningandfinancialmanagement.

CBMCsareabletosuccessfullymobiliseresourcesfromthecommunitiesbutsignificantresourcegapsremain.TheperceptionofIQSstakeholdersislessthatparentsareunwillingtocontributefunds,butmorethatparentshavelimitedfundstogiveduetopoverty,whichisoneofthekeyreasonswhyparentssendtheirchildrentoIQS.CBMCtrainingonresourcemobilisationshouldconsiderthemobilisationofalternativeresources,giventhatthereappearstobelimitedscopetoraiseresourcesfromparents.

CommunitysupportofIQSsandCBMCsisevidencedbycommunitymembershipintheCBMC,andbycommunitymembersdonatingmoneyandotherresourcesforschoolimprovementactivities.HeadteachersandproprietorsareusuallypartoftheCBMC,andoftenchairtheCBMC.ThistestifiestotheiracceptanceoftheCBMC’sinvolvementinschoolmanagement.SinceIQSschoolmanagementisoftenseenascollectiveinthecommunity,GEP3willneedtoconsiderhowtoensurethatallrelevantstakeholdersareincluded,andthatconflictsofinterestareavoidedwhenproprietors/headteachershaveoverlappingroles.CBMCmembersseetheirrolesasbroad,coveringarangeofresponsibilities,butfeelthattheyhaveweakcapacityandlimitedresourcestofulfilthem.

Q5:HowwellhaveCBMCsbeen

abletoadequatelymanagemini-

grantsandinvesttheseresources

intheimprovementofagirl-

friendlyschoolenvironment?

IQSsareabletoinvestintheschoolenvironmentanddoconsidermakinggirl-friendlyinvestments,butthefinancialresourcestheyhaveaccesstoaregenerallysmall.Additionaltrainingandsupervisioninfinancialmanagementareneeded.

IQSsareuniversallydeficientin‘soft’and‘hard’infrastructure,andprovideapoorschoolenvironmentfortheattendingchildren,particularlygirls.IQSshavesevereshortagesofhygienefacilitiesforgirlsandofferfewspaceswheregirlsandboyscanparticipateinshapingeducationorvoicetheirneeds.CBMCsreportedutilisinganaverageof80%ofthetotalfundsraisedinthepastyearsonschoolimprovements.ThequalitativecasestudyfindingsindicatethatCBMCshaveconsideredtheattendanceandretentionofgirlswhenallocatingresources.However,almost50%ofCBMCsreportednothavingaccesstoanyfundsduringthelastyearandthemedianamountraisedislimited(NigerianNaira(NGN)20,000).Financialmanagementpracticesarepoorandnecessitatefurthertraining.GovernmentmonitoringandsupervisionislimitedandconsideredunreliablebyIQSstakeholders.

©EDOREN 60

SchoolmanagementandtheroleoftheCBMC

Schoolmanagementisseenascollective,withoutclearlydefinedanddelegatedroles.Aswasdiscussedintheprevioussection,schoolleadershipinIQSsiscomplex,whichisreflectedinschoolmanagement.Whilethequantitativesurveyshowsthat90%oftheIQSshaveasoleproprietor,thequalitativecasestudiessuggestthatthemanagementoftheIQSisnotanindividualresponsibilitywithclearlydefinedanddelegatedresponsibilities.Thiscreatesconfusionaroundwhoisinchargeofwhat.Severalactorsareperceived—andperceivethemselves—tobepartofthesuccessfulrunningoftheschool,inmanycasesmeaningthatmanagementbecomescollective.Theseactorsincludenotonlythosewhoareactivelyinvolvedwiththeschool,butalsostakeholderssuchascommunityleadersandimamswhooftenhaveinfluenceovertheschool’svisionandthecommunity’sacceptanceofintegration.Suchperceivedcollectiveresponsibilityovertheschoolprovidesanopportunityforacommunity-basedmanagementstructureliketheCBMC,whereresponsibilitiescanbeheldcollectively.TheimportanceofexistingsocialstructuresandnormswithintheIQScommunitiesmeanthatGEP3willneedtoconsiderhowtoensurethatallrelevantstakeholdersareincluded,andfortrainingtoespeciallytargetthosewhoholddecision-makingauthoritywithincommunities.

MostCBMCsareestablishedandactive,andalargesharehavereceivedtraining.TheCBMCcanonlycontributetotheimprovementoftheschoolenvironmentundertheassumptionthatitisactuallyestablishedandactive.In13%oftheIQSssurveyedtheCBMCwasnotestablished.Thiswasthecasein20%ofIQSsinBauchi,versus7%inNiger.WhilemostCBMCsappeartobeactive(asindicatedbyhavingmetduringthepastschoolyear),theaverageattendancerateatthelastmeetingwaslessthanhalfofthemembers(seeTable7).WhileCBMCsinBauchimetalmosttwiceasoftenasthoseinNiger,theattendanceratewaslower.Approximately60%ofCBMCshadreceivedsomesortoftrainingduringthelasttwoyears.

CBMCshaveconsiderablecommunityrepresentation,althoughwomenareaminorityandchildrenare

hardlyincluded.ParentsmakeupmostofthemembersinatypicalCBMC,althoughinthemajorityoftheIQSssurveyedtheCBMCisnotchairedbyaparentbutbytheproprietororheadteacher(seeTable7).InallIQSsincludedinthequalitativestudy,theMallam—oftenalsobeingtheproprietor—holdsakeypositionintheCBMC,suchaschairperson.ThereisthusaquestionaroundpotentialconflictsofinterestinregardtotheCBMC’smonitoringroleandabilitytoholdschoolleadershiptoaccount.Womenformaminorityamongthemembers,although87%ofallCBMCshadatleastonefemalemember.Theaveragefemaleattendancerateatthelastmeetingwas35%,whichisbelowtheattendancerateforallmembers.Almostnochildren(definedasanyoneundertheageof15years)areincludedinCBMCmembership.ThisindicatesthatatpresentCBMCsdonotprovideaforumforgirls’voicesandneedstobeheard,andforgirlstoinfluenceinvestmentsintheschoolandbroaderdecision-makingrelatedtoschoolmanagement.InterviewedCBMCmembersmostlyconsiderwomentobeanimportantresourceoftheCBMCastheyhavethepotentialtoconvincemotherstosendtheirchildrentoschool.However,genderrolesandresponsibilitiesunderminetheextentofwomen’sinvolvementinCBMCs.

Table7: SummaryofCBMCcharacteristics

ForsurveyedCBMCstheaverage… OfCBMCs…

• …yearssinceCBMCestablishmentisthreeyearsago;and

• …membershipis14members(16inBauchi,13inNiger),amongwhich:fourwomen,nochildren,10parentsandthreefacilitators;and

• …attendancerateatlastCBMCmeetingwas41%(36%inBauchi,46%inNiger).*

• …37%wereestablishedbeforeintegration,ofwhich70%wereestablishedtheyearbeforeintegration;and

• …55%havetheproprietororheadteacheraschair(66%inBauchi,45%inNiger);

• …90%hadameetingduringthelastschoolyear,withfourmeetingsonaverageperyear(sixinBauchi,threeinNiger);and

©EDOREN 61

• …63%receivedtraining(54%inBauchi,71%inNiger).

*AttendancerateisestimatedforthoseCBMCsthatkeptmeetingrecords,whichwastwo-thirdsofCBMCs.

Source:GEP3Baselinesurvey2015

CBMCmembersseetheirrolesasbroadandcoveringarangeofresponsibilities.CBMCsareactivein

termsofmonitoringbutthereisscopeforimprovementinschoolplanningandfinancialmanagement

practices.ThequalitativeresearchfoundthatCBMCmembersgenerallyunderstandtheirrolesandresponsibilitiesbuthaveweakcapacityandlimitedresourcestoactonthem.CBMCsseetheirroleasbroadandperceivetheirmainresponsibilitiestobeincreasingtheawarenessoftheimportanceofeducationforbothboysandgirls,identifyingwhychildrenarenotenrolledandlearning,andto‘solve’theseproblems.Whiletheyalsoperceivepromotingfacilitatorattendancetobepartoftheirresponsibilities,monitoringsometimesbecomessimplysymbolicasCBMCmembersmonitorfacilitatorperformancewithoutpriorteachingexperienceandwithalackofguidance.ThequantitativedataindicatethatCBMCsareactiveintermsofmonitoring.AlmostallCBMCchairsindicatedhavingvisitedtheschooltomonitoritsperformanceduringthelastterm,withtheaveragenumberofreportedvisitsbeingseven.Over80%ofCBMCmembersreportedlymonitoredpupilandteacherattendanceandtookactiontoimprovepupilandteacherattendanceduringthelastterm.Areasforimprovementareschoolplanningandfinancialmanagement.Only20%ofCBMCssurveyedhadcompletedaWCDP,ofwhichtwo-thirdscouldprovideevidenceabouttheWCDP.Intermsoffinancialmanagement,only27%ofCBMCsreportedusingacashoraccountbook,ofwhichhalfcouldpresentanupdatedcashbook.70%ofCBMCsdidreportusingabankaccounttostorefunds,butonly40%indicatedthattheykeptrecordsofbankdepositsandwithdrawals.

Resourcemobilisation

CBMCsareabletosuccessfullymobiliseresourcesfromthecommunities.ThemajorityofCBMCssurveyedundertaketheresponsibilityofmobilisingresources(65%ofCBMCsreportedcashmobilisingeffortsinthepreviousyear)andtheyarereasonablysuccessfulindoingso(over80%ofthoseundertakingtheseactivitiesreporteddoingsosuccessfully).Inallinstances,mostoftheresourcesweremobilisedfromthecommunity,withasmallsharereceivedfromthegovernment,NGOsandotherexternalagencies.40%ofCBMCsinBauchiand30%inNigerreportedhavingreceivedaGEPschoolgrantduringthelastschoolyear.Inaddition,about40%oftheCBMCsalsomobilisednon-cashresources,whichcameexclusivelyfromthecommunity.

SignificantresourcegapsexistinIQSsandCBMCs’perceivedroleisunderminedbyalackoffunding.ThemajorityofCBMCsurveyedhadmadeanefforttomobilisefundsduringthepreviousyear.Nevertheless,almosthalfreportedthattheydidnothaveaccesstoanyfundsduringthelastyeareitherbecausetheyhadnottriedtomobilisefundsorbecausetheireffortshadbeenunsuccessful,andbecausetheyhadnotreceivedanyfundsfromothersources.Also,themedianamountraisedwasonlyNGN20,000,whichissmallcomparedtothevalueoftheschoolgrants,whichmostlyamountedtoNGN120,000orNGN150,000.ThequalitativeresearchhighlightshowCBMCsperceivealackoffundstobehinderingtheirabilityto‘solve’theproblems.Atthesametime,CBMCsacknowledgethatparentshavelimitedcapacitytocontributefunds.Theperceptionislessthatparentsareunwillingtocontributefunds,butmorethatparentshavelimitedfundstogive.ThisisunderlinedbythefactthatoneofthekeyreasonswhyparentssendtheirchildrentoIQSsappearstobeduetopoverty,andaninabilitytoaffordtosendchildrentothepublicprimaryschool.AsIQSsareprivatelyrun,eitherbyaproprietororbyasmallcommunity,theyrelymoreonresourcemobilisationthanpublicschools.Byrelyingoncontributionsfromcommunitymembers,whomaynothavefundstogive,thereisariskthattheworkofCBMCscouldbediminishedas‘externalresources’runout.CBMCtrainingthatfocusesonbuildingcapacitytomobilisecommunityresourcesmaycreateunrealisticexpectations.Alternativeresourceswillneedtobeconsidered,althoughCBMCs’capacitytoadvocateforgovernmentresourcesmaybelimitedastheIQSsectorisalreadyunder-funded—asevidencedbythelackoffacilitatorremuneration—andCBMCs(unlikeSBMCs)donothaveLGA-levelandstate-levelrepresentativestovoicetheirconcerns.

©EDOREN 62

Communityinvolvement

ThereisgoodcommunitysupportforIQSs.Asmentionedabove,thereappearstobeconsiderablecommunitysupportforintegratedschools.Thisisreflectedinmanyways,suchascommunitymembersbeingpartoftheCBMCanddonatingresources.Supportbythecommunityisnotlimitedtofinancialcontributionsalonebutalsoencompassesmoralsupportandtrust,aswellasCBMCmembersbeinginvolvedinschoolmonitoring.ThequalitativecasestudiesindicatethatgiventheIQSs’embeddednesswithinthecommunity’ssocialstructurestheactivesupportofcommunityandreligiousleadersseemsvitalforsuccessfulintegrationandaidstremendouslyintermsofgainingparents’support.

However,morecommunityengagementwithCBMCsdoesnotalwaysmeancommunitydecision-making

power.Althoughparentsmaybeinvolvedindecisions,andactivelyengagewithdiscussions,thatdoesnotnecessarilymeanthattheyhavedecision-makingpower.SincetheMallamoftenhasahighstatus,throughbeingtheproprietor,theheadteacher,aCBMCmemberandareligiousauthority,hisdecision-makingauthoritytendstobestrong.Moreover,theinvolvementofmenandwomendifferssignificantlyintheIQScontext.MenmakeupthemajorityofCBMCmembers.Whenwomenarememberstheymaynotattend,asillustratedbysomeofthecasestudyschoolsandtherelativelylowerCBMCattendancerate.

RolesandresponsibilitiesmaybewellunderstoodbyCBMCmembers,buttheymaynotalwayshavethe

capacitytoenactthem.Thiscouldpartlyexplainwhyincreasedcommunityinvolvementdoesnotalwayscorrelatewithimprovedactivitieswithinschools.WhilstCBMCsgenerallyfeelthattheyaredoingagoodjob,theybelievethatiffacedwithfewercapacityconstraints—mainlyidentifiedasalackoffundstoimproveinfrastructureandtopayfacilitators—theycoulddobetter.WhilstCBMCmembersshowawillingnesstoadoptnewskillsandknowledge,theyfacenumerouspracticalchallengeswhenitcomestomobilisingresourceseffectively.Contextualconstraints—suchassocioeconomicfactors,theirownqualifications,andthechallengesofnavigatingengrainedsocialnorms—needtobetakenintoaccountwhendesigningcapacitybuildinginresourcemobilisationforCBMCs.ThecontextinwhichCBMCsoperatealsoneedstobetakenintoaccountwhenevaluatingtheirefficiencyandeffectiveness:thecaseofCBMCscannotbecomparedeasilytoSBMCsinpublicschools.Moreover,CBMCstendtobehighlyembeddedintheschoolstructure,dealingwithaspectsofreligiouseducationandgeneralcommunitywellbeing,inadditiontoaspectsofintegration.Asaresultofthisbroaderremit,CBMCswilllikelybeengagednotonlyindiscussingandmanagingIQSmatters,butalsodiscussingandmanagingablendofbothQur’anicandsecularlearning,aswellascommunityissuesexternaltoeducationmatters.

Theschoolenvironment

IQSsareuniversallydeficientin‘soft’and‘hard’infrastructure,andprovideapoorschoolenvironmentto

theattendingchildren.Intermsof‘soft’infrastructure,only3%ofIQSssurveyedhavealibraryandonlyathirdhaveaplayground.Only3%oftheschoolshavesomeformofmothergroup,teacher–studentassociationorpupilgroupwherestudentscancometogethertoshareconcerns.ThequalitativecasestudiesindicatethatsomecorporalpunishmentisusedinallsixIQSvisited,whichpupilsperceivetobethemainthingtheydislikeaboutschool,andwhichdetersthemfromattendingschool.However,thisappearstobealsothecaseinpublicprimaryschools,whichpupilsconsidertobelessofasafespace.Physicalinfrastructureislackinginthemajorityofschoolsinbothstates—withmoreschoolsinBauchiinneedofrepairs,andchildrennothavingaccesstofunctionaltoiletsanddrinkingwateratschool.90%oftheschoolsarereportedasbeing‘inneedofmajorrepairs’.Only40%oftheschoolshaveanelectricityconnectionandamongthoseconnectedtothegrid,only3%actuallyhadelectricityonthedayofthesurvey.Only3%oftheschoolshaveaccesstoawatersource,20%oftheIQSsdonothaveaphysicalclassroomstructureforthestudents,andonly25%oftheschoolshaveafunctioningtoiletfortheirpupils.Bothfacilitatorsandpupilsacknowledgediscomfortwiththepoorschoolconditions.

MostIQSsdonotcurrentlyofferagirl-friendlyschoolenvironment.IQSshavesevereshortagesofhygienefacilitiesforgirls—amongthelessthan25%schoolswithafunctioningtoiletforpupils,only8%have

©EDOREN 63

functioningtoiletsforgirls—whichcanleadtogirlsreturninghometousefacilitates,andsubsequentlymissingclasses.Whereschoolshaveplaygrounds,girlscouldbeseentobeusingthemduringobservations,butthequalitativecasestudiessuggestthatboyshavemorefreedomtoplayandgirlsexpressadesiretobeabletoplaylikeboys.Asmentionedabove,only3%schoolshaveanyinteractivegroupswherepupilscancometogethertoshareconcerns.NeithergirlsnorboysarerepresentedinCBMCsandthereisverylowfemalerepresentationamongfacilitators,headteachersandproprietorsofIQSs.WhilewomenaremostlyrepresentedontheCBMC,theyrepresentaminorityandattendlessthanmen.Thepresenceofastrongfemalerolemodelcanalterperceptionsandself-confidenceamongthegirlsandtheirmothers,asshowninoneofthecasestudyIQSsledbyafemaleMallama.However,thepresenceoffemalefacilitatorsdoesnotnecessarilymeanthatgirlswilldisplayahigherlevelofconfidence.Themostimportantfactorinimprovinggirls’agencyappearstobeapositiveattitudeonthepartofthefacilitatortowardsgirls,andwhetherhe/sheisempowered,ratherthanthesexofthefacilitatorhim/herself.Asmentionedabove,gender-biasedclassroompracticesandattitudesprevail,whichcanaffectgirls’learningopportunitiesandself-confidence,asreflectedintheirperceivedshynessinclass.However,thereappearstobeawarenessthatsuchinstitutionalculturesmayaffectgirls’education.andthereappearstobeawillingnessamongschoolleaderstoreflectonandconsidergenderequityinschoolplanning.Furthermore,itappearsthatgirlsarenolesslikelytobeattendinganIQScomparedtoboys.Thiswasindicatedduringthemanualcountingofpupilspresentonthedayofthesurvey,aswellasduringthelessonobservations,thoughpoorenrolmentandattendancedatamakesitdifficulttomakecomparisonsacrossallthesampledschools.

Mobilisedresourcesareinvestedintheschoolandgirl-friendlyinvestmentisconsidered,butamounts

appearinsubstantialgiventhepoorconditionsoftheschoolenvironment.CBMCsreportedutilisinganaverageof80%ofthetotalfundsraisedinthepastyearsonschoolimprovements,althoughinthemajorityofcasesnoevidencecouldbeprovidedoftheinvestment.Fundsaremostlyinvestedinconstructionworks,suchastheconstructionofnewbuildings,renovationofbuildingsorconstructionoftoiletsandwaterfacilities.Secondinorderistheprocurementofteachingandlearningmaterials.Itisinterestingtonotethatdespitefacilitatorsreceivinglittleremunerationand,often,fewfacilitatorsbeingavailable,fundsarerarelyinvestedinfacilitatorrecruitmentorremuneration.ThequalitativecasestudyfindingsindicatethatCBMCshaveconsideredtheattendanceandretentionofgirlswhenallocatingresources,withexamplesofinvestinginschooluniforms(hijabs),toiletsandwaterfacilities.However,itisnotcleartowhatextentCBMCsareawareoftheirroleinraisingadditionalresourcesforgirls’education,orensuringthatGEP3resourcesareusedforschoolinfrastructurethattargetsgirls’educationspecifically.Ingeneral,theamountofmobilisedresourcesappearstobeinsubstantialgiventhepoorconditionsoftheIQSschoolenvironment.

Governmentmonitoringandinteraction

ThereisavariationintheextentofinteractionbyIQSswithgovernmentbodies,althoughingeneralitis

limited.Respondentsinthequalitativestudyperceivegovernmentauthoritiesand‘externals’tobethemainactorsresponsibleforprovidingfurthersupporttotheIQSs.However,IQSsarereportedlyrarelyvisitedbyLGEAofficialsor‘government’,andthiswasconfirmedbyLGEAofficers.BauchiIQSappearstohavegreaterinteraction.AlargershareofsurveyedIQSsinBauchi(44%)reportedmonitoringvisitsduringthelastterm,comparedtoschoolsinNiger,whereonly33%ofIQSsreportedmonitoringvisits.ThisdoesnotseemtobeassociatedwiththefactthatBauchicountsmoreurbanIQSsbecauseurbanIQSsinthesampleareaslikelytohavereceivedamonitoringvisitasruralIQSs.Also,moreheadteachersinBauchireportedtrainingconductedbySUBEBandtheLGEA.ThisholdstruefortrainingreceivedbyCBMCmembersaswell.ThetwohigherperformingcasestudyIQSsperceivegovernmentofficialstobemorepresent.

Governmentofficialscitechallengesinreachingalltheschoolsonaregularbasis,whichundermines

theircontributiontoefficientlymonitoringandsupportingtheIQSs,andnegativelyaffectscommunities’

perceptionsaboutgovernmentsupport.Officialsperceivethechallengeofattemptingtovisit30schools

©EDOREN 64

andmoreonaregularbasistobeunmanageable,giventhedistancesconcerned,thelackoftransportationinmanycases,andtheirworkloads.Whenvisitsdooccur,localgovernmentofficers’engagementwithcommunitiesoccursthroughtheheadteacher/Mallam/proprietor,andtheyseetheirroleasprovidingsupervisionandadviceonteachingandmanagementpractices,andofferinginstructionandcorrectionbasedontheirobservations.Communitiesgenerallyviewsupportfromthegovernmentnegatively,consideringgovernmentactorstobeunreliable.SinceIQSslooktogovernmentforfinancialsupportorsupportoffacilitators,thelackofgovernmentinteractionandsupportmayhaveimplicationsforcommunityattitudestowardsintegration.

5.3.3 ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-

friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamong

girls

Thissectionfocusesonthebaselinedataofpupils’learningoutcomesinIQSs,themainfinaloutcomevariableintheToCofGEP3’sIQSS.TheIQSSinterventionisexpectedtoimprovelearningoutcomesinbasicliteracyandnumeracy,especiallyforgirls.Itisalsoexpectedtoimproveretentionand,toalesserextent,enrolmentofgirls.Tostartwith,weprovidebackgroundinformationonpupilcharacteristics,attitudesandschoolparticipation,whichcorrespondswithsomeoftheassumptionsunderlyingContributionClaim3.Next,wepresentthebaselinelearninglevelsofpupilsinHausaandEnglishliteracyandnumeracy.Finally,wediscussqualitativefindingsaboutfactorsthatinfluenceretention,asecondaryoutcomeintheToC.

Summaryofevidencerelatedtoevaluationquestions

Q7:Towhatextenthavepupil

literacyandnumeracyskills,

especiallyofgirls,improvedin

GEP3-supportedIQSs?Howhas

GEP3contributedtosuch

improvement?

TowhatextentdoestheIQSS

interventioncontributeto

reducingthegapbetween

learningoutcomesandexpected

learningoutcomes,asexpressed

inthecurriculum?

Pupilperformanceisverylow,particularlyinliteracy,whichmaybeassociatedwithfacilitators’lowteachingcompetenciesandadeficientschoolenvironment.Barrierswithregardstopupils’continuedparticipationintheeducationprocess,attitudestowardsgirls’education,andlanguagediversityarelikelytoalsoaffectlearning.IQSmobilityandpupilsattendingotherschoolsrequirefurtherinvestigation.

Pupils’literacylevelsareverylowinbothHausaandEnglish.ThevastmajorityofGrade2pupilsassesseddisplayedliteracylevelsassociatedwithpre-schoolpupils.Furthermore,mostofthesepupilsarealongwayfromthecut-offassociatedwithemergingliteracyskills(thoseassociatedwiththeP1curriculum).Numeracyoutcomesarebetter,with69%ofchildrendisplayingemergingnumeracyskills.However,asmallminority(11%)hadP2grade-levelnumeracyskills.Theverylowleveloflearningisgenerallyconsistentacrossgroups,althoughtheperformancegapbetweengirlsandboyswidensatthestageinthelifecycletypicallyassociatedwithpuberty.

GirlsandboysappeartobeattendingIQSsequally.However,bothfacechallengesinregardtoattendingandremainingfullyengagedwiththelearningprocessbecauseofout-of-schoolresponsibilities,whicharelinkedtohouseholdpovertyandtheneedforchildrentosupportthefamily’slivelihoodactivities.

AlthoughpupilslargelyreportedspeakingHausaathome,andclassroominstructionappearstobetakingplaceinHausaaswell,teachingisnotalwaystakingplaceinthemothertongueofthepupils,inparticularinNigerwherealmostofhalfofpupilsreportspeakingNupeathome.

TotheextentthatIQSsaremobile,girls’learningislikelytobeaffected.IQSmobilityneedstobeverifiedatmidlineandmonitoredregularly,withspecialattentiontoTsangayaIQSsinBauchi,whichhavetraditionallyhadanomadicculture.

AlargeshareofchildrenstudyinginIQSsattendanotherschoolaswell.Thismayaffectthetimethatpupilshaveavailabletoattendtheintegrated

©EDOREN 65

curriculumclasses.Itmayalsoconstituteanalternativeexplanationforchangesinlearningoutcomes.

Q8:Whataretheattitudesof,

andacceptanceby,community

membersandleaderstoward

integratingformalsubjectsin

Qur’aniceducation?Howhave

theseattitudesandhowhasthis

acceptancechangedduringthe

intervention?Why?Howhave

attitudeschangedinregardto

girlsreceivingformaleducation?

Thequalitativecasestudiesindicatethatintegrationhasgainedacceptancebutthatitneedstohappengradually,withoutaffectingreligiouseducation.Attitudestowardsgirls’educationareperceivedtobechangingbutbarriersremain.

Theintegratedcurriculumhaslargelygainedacceptanceinthecasestudycommunities.IntegratedQur’aniceducationnowseemstobeanintegralpartoftheschoolsinwhichitissituated,althoughthedepthofintegrationvariesamongIQSs.Communitiesperceivetheacceptanceofformaleducationtohaveincreased.Thisisowingtoacombinationof‘outsiderscomingandexplaining’(intheformofsensitisationcampaigns)andpeopleseeingthepositiveoutcomesofreceivingformaleducation.However,itisseenasimportantthatintegrationhappensgradually,andfortheIQSsnottotransformintopublicprimaryschools,whichcouldnegativelyaffectreligiousteaching.

Boysandgirlsviewthevalueofeducationfortheirfuturelifedifferently.However,IQSmanagementandfacilitatorssaythatgirls’interestinformalsubjectshasincreased.Barriersrelatedspecificallytoattitudestowardsgirls’educationpersist,althoughattitudestowardsgirls’educationareperceivedtobechanging,andparentsandcommunitiesdoappeartobewillingtoreflectonandreconsidertheroles,responsibilitiesandcapabilitiesofgirls.Earlymarriageofgirlsconstitutesanimportantbarriertocontinuededucationforgirlsovertheageof10.

Q9:Whatunintended

consequencesdoesIQSShavefor

teachers,headteachers,pupils

andproprietorswithintheIQSs,

aswellasforthebroaderschool

community?

TobeansweredafterIQSSintervention.

Pupilcharacteristics,attitudesandschoolparticipation

Hausaisthemostcommon,butnottheonly,languageoftheimmediateenvironment.AmajorityofthepupilsreportedspeakingHausaathome(75%),followedbyNupe(22%)andFulfude(2%),withnostudentreportingspeakingEnglishathome.AsdepictedinFigure13,thereareconsiderablecross-statedifferencesintermsoflanguagespokenathome.WhilealmostallpupilshaveHausaastheirmothertongueinBauchi,thisisthecaseforonly54%ofpupilsinNiger.InthisstateNupeisalsocommonlyspokenathome.

©EDOREN 66

Figure13: Languagespokenatpupils’homes,bystate

Teachingwithinschoolsisnotnecessarilytakingplaceinthemothertongueofthepupils.ClassroomobservationsandheadteacherinterviewsindicatethatHausaandEnglisharethetwomainlanguagesusedintheclassroom.Therefore,teachingattheIQSsisnotnecessarilytakingplaceinthepupils’mothertongue.InparticularinNiger,Nupeisunder-usedrelativetotheproportionofpupilswhospeakitathome.

BothboysandgirlsseemtoattendIQSsequally,andgirlsareonaverageolderthanboys.Thegenderratioappearsquitebalancedacrossbothstates,althoughduetothedifficultyofnothavingformalenrolmentrecords,thisfindingisnotrepresentativeofthesampleaswhole.40Schoolattendancedatacorroboratethisfinding:justasmanygirlsasboyswereattendingintegratedsubjectclassesinIQSsonthedayofthesurveyvisit.ThisfindingsupportstargetingIQSsasameansofeducatinggirls.Atanyparticularlevel,girlsareslightlyolderthanboys,whichsuggeststhattheyareenteringschoollateroraremorelikelytorepeatayear.

ThenumberofpupilsenrolledorattendingformalclassesvariesconsiderablyacrossIQSs.AmongIQSswithenrolmentrecords,enrolmentrangesfromaslittleas20pupilstoasmanyas400pupils,amajorityofwhichareattheP1–P3(earlygrade)level.Similarly,atthetimeofthelessonobservationsonefacilitatortaughtbetweenfourand147children,withamedianof40pupilsperfacilitator.

AlargeshareofchildrenstudyinginIQSsattendanotherschoolaswell.40%ofpupilsreportedcurrentlyattendinganotherschoolbesidetheIQS.Thisdidnotvarymuchbygender.However,thereweresignificantcross-statedifferences,with72%ofpupilsinBauchiascomparedtoonly13%pupilsinNigerreportingattendingotherschools.InBauchi,mostofthesechildrenwereattendingapublicprimaryschool(94%)ascomparedto55%inNiger.ThequalitativedatasupportthefindingofchildrenattendingIQSs,alongwithpublicschools.Lackoffundsisthekeyfactorwhychildren,andinparticulargirls,mayattendonlytheIQS,althoughthequalitativeresearchindicatesthatsomegirlsdonotattendpublicprimaryschoolsduetoparentsnot‘valuingeducationforgirls’.Incaseswhereparentsmaynotwishtosendgirlstothepublicprimaryschool(eitherforculturaloreconomicreasons)theIQSoffersanopportunityforgirlstoaccessformaleducation.However,thereissomeriskthatfollowingtheintroductionofsecularteachingattheIQS,parentsmayrefrainfromsendingtheirdaughterstopublicprimaryschoolsandsendthemtotheIQSonly,inordertosavemoney.Itisimportanttonotethefactthatpupilsareattendingotherschools,

40Only30%ofIQSssurveyedhadenrolmentrecords.

3%

93%

3% 0% 1%1%

54%

0%

43%

2%

Fulfulde Hausa Kanuri Nupe Others

Bauchi Niger

Source:GEP3Baselinesurvey

©EDOREN 67

particularlyinBauchi,becausethereissomelikelihoodofspillovereffectsofinterventionsatthelocalprimaryschoolsonoutcomesattheIQSs.Furthermore,this‘doubledipping’mayaffectthetimethatpupilshaveavailabletoattendtheintegratedcurriculumclasses.

Boysandgirlsviewthevalueofeducationfortheirfuturelifedifferently,althoughgirls’interestin

formalsubjectsisperceivedtohaveincreased.Inallschools,pupilsconsiderformalsubjectstobeimportantforbothboysandgirls.InseveralIQScommunities,stakeholders(includingboys)perceivegirlsasbeingtoofocusedonmarriage,andtheybelievethatthisaffectstheirattendanceandinterestinschool.However,IQSmanagementandfacilitatorsperceiveanincreasedinterestinformalsubjectsamonggirls.TheyattributethistotheactivitiesofCBMCsandfacilitatorsencouraginggirls.

Pupils’learningoutcomes

Pupils’literacylevelsareverylowinbothHausaandEnglish.Only2.5%ofthepupilsinBauchiandNigerwereabletocompletesomeofthetaskswithinthebasicHausaliteracyrange.Afurther1.8%ofpupilswereabletodemonstrateemergingliteracyskills,whilethevastmajorityofpupilswerefoundtohaveonlypre-literacyHausaskills.Figure14representsthedistributionofperformanceinHausaliteracy.TheaxisrepresentstheHausascalescorederivedfromthepsychometricanalysisofpupilperformanceontheassessment.Thepeakofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthatmakinglargegainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracywillrequiresubstantialeffort.AsimilarpatternexistsforEnglishliteracy.Again,alargemajorityofpupilshaveyettoacquireanyknowledgeorskillsbeyondpre-literacy,while3.4%ofpupilswereabletodemonstratesomeoftheknowledgeandskillsthatfallwithintheemergingliteracyrangeand2.4%wereabletodemonstratesomeoftheskillsthatfallwithinthebasicliteracyrange.ThissuggeststhatpupilproficiencyinEnglishafterayearofschoolingisslightlyhigherthanpupilproficiencyinHausa.

Manychildrenareonthecuspofachievingemergingnumeracythoughgainsinthepercentageofpupils

achievingbasicnumeracywillbemorechallengingandwillrequiremoreeffort.AscanbeseenfromFigure15,thecuspofthedistributionfornumeracyachievementfallsjustabovethecut-offpointbetweenpre-numeracyandemergingnumeracy.Thisindicatesthatlargegainsinthepercentageofpupilsfallingwithintheemergingnumeracyrangecouldbeobservedasmanychildrenareonthecuspofachievingemergingnumeracy.Achievinggainsinthepercentageofpupilsachievingbasicnumeracywillbemorechallengingandrequiremoreeffort.

©EDOREN 68

Figure14: DistributionofHausaliteracyproficiency

©EDOREN 69

Figure15: Distributionofnumeracyproficiency

Theverylowleveloflearningisgenerallyconsistentacrossgroups,althoughtheperformancegap

betweengirlsandboyswidensatthestageinthelifecycletypicallyassociatedwithpuberty.AllagegroupstypicallyachievewithintheHausapre-literacyproficiencyrange,andtherearenosubstantialdifferencesbetweentheachievementofboysandgirlswithineachagelevel(withtheexceptionoffour-andfive-year-olds).ThistrendissomewhatdifferentforEnglishliteracyandnumeracy.ForEnglishliteracymeanscalescoresforallagecategoriesareagainbelowthecut-offpointbetweenpre-literacyandemergingliteracy.However,girls’performancedeclines,ascomparedtoboys,aroundpuberty(aged12orso).Similarly,thenumeracyperformancegapbetweenboysandgirlswidensaroundpuberty,withgirls’performancedecliningascomparedtotheirmalecounterparts.ItisalsonoteworthythatacrossHausa,Englishandnumeracyassessments,pupilsinBauchiperformedslightlybetterthanpupilsinNiger,whichmayberelatedtothegreaterlanguagediversityinNigerormorepupilsinBauchiattendingpublicprimaryschoolsatthesametimeasattendingtheIQS.

Barrierstoattendanceandretention

Continuedbarrierstogirls’educationalaccessandretentionexist–amongothersrelatedtoattitudes

towardsgirls’education(althoughattitudesareperceivedtobechanging).Thequalitativeresearchhighlightsthatgirls’attendanceandretentioninIQSscontinuestoremainachallenge,withpersistentbarriersrelatedspecificallytoattitudestowardsgirls’education.Lackofawarenessoftheimportanceofeducation,inparticularforgirls,waswidelycitedasareasonforlowattendance.However,stakeholdersperceivedattitudestowardsgirls’educationtobechanging,andparentsandcommunitiesdoappeartobewillingtoreflectonandreconsidertheroles,responsibilitiesandcapabilitiesofgirls.

Financialcapacityconstraintshaveacriticalbearingonattendanceandretention.Changingattitudesalone,saythroughCBMCactivities,maynotbesufficienttobringaboutchangeinbehaviouraspovertyisoftencitedasacriticalreasonwhyparentsdonotsendanyoftheirchildrentoanyschool,publicprimary

©EDOREN 70

orIQS.WhileIQSsmaynotchargeafeeorrequireinvestmentinschooluniforms,povertyremainsabarriertoaccessforsomechildren,owingtotheopportunitycostofschooling,whichislinkedtotherolethatchildren(andparticularlygirls)playinsupportingthefamily’slivelihoodactivities(seebelow).CBMCandschoolleadersinallsixcasestudyschools,forexample,feltthattheycouldnotinsistonpaymentfromparentsasthiswouldturnparentsaway.TheybelievethatresourceconstraintsofthecommunitylimitthecapacityofschoolmanagementtoinvestinIQSs,whichmay,inturn,affectthequalityofeducationprovidedandthecapacitytoretaingirls.

Out-of-schoolresponsibilitiesaffectchildren’sschoolattendance.Acrossthesixcasestudyschools,boysandgirlswereallresponsibleforincome-generatingactivities,orforhelpingwithhouseholdchores.Whilstschoolstakeholdersattributedthistoaperceivedlackofprioritisationofchildren’seducation,parentstendedtoviewthisasafinancialandpracticalnecessity.Mothersareperceivedtobelesspositiveaboutgirls’educationandmoreoftenhindertheirattendingschoolbecausegirls’schoolattendancemeansthemothercannotgenerateasmuchincomeasisneeded.Unmarrieddaughtersfulfilakeyroleintherunningofhome-basedbusinessesinnorthernNigeria(eitherbydirectlyengaginginincome-generatingactivities,suchashawking,ortakingcareofhouseholdchoressoastoallowthemotherstoengageinincome-generatingactivities),wherewomenarekeyincomegeneratorsforthefamily.ThegenderedpoliticaleconomyofthisrealityisonethattheToCofGEP3’sIQSSdoesnotaddresswhenitcomestoensuringthatgirlsareabletonotonlyaccessIQSs,butalsotoremainfullyengagedconsistentlyandlongenoughforlearningtooccur.

Oneofthemainreasonsstatedforlowenrolmentandretentionofgirlsisearlymarriage.Whilsttheacquisitionofformaleducation,includingforgirls,maybegainingacceptanceinthesixIQScasestudyschools,earlymarriageisstillprevalent.Thisisparticularlyproblematicforgirlsovertheageof10whoareacquiringabasiceducationforthefirsttime.Whileprovisionsareoftenmadeforgirlstocontinueeducationaftermarriage,thisisdependentonthewillingnessandattitudeofthehusbandinregardtoallowinghiswifetoattendschool.Thus,thoughattitudechangesbyparentsareimportantintheyearsbeforemarriage(andinnegotiatingamarriage),changingtheattitudesofmenlikelytomarrygirlsinthecommunityislikelytobeequallyessential.

ThenomadiccultureofsomeIQSscouldhavedirectimplicationsforlearningoutcomes,especiallyfor

girls.IfanIQSweretomove,therecouldbedirectimpactsonlearningoutcomesastheboysmovingwiththeMallammayresultinthemnotreceivinganyformaleducationformonths,unlesstheMallamwerealsoteachingformalsubjects.Forgirls,similarly,therewouldbeuncertaintyas,iftheydonotmovewiththeschool,theiraccesstoformaleducationwoulddependonwhetherformalsubjectteachingcontinuedintheabsenceoftheMallamandthemalepupils.Itisimportanttoexplorethisfurtheratmidline,and,aspartoftheimplementationofGEP3goingforward,itmaybenecessarytoconsiderhowthisinformationcanbeverifiedandtrackedaspartofthemonitoringprocessattheLGAlevel.

©EDOREN 71

6 Conclusionsandrecommendations

Thisreporthassynthesisedavarietyoffindingsfromthebaselinedatacollectionofthemulti-yearevaluationofGEP3.ItisaccompaniedbytheBaselineTechnicalReport.Thefindings—somedrawingattentiontotheweaknessesof,andotherssupportiveof,thekeyaspectsunderpinningGEP3’sToC;animpactevaluationofGEP3’searlylearningintervention;andaperformanceevaluationofGEP3’sIQSS—aimtocapturetheeducationsituation,answerspecificevaluationquestionsandtoprovideevidencetoinformtheToCandprojectimplementation.Herewereiterateanddiscusssomeofthekeyfindings,andofferrecommendationsforimprovementsthatreflecttheimplicationsofthebaselinefindingsforGEP3.

6.1 Conclusions

Keyconclusions:GEP3ToC

Theplausibilityofthere-designedGEP3ToCisbackedbyitspotentiallysynergetic,coherentlogicand

stakeholders’commonunderstandingofitsmainobjectivesandoverarchingstrategy.GEP3seekstoholisticallyimproveeducationaccess,qualityandgovernance,witheachoutputhavingthepotentialtostrengthentheachievementofeachotheroutput.Therearealsosynergiesacrossinterventions:inparticular,SBMCcapacitydevelopmentcanreinforceinterventionsrelatedtogirls’enrolment,girls’retentionandeffectiveteaching.StakeholderscommonlyrecognisethatGEP3isseekingtoimprovegirls’educationaloutcomes,particularlyincreasedgirls’enrolment–representedbyitstargetofgetting1millionadditionalgirlsintoschool.StakeholdershaverelativelygoodknowledgeofGEP3’soverallstrategyandacknowledgethatthere-designhasimprovedtheproject’sfocus.

However,criticalassumptionsremainhighlyuncertain.Criticalassumptionsrelatedtothemanagementandresourcingoftheeducationsystemarehighlyuncertain–inparticular,thereleaseofgovernmentfunding,schoolinvestmentandhumanresourcecapacityremainingonaparwithincreasedenrolment,effectivemonitoringattheschoolandinterventionlevel,andSBMC’sabilitytodeliveronitsmultipleprojectresponsibilities.Thefactthatsomeoftheseassumptionsaretargetedwithstrengthenedeffortaspartofthere-designenhancesplausibility.However,theeffectivenessofsucheffortsdependsonoperationalisationandeffectivedelivery,thescopeandstrategyofwhicharenotalwayscleartostakeholders,andwhicharesubjecttogovernmentcapacityandcontextualconstraints(suchassecurityconcerns).

SBMCsandMAshaveapivotalroletoplayinthesuccessofmanyinterventions.SBMC/CBMC/MAfunctionalityandcapacitytoundertakethemultiplerolesassignedaspartofGEP3interventionsisacriticalassumptionofGEP3’sToCandrequiresclosemonitoring.TheexpectationsplacedonSBMCsarehighandthecapacitybuildingandempowermentprocessisstartingfromalowbase,particularlyinIQSs.TheyfacegreaterchallengeswithinIQSs,astheirempowermentisbasedonalargernumberofassumptions(suchastheassumptionthattheyareactuallyestablished,inthefirstinstance,andtheassumptionthatthereisacceptanceofintegration).

Improvedcapacityofteacherstodelivereffectivelearningforgirlsisunderstoodtobevitaltoimproving

learningoutcomes,butstakeholderswerenotwellawareofthetrainingapproachandoutcomes.Teachingqualityisgenerallyconsideredtobelowandarangeoffactorsarelikelytoplayanimportantroleinimprovingit.Theseareoftennot–oronlyindirectly–underthecontroloftheproject,suchasteachers’initialcompetencylevel,theirdeploymentandtheirremuneration.IQSspresentgreaterchallenges,whichrequireattentionfromGEP3andneedtobetakenintoaccountinthedesignofinterventions.

TheplausibilityofGEP3’sscale-upapproachisdependentonuncertaingovernmentfundingandvaries

acrossinterventions.Scalingupinterventionsismoreplausiblewheretheirobjectivesarewellrecognisedbystatepartnersasbeingimportant,asupportinginstitutionalframeworkisinplaceandfundingsourcesarerelativelywelldefined.Plausibilityofscale-upisunderminedbythefactthatstakeholdersdidnothave

©EDOREN 72

aclearunderstandingabouthowscale-upwouldhappen.Thescale-upoftheIQSSisparticularlyuncertainbecausetheinstitutionalmandateoverIQSsisunclear,andduetotheSAMEs’limitedanduncertainaccesstoresources,thelimitednumberofwell-establishedIQSsavailableforscale-up,andsupply-sideconstraints—particularlywithregardstofacilitators.

Whilsttheprojectmakesaconcertedefforttoinvolveavarietyofstakeholderswithandthroughstateeducationpartners,theirinvolvementinoperationalplanningremainsunevenacrossinterventions.Governmentinvolvementremainsexternallydrivenandinfluencedbychangesinstaffingandotherexternalconditions.Nevertheless,theinvolvementofabroadrangeofstakeholdersisaperceived

strengthofGEP’simplementationstrategy.Identifyingandaddressingwheregovernmentcapacitybuildingneedsarethehighest,especiallyatthelocallevel,isessentialforGEP3’ssuccessfulimplementation.MonitoringcapacityisrightlyemphasisedinGEP3’sre-designasitisconsideredakeyoperationalprocessforproject,butitsoperationalisationhasyettoadvance.SeveralGEP3interventionsaredesignedtobespecificallyequityenhancing,empiricalverificationisrequiredastowhetherthemostvulnerablehouseholdsandgroupsareabletobenefitandactivelyparticipateinGEP3interventions.GEP3isnolongeroperatinginLGAswiththehighestgendergapinallstates.

Keyconclusions:Earlylearningintervention

TheevidenceonlearningoutcomespointstoverylowlevelsofHausaandEnglishliteracy.AlargemajorityofpupilshavenotacquiredtheknowledgeandskillsexpectedbytheNigeriancurriculuminHausaliteracy,norhavetheyacquiredeitheremergingorbasicEnglishliteracyskills.Mostpupilsscorewellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthatachievinglargegainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracywillrequiresubstantialeffort.KnowledgeofphonicsinparticularisfoundtobemostchallenginginregardtobothHausaandEnglishliteracy.Theearlylearninginterventionwillneedtotakethesecaveatsintoaccountwhenpromotingaphonicsapproachtoteaching.

Learningoutcomesaredifferentiatedbyage,genderandtypeofschool,witholdermalestudentsin

particularperformingbetterthanoldergirls.Regressionanalysisindicatesthatwhilstteachercharacteristicsdonotappeartobeassociatedwithlearningoutcomes,pupils’individualcharacteristicsandsocioeconomicbackground,aswellasschooltype,areassociatedwithpupilachievement.Thelackofassociationbetweenteacher-levelfactorsandlearningoutcomescouldreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetectedifthereweregreatervariationincompetencylevelsacrossteachers,andthattheirabilitytoinfluencepupils’learningoutcomeswouldbehigher.Thefindingthatwealthierchildrenmayperformbetterthantheirpoorercounterpartsisanindicationthatpovertycouldbeacriticalfactornegativelyinfluencingchildlearning.ThefindingthatstudentsinIQSsperformbetterthanthoseinpublicprimaryschoolsaftercontrollingforseveralfactorsisinterestingbutdifficulttoexplain.

IQSsandpublicprimaryschoolsoffergreatlydifferentcontexts.Teachercharacteristicsvaryacrossthetwoschooltypes,asdoschoolsize,schoolmanagement,externalschoolmonitoring,accesstoinfrastructure,andpupilcharacteristics.Teacherandheadteachercapacitydevelopmentwillneedtobeappropriateforthesedifferentcontexts.AconsiderablenumberofIQSsonlyhaveasinglefacilitatoroftheintegratedcurriculumsubjects,whichhasimplicationsforimplementingRANA’sschool-basedpeermentoring.

Teachersdemonstrateverylimitedknowledgeandskillsinthemajorityoftheareasrequiredtofunction

effectivelyasateacher.Almostnoneoftheteachersdisplayanypedagogicalknowledgeandskills,andmostappearunabletoascertainwhichactivitiesarebestsuitedtoimprovepupilperformanceorsupportgirlsintheirlearningwithinclassrooms.Over60%ofteachersareunabletodisplaycompetenceinGrade1and2-levelHausa.Hardlyanyteachersarecompetentinwritingandinterpretingwordsandphrases.ThelowHausaknowledgeissignificantforaninterventionthatfocusesonteachinginHausabecauseitislikelytonegativelyaffectteachers’abilitytopasstheirknowledgeontopupils.Thereareveryfewdifferencesinthelevelsofknowledgedemonstratedbydifferentgroupsofteachers.Forexample,nodifferenceswere

©EDOREN 73

notedbetweenteachersinpublicprimaryschoolsversusIQSs,orbetweenteachersthatattendedpreviousGEP/UNICEFtrainingandthosethatdidnot.

Hausaisusedextensivelyasthemainlanguageinteaching,buttheuseofHausateachingandlearning

materialsisveryrare.AllteachersreportspeakingHausaandamajorityreportspeakingEnglish.AllpupilsreportspeakingHausaathome.Hausawasusedinallclassesobservedduringthesurvey,whileEnglishwasalsoused,particularlyinpublicprimaryschools.ThisvalidatestheassumptionthatHausaisthelanguageoftheimmediateenvironmentforthepupilsandthemostcommonlanguageusedinclass.TheavailabilityanduseofHausamaterialsinclasswasrarelyobserved.TheemphasisonHausa-basedteachingandthedistributionofHausateachingandlearningmaterialsisthereforerelevant.AkeypointofattentionisthattheteachertrainingandteachingmaterialsneedtobecarefullyadaptedtothelowlevelofHausaliteracyskillsamongtheteachers.

Overall,teachersconsidertheirroletobeimportantandenjoyworkingasteachers,whilstbeing

consciousofthefactthattheyarelimitedintheirabilitytocontributetopupils’learning.Theevidencepointstoteachermotivationbeingrelativelyhomogenousacrossdifferentdisaggregations(rural/urban,schooltypeetc.).Highteacher-to-teacherinteractionscoressuggestarelativelyhighlevelofcollaborationamongteachers,whichcouldhelpthedevelopmentofspillovereffectswithinschoolsofanyteacher-specificinterventions.

Keyconclusions:IQSS

1. ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs

GEP3’scontributiontomoreeffectiveteachingassumesthatIQSfacilitators’knowledgeandskillsandgender-sensitiveclasspracticescanbeimprovedthroughtrainingandmentoring.Baselineevidencesuggeststhatthereisaneedforfurtherexpansionanddeepeningoffacilitatortraining.Facilitatorsdemonstrateverylowlevelsofpedagogicalknowledge,somethingthatwillnotonlyhindereffectiveteachingbutthatmustalsobeborneinmindindesigningeffectivetrainingprogrammes.Subjectknowledgelevelsarealsolow,with66%offacilitatorsfailingtodisplaycompetenceinGrade1and2-levelHausa.GEP3alsoneedstotakeaccountofthefactthatfacilitatorsthemselves,aswellascommunitymembers,maynotperceivetheirlackofcompetencetobeacauseoflowpupilperformance.

Themodalitiesofthistrainingcanalsobeimproved.Forexample,itwasfoundthatappropriatefacilitatorsdonotalwaysgettrained.Knowledgeanddisseminationstrategiesatschoollevelalsoneedtobeconsidered,toreinforceimpact,althoughitalsoneedstobeconsideredthataquarterofIQSsonlyhaveonefacilitatorofnon-religioussubjects,whichwillconstrainamentoringprocessthatisbasedonschool-basedpeerinteraction.

Moreover,evidenceatbaselinesuggestsdeeplyingrainedgenderbiasesamongfacilitators,whoaremostlymale,towardsgirls’education.Andwhilstfacilitatorsdisplaygender-sensitivetechniques,theclassroomspacecontinuestoimposeagenderbias.Itis,therefore,importanttodelivergender-sensitivetechniquesintheclassroom,withthesupportofincreasinglytransformedgender-sensitiveattitudesamongstthefacilitatorsthemselves.

AnotherassumptionunderpinningGEP3’scontributiontoeffectiveteachingisthatmentoringoffacilitatorscanincreasetheirmotivation,leadingtomoreeffectiveteaching.Thisassumptionisborneouttosomeextent—thebaselinesurveyfoundthatpedagogicalleadershipinfluencesmotivation,asfacilitatorswhomeetheadteachersindividuallyreportedbeingmoremotivatedthanothers.

Facilitators’motivationisshapedbyissuesofremuneration,socialstatusandcommunityrolesandrelations.Motivation,anditstranslationintoteachingefforts,isinfluencedbytheverybasicandpracticalunderpinningofanIQSfacilitator’seconomicneedsandresponsibilities,andhow,intheabsenceofadequateremuneration(andthestatusthatthismakesmorelikely),thesemayconflictwithfacilitators’

©EDOREN 74

well-intentionedcommitmenttobeingengagedinteachingtheintegratedcurriculum.Thelackofremunerationislikelytoaffectfacilitators’motivationtoteach,andmakesitdifficulttoattractqualified

facilitatorsandholdthemaccountable.

IntegratededucationinIQSsappearstohavegainedacceptancebutitisimplementedtovarying

degrees.AsubstantialportionofIQSscontactedforthesurveywerefoundtobenon-integratedandfewIQSsprovideallcoresubjectsoftheintegratedcurriculum.Thismayberelatedtothefactthatintegrationhasoccurredrelativelyrecent,andthatonlyalimitednumberofhoursarededicatedtonon-religioussubjects.Also,facilitatorattendanceseemstobeflexible,withnon-religioussubjectsbeingtaughtasandwhenafacilitatorisavailable.Amorestructured,intensifiedteachingprogrammewillbehardtoenforcebecausebaselineresultssuggesttheneedforintegrationtobegradualandfortheschooltoretainitsQur’aniccharacter.Moreover,itisdifficulttoholdunremuneratedfacilitatorstoaccount.Nonetheless,thediverseIQScontextmayoffertheopportunitytoidentifyIQSsthatpositivelydeviateandcanactasrolemodelforothers.

GEP3assumesthatthedistributionofHausateachingandlearningmaterialscansupporteffectiveteaching.ThebaselinefoundthattheavailabilityofteachingandlearningmaterialsisverylowinmostIQSs,whichsuggeststhatthereisaneedforsuchmaterials.However,thesematerialswillneedtobe

carefullytailoredtotheverylowlevelsofskillsandknowledgeamongstbothfacilitatorsandpupils,in

orderforthemtoplayausefulrole.Furthermore,itisimportanttoconsiderthatforaconsiderableshareofpupilsinNigerNupeisthelanguageoftheimmediateenvironment.

Theappointmentofaheadteacherappearstodependonsocialstatusratherthanqualifications,which

raiseschallengesrelatedtowhototargetforheadteachertraining.Thischallengearisesbecausetheassignedheadsmaynotbethemostqualifiedtotakeupthepedagogicalleadership,buttheymayhavethesocialstatusandcloutneededtoundertakemoralleadershipduties.Pedagogicalleadershipseemstobesharedamongstmanystakeholders,withCBMCmembersguidingheadteachers,headteachersadvisingfacilitators,andproprietorsandcommunityleadersofferingtheirperspective.Alackofclarityaboutleadershiprolesmakesitdifficulttoidentifywhoseleadershipcapacitiestobuild.Furthermore,itmaybethatthereisnoclear‘candidate’forheadteacher/pedagogicalleader,asinsomecasesnooneinthecommunityhasanyformaleducationbeyondsecondaryschool.Headteachers’record-keepingabilitieswerenotablypoorandthiscouldbeanareatofocusongoingforward.

2. ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentin

IQSs

ThemainpathwaytoachieveincreasedcommunityinvolvementinIQSsandtomobilisegreaterresourcesisthestrengtheningofCBMCsandschoolmanagement.BaselineevidencesuggeststhatschoolmanagementinIQSsisseenasacollectiveeffort,withoutclearlydefinedanddelegatedroles.Itwill,therefore,beimportantfortheCBMCtotakeintoaccountandincorporatethevoicesofkeycommunityfigures,giventheinfluencetheymayhaveonthesuccessofintegration.TheCBMCcanonlycontributetoimprovementsifitisestablishedandfunctional,andthisisnotcurrentlythecaseforallGEP3IQSs.Itwillalsobeimportantforwomenandchildren’svoicestobeheardinCBMCs,toensuregirl-friendlyschoolinvestments.Thebaselinefindingspointtodeficienciesonthisfrontatpresent,partlybecausegendernormsseemtoaffectwomen’sparticipationinCBMCs.

OneofthecriticalrolesofCBMCsisinmobilisingfunds.WhileCBMCsareabletomobiliseresourcesfrom

thecommunity,significantresourcegapsremaininIQSs,andCBMCs’perceivedroleisundermineddue

toalackoffunding.Gapsinresourcemobilisationcanbetracedtoparents’inability,ratherthanunwillingness,tocontributefunds.TheselimitationsinthescopeforraisingresourcesfromthecommunityneedtobetakenintoaccountbyGEP3whendesigningitstrainingmoduleonresourcemobilisationforCBMCs.

©EDOREN 75

WhileCBMCtrainingcanbeausefultooltofurtherguideCBMCsintheimplementationoftheirrolesandresponsibilities,CBMCmembersmaynotalwayshavethecapacitytofulfiltheserolesandresponsibilitiesduetoresourceconstraints.FurthertrainingoftheCBMCsshouldgobeyondtheirdutiesinregardto

sensitisation,whichseemtobewellunderstood,andshoulddevelopmoretheircapacityinmonitoring,

formalschoolmanagementpracticesandensuringthatgirls’educationisactivelysupportedintheIQSs.Thiscanincludetheabilitytomorerigorouslyengagewiththefacilitatorsontheirattitudesandteachingbehaviours,andagreatersenseofaccountabilityonthepartoftheCBMCstoensureagirl-friendlylearningenvironment.

IQSsareuniversallydeficientin‘soft’and‘hard’infrastructure,andprovideapoorschoolenvironment

fortheattendingchildren.Theschoolconditionsarenotalwaysconducivetoagirl-friendlyenvironment,despitethefactthatgirlsappearnolesslikelytobeattendinganIQScomparedtoboys.EvidencesuggeststhatschoolleadersandCBMCmembersarewillingtoreflectonwaystoimprovegenderequityinschool

planning,andthisshouldbenotedinanyplanninggoingforward.Moreover,thebaselinestudyrevealsthatthepresenceofastrongfemalerolemodelcanalterperceptionsamongmothersandgirls.Themost

importantfactorimprovingagirl’sagency,however,appearstobetheattitudeofthefacilitator,andwhetherhe/sheisempowered.

ChallengesremaininthemonitoringofIQSsbylocalgovernment,withofficialsindicatingthattheystruggletoregularlyvisitthelargenumberofschoolsallocatedtothem.However,thismeansthatfollow-throughonCBMCeffectivenessisdifficult,witheachIQSoperatinginwhatappeartobelargelyunstructuredandcontextuallydrivencircumstances.Communityperceptionsregardingtheroleofthegovernmentalsotendtobenegative.

3. ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendly

learningenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls

GirlsandboysappeartobeattendingIQSsequally.Atanyparticularlevel,girlsareslightlyolderthanboys,whichsuggeststhattheyareenteringschoollaterorarebeingheldbackmorethanboys.Amajorityofthe

pupilshaveyettoacquirebasicliteracyinHausaortheknowledgeandskillsexpectedinEnglishliteracy.

Manychildrenareonthecuspofachievingemergingnumeracythoughgainsinthepercentageofpupils

achievingbasicnumeracywillbemorechallengingandwillrequiregreatereffort.Girls’performanceinEnglishliteracydeclines,ascomparedtoboys,aroundpuberty(aged12orso),andperformancegapsinnumeracyalsoappeartowidenaroundthisagegroupaswell.

Girls’interestinimprovingtheirlearningoutcomesandboysrespectinggirlsbeingeducatedareassumptionsunderlyingContributionClaim3.ThebaselinefindingssuggestthatCBMCs’activitieshavecontributedtoanincreaseingirls’interestinformalsubjects.However,boysandgirlscontinuetoperceivethevalueofeducationdifferently.Stakeholders(includingboys)perceivegirlsasbeingtoofocusedonmarriage,andbelievethatthisaffectstheirattendanceandinterestinschool.

Alargeshareofchildrenattendmorethanoneschoolatthesametime,particularlyinBauchi.Thismeansthatchangesinthequalityofeducationintheotherschoolmayformanalternativeexplanationforimprovementsinlearningoutcomes.Theresearchidentifiespovertyandparentalattitudesaskeyfactorsrelatingtowhychildren,andinparticulargirls,mayonlyattendanIQS.Whenfacedwithbindingeconomicorculturalconstraints,IQSsofferanopportunityforchildren’s,particularlygirls’,exposuretoformaleducation.However,thereissomeriskthatfollowingtheintroductionofsecularteachingattheIQS,parentsmayrefrainfromsendingtheirdaughterstoapublicprimaryschoolandsendthemtotheIQSonly,inordertosavemoney.

Evidencepointstotwodiscrete,albeitinteracting,issuesthatunderminegirls’accessandretentionin

school–thecostofschooling,andsomepersistingresistanceamongstparentstosecularschoolingfor

girls.However,attitudestowardsgirls’educationareperceivedtobechanging,andparentsandcommunitiesappeartobewillingtoreflecton,andreconsider,theroles,responsibilitiesandcapabilitiesof

©EDOREN 76

girls.Changingattitudesalone,saythroughCBMCactivities,may,however,notbesufficienttobringaboutbehaviourchangeasthefinancialconstraintsofparentsarehighlightedashavingacriticalbearingoneducationalaccess.Givensuchconstraints,financialconstraintswilllimitthecapacityofschoolmanagementtoinvestinIQSs,whichmayultimatelyimpactthequalityoflearning,aswellasthecapacitytoretaingirls.

Theroleofwomenaseconomicagentsandaskeyincomegeneratorsforthefamily,withresultantresponsibilitiesforgirlstocontributetorunninghome-basedbusinessesinnorthernNigeria,affectsgirls’

schoolattendanceandlearningprocess.Whilstsensitisingwomeniscriticaltoimprovinggirls’involvementineducation,onitsownitisunlikelytobesufficientgiventhegenderedpoliticaleconomyofthecontextinwhichGEP3operates.

Menalsoplayacriticalroleingirls’lives,particularlyinrelationtoearlymarriage.Thequalitativecasestudiesindicatethatearlymarriageisprevalent,andisstatedasoneofthemainreasonsforlowenrolmentandretentionofgirls.Whenfacedwithearlymarriage,provisionsmaybemadeforthegirltocontinuehereducationpost-marriage,butthisisheavilydependentonherhusband’sdiscretion.Thus,thoughattitudechangesbyparentsareimportantintheyearsbeforemarriage(andinnegotiatingamarriage),changingtheattitudesofmenlikelytomarrygirlsinthecommunityislikelytobeequallyessential.

6.2 Recommendations

GEP3-widerecommendations

1. Theprojectpartnersshouldcontinuetheirpoliticalengagementtoensuregovernmentfundingis

leveraged,withanincreasedemphasisontheprojectscale-upandIQSS.Inlinewiththerecommendationsofthe2015AnnualReviewandtheEDORENpoliticaleconomyanalysis,theprojectpartnersneedtoputpoliticalengagementforeffectiveresourceprovisionforgirls’educationatthecentreofGEP3’sToC.Theprojectteamshouldsystematicallytrackandreportongovernmentprojectexpendituretofacilitatethisengagementandaccountability.Projectpartnersshouldcometoacommonunderstandingofhowprojectscale-upcanhappen,planforagreedtargetsaspartofinstitutionalisedplanningprocesses,andmonitorandaccountfortheirprogress.AnagreementonhowIQSScanbescaledupandfundedisparticularlyneeded.TheGESCshouldbeusedasaregularplatformtocoordinateadvocacyforgovernmentfundingandtofacilitatethemonitoringofkeyprojectassumptions.Aspartofgovernmentengagement,learningandretentionneedtoreceivestrongerattentionascoreprojectobjectives,togetherwiththecriticalassumptionthatsupply-sideinvestmentneedstorespondtoincreasedgirls’enrolment.

2. Theprojectshouldspecifyandcommunicateoperationalobjectivesandstrategies,strengthentheir

monitoringandstrengthenthemonitoringofunderlyingassumptions,andfeedbackmonitoring

informationtofacilitatelearningandaccountability.Governmentandotherimplementingpartnersthatareinvolvedin,oraresupervising,interventionexecutionshouldbemademoreawareof,andneedtoagreeon,whotheinterventionsintendtoreach,how,whenandwithwhatspecificobjectives.Themonitoringofinterventiondelivery,ofitsoperationalassumptions(e.g.theassumptionthattheinterventiontargetgroupwillbereached)andofitssupportsupervision,needstobestrengthened.Themonitoringdataneedtobesystematicallydiscussedamongprojectpartners,toenablelearningandaccountability.Theuseofdataingeneral–includingASCdata,amongotherkinds–shouldnotbetakenforgranted.ActivitiestopromotedatauseshouldbeembeddedinGEP3’sstrategy.

3. SBMC/CBMCmonitoringandsupportsupervisionneedcomprehensiveandsystematicemphasis.WesupportGEP3’semphasisonSBMC/CBMCcapacitybuildingandeffectivenessmonitoringbecauseoftheirpivotalroleintheproject’sToC.ThemonitoringandsupportsupervisionneedstobecomprehensivewithregardstothemultiplerolesthatSBMC/CBMCsplayinGEP3-supportedactivitiesandtheschool-levelresultsthatSBMC/CBMCsareexpectedtoachieve.Themonitoringshouldalso

©EDOREN 77

collectfeedbackaboutthesupportsupervisionitself,particularlyinIQSs,sincethisisanimportantcomponentofthecapacitybuildingapproach.CBMCeffectivenessmonitoringneedstoreceiveparticularattention,giventheirrecentinstitutionalisationandthecomplexcontextinwhichtheyexist.Themonitoringandsupportsupervisionneedstoberegular,backedbyadequateresourcesandreportedonsystematically.MonitoringfindingsneedtobediscussedwithstateandresponsiblelocalgovernmentofficersinvolvingSBMClocalandstaterepresentationstructures.TheempowermentoftheseSBMCstructuresisapromisingmechanismbywhichtofacilitatecontinuousandgraduallearningamongpeersandtoprovidevoiceandasenseofself-efficacytotheSBMCsandMAs.

Recommendationsfortheearlylearningintervention

1. Theteachertrainingandteachingmaterialsneedtobedesignedtakingintoaccountthelow

competencylevelsoftheteachersandteachers’subjectspecialisations.Thebaselinefindingshighlightthatteachers’knowledgeandskilllevelsareverylow,includinginbasicHausaliteracyandinassessingpupils’performance.TheRANAinterventionwillneedtobecarefullytailoredtoexistingknowledgeandskilllevelstoensurethattrainingcontentandmaterialsarepitchedattherightlevel,andthattrainingissufficientlyintensivetofillthelargegapsinteachers’subject,pedagogicalandcurriculumknowledge.Inaddition,theinterventionteamneedstocarefullyconsideritsteachertargetingapproach,asitcannotbeassumedthatallteachersareinvolvedinHausaliteracyteaching.Thebaselinefindingsindicatethatalargeshareofteachersonlyteachonesubject,andthatmathematicsisoneofthetwomainsubjectstaught.

2. TheRANAteamneedstoassesswhetheritsapproachtopupilliteracyimprovementisappropriate

fortheverylowpupillearningoutcomelevelsandteachercompetencylevels,andtoadaptteacher

trainingandlearningmaterialsifneeded.Pupillearningoutcomelevelsarecurrentlyverylowandlevelsfallwellbelowthecut-offpointbetweenpre-literacyandemergingliteracy.AssessmentitemsthatrequireknowledgeofphonicsrankasthemostdifficultitemsinboththeHausaandEnglishassessments.Theinterventionteamneedstotakethisintoconsiderationinitsapproachtopupilliteracyimprovement.

3. Teachercapacitydevelopmentneedstoincorporateactionstochangeteachers’awarenessand

perceptionsofteachers’self-efficacy.Teachersperceivepupilefforttobethecauseoflowpupilperformance,ratherthantheirowncompetenciesandteachingpractices.However,itislikelythattheirownlowlevelofcompetencycontributestolowpupilperformance.Thecapacitybuildingshouldsupportteachers’reflectionaboutthelinkbetweentheirowncompetenciesandpupilperformance;andshouldbuildteachermotivationandaccountabilityforimprovingpupillearningoutcomes.

4. TheRANAcommunityengagementcomponentandtheteachertrainingshould,totheextent

possible,addresspupilandhouseholdfactorsthatinfluencelearning.Theregressionmodelindicatesthatpupilandhouseholdcharacteristicsareassociatedwithlearningoutcomelevels.RANAiswellplacedtooffsetsomeoftheeffectsofsocioeconomicbackgroundonlearningoutcomesbypromotingamoresupportivehomelearningenvironmentforallchildrenthroughitscommunityengagementcomponent.

5. Thepeermentoringapproachneedstobeadaptedtoschoolswithonlyonetargetedteacher.Thebaselinefindingsindicatethat40%ofIQSshaveonlyasinglefacilitatoroftheintegratedcurriculumsubjects.Therefore,theschool-levelpeermentoringinitscurrentformisnotapplicabletoalargeproportionofIQSs.Itisrecommendedthattheprojectidentifiesalternativewaysofensuringthattrainedteacherscanprovidepeersupport,possiblybyengagingwithteachersfromdifferentbutnearbyschools.

6. RANAneedstosufficientlystrengthenthecapacityofIQSgovernmentstakeholderstoensurethat

theyprovideeffectivemonitoringandsupportsupervisiontoIQSs.Thebaselinedataindicatethat

©EDOREN 78

IQSsreceiverelativelylimitedexternalmonitoringandsupervisionvisits.Giventheircomplexcontext,theyarelikelytoneedcontinuoussupportandfeedbackevenmorethanpublicprimaryschools.RANAshouldnotassumethatIQSgovernmentstakeholdershavethecapacitytoconductmonitoringandsupportsupervision,andtheyshouldactivelysupportthis.Theroles,responsibilitiesandprocessformonitoringandsupportsupervisionshouldbewelldefinedandexplainedtoIQSstakeholders,inorderforittobeacceptedandaccountedfor.

7. RANAshouldtrytoattractthemostsuitablecandidatesatIQSsforitspedagogicalleadership

training.Thebaselinefindingsindicatethatheadteachersupportivepedagogicalactionsarerelativelylimited,particularlyinIQSs.ThecomplexityofIQSleadershipmaycontributetothis.ItwillbeimportantforRANAtotargettheIQSleadershipthatismostrelevantforthechangesinteachingpracticeswhichareaspiredto.

RecommendationsfortheIQSS

1. Selectionoffacilitators,headteachersandCBMCmembersrequirescloseattention,verificationand

monitoring.GEP3needstotargetfacilitatorsfortrainingwhoregularlyteachtheintegratedsubjects,whichrequiresverificationandsystematicmonitoring.Giventhat,ofthefiveintegratedcoresubjects,mathematicsandlanguagesarethemostfrequentlytaught,GEP3maywanttotargetfacilitatorsteachingthesesubjectsinthefirstinstance.SincetherolesofpedagogicalleadershipandschoolmanagementmaybesharedamongIQSstakeholders,andheadteachersmaybechosenforreasonsotherthantechnicalqualifications,theselectionofheadteachersfortrainingneedstobebasedonanexplanationoftheexpectationoftheroleintermsoftechnicalcapacity,inordertoattracttheappropriatecandidatefortraining.SimplyaskingtheIQStonominateandsendsomeonebasedonthetitleofheadteacher/Mallammaynotyieldpositiveresults.InthecaseofCBMCs,thereisacaseforGEP3totargetmemberswhocanmosteffectivelyfulfiltheresponsibilitiesrequiredfromtheCBMCs,whilebalancingthiswithequityconsiderations.ThesemembersmaynotnecessarilybethoseCBMCmemberswhoholdkeypositions,asrolesarenotalwaysclearlydefinedanddelegated.CBMCsappeartoneedfurthertrainingthatgoesbeyondtheirdutiesinsensitisation,anddevelopsmorecapacitiesinmonitoring,formalschoolmanagementpracticesandensuringthatgirls’educationisactivelysupportedintheIQS.Thiscanincludetrainingonhowtoengagewithfacilitatorsontheirattitudesandteachingbehaviours,andagreatersenseofaccountabilityonthepartoftheCBMCtoensureagirl-friendlylearningenvironment.

2. Monitoringandsupportsupervision,particularlyinthecaseofIQSs,needsclosefollow-upand

capacitysupport.TheIQScontextisdiverse,flexibleandstillunderdevelopment.InordertoadapttheIQSSinterventiontosuchacontext,quicklearningandfeedbackbasedonmonitoringdataisneeded.However,externalmonitoringandsupportsupervisionofIQSshasbeenlimitedinthepast.Hence,thisneedsclosefollow-upandcapacitysupportbyGEP3.Specialattentionisrequired,tomonitorschoolmobility,facilitatortransferandremuneration,pupilenrolmentrecords,pupilattendanceinotherschools,investmentinagirl-friendlyschoolenvironmentandtheschedulingofintegratedsubjectswithintheIQSclassschedule.

3. Facilitatortrainingandmentoringneedstobewelladaptedtotheverylowlevelsoffacilitator

competencyandtotheIQScontext.Facilitatortrainingwillneedtoaddressboththelowlevelsofpedagogicalknowledgeaswellasthelackofsubjectknowledge,particularlyinregardtoHausaliteracy,requiredtoeffectivelyusepedagogicaltechniquesandmaterials.Whilegenerallylow,facilitatorqualificationsvaryacrossIQSs,whichwillrequirethetrainingprogrammetobeabletodealwiththisvariation.Itisworthincreasingfacilitators’awarenessaboutthelinkbetweenteachercompetencies(orlackthereof)andpupilperformanceinorderforfacilitatorstobeabletobetterself-assesstheirrequiredcompetencies.Peermentoringwillrequiresupportingpeer-to-peerinteractionbetweenschoolssinceaquarterofIQSsonlyhaveonefacilitator.Knowledgedisseminationstrategies

©EDOREN 79

needtobeconsideredtoreinforceimpact,becausefacilitatorswithinthesameIQSarenotnecessarilyawareofeachother’sparticipationintraining.SincetheIQSteachingcontextisdiverse—withadequateinfrastructureoftenlacking,differentclassorganisations,andvaryingpupil–teacherratios—thetrainingneedstobecontextuallysensitive,totakeintoaccountthisteachingreality.

4. Trainingandmentoringongender-sensitiveclasspracticesneedstogobeyondthedisplayof

gender-sensitivetechniques,andneedstoaddressentrenchedgenderedattitudesaboutgirls’abilitytolearnandtheimportanceofeducationintheirfuturelife.

5. GEP3needstoadvocateforadequatefacilitatorremunerationandtopromotethemobilisationof

resourcesbeyondthecommunity.AverysmallshareofIQSfacilitatorsareremunerated,whichisagainsttherecommendationofpolicydocumentssuchastheIQTEBenchmark.Thelackofremunerationislikelytoaffectfacilitators’motivationtoteach,andmakesitdifficulttoattractqualifiedfacilitatorsandholdthemaccountable.WhileCBMCsareabletomobiliseresourcesfromthecommunity,significantresourcegapsremaininIQSsandCBMCs’perceivedroleisunderminedduetoalackoffunding.TheprojectshouldnotassumethatCBMCswillbeabletomobiliseconsiderableresourcesfromwithinthecommunity,andthereforeitshouldlookinto,andpromoteaccessto,otherresources.

6. Femaleandgirls’participationinshapingeducationandtheschoolenvironmentneedsfurther

investigation.CBMCsstruggletoattractwomenandensurewomenparticipateintheCBMC.Girls’voicesarehardlyrepresented.Increasingwomen’sinvolvementmayproveachallengesincetheirabsenceseemstoberelatedtoentrenchedgenderrolesandresponsibilitiesratherthanalackofrecognitionabouttheirpotentialcontributiontoschoolmanagement.Therefore,GEP3willneedtofurtherinvestigatemechanismsforensuringwomen’sandgirls’engagementinschoolmanagementthataresensitivetosuchrolesandresponsibilities.

7. Learningandteachingmaterialsneedtobeadaptedtotheverylowlevelsofskillsandknowledge

amongstbothfacilitatorsandpupils,andtothelanguageoftheusers.WhilealmostallfacilitatorssurveyedreportedspeakingHausa,Hausaliteracylevelsamongfacilitatorsaregenerallylow.Pupils’HausaliteracyproficiencyisextremelylowandinNiger43%ofpupilsspeakNupeathome.Learningandteachingmaterialsneedtobeadaptedtothislanguageandliteracyreality.

©EDOREN 80

Bibliography

Antoninis,M. (2012) Tackling the largest global education challenge? Secular and religious education inNorthern Nigeria. CSEA Working Paper WPS/2012-17. Oxford: Centre for the Study of AfricanEconomies.

DFID.(2011)DFIDEthicsPrinciplesforResearchandEvaluation.DFID.

Delahais,T.andToulemonde,J.(2012)Applyingcontributionanalysis:Lessonsfromfiveyearsofpractice.Evaluation,18(3),pp.281–293.

EDOREN.(2015)EvaluationFrameworkandPlan,UNICEF’sGirls’EducationProjectPhase3(GEP3).Oxford,UK:EDOREN.

Graham, A., Powell, M., Taylor, N., Anderson, D. and Fitzgerald, R. (2013) Ethical Research InvolvingChildren.Florence:UNICEFOfficeofResearch–Innocenti.

Mayne,J.(2012)Contributionanalysis:Comingofage?Evaluation,18(3),pp.270–280.

©EDOREN 81

AnnexA ToCoftheearlylearningintervention

Figure16visualisestheToCoftheearlylearningintervention.ThemaincausalassumptionunderlyingtheToCisthatliteracylearningoutcomesofgirlsandboys,particularlyinHausaasamothertongue,willimproveinearlygradesifteachingpracticeimprovesthroughtheuseofimprovedteachingandlearningmaterials(useoftheRANALiteracyPackage)andthepresenceofmoreknowledgeable,skilledandgender-sensitiveteachers.Numeracyisalsoexpectedtoimprove,buttoalesserdegree.Numeracywillbeinfluencedindirectlybyincorporatingnumeracythemesintothereadingcontent,exercisesandmaterials.

Theabovecentralassumptionisconditionalonschoolstakeholders(suchasparents,communityleaders,IQSproprietors,headteachers,teachersandgovernmentstaff)beingsupportiveofanincreasedemphasisonmothertongueinstructionandliteracyacquisition;andonteachersadheringtothemothertongueduringinstruction.Tothisend,theprojectwillengagethecommunity,championliteracyinthecommunities,trainheadteachersandadvocateforgovernmenttoimplementenablingpublicpolicies.OtherfactorsthatwillinfluenceimprovedHausa-basedteachingpracticeareregularpupilattendanceandtheclassandschoolenvironmentsbeingconducivetolearning.ThefocusonHausaassumesitisthemothertongueandthelanguagethatpupilsknowandunderstandbest.

Improvementofteacherknowledgeandskillsthroughin-serviceteacherdevelopmentisassumedtobecentraltomoreeffectiveteachingintheearlygrades.41ThisassumesthattheimprovedHausaliteracycurriculum,thematerials,thepedagogicalmethodologyandthein-servicetrainingandmentoringapproacharewelltargetedtotheneedsandcapacitylevelsofteachers,aswellastotheirteachingenvironmentinpublicschoolsandIQSs.Tooptimiseteacherlearning,theprojectenvisionsateacherdevelopmentapproachbasedonprogressiveprofessionaldevelopmentandschool-basedpeermentoringcomplementedbysupportivesupervisionandmonitoringbyMasterTrainersandgovernmentstaff.ThesuccessofthisapproachdependsonteachersbeingliterateinHausa,onweeklypeermeetingsbeingfeasiblegiventhecontextoftheschool(forexample,teachersbeingabletomeetregularlywithotherteachersgiventhesizeoftheschooloritslocation),LeadTeachershavingthecapacity,motivationandavailabilitytoleadthemeetings,andMasterTrainersandgovernmentstaffhavingtheresources,competenciesandincentivestoconducteffectivesupportivesupervision.Throughgender-sensitivetrainingthataddressesequityintheclassroom,teachersareexpectedtotargetgirlsmorespecificallyinclass,whichismeanttoincreasethelikelihoodofgirlsbenefitingfromtheeducationprovided.

Inorderforteachertrainingtoresultinimprovedknowledge,skillsandteachingpracticesteachersareassumedtobemotivatedtolearnandtotranslatenewknowledgeandskillsintopractice.Peer-to-peerinteraction,supportivesupervisionandotherincentives(e.g.trainingcertificates)aremeanttocontributetothis.Theextenttowhichteachermotivationtranslatesintoimprovedteachingmayalsobeinfluencedbyteachers’remunerationandtheirworkingconditions.Furthermore,actualimprovedteacherefficacyandlearningoutcomesmayimproveteachers’perceivedefficacy,whichinturnmayincreasetheirmotivation.

ImprovedteachingandlearningisassumedtobefacilitatedbythedistributionandusageofrelevantteachingandlearningmaterialsinHausa.Thisrequiresthattheteachershavethepedagogicalknowledgeandskillsneededtoeffectivelyusethematerialsduringteachingpractice.Furthermore,itassumesthatthematerialsarewellalignedwiththecurriculumandwiththecompetencylevelsoftheteachersandpupils.

Whilethecentralobjectiveoftheearlylearninginterventionistoimprovepupils’literacyand,toalesserdegree,numeracyskills,itisassumedthatthiswillfacilitatetheacquisitionofEnglishasasecondlanguage,andthetransitiontoEnglishinlatergrades.Furthermore,improvedlearningintheearlygradesisexpectedtocontributetohigherretentionratesbecausechildrenwhoperformwellearlyonareassumedtobemore

41Wedistinguishbetweenthreetypesofknowledge:subjectknowledge,curriculumknowledgeandpedagogicalknowledge.SeeSection3.2.9.2intheBaselineTechnicalReport.

©EDOREN 2

motivatedandsupportedbytheirparentstoremaininschoollonger,alinkthatmaybeparticularlyimportantforgirls,whofaceahigherriskofdroppingout(UNICEF,2014d).Apotentialfeedbackloopexistsbetweenincreasedretentionandteachingqualityinearlygrades.Thisfeedbackloopcanbenegativeorpositive.Ifhigherretentionisnotaccompaniedbyincreasedteachingandschoolresources,itcanleadtohigherpupil–teacherratios,whichmaynegativelyaffectteachingquality.Ontheotherhand,teachersmaybeincreasinglymotivatedbythehigherretentionoftheirstudents,whichmaypositivelyinfluenceteachingquality.

©EDOREN 3

Figure16: DiagramdepictingToCoftheearlylearningintervention

©EDOREN 4

AnnexB Evaluationquestions

B.1 Evaluationquestionsfortheearlylearninginterventionevaluation

Box6.EvaluationquestionsforGEP3’searlylearningintervention

1. TowhatextentdoestheearlylearninginterventionimproveHausaliteracyandEnglishlanguagelearningoutcomesamonggirlsandboysintheearlygradesinprimaryschoolsandIQSs?Towhatextentdoestheearlylearninginterventionreducethegapbetweenthelearningoutcomesofthelowestperformingpupilsandtheexpectedlearningoutcomes,asexpressedinthecurriculum?

2. Towhatextentdoesteachers’knowledgeinliteracyandlanguageacquisitioninearlygradesimproveasaresultoftheintervention?

3. Towhatextentdoteacherskillsinearlygrade,gender-sensitiveinstructionimproveasaresultoftheintervention?

4. Towhatextentandhowdoteachersadjustandchangetheirclassroompracticesasaresultoftheintervention?

5. AremorereadingandlearningmaterialsinHausausedintheclassroomduetotheintervention?Dotheycontributetomoreeffectiveteachingandlearning?

6. Towhatextentdoestheearlylearninginterventionimprovepupilretention,especiallyretentionofgirls?

B.2 EvaluationquestionsfortheIQSSevaluation

Box7.EvaluationquestionsforGEP3’sIQSS

1. HowwellhasGEP3’steachercapacitydevelopment(trainingandmentoring)contributedtoimprovedteacherknowledgeandskills,andmoreeffectiveteachingintheclassroom?

2. HowwellhasGEP3’steachercapacitydevelopmentcontributedtoanimprovementingender-sensitiveteaching?

3. HowwellhasGEP3’sheadteachercapacitydevelopment(trainingandmentoring)contributedtoimprovedpedagogicalleadershipandschoolmanagement?

4. HowwellhasGEP3’sCBMCcapacitydevelopmentcontributedtoimprovedschoolmanagementandincreasedmobilisationofresourcesforschoolinvestment?

5. HowwellhaveCBMCsbeenabletoadequatelymanagemini-grantsandtoinvesttheseresourcesintheimprovementofagirl-friendlyschoolenvironment?

6. Howwellhaveteachingandlearningmaterialssuppliedthroughtheinterventionbeenperceivedbyteachersandheadteachersasappropriateandwelltargeted?Havetheybeenusedformoreeffectiveteaching?

7. Towhatextenthavepupilliteracyandnumeracyskills,especiallyofgirls,improvedinGEP3-supportedIQSs?HowhasGEP3contributedtosuchimprovement?TowhatextentdoestheIQSSinterventioncontributetoreducingthegapbetweenlearningoutcomesandexpectedlearningoutcomes,asexpressedinthecurriculum?

8. Whataretheattitudesof,andwhatisthelevelofacceptanceby,communitymembersand

©EDOREN 5

leadersinregardtointegratingformalsubjectsinQur’aniceducation?Howhavetheseattitudesandhowhasthisacceptancechangedduringtheintervention?Why?Howhaveattitudeschangedinregardtogirlsreceivingformaleducation?

9. WhatunintendedconsequencesdoesIQSShaveforteachers,headteachers,pupilsandproprietorswithintheIQSs,aswellasforthebroaderschoolcommunity?

©EDOREN 6

AnnexC Samplesizeofbaselinesurvey

C.1 Samplesizeofearlylearninginterventionbaseline

Table8presentsthetargetedandfinalsamplesizesofthebaselinesurveyoftheearlylearninginterventionevaluation.Intotal,240schoolsweretargetedandsurveyedacrossthetwoschooltypesandthetwostates.Thesamplewasstratifiedaccordingtostateandschooltype,withequalsamplesizesof60schoolsineachstratum(60publicprimaryschoolsand60IQSsineachstate).

Ineachschool,thetargetsamplesizeforpupilsissixgirlsandsixboys,resultinginatotaltargetedsamplesizeof2,880pupils(1,440girlsand1,440boys).ThetargetedsamplesizeforteachersisthreeteachersperpublicprimaryschoolsandtwoteachersinIQSs.ThenumberofteachersislowerinIQSsbecauseweanticipatedonlytwofacilitatorsonaverageperIQS.Hence,thetotaltargetedteachersamplesizeis360primaryschoolteachersand240IQSfacilitators.Thefinalpupilsamplesizeis96%and88%ofthetargetinpublicprimaryschoolsandIQSs,respectively,whilethefinalteachersamplesizeequals83%and74%inpublicprimaryschoolsandIQSs,respectively.Therelativelylowerpercentageofteacherssurveyedcomparedtothetargetedsamplesizewasduetothefactthatsomeschoolsdidnothavethetargetednumberofteachers,whichwasparticularlythecaseintheIQSsinKatsina.

Table8: Targetedandfinalsamplesizesoftheearlylearninginterventionbaselinesurvey

Population

Publicprimaryschools IQSs

Targetsample Finalsample % Targetsample Finalsample %

Schools 120 120 100% 120 120 100%

Pupils 1,440 1,389 96% 1,440 1,264 88%

Teachers 360 299 83% 240 178 74%

C.2 SamplesizeofIQSSbaseline

Table9presentsthetargetedandfinalsamplesizesofthebaselinesurveyoftheIQSSevaluation.Intotal,60schoolsweretargetedandsurveyedacrossthetwoschooltypesandthetwostates.ThesamplewasstratifiedaccordingtoLGAs,withequalsamplesizesoffiveschoolsineachLGA,andtherefore30perstate.

IneachIQS,thetargetsamplesizeforpupilsissixgirlsandsixboys,resultinginatotaltargetedsamplesizeof720pupils(360girlsand360boys).ThetargetedsamplesizeforfacilitatorsistwofacilitatorsperIQS.Hence,thetotaltargetedfacilitatorsamplesizeis120IQSfacilitators.Thefinalpupilandfacilitatorsamplesizesbothequal80%ofthetarget.Inaround20%oftheIQSsonlyhalforlessofthetargetnumberof12pupilscouldbesurveyedbecauseonlyalimitednumberofpupilswaseligibleforthesurvey.Thebelowtargetachievementoffacilitatorssurveyedwasduetothefactthattheschooldidnothavethetargetednumberoffacilitators.

©EDOREN 7

Table9: TargetedandfinalsamplesizesoftheIQSSbaselinesurvey

Population

IQSs

Targetsample Finalsample %

Schools 60 60 100%

Pupils 720 576 80%

Facilitators 120 96 80%

CBMCs 60 52 87%

©EDOREN 8

AnnexD Randomisationchecksfortreatmentandcontrolgroup

Table10: School-levelbalance(allheadteachersinpublicprimaryschoolsandIQSs)

Variable Treatmentmean

Controlmean

Totalmean TotalN P-value

Headteacherisfemale 0.0 0.0 0.0 240 0.653

Whetherheadteacherhadbeenabsentinthelastterm? 0.6 0.7 0.6 225 0.114

Numberofdaystheheadteacherwasabsentinthelastterm 7.7 8.0 7.9 137 0.911

Shareofschoolsthathaveseparatefunctioningtoiletforgirls 0.3 0.2 0.3 239 0.583

Numberofyearssinceintegration 1.8 2.0 1.9 119 0.652

TotalnumberofboysenrolledinintegratedsubjectsfromP1toP6 173.8 201.1 187.6 115 0.522

TotalnumberofgirlsenrolledinintegratedsubjectsfromP1toP6 121.4 145.2 133.4 115 0.487

Ratiooftotalnumberofchildrenstudyingintegratedsubjectstoteachers

42.4 42.2 42.3 114 0.978

RatiooftotalnumberofchildrenstudyingintegratedsubjectstoteachersinP1–P3

39.6 42.2 40.9 115 0.694

Ratioofgirlstoboysenrolledintotal 0.7 0.7 0.7 111 0.347

Headteachersatinonanylessonfortheentireduration 0.5 0.5 0.5 225 0.846

Schoolrepairs 0.9** 1.0 0.95 239 0.047

RatioofgirlstoboysinZamfarainP1–P3 0.6 0.6 0.6 55 0.841

RatioofgirlstoboysinKatsinainP1–P3 0.8 0.9 0.8 56 0.354

Ratiooftotalpupilsenrolledtototalroomsusedforteachingontheday 49.9 77.7 64.4 107 0.182

Fractionoffemaleteachersofintegratedsubjects 0.1 0.1 0.09 240 0.172

OverallF-test,F-stat 1.875 P-Value: 0.1722

Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.

©EDOREN 9

Table11: Teacher-levelbalance(allteachersinpublicprimaryschoolsandIQSs)

Variable Treatmentmean

Controlmean

Totalmean TotalN P-value

Teacherisfemale 0.1 0.1 0.1 477 0.613

DoestheteacherhaveanSSCE? 0.7 0.7 0.7 477 0.515

DoestheteacherhaveanNCE? 0.6 0.5 0.5 477 0.330

DoestheteacherhaveaGrade2Certificate? 0.1 0.1 0.1 477 0.344

Hastheteacherbeenabsentinthelastterm? 0.7 0.7 0.7 450 0.862

Numberofdaystheteacherwasabsentinthelastterm 6.1 5.0 5.5 449 0.341

DoestheteacherspeakHausa? 1.0 1.0 1.0 356

Hastheteacherattendedtraininginthelasttwoyears? 0.4 0.5 0.5 473 0.238

Istheteacherabletoidentifylowperformers? 0.4 0.5 0.5 463 0.430

Istheteacherabletogiveevidenceforjudgementsanddiagnose? 0.1 0.1 0.1 463 0.556

Doestheteacherhavewritingskills? 0.2 0.2 0.2 463 0.578

DoestheteacherhaveHausaknowledge? 2.9 2.8 2.9 463 0.763

Doestheteacherhavecomprehensionskills? 2.1* 2.3 2.2 463 0.091

Istheteacherabletointerpretwordsandphrases? 0.2 0.1 0.2 463 0.652

OverallF-test,F-stat 0.2037 P-Value: 0.6519

Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.

©EDOREN 10

Table12: Pupil-levelbalance(allpupilsinpublicprimaryschoolsandIQSs)

Variable Treatmentmean

Controlmean

Totalmean TotalN P-value

TotalscoreinEnglishassessment 354.47 356.95 355.66 2649 0.429

TotalscoreinHausaassessment 507.34 509.67 508.46 2649 0.600

SpeaksHausaathome 0.99 0.99 0.99 2623 0.381

Femalepupil 0.46 0.44 0.45 2651 0.498

Ageofpupil 9.61* 9.35 9.48 1677 0.077

Hasachair 0.9*** 0.90 0.88 2578 0.001

Hasamotorcycle 0.67 0.66 0.66 2581 0.352

Hasacar 0.18 0.19 0.19 2578 0.256

HasaTV 0.38 0.39 0.38 2579 0.587

Hasacomputer 0.05 0.05 0.05 2574 0.92

Hasacamera 0.05 0.04 0.05 2576 0.194

Hasamobilephone 0.9** 0.95 0.94 2580 0.025

Hascattle 0.56 0.53 0.54 2579 0.187

Hasagoat 0.87 0.87 0.87 2579 0.973

Hasahorse,donkeyormule 0.18 0.16 0.17 2579 0.352

Hasasheep 0.7** 0.68 0.70 2580 0.018

Hasachicken 0.92 0.91 0.92 2581 0.356

OverallF-test,F-stat 0.8539 P-Value: 0.3555

Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.

©EDOREN 11

AnnexE Regressionoflearningoutcomesonkeyinfluencingfactors

ThetablesbelowreporttheresultsobtainedwithourmainregressionmodelsonfactorsinfluencingHausascaledscoresandEnglishscaledscores,respectively,intheearlylearningpupilsample.Thetablesreportthecoefficientestimatedforeachexplanatoryvariable,theirassociatedstandarderror,t-statisticsandp-values,whichprovideanindicationofthelevelofsignificanceinthecorrelationbetweeneachexplanatoryvariableandtheoutcomevariableofinterest,eitherHausaorEnglishscaledscores.

Table13: RegressioncoefficientsandtheirstatisticalsignificanceforthemainfactorsinfluencingHausascaledscores

Variable Coefficient Stderror t-stat p-value

Gender(pupilisfemale) -9.212 6.294 -1.464 0.145

Agedfrom7to10 23.376 6.828 3.423 0.001***

Agedfrom11to15 114.695 8.856 12.951 0.000***

Agedover16 178.553 20.394 8.755 0.000***

PupilinsecondHWItertile 13.769 7.181 1.917 0.056*

Pupilinthird(top)HWItertile 37.849 6.713 5.638 0.000***

Publicprimaryschool(notIQS) -57.353 11.026 -5.201 0.000***

Schoolhasgirls’toilets 8.479 7.486 1.133 0.259

Schoolhaswater 8.971 7.026 1.277 0.203

Pupilattendsotherschool -5.628 6.903 -0.815 0.416

Schoolinruralareas -32.710 11.559 -2.830 0.005**

SchoolinKatsina 3.438 6.683 0.514 0.607

Teachermotivation 12.749 16.729 0.762 0.447

Teacherknowledge1 25.998 14.323 1.815 0.071*

Teacherknowledge2 2.246 4.721 0.476 0.635

Teacherpedagogy1 -3.103 24.650 -0.126 0.900

Teacherpedagogy2 0.414 0.364 1.137 0.257

constant 446.139 57.053 7.820 0.000***

Numberofobservations:1,613;R-squared0.339

Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.

©EDOREN 12

Table14: RegressioncoefficientsandtheirstatisticalsignificanceforthemainfactorsinfluencingEnglishscaledscores

Variable Coefficient Std.error t-stat p-value

Gender(pupilisfemale) -6.752 5.706 -1.183 0.238

Agedfrom7to10 18.687 6.733 2.775 0.006**

Agedfrom11to15 82.448 9.185 8.977 0.000***

Agedover16 126.046 14.208 8.871 0.000***

PupilinsecondHWItertile 16.665 5.759 2.894 0.004**

Pupilinthird(top)HWItertile 27.566 5.902 4.671 0.000***

Publicprimaryschool(notIQS) -45.609 8.715 -5.234 0.000***

Schoolhasgirls’toilets 13.246 6.512 2.034 0.043**

Schoolhaswater 7.182 5.873 1.223 0.223

Pupilattendsotherschool 3.371 6.704 0.503 0.616

Schoolinruralareas -28.869 9.372 -3.080 0.002**

SchoolinKatsina 4.834 5.710 0.847 0.398

Teachermotivation 21.505 15.426 1.394 0.165

Teacherknowledge1 15.479 12.668 1.222 0.223

Teacherknowledge2 -0.142 3.710 -0.038 0.970

Teacherpedagogy1 -18.557 20.066 -0.925 0.356

Teacherpedagogy2 0.192 0.350 0.550 0.583

constant 278.692 53.062 5.252 0.000***

Numberofobservations:1,613;R-squared:0.332

Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.

Inadditiontothemainregressionmodel,theanalysiswasextendedbyusingaclusterfixedeffectsmodel,withtheschoolbeingthecluster.Thisenablesustocontrolforallschool-levelinfluencingfactorsatonce.

©EDOREN 13

Table15: Hausaschoolfixedeffectsmodel

Variable Coefficient Std.error t-stat p-value

Gender(pupilisfemale) -13.161 6.266 -2.100 0.037**

Agedfrom7to10 15.989 5.891 2.714 0.007**

Agedfrom11to15 83.130 10.829 7.677 0.000***

Agedover16 133.901 23.743 5.640 0.000***

PupilinHWItertile -0.598 7.513 -0.080 0.937

PupilinthirdHWItertile 20.247 6.027 3.360 0.001***

Pupilattendsotherschool 3.783 7.179 0.527 0.599

Numberofobservations:1,639;R-squared:0.606

Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.

Table16: Englishschoolfixedeffectsmodel

Variable Coefficient Std.error t-stat p-value

Gender(pupilisfemale) -10.366 5.698 -1.819 0.070*

Agedfrom7to10 15.998 7.250 2.207 0.028**

Agedfrom11to15 61.565 10.618 5.798 0.000***

Agedover16 96.922 17.944 5.401 0.000***

PupilinHWItertile 9.296 5.917 1.571 0.118

PupilinthirdHWItertile 14.186 5.770 2.458 0.015**

Pupilattendsotherschool 11.474 7.783 1.474 0.142

Numberofobservations:1,639;R-squared:0.606

Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.

©EDOREN 14

AnnexF UpdatedToCofIQSSintervention

TheinterventionToChasanimportantmethodologicalroleintheevaluationofGEP3’sIQSS.Whilecausalinferenceinthecaseoftheevaluationoftheearlylearninginterventionisderivedfromcounterfactualcomparisonusingacontrolgroup,contributionanalysisinferscausalityfromareasonedToC.Therefore,inthissectionwepresentanin-depthdiscussionoftheToCoftheIQSS,startingfromtheToCincludedintheGEP3EvaluationFrameworkandupdatedbasedoninsightsfromthebaselinedatacollection.SincetheoverallToCresultchainpresentedintheEvaluationFrameworkwaswell-founded,littleupdatinginthisregardhasbeenrequired.Thereviewmostlyfocusesonfurtherexplainingthemechanismandassumptionsunderpinningtheresultchain.

Error!Referencesourcenotfound.visualisestheToCofGEP3’sIQSS.InthefollowingsubsectionswewilldiscusstheToCaccordingtothreecausalpackagesthatmakeupthisToC.Eachcausalpackageconsistsofcausalfactorsinterlinkedviacausalmechanisms,plustheirunderlyingassumptions.Acausalpackageisassumedtobesufficientforgeneratingkeyintermediaryorfinaloutcomesontheconditionthatthecausalchainandassumptionshold.ForeachcausalpackagerelatedtoGEP3’sIQSStheremaybealternative,non-GEP3relatedexplanationsfortheoutcomeoccurring,whichwillrequireexaminationaspartoftheevaluation.

Eachcausalpackagepresentedbelowcorrespondstothecontributionclaimspresentedinthemethodologicalsection.42

42(Mayne,2012).Thecausalpackageisdefinedasbeingsufficienttocauseanoutcome,whileitsindividualelements,inparticulartheinterventionelements,arenecessarypartsforthepackagetomakeadifferencetotheoutcome.

©EDOREN 15

Figure17: DiagramdepictingToCoftheIQSS

©EDOREN 16

ContributionClaim1GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs

Causallinkagesandmechanisms

GEP3’sstrategyforintegratedQur’aniceducationaimstoprovideaccesstoqualityeducationinIQSs,particularlyforgirls,byimprovingeffective,gender-sensitiveteachingoftheformalsubjectsincludedintheharmonisedintegratedcurriculum.MoreeffectiveteachingisconsideredakeyintermediaryoutcomeoftheIQSS.Weconceptualiseeffectiveteachingasacombinationofthefollowingattributes:1)demonstratedteachingcompetency,drawingonthreetypesofknowledge(subjectknowledge,pedagogicalknowledgeandcurriculumknowledge);2)useofapupil-centredlearningapproach;3)timeontask;4)effectiveuseofteachingandlearningmaterials;and5)noobservablegenderbiasduringteachingpractice.

ThecorecausalpathwaytoachievemoreeffectiveteachingisbyimprovingIQSfacilitators’knowledgeandskillsthroughtrainingandmentoring.Thetrainingismeanttoimprovetheirknowledgeandskillsintermsofthecoresubjectsoftheharmonisedintegratedcurriculum,pedagogy,andtheintegrationprocessinQur’aniceducation.Thetrainingisalsomeanttoimpartabetterunderstandingofgender,equityandgender-sensitiveteachingmethods..Thementoringprocessisassumedtoplayanimportantroleinfacilitatorseffectivelyacquiringthisknowledgeandskills,andputtingitintopractice,byactivatingthefollowingmechanisms:

• peerinteractionthatencouragesfacilitatorstodiscussacquiredknowledgeandskills,shareexperiencesandprovidefeedbackbasedonacollaborativerelationshipamongpeers;

• school-basedsupportthatallowsfacilitatorstoaccessandrefreshtheirknowledgeandunderstandingatanytimewhilstusingtheprovidedmaterials;and

• continuousface-to-face,practicalsupportbyanexperiencedmentorbasedonalong-term,personalrelationshipthatreinforcesthelearninganddeliveryofnewlyacquiredknowledgeandskillswithconfidence.

Mentoringisalsoassumedtoincreaseteachermotivation.Whileincreasedteachermotivationisnotadirectintendedobjectiveoftheintervention,itisassumedtobeakeycontributingfactortomoreeffectiveteaching.Theunderlyingtheoryisthatpeerinteractionandongoingsupportincreasethefacilitators’perceivedteachingefficacy,self-confidenceandself-esteem,whichresultsinincreasedmotivation,asexpressedin(i)moreinterestandenjoymentintheirwork,(ii)moreeffortbeingmade,(iii)moreimportanceattachedtotheirwork,and(iv)lesspressureandtensionexperiencedinrelationtotheirwork.43Thisinturnsenhancesthelikelihoodoffacilitatorsactuallyapplyingtheirimprovedknowledgeandskills,andspendingincreasedtimeontaskonactivitiesthatbenefitstudentlearning.

ThedistributionofHausateachingandlearningmaterialscontributestomoreeffectiveteachingasmaterialsprovideguidancefortheteacherduringtheteachingprocessandfacilitatetheknowledgetransferbetweenteachersandpupils.Thisrequiresteacherstohavethepedagogicalknowledgeandskillstoeffectivelyusethematerialsduringteachingpractice.Pupilworkbooksandnotebooksalsoallowpupilstopracticeandallowteacherstoassesspupilsinwriting.TheprovisionoflearningmaterialsmayalsoensuretheinclusivenessofintegratedQur’aniceducation.However,itmustbenotedthatpovertyhasbeenidentifiedasoneofthekeybarrierstoaccessingpublicprimaryschool43ThistheoryisbasedonthetheoryofmotivationusedbytheTDPinNigeria.

©EDOREN 17

andtotheextentthattheintegrationprocessforcesparentstoinvestinlearningmaterials,comparedtothesituationwithoutintegration,parents’inabilitytodosomaycausepupilstodropout.

Headteachertrainingseekstosupportamoreeffectiveteachingprocessinseveralways.First,thetrainingisassumedtoincreasepedagogicalleadershipintheIQSbyprovidingtheheadteacherwithknowledgeandskillsintermsofpedagogyandleadership.This,inturn,willprovidethefacilitatorswithschool-based,continuousguidanceinhowtobetterdeliveryinstruction.Second,leadershipcaninspireandmotivatethefacilitator,whichcontributestoimprovedattendanceanddemonstratesitselfinincreasedeffortandenjoymentofwork.Third,thetrainingalsoaimstoimprovetheheadteacher’sschoolmanagementpractices,suchasteacherattendancemonitoringandclassscheduling,whichinturncancontributetoincreasedfacilitatorattendanceandtimeontask.Improvedschoolmanagementcanalsoincreasetheresourcesavailableforfacilitatorremuneration,whichcanaffecttheirattendance,andcansubsequentlycontributetomoreeffectiveteaching.

Animproved,moregirl-friendlyschoolenvironmentandimprovedmonitoringandsupportsupervisionbylocalgovernmenteducationstaffisassumedtocontributetomoreeffectiveteaching.Abetterschoolenvironment,intermsofinfrastructure,classroomconditions,safetyandrelationshipsbetweendifferentactorsintheschool,canimproveteachers’andpupils’comfortandfeelingofwellbeingintheschool.Itcanalsoleadtoincreasedinstructionaltimebycreatingthephysicalconditionsrequiredforteaching(e.g.light).Girl-friendlyschoolfeatures,suchasthepresenceoffemaleteachersorfemaleparticipationinschoolmanagement,cancontributetoanincreaseingender-sensitiveteachingpracticesbycreatingaclimatethatismoresupportiveofgirls.Improvedgovernmentmonitoringandsupportsupervisioncancontributetobetterschoolmanagementandcanmotivatefacilitatorsandproprietorstocontinueprovidingtheintegratedcurriculumasitsignalsgovernmentsupportforintegration.Furthermore,sustainedfinancialgovernmentsupportcansupportfacilitatorremuneration,whichwillcontributetofacilitators’motivationandattendance.

Whilemoreeffectiveteachingisassumedtocontributetoimprovedlearningoutcomes,improvementinlearningoutcomescaninturncontributetoanincreaseinfacilitators’perceivedteachingefficacy,whichisexpectedtostrengthentheirmotivation.Anincreaseingirls’enrolmentandretentionmayalsoaffectteachingpractices:first,itmaycontributepositivelybycreatingamoregirl-friendlyschoolenvironment,whichinturncansupportgender-sensitiveteachingpractices;second,itmayhaveanegativeeffectbyincreasingpupil–classroomandpupil–teacherratios.

Assumptions

Assumptionsaredefinedhereasinfluencingconditionsthatmightenablecausallinkagesandmechanismstowork,ormayimpedethoselinkagesandmechanism.KeyassumptionsunderlyingthecausalchainofContributionClaim1arepresentedinFigure18.

Implementationassumptionsarenotincludedinthediagram.Itisassumedthatthetrainingandmentoringareimplementedasplannedandcoverallthecomponentsofthetrainingcurriculuminaqualitymanner,andthattheteachingandlearningmaterialsaredistributed.Theseareimportantassumptions,inparticularwithregardstothementoringsinceGEP3’scapacitydevelopmentapproachforfacilitatorsstronglyemphasisescontinuous,school-basedfollow-upbyexperiencedmentorscomplementedwithpeerinteraction.

©EDOREN 18

Figure18: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’

©EDOREN 19

Alternativeexplanations

Weconsidertwoalternative,non-GEP3-relatedexplanationsthatcouldalsobesufficientforthe

intendedchangeinteachingintheIQSstobeobserved:

• Facilitatorsbuildcapacityinotherways.IffacilitatorsintheGEP3IQSsparticipateinothertrainingorcapacitybuildingprocessesduringtheGEP3projectperiod,teachingcouldbecome

moreeffectiveregardlessofGEP3.Suchcapacitybuildingmaytakeplacewhenfacilitatorsare

alsoteachersinpublicprimaryschools.Furthermore,ifnewfacilitatorswhohaveahigher

teachingcompetencylevelareemployed,teachingmayalsobecomemoreeffective.

• Facilitatorsaremotivatedinotherways.Iffacilitatorremunerationorworkingconditions

improvewithoutanyinfluencebyGEP3interventions,facilitatorsmaybecomemoremotivated

andteachmoreeffectivelyregardlessofGEP3.

ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs

Causallinkagesandmechanisms

MainlythroughitssupporttoCBMCsGEP3seekstoimprovetheschoolenvironmentoftheIQSandmakeitmoregirl-friendly.Weconsidertheschoolenvironmenttoencompassthephysical

environment,intermsofinfrastructureandclassroomconditions,itssecurityandperceivedsafety,

andtheschool’sorganisationandinstitutionalculture,intermsofleadership,organisational

processes,relationshipsbetweenschoolstakeholders,andtraditions.Theschoolenvironment

affectsthelearningexperienceofthepupils,aswellastheteachingexperienceofthefacilitators.

GEP3seekstocontributetothegirl-friendlinessoftheschoolenvironmentintheIQS,promoting

investmentininfrastructureandresourcesthatimprovetheschoolexperienceofgirlsandaschool

organisationthatsupportsgenderequity.

Thecorecausalpathwaytoimprovingtheschoolenvironmentisbyimprovingschoolmanagement

andsupportingthemobilisationofmoreresources.Additionalresourcesthataremobilisedcanbe

investedincriticalinputsthataddressbarrierstogirls’educationandthatareprioritisedinschool

developmentplans.Betterschoolmanagementisassumedtoimprovetheschoolenvironment

because,throughthedevelopmentofWCDPs,strengthsandweaknessesintheschoolenvironment

areassessed,prioritiesforimprovementidentified,andresourcemobilisationactionsoutlined.

ThroughtheactiveparticipationoftheCBMC—whichismeanttorepresentthedifferentschool

stakeholders—theprioritiesandpreferencesofvariousstakeholdersaretakenintoaccount:in

particularthepreferencesofwomenandgirls.Itisassumedthatwomenandgirls’participationin

theschoolmanagementbettersignalsgirls’needsandstrengthensthedemandforgirl-friendly

schoolinputs.Furthermore,betterschoolmanagemententailsmoreadequatefinancial

managementandrecord-keeping,whichsupportsplanningandtransparency,andhencemore

effectiveuseofresourcesprovidedbythecommunityorotherexternaldonors.Giventheoftenlow-

resourcesettingoftheIQSsresourcemobilisationisanecessaryconditiontotranslatedevelopment

plansintoinvestments.

ThroughafunctionalCBMCcommunityrepresentativesareinvolvedintheIQS.Thisisassumedto

increasecommunitysupportforintegratedQur’aniceducation,particularlyforgirls,becauseitmakescommunitymembersmoreconsciousoftheintegrationandempowersthemtohaveasayin

itsimplementation.Thiscontributestoincreaseddemandforgirls’education.Inaddition,ifthe

communitybecomesmoreinvolvedinthemanagementanddecision-makingoftheIQSthenitis

assumedthattheywillbemorelikelytosupportitfinanciallyorwithotherresourcesbecausethere

©EDOREN 20

willbecommunityownership,oraperceptionofownership,overtheIQS.Finally,community

membersareabletobetterholdschoolleadersandteachersaccountableforthequalityofthe

schoolenvironmentandattendancesince,throughtheirinvolvement,theyaremoreawareofwhat

occursintheschool,andagainbecauseoftheirstrongersenseofschoolownership.

Themainmeanstoachievetheaboveintermediaryoutcomesisthetraining,mentoringandmonitoringoftheCBMCmembers.Thetrainingaimstoimprovetheirunderstandingoftheirroles

andresponsibilities,whichcanenableschoolmanagementandresourcemobilisationbecause

managementandsupervisorytasksandfunctionswillbebetterdefined(e.g.teacherattendance

monitoring,planning,andfundraising).BytrainingCBMCmembersonwholecentredevelopment

planningtheirskillsinschoolplanningaremeanttoimprove.Theinclusionofgenderinthetraining

curriculumaimstocreateawarenessoftheissue,whichissubsequentlyassumedtoresultinmore

gender-responsivewholecentredevelopmentplanning.Trainingonresourcemobilisationand

financialmanagementismeanttomakeCBMCmembersmoreabletomobiliseandmanage

resources.Theperiodicmentoringvisitscanreinforceandrefreshthelearningfromthetraining

workshops,providingpracticalfeedbacktailoredtothecontextofeachIQS.Itislesscleartowhat

extentthementoringvisitswillre-enforcegendersensitisationtoensurethatCBMCmembersfollow

uponandmonitorthegirl-friendlinessoftheschoolenvironment.Monitoringcansignal

deficiencies,basedonwhichactioncanbetaken,andcanincreaseaccountabilitybydirecting

attentiontotheachievementofresults.GEP3willsupportgovernmentstafftocarryouttermly

CBMCeffectivenessmonitoring,providingatoolthatismeanttoimproveCBMCs’monitoring

capacity.

BeyondtheroleoftheCBMC,theheadteachertrainingseekstoincreasetheheadteachers’skillsinschoolmanagement,whichisalsoassumedtocontributetomoreeffectiveschoolmanagement.

Themini-grantsdirectlycontributetoanincreaseinschoolresourcesand,giventhattheyaretargetedatgirl-friendlyinvestments,canimprovethegirl-friendlinessoftheschoolenvironment.

ThegrantscanalsoworkasanincentiveforCBMCmemberstodeveloptheircapacity,asmini-

grantsareprovidedontheconditionthatCBMCmembershaveparticipatedinthetraining.

Furthermore,themanagementofthegrantcanimproveoverallcommunity-basedschool

managementthrougha‘learning-by-doing’capacitybuildingapproachfortheCBMC,whichismeant

tomanagetheresources.

Gender-sensitiveclasspracticesandtheincreasedpresenceofgirlsintheIQS,duetoincreased

enrolmentandretention,canstrengthenaschoolclimatethatsignalsanopennesstogirlpupilsand

cancreateasocialenvironmentwheregirlsfeelwelcomeamongtheirpeers.

Assumptions

Figure19presentstheassumptionsabouttheconditionsthatneedtobeinplaceforIQSS

interventionstoactuallyresultinanimproved,moregirl-friendlyschoolenvironment.

©EDOREN 21

Figure19: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’

©EDOREN 22

Alternativeexplanations

Weconsiderthefollowingalternative,non-GEP3-relatedexplanationsthatcouldimprovetheschoolenvironmentandmakeitmoregirl-friendly:

• CBMCmembersandschoolleadershipimprovetheircapacitythroughothermeans.SchoolmanagementandresourcemobilisationcanimprovebynewstakeholdersbecominginvolvedintheIQS,regardlessofGEP3support.Forexample,aphilanthropistmaytakeaninterestintheIQSortheIQSmaystartbenefittingfromtheinfluenceorcompetenciesofahighlyeducatedor/andrespectedcommunitymember.ItisnotonlydifficulttoidentifyallsuchinfluencesbutitwillbealsohardtoseparateoutGEP3’sinfluenceinsuchinstancessinceCBMCtrainingpromotessuchinvolvement.SomeCBMCmembersmayalsobemembersoftheSBMCintheneighbouringpublicprimaryschool,andmaybuildtheircapacitythroughtheirparticipationinpublicprimaryschoolmanagement.

ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlylearningenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls

Causallinkagesandmechanisms

TheprincipalfinaloutcomethatGEP3aimstoachievewithitsIQSSisforpupilslearningoutcomestoimprove,particularlyforgirls.Moreeffectiveteachingisconsideredacriticalcontributingfactor,whichwillbesupportedbyanimproved,moregirl-friendlyschoolenvironment.Asthesetwofactorsimproveandresultinbetterqualityeducationgirls’retentionisalsoassumedtoincrease.Inaddition,girls’enrolmentisexpectedtoincreasebecauseofanincreaseddemandforgirls’educationasaresultofmorecommunityinvolvementandsupportforIQSs.Girls’retentionandenrolmentarenotfurtherdiscussedbelow,astheyarenotthefocusofthisevaluation.However,itisimportanttonotethatincreasedretentionandenrolmentcaninturninfluenceeffectiveteachingandtheschoolenvironment,aswasdiscussedintheprevioussections.

Internationalstudiesindicatethatwhatteachersknow,whattheydoandhowmuchtheycareaccountsformorevarianceinpupilachievementthananyotherpolicy-amenablevariable(Hattie2003).Severalmechanismsareatplaythatmaycausemoreeffectiveteachingtoleadtoimprovedlearningoutcomesforgirls.First,thecombinationofimprovedsubjectknowledgeandpedagogicalskills,supportedbyfit-for-purposeteachingandlearningmaterials,canenablemorecompetentandequippedfacilitatorstobettersupportpupils’learningachievementintheclassroom.Byteachinginamoregender-sensitivemannertheteacher’sinteractionwiththegirlpupilimproves,whichcanimprovethequalityofthegirl’seducationinparticularandprovideadditionalmotivationforhertolearn.Second,student-centredteachingmethods—acompetencythatGEP3trainingintendstostrengthen—encourageschildren,especiallygirls,toattendschools,whichincreasesthepupil’sactualinstructionaltimeandcan,therefore,increaselearning.Third,effectiveteachingalsoentailsfacilitatorsattendingandteachingwheninschool(asaresultofbetterschoolmanagement).This,again,canincreasetheactualinstructionaltime,andhencepositivelyinfluencelearningoutcomes.

Totheextentthattheschoolenvironmentismoreconducivetogirlslearning,learningachievementisfurtherstrengthened.Again,thecausalmechanismshereareseveral.First,improvedphysicalconditionsunderwhichteachingandlearningtakeplace(e.g.lessovercrowdedclassroomsorteachinginashadedlocation)enhancethedegreeofsupportthatteacherscanprovideandtheattentionthatpupilscangivetolearning.Second,thephysicalconditionsinfluencetheinstructionaltimeasprotectionagainstrain,orlightduringeveningclasses,enableclassestotakeplace.Third,asafe,comfortableandenjoyableschoolenvironmentaffectspupils’experienceofschooling,whichislikelytoleadtomorefrequentattendanceandgreaterengagementintheschoolingenvironment.Totheextentthattheschoolenvironmentismoregirl-friendly,girls’schoolexperienceisparticularlyenhanced.

©EDOREN 23

Assumptions

Figure20presentstheassumptionswithregardstotheconditionsthatneedtobeinplaceformoreeffectiveteachingandanimproved,girl-friendlyschoolenvironmenttoactuallyresultinimprovedlearningoutcomes,particularlyforgirls.Wedonotfocushereonassumptionsrelatedtoretentionbutmanyofthepresentedassumptionsarerelevanttoachievingtheretentionofgirls.

Therearemanyfactorsthat,inturn,influencewhethertheconditionshold.Thedifferentassumptionsarealsointerdependent.Pupils–inparticular,girls–actuallyattendingclassisinfluencedbyexternalfactors,suchastheiroutsideschoolactivitiesandresponsibilities,publicprimaryschoolattendance,economicconstraints,schoolaccessibility,andtheattitudesofthepupilsandtheirsocialenvironment.Girlsareassumedtobeinterestedinacquiringaformaleducation,whichwilldependontheiraspirationsandthevaluetheyseeinformaleducation–given,inparticular,theexpectationofearlymarriage,whichisanimportantbarriertogirls’retention.

Figure20: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’

Alternativeexplanations

Weconsiderthefollowingalternative,non-GEP3-relatedexplanationsthatcouldalsobesufficienttoexplainimprovedlearningoutcomes:

• Improvedaccessandqualityeducationinotherschools.ThebaselinedatademonstratethatIQSsarenottheonlyalternativetoaccessingformaleducation.PupilsareenrolledinbothIQSsandpublicprimaryschools.Theextentthatpupilsattendthetwoschoolsdependsonmanyinteractingfactors,suchasaccessibilityofthepublicprimaryschool,therelativecostofattendingpublicprimaryschoolsandpovertyoftheparents,aswellasculturalbeliefsandsocialnormswithregardstoformaleducationandgirls’accesstothis.Iftheseunderlyingfactorschangepublicprimaryschoolattendancemay

©EDOREN 24

increaseandlearningoutcomesmayimprove,regardlessoftheintegrationofformaleducationintheQur’anicschools.Thequalityofeducationalsoplaysarole.Accesstopublicprimaryschooldoesnotnecessarilyresultinimprovedlearningoutcomes.TotheextentthatthequalityofeducationinthepublicprimaryschoolsthatIQSpupilsareattendingbecomesbetter,theirlearningoutcomescanimprove.Furthermore,inlessremoteareaspupilsmayattendvocationalorothertraining,whichmayprovidesomeformofbasiceducationormotivatepupilstolearn.Finally,inmoreurbanareasformalprivateschoolsmayalsoattractpupils,whichcanaffecttheirlearningoutcomes.

• Changesinthecompositionofthepupilpopulation.Overtime,theIQSmayattractadifferentpupilpopulation:forexample,youngerpupilsstarttheintegratedcurriculumormorepupilswithpreviouseducationexperienceareattractedtotheIQS.TotheextentthattheirlearningoutcomelevelsaredifferentfromthelearninglevelsofthebaselinepopulationchangesintheaveragelearningoutcomeintheIQSmaybeobserved,regardlessofimprovedteachingoranimprovedschoolenvironment.

EDOREN is a consortium of leading organisations in international development and education: Oxford Policy Management (OPM), and the Institute of Development Studies (IDS) at the University of Sussex, and is supported by UK Aid. EDOREN cannot be held responsible for errors or any consequences arising from the use of information contained in this report. Any views and opinions expressed do not necessarily reflect those of OPM, IDS and EDOREN or any other contributing organisation.