TURKEY:EVALUATING THE IMPACT OF İŞKUR’SVOCATIONAL TRAINING PROGRAMS
August 2013
Human Development Sector UnitEurope and Central Asia Region
Report No: 82306-TR
WORLD BANK
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 1
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS
August 2013
Human Development Sector UnitEurope and Central Asia Region
Report No: 82306 - TR
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS2
ACRONYMS AND ABBREVIATIONS
ALMP Active labor market program GDP Gross domestic product IRR Internal rate of return ITT Intent-to-treat effect İŞKUR TurkishEmploymentAgency LATE Local average treatment effect MHI MentalHealthIndex MIS Managementinformationsystem OECD Organization for Economic Co-operation and Development OTJ On-the-jobtraining PISA Program for International Student Assessment SIEF SpanishImpactEvaluationFund TL TurkishLira TUIK TurkishStatisticalInstitute UN United National VQI Vocational Qualification Institution WAP Working age population WB World Bank
CURRENCY EQUIVALENTS(ExchangeRateEffectiveAugust,2013)CURRENCY = TLU$ 1.00 = 1.92 TL
Vice President: PhilippeH.LeHouerou,ECAVP CountryDirector: MartinRaiser,ECCU6 Sector Director: AnaRevenga,ECSHD Sector Manager: RobertaGatti,ECSHD Task Team Leader: CristobalRidao-Cano,ECSHD
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 3
CONTENTS
Acknowledgements v
Overview vii
Section1:TheContext:Jobs,SkillsUpgrading,andİŞKUR 1
Section2:TheEvaluationofİŞKURTrainingPrograms:Design,Data,andMethods 5
Design 5
Data 7
Methods 9
Section3:TheEvaluationofİŞKURTrainingPrograms:Results 13
ProfileofİŞKURtraineesinthesampleunderthestudybeforecoursesstarted 13
ImpactsofİŞKURvocationaltrainingunderreview 20
Whydoesthetrainingunderreviewhavealimitedimpact onemploymentandwhattypesoftrainingworkbetter? 26
Cost-benefitanalysisofİŞKURtrainingunderreview 31
Section4:StrengtheningtheImpactofİŞKURTrainingandServices 33
References 37
Annex1:EstimationMethodology 39
Annex2:DefinitionsofKeyOutcomeVariables 40
Annex3:EstimationResults 42
LIST OF BOXES
Box1:İŞKURVocationalTrainingProgram 4
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS4
LIST OF FIGURES
Figure1: Moreeducatedentrantsbutlarge“stock”oflow-skilledworkers 2
Figure2: Geographicdistributionofprovinces 6
Figure3: HigheducationalattainmentamongİŞKURtrainees 14
Figure4: LimitedjobsearchamongİŞKURtrainees 17
Figure5: İŞKURservicesareusedbymostjobseekers 19
Figure6: TraineesattachgenuinevaluetoİŞKURtraining 21
LIST OF TABLES
Table1:ProfileofİŞKURtraineesrelativetotheunemployedandthejoblessinurbanareas 16
Table2:LargeexpectedimpactofİŞKURtrainingonthelabormarket 20
Table3: İŞKUR’strainingshavenegligibleoverallimpactsonemploymentandsmall impactsonjobquality 22
Table4:NosignificantvariationintheimpactofİŞKURtrainingsonemployment bygenderandage 24
Table5:Bigmismatchbetweenexpectationsandrealities 25
Table6:Modestimpactsoftrainingonothermeasuresofindividualandhousehold well-being 26
Table7:Coursecharacteristicsassociatedwithbetterimpacts 28
Table8: Itisreallythecombinationofprivateprovisionandcompetitionthatworksbest? 29
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 5
ACKNOWLEDGEMENTS
ThisstudyistheresultofarequestbytheTurkishEmploymentAgency(İŞKUR)totheWorldBank(WB)toevaluatetheimpactofvocationaltrainingprogramsfortheunemployedand to identifyways to strengthen theseprograms.The studycouldnothavebeenpossiblewithoutthetechnicalsupportofİŞKURthroughoutits design and implementation. The WB team included Cristobal Ridao-Cano,RitaAlmeida,SarojiniHirshleifer,DavidMcKenzie,andA.LeventYener,withtheinvaluableassistanceofAyçaDönmez,ElçinKoç, andElifYükseker.The İŞKURteamcomprisedMehmetAslan,FeridunGiresun,AbdülkadirYanıcı,SerkanYücel,FeridunKaya,EmreÖzen,andDeryaDuyarCoşkun.OverallguidancewasprovidedbyMartinRaiser(CountryDirectorforTurkey,WB),UlrichZachau(formerCountryDirector for Turkey, WB), Roberta Gatti (Sector Manager, Human DevelopmentEconomics, Europe and Central Asia, WB), Jesko Hentschel (former SectorManager,HumanDevelopment Economics, Europe andCentral Asia,WB),AnaRevenga (Director,HumanDevelopment, Europe andCentralAsia,WB),NusretYazıcı(DirectorGeneralofİŞKUR),AsımGökerKeskin(DeputyDirectorGeneralofİŞKUR),MustafaKemalBiçerli(FormerDGofİŞKUR),NamıkAta(FormerDeputyUndersecretaryofMoLSS,andformerDGofİŞKUR),andBirolAydemir(FormerUndersecretaryofMoLSS).
ThefinalreportgreatlybenefitedfromcommentsprovidedbyChristianBodewig(WB),DavidRobalino(WB),andStefanieBrodmann(WB),aswellasfromfeedbackreceivedatnumerouspresentationsofthedesignandpreliminaryresultsofthestudyinWashingtonDC,Ankara,andIstanbul.ThisstudywasfundedbytheSpanishImpactEvaluationFund(SIEF),theGenderActionPlan,theWB,andİŞKUR.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS v
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS6
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 7
OVERVIEW
Turkey’seconomicperformanceduringthe 2000s is a global success story;high-incomestatusisnowwithinreachbutthiswillrequirecreatingmoreandbetterjobs.TheTurkisheconomygrewbyanaverageof5.5percentbetween2002and2011.Turkeytodayisthe16thlargesteconomyintheworld,withpercapita income exceeding US$ 10,000.Economic growth has been rapid andinclusive,asthewelfareofthepoorest40percentofthepopulationgrewfasterthan the average during the 2000s.Inclusivegrowth ispartly the resultofrapid urbanization generating more and betterjobopportunities.However,untilrecently these opportunities have notkept pace with the increasing supplyof (mostly) young and low-skilledworkers looking for jobs in cities,resulting in stagnant and even declining employment rates during the last twodecades.InTurkeytoday,lessthanhalfoftheworkingagepopulation(15-to64-year-olds)isemployedand42percentofworkersareinformal.Theoutstandingemployment growth after the 2008-09crisismayheraldachangeintrend(theWorld Bank is currently carrying outmoredetailedanalysisintothereasonsfor recent rapid employment growth)but with reported skill shortages,unemploymentratesaround10percentand lowemployment ratescoexisting,
policies to improve the functioning oflabormarketsremainapriority.
Upgradingtheskillsofthecurrentlaborforce is crucial for creating more and better jobs. Although the young arebecomingmore educated and skilled,half of the working age population(WAP)stillhaslessthanbasiceducation,accountingformostofthejoblessandinformalworkers,whilethedemandforskills in formal non-agricultural sectors continuestoincrease.Evenrecenthighschool and college graduates havedifficultyfindingjobsforwhichtheyareinprinciple“qualified,”despitethefactthat businesses report to have a hardtimefindingworkerswiththerightskills,indicatingaskillsmismatch(McKinsey2012).While the evidence shows thatskills are best acquired the first timearound,Turkeyalsoneeds toenhancetheskillsofitsexistinglaborforce.
The Turkish Employment Agency(İŞKUR)playsaleadingroleinupgradingthe skillsof jobseekers and facilitatingtheiraccesstoproductiveemploymentby providing vocational training andother employment support services.İŞKURhascomealongwaysince2008,significantly expanding its coverage(from30,000traineesin2008to464,000in2012,representing19.6percentofthe
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS vii
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS8
registeredunemployed).Theexpansionof İŞKUR’strainingcoveragehasbeencomplementedwithfurtherreformstoimprovethequalityandeffectivenessofvocationaltrainingprograms,includingthe introduction of quality criteria inthe selection of training providers aswell as vocational and job counselingservices.
The increasing importance of İŞKURvocational training prompted thegovernmenttocommissionthepresentstudy to evaluate its impact and toidentify ways to enhance it. It is themost recent product of a long standing collaboration between the Ministryof Labor and Social Security (MoLSS)and İŞKUR with the World Bank toexaminepolicyoptionstohelpimproveemploymentprospectsforjobseekers,particularlyamongwomenandyouth.The study could have been possiblewithoutthetechnicalsupportprovidedbyİŞKUR.
The study evaluates themain type ofİŞKUR’svocationaltrainingcoursesataparticularpointintime,andthusitdoesnot take into account recent reforms.The study evaluates a representativesample of general vocational training courses,whichaccountforabouttwo-thirdsofİŞKURvocationaltrainees(theotherone-thirdmostlytakecourseson-demandbyemployers)thattookplacebetween December 2010 and June
2011.However, approximately80%ofthe2012courseswerejobguaranteedcourses. The active labor marketservices regulation was amended in 2013 to increase the rate of jobguaranteed courses. In this respect, itis expected that the2013courseswillhaveasimilarrate.
The evaluation has an experimentaldesign, exploiting the excess demandfor İŞKUR vocational training coursesto randomly assign eligible trainingapplicants into those who receivetraining (treatment group) and thosewho do not (control group). Originaldatawascollectedfrom5,902applicants(henceforth called İŞKUR trainees)(evenly split between treatment andcontrol groups) between September2010andJanuary2011(beforetrainingstarted) and approximately one yearlater. The experimental design allowsthedifferenceinemploymentoutcomesafter the training between treatmentand control groups to be attributed to training, and training only. This is thefirst randomized evaluation of a large-scalevocationaltrainingprogramfortheunemployedinadevelopingcountry.
Thestudyinvestigates:(i)theprofileandjobsearchbehaviorof İŞKUR traineesin the sample before courses started,as well as their assessment of İŞKURservicesbeforethestartofthecourses;(ii) the average impact of training on
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMSviii
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 9
employment;(iii)differentialimpactsfordifferentgroupsoftrainees,particularlyintermsofgender,age,andeducationlevel;(iv)theprocessbywhichtrainingleads to employment outcomes,including identificationof the typesoftraining that have a higher impact onemployment; and (v) the costs andbenefitsof İŞKURtraining (overallandfor different types of training). Finally,the study makes use of the findingsfrom this evaluation, complementedby evidence from other countries, tosuggestanumberofpolicyoptionstofurtherstrengthentheimpactofİŞKURtraining and services.
The profile of İŞKUR trainees in the sample before courses started
İŞKUR trainees are significantlyyounger, have less work experience,andaremoreeducatedthantheaveragejobseeker in urban areas. Women are overrepresented among İŞKURtrainees (63 percent) compared tourban jobseekers (30percent).1 İŞKURtrainees are significantly younger (45percent are between the ages of 15and25comparedto31percentamongurban jobseekers) andhave lesswork
experience.Themoststrikingdifferenceis that İŞKUR trainees are remarkablymoreeducatedthanurbanjobseekers:74 percent of İŞKUR trainees havecompletedatleasthighschool,versus42 percent of urban jobseekers. Thisdifference persists even after limiting the urban sample to 20- to 29-year-olds.
Thesignificantlyhighereducationlevelof İŞKURtraineesismostlyduetothedesign and implementation of İŞKURtraining. The difference in educationalattainmentbetweenİŞKURtraineesandthe average jobseeker in urban areascould be explained by a number offactors:(i)İŞKURtraineesmusthaveatleastprimaryeducationandmustmeetother skill prerequisites dependingon the course; (ii) many courses aredesignedforpeoplewithmediumlevelsofeducation;(iii)intheeventofexcessdemandforcourses,trainingproviderstend to prefer individuals with higherlevels of education;2 and (iv) moreeducatedjobseekersaremorelikelytoapplyforİŞKURtraining.3
İŞKUR trainees engage in little jobsearch, although most of those who
1- DataonurbanjobseekersarefromtheLaborForceSurvey.
2- Amongotherthings,moreeducatedindividualsareperceivedtobemorelikelytocompletetraining,whichiswhatdeterminespaymenttoproviders.Until2008,paymenttoproviderswastiedtojobplacementtargetsandmoreeducatedpeopleareperceivedtodobetterinthelabormarketregardlessoftheeffectivenessoftraining.
3- Thiscouldbebecausethecoursesaremoreattractiveforbettereducatedjobseekers,theyaremoreinformedaboutthem,ortheyexpecttogetahigherreturnfromthem(althoughasshownlater,wefindlittleevidenceofthelatter).
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS ix
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS10
do tend to use and value İŞKURemploymentservicestohelpthemfindajob.Overthelastfourweekspriortotheface-to-facebaselinesurvey(beforecourses started), only about half ofİŞKUR trainees not currently workingor going to school actively searchedfor a job.Interestingly, İŞKUR traineesare less likely to be looking for a jobthan urban jobseekers, even thoughany jobseeker registered in İŞKUR issupposed to look fora job inorder tobenefit from İŞKUR services. Traineesthatdosearchdevotesignificanttimetoitmostlylookforhigh-paying,full-timewageemployment.About74percentoftraineessearchingforajobuseİŞKURservices.. In particular, İŞKUR servicesareregardedasthesecondbestchanneltofindwageemploymentafterapplyingdirectlytoemployers,andcomebeforethenetworksoffamilyandfriends.
İŞKUR trainees attach a great exantelabor market value to İŞKUR training,evenrelativetoothertrainings,becausethey perceive it as being of qualityand valued by employers. Before courses start, 80 percent of traineesbelieve İŞKUR training will help themdo better in the labor market. Theexpected impacts after oneyear arelarge: trainees expect İŞKUR trainingtoincreasetheirprobabilityoffindingajobby32percentagepointsrelativetonotrainingandby14percentagepointsrelative to other trainings. Almost all
traineesthinkİŞKURtrainingimprovesjob-readinessskillsandtheknowledgeof the profession and are confidentabout the quality of İŞKUR training,particularly if provided by publicinstitutions. Almost all trainees also find trainingusefulbecause they thinkemployersvalue the İŞKURcertificate.Despite thegenuinevalueattached toİŞKURtraining,forone-thirdoftraineesthe stipend received is an importantreasonfortakingthetraining,andtwo-thirdswould not take it if they had topayforit.
Impacts of İŞKUR vocational training under review
The overall impact of İŞKUR trainingcourses considered under this studyon employment is negligible, butcourseshavea small but significantimpact on the quality of employment,withlittlevariationofimpactacrossage,gender, or level of education. İŞKURtrainingcoursesconsideredunderthisstudyare found tohavenosignificantimpactonthelikelihoodofworking,thenumberofhoursworkedperweek,orthemonthlyincomereceived.However,coursesare found to have a small butstatistically significant impact on thequality of employment. In particular,İŞKURtrainingincreasestheprobabilityofworkingintheformalsectorbyupto3percentagepoints,whichisequivalentto a 10 percent increase in formal sector
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMSx
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 11
employment. And training increasesincomefromformalemployment(byupto13percent)andoccupationalquality.4 Theimpactoftrainingonemploymentappears larger formen over 25 yearsofage,butthedifferencesbyageandgenderarenotsignificant,norare thedifferences by level of education.Thesmall impacts of İŞKUR training onemploymentareinlinewithfindingsforsimilarprogramsaroundtheworld.
The actual impacts of İŞKUR trainingunder review differ significantly fromthe expectations formed before thetrainings, which may in part indicatelimited information about İŞKURtrainings. On average, individualsare reasonably close in terms of theirexpectations of what employmentlevels will be like in the absenceof training. However, individualssubstantiallyoverestimate thebenefitsfrom training.Thiscouldbeexplainedbythenaturaltendencytooverestimatefuture benefits and underestimate future costs of action taken today,or changes in context from the timewhen expectations were formed. Itmay also suggest, however, that theaverage trainee does not have muchinformationabout trainingsofferedbyİŞKURandhencethetrainingmaynotbe sufficiently targeted. Interestingly,thehighevaluationofİŞKUR’sservices
persists even after the training iscompleted, and even though for theaveragetraineethetrainingmakeslittledifference in terms of employment orearnings. Possibly, İŞKUR’s serviceshaveanintrinsicvaluenotcapturedbyactualemploymentstates.
Why does training have a limited impact on employment and what types of training work better?
Thesmall impactofthetrainingunderreviewdoesnotseemtobeexplainedby the labormarketcontext.Althoughat the time of training there was awide variation in unemployment ratesacross the 23 evaluation provinces,trainees from high unemploymentprovinces did not do better or worse atfindingemploymentthanthosefromlow unemployment provinces. Thisresult suggests that the small impactof training on employment does notappear to be driven by the buoyantlabormarketat the time the follow-updatawerecollected(early2012).
LowlevelsofactivejobsearchamongİŞKURtraineesmaybeonereasonthetraininghaslittleimpactonemploymentoutcomes. Wage and employmentexpectations among trainees appeartoexceed themeanof those inactualemploymentbyaconsiderablemargin.
4- Thisisacontinuousmeasurerangingfrom16(e.g.,domestichelpers)to90(e.g.,judges),andiscodedaszeroforindividualsnotworking(seeAnnex2fordetails).
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS xi
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS12
Trainees may decide not to activelyseek a job or not to take the jobson offer post-training, even if theiremployment prospects improved.Improved targeting of training and improved incentives to encourage active job search are part of İŞKUR’srecentreformsandtheevidencelendssupporttotheirimportance.
İŞKUR courses contracted to privateproviders, particularly those facingmore competition5 have a largepositive impact on employment. Theresults show that there is a higherreturn from taking a course offered by private provider or a provider thatfacesmorecompetition.Butitisreallythe combination of private provisionand intense competition (having twoor more competitors) that makesthe biggest difference: 17 percent ofthe courses included in the study fitthesecriteria,andthetraininginthosecourses increases the likelihood ofworkingfor20ormorehoursperweekby9percentagepoints.Whilesomeofthisdifferentialimpactrelativetoothertypes of training is due to differencesin the distinctive courses offered byprivateprovidersandthetraineestheyattract, the differential impact remainseven after controlling for course and
trainee characteristics. This suggests
that private provision may be one
route to ensure training courses meet
the needs of the labor market and
hence improve their effectiveness in
overcomingmismatches.
Overall, İŞKUR training courses under
review have a negligible net return,
but İŞKUR courses offered by private
providers that face more competition
have a big net return. The total cost
to İŞKUR of providing a training
course averages 2,429TLper person.
The average cost per person is 2,671
TL if the İŞKUR course is offered by
privateproviderssubject tosignificant
competition.6 The average gain in
monthlyincomefromİŞKURtrainingis
closetozero,sotheannualinternalrate
ofreturn(IRR)over30yearsisalsoclose
tozero.Incontrast,thegaininmonthly
income for İŞKUR trainees taking
courses offered by private providers
facing significant competition is 128
TLpermonth.Basedonthisestimated
gain, it would take 21months for the
gain in income to offset the costs of
provision, and the IRR over 30 years
wouldbe58percent(or48percentwith
10percentperyeardepreciationofthe
gains).
5- Thecompetitionvariableisbasedonthequestiontoprovidersaboutthenumberofcompetitorstheyface:(i)zero;(ii)one;(iii)twotofive;(iv)sixtonine;or(v)10ormorecompetitors.
6- Thetrainingcostreflectsthecostoftrainingduringthestudywasimplemented.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMSxii
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 13
Strengthening the impact of İŞKUR training and services
After thedata for thisevaluationwerecollected, İŞKUR continued expandingand introduced reforms to address some ofthechallengesidentifiedinthisstudy.Indeed, two recent reforms includingthe selection of training providerson the basis of specific quality andperformancecriteria(notjustcost)andthe introduction of job and vocationalcounselors will help address at leasttwoissuesidentifiedbythisevaluation:(i) thequalityandemployment impactof training; and (ii) the informationtrainees have about the courses theytake. In2012, İŞKURstartedhiring joband vocational counselors to advise jobseekers on occupational choiceand training courses. A new regulation approvedinMarch2013assignsmoreweight to quality in the selection ofproviders and introduces modules to the training programs to increaseemployability. The regulation alsoextends jobs placement requirementsto providers of general vocational training, and incentivizes employersto hire and keep young and femaletrainees employed in occupations forwhichtheyaretrained.
This section builds on these goodinitiativesandachievementstosuggestsome options to further streng thenİŞKUR training and services based on
theresultsoftheevaluationandlessonsfrom international experience. Reformcanonlyhappengradually,introducinga few new initiatives and evaluating thembeforemovingtothenextsetofreforms.ThisistheapproachİŞKURistakingandthisjointevaluationisagoodillustrationofthis.
Improving the relevance of skills training. The study finds that the lowoverall impactofİŞKURtrainingunderreviewonemploymentmayinpartbedue to the low value-added of thesecourses in terms of the skills theyhelp tobuild.As İŞKURdevelopsnewmodules to be added to the trainingprogram, it may consider puttingmore emphasis on behavioral skills,whicharehighlyvaluedbyemployersin Turkey (McKinsey 2012). Andthere is increasing evidence of theemploymentimpactofbehavioralskillstraining (Almeida et al. 2012). Goingforward,itisalsoimportanttocontinuestrengtheningthelinkbetweenİŞKUR,trainingproviders,andlocalemployers,building on the experience of theProvincial Employment and TrainingCouncils.
Incentivizing and supporting more job search. Thelowincidenceofjobsearchamong İŞKUR trainees in the studysuggests theneedtoencouragemorejob search and expand employmentservices (job placement, counseling,
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS xiii
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS14
job-search assistance). The newregulation linking training to jobssearch is a step in the right direction.International evidence shows that it ismore cost-effective to first encourage jobseekers to look for a job andto assist them in this task throughemployment services before theyget any training than to offer trainingfirst (as in Turkey). To encourage jobsearch,itisimportanttotiethereceiptof unemployment or social assistancebenefitstoit—Turkeydoesthisalready,but perhaps compliance could beimproved.Employmentservices,whicharelimitedinTurkey,shouldbecentralto employment activation efforts. Forexample,intheU.K.’sjobcenterplus,allregistered unemployed people areoffered some employment services(rangingfromminimumjobplacementservices to job-search assistance forthe hard-to-employ) and are requiredto take individual actions to find a job (i.e., the market test– receipt ofbenefits is conditional on that) beforeit is determined whether they needadditionalservices(includingtraining).
Defining the priority groups for training. ThelowoverallimpactofİŞKURtrainingunder review on employment couldalsoberelatedtowhoİŞKURisactuallytrainingandwhether these jobseekersaretheoneswhocanbenefitthemostfromtrainingrelativetoothergroupsofjobseekers. İŞKURis trainingthemost
educated jobseekers. Aside from theminimum requirement to have basiceducation,thisismostlytheresultofthecoursesİŞKURoffersandtheselectionofapplicantsbyproviders.Theevaluationdoes not showdifferential impacts bygenderandage,buttheseresultsrefertothecurrentprofileofİŞKURtrainees(e.g., highly educated) as well as thetraining and services İŞKUR currentlyoffers. Low-skilled workers account for most of the labor force and face thegreatest jobschallenge.Andtheyfacebarriers to productive jobs other thanskills, including information. Countrieswithwell-developedpublicemploymentservices (e.g., the U.K., Germany, andAustralia) serve all jobseekers thatregister,butthebulkoftheirresources(includingskillstraining)supporthard-to-employ jobseekers. İŞKUR hasrecently started to serveone segmentof the hard-to-employ, namely thosereceiving social assistance benefits and able to work.
ThefocusofİŞKURtrainingsonwomenand youth seems a priori appropriatefrom a policy perspective. Activityrates are especially low for womenand youth. International evidenceshows that well-designed skills trainingprogramshaveahigherreturnforyouththan others (Betcherman et al. 2007),mainly because it is easier to learnwhenyoung.Successfulprograms foryouthintheU.K.,theU.S.,andseveral
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMSxiv
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 15
Latin American countries (e.g., theJovenesprogram)targetdisadvantagedout-of-schoolyouth(typically15-to29-year-olds with less than a secondaryeducation). And in countrieswith lowfemale employment, like Colombia(Attanasio et al. 2011) and theDominican Republic (Ibarraran 2012),well-designedprograms foryouthcanhavealargepayoffforyoungwomen.
Better information on jobseekers to adjust services to needs.Thenew joband vocational counselors hired byİŞKUR will most certainly result in abetter match of trainees to courses(a possible problem identified in thisevaluation),buttheydonotassesstheemployability of jobseekers and thustheirneedfortraining(orotherservices)to begin with. Countries with well-developedpublicemploymentserviceslike the U.K. and Australia do makean initial employability assessmentof jobseekers, which is then usedto “profile” jobseekers into differentgroupsreceivingdifferentemploymentsupport packages, with the bulk ofresourcesgoingtothehard-to-employ.
More contracting of services to the
private sector while ensuring quality.
The high return of İŞKUR courses
offeredbyprivateprovidersfacingmore
competition suggests that increasing
the share of courses subcontracted
to private providers and increasing
competition among them would
significantly increase the employment
impactofİŞKURtraining.However,itis
alsoimportanttoensurethequalityof
providersandmakethemaccountable
for results. İŞKUR has already taken
a number of measures to increase
the quality of providers through the
selection process. İŞKUR has also
recently extended jobs placement
requirements to providers of general
training.Thereisstillroomtostrengthen
thecontractswithprivateproviders to
improve the impact of training. And
to avoid “creaming” off the easy-to-
employ by private providers, and to
take into account the higher cost of
helping the hard-to-employ become
employed, the contracting of services
forthemcouldbedoneseparately,asis
doneintheU.K.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS xv
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS16
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 1
SECTION 1: THE CONTEXT: JOBS, SKILLS UPGRADING, AND İŞKUR
1. Turkey’s labor market continues to
be characterized by low employment
rates—particularly amongwomenand
youth—and low labor productivity.
Despite the remarkable upturn in
employment after the crisis, still less
thanhalfoftheworkingagepopulation
(15-to64-year-olds)wereemployedas
ofmid-2012and theemployment rate
among working age women was under
30percent.About35percentofyouth
(15- to 24-year-olds), mostly women,
are neither working nor attending
school—the highest share of inactive
youth among OECD countries. Job
informality (defined as jobs without
socialsecuritybenefits)hascomedown
significantlysinceitwasfirstmeasured
in2005,butitstillaffected42percentof
workers in2011(28percentexcluding
the agricultural sector), contributing
to Turkey’s lower labor productivity
compared with the OECD and other
peercountries(WorldBank2013a).
2. Thelarge“stock”oflow-skilledworkersisakeystructuralfactorbehindthejobschallenge inTurkey.New entrants into the labor force are quickly becomingmoreeducated:theproportionof25-to34-year-oldswithatleastahighschooleducation increased from 26 percent in2000toabout41in2010(Figure1).These remarkable changes, however,areslowtoaffectthestockofeducationin the labor force:half of theworkingagepopulation(WAP)stillhaslessthana basic education. As firms strive to staycompetitiveinaglobalmarket,the“skills bar” in formal non-agriculturalsectorsisincreasing—skillsarealreadythe thirdmost important constraint tobusiness operations cited by Turkishfirms.7 The large supplyof low-skilledworkers and the decreasing demandfor them are the main forces behindthepoorjobperformanceofthisgroup,which accounts for 64 percent of thejobless and 65 percent of informalworkers.
7-BusinessEnvironmentandEnterprisePerformanceSurvey(BEEPS)for2008.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS2
3. In addition, there are indications ofamismatchbetweentheskillsobtainedineducationbyyoungworkersandtheneeds of the labormarket.According to a recent comparative study byMcKinsey(McKinsey,2012)8,56percentofemployersinTurkeysaytheycannotfind workers with the right skills andthis despite the fact that more thanonethirdofyouthareneitherworkingnorattendingschool.Themainreasonbehind this apparent paradox is themismatch between the skills suppliedby youth and those demanded byemployers. According to the samestudy,youthinTurkeyoftendonotgoto collegebecause theybelieve it haslittle valueadded.This is validatedby
the experience of those who actuallywenttocollege(54percentthinkcollegeeducation did not improve their jobopportunities) and by actual data (thereturntocollegeeducationinTurkeyisamongthelowestintheOECD).Finally,theskillsthatemployersacrossTurkeyvaluethemostarebehavioral(e.g.,workethics, teamwork, and communicationskills),not technical.This isconsistentwith findings for the other countriesincluded in the study, which highlightthat as countries move up the valuechainhigher-levelcognitiveskills(e.g.,problemsolving,communication),andbehavioral skills (e.g., perseverance,self-discipline, teamwork) becomeincreasinglymoreimportantthatbasic
8- Theother7countriescoveredincludeBrazil,Germany,India,Mexico,Morocco,SaudiArabia,theU.K,andtheU.S.
FIGURE 1
Moreeducatedentrantsbutlarge“stock”oflow-skilledworkers
New entrants are more educated(%of25-34-year-oldswithatleastahighschooleducation)
But there is a large “stock” of low-skilled workers(EducationlevelsoftheWAPin2010,bygender)
Source:TUIK(LaborForceSurvey),UN(urbanization,populationprojections),andauthors’calculations.
LessthanbasicBasicSecondaryCollage
45
40
35
30
25
20
15
10
5
02000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
60
50
40
30
20
10
0
52
43
56
1922
1619
23
16
1012
8
Total Men Women
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 3
cognitive (numeracy and literacy) andtechnicalskills.
4. The government has launchedsignificant education reforms, withsignificant achievements already interms of increased coverage and student learning.9 But the impact ofthese education reforms will onlymaterialize in the long run, while thegrowthpotentialoftheTurkisheconomyis hampered by the large segment ofthecurrentlaborforcethatisinneedofskills upgrading.
5. The Turkish Employment Agency(İŞKUR)playsakeyroleinupgradingtheskillsof jobseekers.Skills upgrading is relevant for existing workers, throughon-the-job training (OJT), and forjobseekers. Despite being one ofthe countries where employers aremore concerned about the skills ofthe workforce, Turkey is among thecountrieswith the lowest incidenceofOJT:only about 29percent of firms inTurkeyprovideOJTtotheiremployees(World Bank 2010). İŞKUR is themainproviderofskillstrainingforjobseekers.Vocational training is one of İŞKUR’smain active labor market programs (ALMP) to help jobseekers findemployment.İŞKURisalsoresponsibleforotherALMPs (employmentservices,on-the-job training, entrepreneurshipservices, public works) as well asfor passive labor market programs
(unemployment insurance, short-timework schemes, employment incentives,andwageguarantees).
6. TheincreasingimportanceofİŞKURvocational training prompted thegovernmenttocommissionthepresentevaluation. As part of the 2008 laborreform, participation in ALMPs by allregisteredjobseekerswasfullyfundedbytheUnemployment InsuranceFundregardless of they are eligible forunemploymentinsurance.Thenumberof vocational trainees increased from 30,000 in 2008 to 464,000 in 2012,representing 19.6 percent of registered jobseekers. Aware of the importanceof İŞKUR training to improve theemployability of jobseekers, theleadership of the Ministry of Laborand Social Security (MoLSS) andİŞKUR partneredwith theWorld Bankto evaluate İŞKUR training programsin order to inform the expansion andimprovement of these programs.The present study evaluates generalvocational trainingcourses (seeBox1for a description of these programs),which is the main type of trainingofferedbyİŞKUR.Theevaluationdoesnot cover recent reforms but some of itsmainfindingsarereflectedinthesereforms. Moreover, as a result of thejoint evaluation İŞKUR’s capacity tocontinuouslymonitor and improve itsportfolioofALMPhasbeenenhanced.
9- Turkeyhasvirtuallyachieveduniversalprimaryeducationandincreasednetsecondaryschoolenrollmentto67percent,whileatthesametimerecordinganimpressivehalfayearofschoolgaininPISAscoresbetween2003and2009(WorldBank2013b).
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS4
BOX 1: İŞKUR Vocational Training Programs
İŞKUR provides two basic types of training for the unemployed: general training,whichaccountsforabouttwo-thirdsofcoursesandbeneficiariesandisthesubjectofevaluationunderthisstudy;andjob-guaranteedtraining.ThelatterinvolvesemployersapproachingİŞKURandrequestingthetrainingofacertainnumberofnewemployeesinaparticularfield.Inthiscase,employersareexpectedtoofferjobplacementstoatleasthalfof the trainees.Theevaluation focusesongeneral trainingcourses,wherethereisnojobplacementrequirement.
However,approximately80%of the2012courseswere jobguaranteedcourses.Theactive labormarketservices regulationwasamended in2013 to increase the rateofjobguaranteedcourses.Inthisrespect,itisexpectedthatthe2013courseswillhaveasimilar rate.
Generaltrainingcourseshaveanaveragelengthofthreemonths,coverawiderangeof fields, and are planned through the İŞKUR provincial offices through the year.Trainingcoursesareeithercontracteddirectlytopublicinstitutions(MinistryofNationalEducation) (about 50percentof trainees)or areopen to competition amongprivateproviders.Trainingparticipantsreceiveastipendof15TLperday(20TLfrom2012)tocovertheindirectcostsofattendingtraining.
Tobeeligible toparticipate in the course, individualsmustbeat least 15 yearsold,haveat leastprimaryeducation,andmeetotherskillprerequisiteswhichdependonthecoursetheywishtoparticipatein(forexample,softwarecoursesmayrequiresomepre-existingITknowledgeorskills).Givenexcessdemandforcoursesandadesiretogettheunemployedintojobs,individualsareonlyallowedtotakeoneİŞKUR-supportedcourse ina24-monthperiod.Coursesareadvertisedandpotential traineesapply tothem.Applicationsarethenscreenedtoensureapplicantsmeettheeligibilitycriteria.Iftherearemoreeligibleapplicantsthantrainingseats(i.e.,ifthereisexcessdemand,asisthecasewithmostİŞKURcourses),thetrainingproviderinterviewscandidatesandchoosesthosebestsuitedforthecourse.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 5
SECTION 2:THE EVALUATION OF İŞKUR TRAINING PROGRAMS: DESIGN, DATA, AND METHODS
Design
1. The study makes use of anexperimental design to evaluatethe main type of İŞKUR’s vocationaltraining courses at a particular point in time. The study evaluates the impactof general vocational training courses using an experimental design thatexploits the excess demand for mostİŞKURvocationaltrainingcourses.Theevaluation focuses on courses thatwereoversubscribed,startedbetweenOctoberand December of 2010, andwere completed by June 2011. Theevaluationwasdesignedtoanswerthefollowingmainquestions:
• Whatistheaverageimpactoftrainingonemployment?
• Whichtraineesbenefitthemostfromtraining?
• Whatarethechannelsthroughwhichtrainingaffectsemployment?
• Whattypesoftraininghaveahigherimpactonemployment?
2. Evaluation provinces.Thegoalwastoensureabroadgeographicdistributionand range of labor market conditions. Theselectionofprovincesbeganwitha list of the 39 provinces that had atleast two significantly oversubscribedtraining courses in 2009. Theseprovinces were first stratified bywhether their unemployment ratewasaboveorbelowthemedianof10percent in 2009. Ten provinces were then randomly selected from eachstrata with a probability proportionaltothepercentageofindividualstrainedin 2009. Three additional provinces(Antalya, Gaziantep, and Diyarbakir)were included in the sample at therequest of İŞKUR because of theirimportance in representing varyinglabormarketconditionsacrossTurkey.Asaresult,23provinceswereselectedfor inclusion in the evaluation (Figure2).
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS6
3. Evaluation courses. The targetsample size of individuals needed to estimate the impact of training wascalculated as 5,700. This target wasdivided amongst the 23 provinces inproportion to the number of traineesintheseprovincesinthepreviousyear.ThusIstanbulaccountsfor21.8percentof the sample; Kocaeli, Ankara, andHatayprovincescollectivelyaccountforanother28percent;andtheremaininghalf of the sample is split among theother 19 provinces. The key criteriausedtodecidewhichcoursestoincludeintheevaluationwere:(i)thelikelihood
of the course being oversubscribed
(which ensures that themost popular
typesoftraining,forwhichtherewould
be demand for further scale-up, are
included); (ii) inclusion of a diversity
oftypesoftrainingproviderstoenable
comparison of private and public course
provision;and (iii)coursestartingand
endingdates, i.e.,courses thatstarted
between October and December 2010
andfinishedbyJune2011,asthistends
to be a timeof yearwhen jobseekers
aremorelikelytoseektrainingthrough
İŞKUR.
FIGURE 2
Geographicdistributionofprovinces
Note:Thenamesofevaluationprovincesareindicatedintheorangerectangles.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 7
4. This resulted in a set of 130evaluation courses spread throughoutTurkey, of which 39 were offered byprivate providers and the rest weregovernment-operated.Thesinglemostcommon course was computerized accounting, for which 24 percent oftrainees applied. Twenty-one percentof trainees were in service courses (babysitter,cashier,waiter,caringfortheelderly); 15.4 percent were craftsmanor machine operators (welder, naturalgas fitter, plumber, mechanic); 14.7percent were in technical courses(computer technician, computer-aideddesign,electricalengineering);and12.2percent were in professional courses (webdesigner,computerprogrammer,IT support specialist). The averagecourse size was 28 trainees, and theaverage course length was threemonths(typicallyaroundsixhoursperday),bothwithsignificantvariation.
5. Evaluation participants. Courses were advertised and potential trainees applied tothemfollowingstandardprocedures.Applications were screened to ensure applicantsmet the eligibility criteria ofİŞKURandthecourseprovider.Trainingproviderswere then asked to select alistofpotentialtraineesthatwasatleast2.2timesthecoursecapacity.Typicallythis involved short interviews witheligible applicants. These individuals’applicationdetailswerethensubmittedinto İŞKUR’sManagement Information
System (MIS). The MIS stratifiedapplicants for each course by genderand age (i.e., whether or not theywere less than 25 years old). Withinthese strata, trainees were randomlyallocatedbytheMISintothreegroups:atreatmentgroupthatwasselectedfortraining,acontrolgroup thatwasnot,andawaitlistedgroupthatthetrainingprovidercouldselectintothetrainingifthereweredropouts.Thefinalevaluationsample consisted of 5,902 applicants,ofwhich3,001wererandomlyassignedtotraining(treatmentgroup)and2,901werenot(controlgroup).
6. Compliance with treatment. Not all those accepted into the course(treatmentgroup) tookup the training(23 percent). Of those who did, mostcompletedthecourse(72percent)andreceived a certification (69 percent).About 3 percent of the control groupactually received a certification in anİŞKURcoursethatwasnotincludedinthesampleofevaluationcoursesduringtheperiodthatevaluationcoursestookplace.Sincepartialtrainingisunlikelytobeofinteresttoemployers,weconsiderthose who received a certification tohavereceivedtreatment.
Data
7. TheMIScontainedbasicinformationaboutthecourseandthegender,age,and education level of applicants. The
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS8
maindatafortheevaluationcamefromsurveysadministeredtotheevaluationparticipants between September 2010 andJanuary2011(baselinesurvey)andapproximately one year later (follow-up survey). Data were also collectedfrom a survey of training providersparticipatingintheevaluation.
8. Thebaselineand follow-upsurveyswere conducted through in-personinterviewsbyathird-partyprofessionalsurvey firm. The baseline surveytook place on a rolling basis between September 13, 2010, and January 31,2011.10 The goal was to conduct thesurveys before courses began, butgiventheshortwindowoftimebetweenselectionofapplicantsandthestartofthecourse,inpracticeonlyone-thirdofthosesurveyedwereinterviewedbeforethestartofthecourse,while46percentof those surveyed were interviewedwithin11daysofthestartofthecourse.Theoverallbaselineresponseratewas90 percent.
9. The follow-up survey took placebetween December 27, 2011, andMarch 5, 2012, which corresponds toaperiod approximatelyone year afterthe end of training. It collected dataon employment outcomes, aswell as
individual and household well-being.The response rate was 94 percent,including472individualswhowerenotinterviewedatbaseline. In total, 5,057individuals were interviewed at bothbaselineandfollow-up.Wefoundverysmall differences between the non-response rates (to the baseline andfollow-upsurveys)ofthetreatmentandcontrol groups.
10. The randomization of evaluationparticipants into treatment and control groupswassuccessful,whichvalidatesthe impact evaluation strategy. Theevaluation strategy (see below formoredetails) involves thecomparisonofemploymentoutcomesbetweenthetreatment and control groups and theattributing of any difference to İŞKURtraining, and İŞKUR trainingonly.Thisstrategy relies on the assumptionthat the treatment and control groupswere statistically equivalent (i.e., theyhad similar average demographiccharacteristics).Thesummarystatisticsforthebaselineandfollow-upsurveysshow that the differences in terms ofdemographic characteristics betweenthe treatment and control groupsinterviewedatbothbaselineandfollow-up were small and not statisticallysignificant(seeTableA1inAnnex3).
10-ApplicantsweretoldthatthepurposeofthesurveywastohelpimprovetheservicesofferedbyİŞKUR,andthattheirparticipationinthesurveyhadnoimpactonbeingacceptedintoanytrainingcourse,norwouldtheirindividualdatabesharedwithanybody.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 9
Methods
11. Estimating the average impacts ofvocational training.Theaverageimpactof vocational training on employmentis estimated using two alternative methods. The first measure, theintent-to-treat effect (ITT), comparesthe average outcomes of individualsassigned to training (treatment) andthosenotassignedtotraining(control)regardless of whether people in thetreatment group actually completedthe training. This comparison alsocontrolsforthefactthattherandomizedassignment of individuals to treatment and control groups was done withingroups of individuals applying to thesame course and belonging to thesame gender and age group (called randomizationstrata).Itisimportanttonote that inbothcases, theestimatedimpactreferstothepeoplethatapplytoandareeligibleforİŞKURtraining,nottothelargerpopulationofjobseekers.Assuch,thestudyevaluatestheimpactofİŞKURtrainingasitisactuallydesignedand implemented.
12.Theotherrelevantmeasure,calledthe local average treatment effect(LATE),comparestheaverageoutcomesof people who completed training tothosewhodidnot.However, to avoidthebiasarisingfromthefactthatpeoplewho are more likely to complete thetrainingarealsomorelikelytodowell
inthelabormarket,trainingcompletionis instrumented in the estimation bytraining assignment (see Annex 1 fordetails). As such, this measure triesto capture the impact of completingtrainingforanindividualwhotakesuptrainingwhenheisassignedtotraining.It assumes,however, that traininghasno impact on employment for thosewhodonotcompletethecourse,whichmay not always hold. This estimationmethodology is subject to the samecontrols as the other one. Both ITTand LATE average impact results are reportedbelow,althoughotherresultsarebasedonITTgiventheassumptionsunderlyingLATE.
13. Outcomes of interest. The studyfocuses on the impact of vocationaltraining on employment through arange ofmeasures, includingwhetherindividualsareemployedatall,aswellashowmuchtheyareworking,howmuchtheyearnfromthiswork,thequalityandformality of this employment, and thecompositeindexofallthesemeasures(the aggregate employment index).The study also looks at the impact ofvocational training on measures of individual and household well-being.Annex2explainshowthekeyvariableswere constructed.
14. Investigating differences in impacts among different groups of individuals. The study also looks at how impacts
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS10
vary across individuals with differentcharacteristicstoanswerthequestion:Who benefits themost from training?The variation in impacts is estimatedby interacting the relevant treatmentvariables (assigned training and completed training) with differentindividual and course characteristics(seeAnnex1fordetails).
15. Understandinghowtrainingaffectsemploymentoutcomes.To understand how training could bemore effective,the study looks at four intermediatesteps inacausal chain throughwhichwemight expect to see selection intoa course influencing employmentoutcomes:
(1) Individuals selected for courses must show up and complete training. Thishypothesis is testedby interacting treatment with thepercent of individuals assigned to acoursewhoattendedthecourseorwhocompletedit.
(2) Higher quality courses should have more impact. The basicquestionis:Whattypesoftrainingworkbest?This is investigatedbyinteracting treatment with coursecharacteristics potentially relatedtoquality,includingcourselength,trainer’seducationandexperience,whether the course provider ispublic or private, and the degree
of competition faced by courseproviders.
(3) Skill acquisition, signaling, or job matching. A third step in thecausalchainisforindividualswhotakecoursesofsufficientqualitytousewhattheyhavelearnedinthecoursetofindjobs.Therearethreemain channels through whichvocational educationmay help inthisrespect.First,itmightincreasehuman capital by teaching newtechnical skills. Second, it mayact tocertifyskills that individualsalreadyhaveandactasasignalingmechanism to employers. Third,it may teach individuals newstrategies for finding jobs ina certain profession, or betteralert them to job opportunities,thereby improving job matching.To examine the extent to whichcoursesareplayingeachof theseroles,andwhichtreatmenteffectsvary with them, the follow-upsurvey asked course participantswhether they thought the coursehad done each of these threethings. The percentage of courseparticipantswho thought that thecoursetaughtnewtechnicalskills,certified existing skills, taughtnew strategies for finding jobs,ormade themmoreawareof jobopportunities was then interactedwithtreatment.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 11
(4) Training impacts and unemployment rates. The finalstep in the causal chain is forindividuals who receive trainingtoobtainjobsthattheywouldnototherwise get. This depends onthe labormarket they face.Whenunemploymentratesarelow,thereshouldbemore jobopportunitiesavailable, making it easier forpeople to use their training tofind jobs. But it is also possiblethat firms facing labor shortagesmight hire workers regardless ofwhether or not they have beentrained. When unemploymentrates are high, if employers arenot hiring, then having new skills
may not help the unemployedfind jobs. But it is also possiblethat employers become choosierandsotrainingmayofferworkersa way to distinguish themselvesfrom other workers competingforthesamejobs.Asaresult,itistheoretically ambiguous whetherweshouldexpecttrainingtohavemore or less impact in situations ofhigherorlowerunemployment.This hypothesis is investigatedby interacting treatment with anindicatorforwhethertheprovincewhere the coursewas conductedhadanunemploymentrateabovethemedianunemploymentrateforthe23provincesunderthestudy.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS12
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 13
SECTION 3: THE EVALUATION OF İŞKUR TRAINING PROGRAMS: RESULTS
Profile of İŞKUR trainees under the study in the sample before courses started
1. This sectionmakes use of the datafrom the baseline survey to look at:(i) the characteristics of evaluationparticipants (İŞKUR trainees) andhow they compare with those ofjobseekers in urban areas;11 (ii) thejobsearchbehaviorofİŞKURtrainees;and (iii) trainees’ assessment of andexpectations about İŞKUR services(includingtraining)beforethestartofthecourses.Themaingoalsofthisanalysisareto:(i)determinewhoreceivesİŞKURtrainingamong the largerpopulationofcomparable jobseekers,which can alsohelptobetterunderstandtheimpactoftrainingonemployment;and(ii)analyzetheextentandnatureofthedemandforİŞKURvocationaltraining.
2. Most İŞKUR trainees are womenandyouth.About63percentofİŞKURtraineesarewomen.Twelveoutofthe130coursesintheevaluationonlyhavefemale trainees (babysitting, weaving,
haircare,andsewingmachineoperator)and17courseshavenofemaletrainees(applied basic electronics, welder,plumber,furnituremanufacture,naturalgasfitter,andforkliftoperator).Womenalso comprise themajorityof traineesin a wide range of other courses,including computerized accounting,computer operator, foreign trade andcustoms professional, computer-aideddesign, computer network design,salesperson, and web designer. Theaverage age among İŞKUR trainees is27years,and60percentarebetween20and29yearsof age.Male traineestend to be somewhat younger thanfemale trainees (26 years old versus28).Partlyasaresultoftheiryouth,only12 percent of trainees are householdheadsand34percentaremarried.
3. Most traineeshave completedhighschool.About 76 percent of men and 73 percent of women trainees havecompletedatleastsecondaryeducation,and 30 percent of men and women trainees have completed at least twoyearsoftertiaryeducation(Figure3).This
11-Thecomparison iswith jobseekers inurbanareasbecause İŞKURtrainingcoursesare restricted tourbanresidents.DataonurbanjobseekersarefromtheLaborForceSurvey.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS14
higheducationalattainmentispartlyduetotheageprofileofİŞKURtrainees,butolder cohorts are also well educated(65percentofmenand59percentofwomen34yearsorolderhaveatleastahighschooleducation).Mosttraineeshave some degree of specializationor technical background. The mostcommon technical backgrounds arecomputer technologies/programmingand accounting. About one-third of
trainees have a background in thefield of training they apply to, whileothers have general degrees (e.g.,business) and are interested in İŞKURtrainingtoacquiresomespecialization.Interestingly, 20 percent of men and29 percent of women trainees havealreadytakenvocationaltraininginthepastfiveyears,mostlyfromİŞKUR(47percent), and in areas closely relatedtothetrainingtowhichtheyappliedatbaseline.12
FIGURE 3
HigheducationalattainmentamongİŞKURtrainees(Distributionoftraineesbyhighestlevelofeducationcompleted,inpercentages)
12- BecauseİŞKURtraineescannotenrollintrainingsiftheyhavetakenacourseinthelasttwoyears,mostofthetrainingstendtohavebeencompletedmorethantwoyearsago,between2008and2010.
Source:Baselinesurveyofevaluationparticipants.
Men Women Total
Primary or less Lower sec. High school Voc HS College
35
30
25
20
15
10
5
0
8
16
1113
30 29 29
18
1315
2931 30
1613
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 15
4. İŞKUR trainees have limited priorexperience. About 37 percent of trainees have never worked before. Amongthose with prior work experience, 20percentworkedfor less thanoneyear(averageexperienceisfiveyears).Menare significantly more likely to haveworkedbeforethanwomen(71percentversus 53 percent). The limited workexperienceoftraineesispartlyexplainedby the age and gender compositionof trainees. More than 90 percent ofpeoplewithpriorexperienceworkedasprivatesectoremployeesandabout58percent had social security coverage.Traineesworked on average 54 hoursper week and received an average of TL 677permonth.
5. İŞKUR trainees are significantlyyounger, have less work experience,and are more educated thanjobseekers in urban areas.13 Women are overrepresented among İŞKUR
traineescomparedto theunemployed
butnotrelativetothe jobless(defined
asneitherworkingnorgoingtoschool)
(Table1).İŞKURtraineesaresignificantly
youngerandhavelessworkexperience
than the average unemployed or
jobless individual. Interestingly, İŞKUR
traineesarelesslikelytobelookingfor
a jobthantheaverageunemployed in
urban areas, and the incidence of job
informality among İŞKUR trainees (for
their most recent job) is higher than
that for the average wage employee
(42 percent versus 24 percent). The
most striking difference is that İŞKUR
traineesaresignificantlymoreeducated
thantheaverageunemployed,jobless,
or even employed individual in urban
areas.And this is not only due to the
younger profile of İŞKUR trainees,
as the difference remains even after
limitingtheurbansampleto20-to29-
year-olds.
13- ThecomparisonwithjobseekersisrestrictedtourbanareastomakeitcomparabletoİŞKURtrainees.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS16
6. The significant difference ineducationalattainmentbetweenİŞKURtrainees and the average jobseeker inurban areas could be explained by anumberof factors: (i) İŞKUR traineesmusthaveatleastprimaryeducationandmeetotherskillprerequisitesdependingon the course; (ii) many courses aredesignedforpeoplewithmediumlevelsofeducation;(iii)inthecaseofexcessdemandforcourses,trainingproviderstend to prefer individuals with higher
levels of education;14 and (iv) moreeducatedjobseekersaremorelikelytoapplyforİŞKURtraining.15
7. Only half of İŞKUR trainees wereactivelysearchingforajobatbaseline. Over the last four weeks prior to theface-to-face survey, only about half ofİŞKURtraineesnotcurrentlyworkingorgoingtoschoolwereactivelysearchingforajob.Theshareofapplicantsactivelysearching increases significantly with
TABLE 1
ProfileofİŞKURtraineesrelativetotheunemployedandthejoblessinurbanareas(Percentages)
Note:Joblessisdefinedasnotworkingandnotgoingtoschool.
Source:Data for theunemployedand joblessare fromthe2010LaborForceSurvey(urbansample).Data for İŞKURtraineesarefromthebaselinesurvey.
14- Amongotherthings,moreeducatedindividualsareperceivedtobemorelikelytocompletetraining,whichiswhatdeterminespaymenttoproviders.Until2008,paymenttoproviderswastiedtojobplacementtargetsandmoreeducatedpeopleareperceivedtodobetterinthelabormarketregardlessoftheeffectivenessoftraining.
15-Thiscouldbebecausethecoursesaremoreattractive,theyaremoreinformedaboutthem,ortheyexpecttogetahigherreturnfromthem(althoughasshownlater,wefindlittleevidenceofthelatter).
Female
Married
Aged 15-24
High school or more
High school or more (20-29)
Worked before
Looking for a job in last 4 weeks
All Men Women All Men Women All Men Women
İŞKUR trainees Unemployed (urban) Jobless (urban)
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 17
8. Trainees that do search devotesignificanttimetoitandlookforhigh-paying, full-time wage employment.Although women are less likely tosearchforjobsthanmen,thosewhodotendtodevotemoretimetoitthanmen
the level of education (Figure 4). Thisshare also varies significantly acrossprovinces: Bayburt andMuş have thelowestratesofjobsearch(2percentand11percent,respectively),whileDüzce,Tekirdağ, and İzmir have ratesabove70percent.Menaresignificantlymore
(15hoursperweekversus10hoursperweek).Mostjobseekersarelookingforfull-time wage jobs (80 percent). Thereservationwage of İŞKUR trainees ishigh: only 38 percentwould accept amonthlywageofTL1000(theaverage
likelytobelookingforajobthanwomen(61 percent versus 42 percent), adifferencethatremainsaftercontrollingfordifferencesineducationlevel,whichmay reflect in part the greater timeconstraintthatwomenfaceasaresultoftheirhouseholdresponsibilities.
FIGURE 4
LimitedjobsearchamongİŞKURtrainees(%activelylookingforajobinthelastfourweeks,bygenderandlevelofeducation)
Source:Baselinesurveyofevaluationparticipants.
Primary or less Lower sec. High school Voc HS College
Men Women Total
80
70
60
50
40
30
20
10
0
72
55
32
43
53
3945
57
4249
6358 60
29
39
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS18
monthlywageinTurkey in2010),withno significant difference by level ofeducation. About 40 percent would not acceptajobwithoutsocialsecurity(20percent among those with less thanhigh school education) and, amongthose who would, their reservationwagewouldbeTL1800permonth(withno significant differences across levels ofeducation).Anumberoffactorsmayexplain the high aspirations of İŞKURtrainees,includingtheirhigheducationalattainment, family background (theycome from families with significantlyhigher per capita household incomethantheaverageunemployedinurbanareas),andworkexperience (onlyhalfof jobseekers come from a previousworkrelationship).
9. Most jobseekers use and valueİŞKURservicestohelpthemfindajob.Although the most common channelfor job search is family and friends,
the second most popular channelis İŞKUR, which is used by mostjobseekers (74 percent), followed bydirect contact with employers (Figure5).16Only12percenthaveusedprivateemploymentagencies,reflectinginparttheirlimitedavailability.Theinternetisalso frequentlyusedand, toa smallerextent, newspaper advertisements.İŞKURtendstobelessfrequentlyusedbywomenthanmen.JobseekerswithaprimaryeducationorlesstendtorelymostheavilyonİŞKURservices,whilecollegegraduatesuseittheleast.Thereare differences in the use of İŞKURservices across provinces,whichmayreflect inpart theavailabilityof İŞKURservices.17 Most jobseekers believethat applying directly to employers isthe best channel to find a wage job,with İŞKUR ranking immediatelyafter:28percentofjobseekersviewİŞKURasthebestchannel tosearch forawagejob.
16- Notethatmultiplechannelscanbeused.
17- In Düzce, Gaziantep, Hatay, and Tekirdağ,more than 92 percent of jobseekers use İŞKUR,while inMuş,Eskişehir,andElazığ,thepercentageisbetween30percentand40percent.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 19
10.İŞKURtraineesattachagreatexantelabor market value to İŞKUR training,even relative toother trainings. About 80 percent of trainees believe İŞKURtrainingwillhelpthemdobetterinthelabor market. The expected impactsafteroneyearare large:relativetonotraining,traineesexpectİŞKURtrainingtoincreasetheirprobabilityoffindingajobby32percentagepointsonaverage(Table2)and to increasewagesby39
percent. The expected impacts arelarger for women than for men, buttherearenosystematicdifferencesbyage. Interestingly, trainees with lessthan secondary education completed,whomake up the minority of İŞKURtrainees,havehigherexpectationsthantrainees with higher education levels.And the expected impact of İŞKURtrainingissignificantlyhigherthanthatforothertypesofvocationaltraining.18
18-Thereisvariationacrossprovinces(traineesinAnkara,Antalya,andIspartahavethehighestexpectedimpacts)andcourses(traineesinwaiter,cook,nurse,andhairdressingcourseshavethehighestexpectedimpacts).
FIGURE 5
İŞKURservicesareusedbymostjobseekers(%useofdifferentjobsearchtools,bygender)
Source:Baselinesurveyofevaluationparticipants.
Employer ISKUR Friends and Private Printed ads Internet relatives employment agencies
Men Women Total
73 75 7468
7989
12
5060
71
86
11
43
6475
87
12
46
1009080706050403020100
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS20
11. Thehighexpectedvalueof İŞKURtraining is explained by the highperceived quality of training and thevalue perceived to be attached byemployerstoİŞKURcertificates. About 94 percent of trainees think İŞKURtraining improves job-readiness skillsand the knowledge of the profession(Figure 6). Almost all trainees areconfident about the quality of İŞKURtraining, particularly if provided bypublicinstitutions(50percentareveryconfidentabouttheirquality).Thequalityof İŞKUR training is also perceived tobe higher than that of other training,particularly other privately providedtraining. In addition to quality, 92percent of trainees also find training useful because they think employersdovaluetheİŞKURcertificate.Despitethe genuine value attached to İŞKUR
training,one-thirdoftraineesadmitthatamainreasonfortakingthetrainingisthe stipend provided to trainees (thisproportiondecreaseswith the levelofeducationofthetrainee).Andthefreenatureof İŞKUR trainingalsomatters:two-thirds of trainees would not payforothertrainingiftheydidnotgetanİŞKUR training spot, the main reasonbeing the lack of savings/cash andlimited access to credit.
Impacts of İŞKUR vocational training under review
12. The overall impact of İŞKURtraining (considered under this study)on employment is negligible, trainingdoeshaveasmallbutsignificantimpacton the quality of employment. Using the methods described in Section 2
TABLE 2
LargeexpectedimpactofİŞKURtrainingonthelabormarket(Probability of finding a job after one year under different scenarios, in percentage points)
Source:Baselinesurveyofevaluationparticipants.
Less than HS
High school
Voc. HS
College
Notraining
Notraining
Notraining
All Women Men
Training Training TrainingİŞKUR training
İŞKUR training
İŞKUR training
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 21
Trainees attach genuine value to İŞKUR training(%agreeingİŞKURtrainingisusefulfordifferentthings)
And perceive its quality to be higher than that of other options
(%confidentthequalityoftrainingwillbegood)
and measurements of employmentoutcomes taken approximately oneyear after the trainings (follow-upsurvey data), İŞKUR training coursesconsidered under this study arefoundto have no significant impact on thelikelihood of working, the number ofhoursworkedperweek,orthemonthlyincome received (Table 3). However,courses are found to have a smallbut significant impact on the qualityof employment. In particularly, beingassigned to İŞKUR training increasestheprobabilityofworkingintheformal
sector by 2 percentage points, withthe LATE impact being 3 percentagepoints. Given that 29 percent of thecontrol group is formally employedat follow-up, this LATE estimate isequivalent to a 10 percent increasein formal sector employment. Andtraining increases income from formal employment (byup to13percent)andas occupational quality.19 The overallimpact on employment, as measuredby the aggregate employment index(seeAnnex2), issmallbutstatisticallysignificant.
FIGURE 6
TraineesattachgenuinevaluetoİŞKURtraining
Source:Baselinesurveyofevaluationparticipants.
Job-readiness Knowledge of Employers’ Networking to Changing Stipend stills the profession value of find jobs profession certification
Agree Strongly agree
1009080706050403020100
94
41
95 92 90
79
344638 37
2811
ISKUR, public Other, public ISKUR, private Other, Private
Confident Very confident
120
100
80
60
40
20
0
97
53
8794
67
3643
21
19- Thisisacontinuousmeasurerangingfrom16(e.g.,domestichelpers)to90(e.g.,judges),andiscodedaszeroforindividualsnotworking(seeAnnex2fordetails).
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS22
13. Theimpactsoftrainingunderreviewonemploymentappearlargerformenover25yearsofage,butthedifferencesbyageandgenderarenotsignificant,nor are the differences by level ofeducation(Table4).Thestudyrandomlyassigned individuals to treatment and controlgroupswithineachcourseandwithingroupsofpeopledefinedbyage(underandabove25yearsofage)andgender, which allows a comparisonof the impacts across these groups.The consequence of using smallergroups(asopposedtothefullsample),
however, is that the estimates of theimpactsbecomelessprecise.Therearenosignificantdifferencesintheimpactoftrainingbetweenthosewithatleasta high school education completedandthosewithlessthanahighschooleducation,althoughthosewithpreviouswork experience or previous trainingseems to benefit more from İŞKURtraining(seeTableA2inAnnex3).
14. TheactualimpactsofİŞKURtrainingunder review differ significantly fromthe expectations formed before the
TABLE 3
İŞKUR’strainingshavenegligibleoverallimpactsonemploymentandsmallimpactsonjobquality
Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.SeeAnnex1fordetailsontheestimationmethodologyandAnnex2forvariabledefinitions.
ITT Estimate
LATE Estimate
Control Group Mean
Control Group Standard Deviation
Sample Size
Employed20 hours+
0.012
(0.013)
0.019
(0.018)
0.361
0.480
5529
Working at all
0.020
(0.013)
0.029
(0.018)
0.420
0.494
5497
OccupationalStatus
0.962*
(0.573)
1.452*
(0.828)
17.128
21.763
5418
MonthlyIncome
17.316
(12.271)
25.989
(17.624)
299.109
464.600
5396
Formal Income
22.166*
(11.965)
33.243*
(17.184)
257.887
448.160
5464
WeeklyHours
0.860
(0.680)
1.294
(0.980)
17.922
25.545
5439
Formal Work
0.020*
(0.012)
0.030*
(0.017)
0.293
0.455
5508
TransformedMonthlyIncome
0.121
(0.093)
0.182
(0.134)
2.541
3.516
5396
AggregateEmployment
Index
0.039*
(0.023)
0.059*
(0.033)
0.001
0.871
5497
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 23
trainings. On average, individualswerereasonablycloseintermsoftheirexpectationsofwhatemploymentlevelswouldbelikeintheabsenceoftraining:themeanresponsewasfora31percentchanceofbeingemployed;inpractice,36 percent of the control group isemployed(Table5).However,asshownearlier(Table2),individualssubstantiallyoverestimatethebenefitsfromtraining:theLATEestimateisfora2percentagepointincreaseinemploymentasaresultoftraining,whereasthemeanexpectedincreaseamongthoseassigned to thetreatment group is for a 32 percentage point increase.20 This overestimationoccurs for all gender and age groups. Three factors may help to explainthis mismatch between expectationsand reality: (i) the natural tendencyto overestimate future benefits and underestimate future costs of action takentoday,particularlyamongyouth;(ii) the different context in whichexpectations were formed relative tooneyearafterthetrainings;and(iii)theaverage trainee does not knowmuchabouttrainingsofferedbyİŞKUR.
15.ThesmallimpactsofİŞKURtrainingunderreviewonemploymentareinline
with the impacts of similar programsaround the world. A review of theevidencefromtheU.S.(Heckmanetal.1999) andEurope (Kluve2010) showsthatvocationaltrainingprogramshaveat most modest positive impacts on adult earnings,withlittleimpactsforyouth.Indevelopingcountries,areviewofnon-experimental studies by Betchermanet al. (2004) also foundmixed resultsof training the unemployed. The fewrandomized evaluations of vocational training programs in developing countries have focused on programsfor youth. Results in the DominicanRepublic(Cardetal.2011)showingnoimpact on employment and modestincreases in incomeare consistent with the earlier literature. Somewhatmore encouraging results are found in Colombia, particularly among youngwomen(Attanasioetal.2011).21Theseresults are, of course, context- andprogram-specific. And in some cases,they do provide clues as to whatmakes these programs more cost-effective (moreon this later).But theydosuggest thatwhile theseprogramscan be beneficial for (at least some)jobseekers, they are not apanacea toreducelarge-scaleunemployment.
20-IncontrastwithTable2,Table4 reportsexpectations for the treatmentgrouponly tooffsetanyconcernsthatindividualsinthecontrolgroupwhohadfoundouttheywerenotselectedfortrainingbythetimeoftheinterviewmightunderstatetheirexpectationsofthevalueoftraining.Inpractice,however,thecontrolandtreatmentgrouphaveverysimilarexpectations.
21-Amongyoungwomen, training increased the likelihoodof employmentby7percent andearningsby20percent.Traininghadnoimpactonthoseoutcomesamongyoungmen.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS24
TABLE 4
NosignificantvariationintheimpactofİŞKURtrainingsonemploymentbygenderandage
Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.Testofequalityteststhenullhypothesisofequalityoftreatment impactacrossthefourageandgendergroups.SeeAnnex1fordetailsontheestimationmethodologyandAnnex2forvariabledefinitions.
ITT for males under 25
ITT for males over 25
ITT for females under 25
ITT for females over 25
p-value for testing equality:
Control Mean: young men
Control Mean: older men
Control Mean: young women
Control Mean: older women
Sample size
Employed20 hours+
-0.034
(0.030)
0.047
(0.031)
0.016
(0.025)
0.018
(0.019)
0.276
0.407
0.548
0.356
0.247
5529
Working at all
-0.021
(0.030)
0.069**
(0.029)
0.011
(0.025)
0.023
(0.020)
0.188
0.489
0.621
0.403
0.295
5497
OccupationalStatus
-0.495
(1.299)
2.997**
(1.332)
0.567
(1.161)
1.034
(0.917)
0.294
19.2
24.3
17.3
12.4
5418
MonthlyIncome
-12.112
(31.384)
35.304
(37.501)
14.938
(20.999)
26.052
(17.321)
0.712
347
518
269
187
5396
Formal Income
20.734
(30.160)
32.019
(36.733)
15.571
(20.519)
23.058
(16.788)
0.982
275
458
238
161
5464
WeeklyHours
-0.952
(1.626)
2.917*
(1.772)
0.952
(1.355)
0.761
(1.001)
0.457
20.1
27.3
18.1
12.0
5439
Formal Work
0.003
(0.028)
0.047
(0.032)
0.017
(0.024)
0.019
(0.018)
0.768
0.307
0.466
0.295
0.196
5508
TransformedMonthlyIncome
-0.241
(0.223)
0.383
(0.244)
0.140
(0.181)
0.174
(0.141)
0.268
2.90
3.94
2.50
1.69
5396
AggregateEmployment
Index
-0.029
(0.052)
0.112*
(0.058)
0.046
(0.045)
0.034
(0.035)
0.340
0.070
0.381
-0.015
-0.215
5497
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 25
16. TheimpactsofİŞKURtrainingunderreviewonothermeasuresofindividualand household well-being are equallymodest. One year after the trainingis completed may not be enoughfor the full impact of the training onemploymenttomanifestitself.Andthebenefitsoftrainingcanextendbeyondemployment,particularlyamongyouth(e.g.,throughreducedcrimeorconflict)(Heckman et al. 1999). To this end,the study collected information fromevaluation participants on future jobsprospects as well as other measuresof individual and household well-being. Indeed,more people expect tobeemployedintwoyears(54percent)
thannow(42percent),but there isnodifference between those who wereassigned to training and those whowere not (Table 6). Training does nothave an impact on trainees’ mentalhealth, but it makes trainees assesstheir well-being somewhat higher. Intermsofhouseholdwell-being,trainingdoes not have an impact on currenthousehold income, but it seems toincrease long-termhouseholdwelfare,asmeasuredbyadurableassetindex.Finally,thereisnoevidencethattrainingempowersindividualsinthehouseholdorinfluencestheirgenderattitudes(seeTableA3inAnnex3).
TABLE 5
Bigmismatchbetweenexpectationsandrealities
Notes: Individualexpectationsarethoseofthetreatmentgroupatbaseline.ActualcontrolemploymentlevelandtheLATEestimateforemploymentimpactarefortheoutcomeofworking20hoursormoreperweek.
Full Sample
Males under 25
Males 25 and over
Females under 25
Females 25 and over
ActualControl
EmploymentLevels
36.1
40.7
54.8
35.6
24.7
Expected percent chance
of being employed ifnot trained(Std. dev)
31.3
(24.1)
35.4
(24.6)
33.5
(26.1)
31.5
(22.5)
27.7
(23.5)
Employment Levels(%)
ActualImpactLATE
Estimate(std. error)
0.019
(0.018)
-0.044
(0.047)
0.081
(0.054)
0.021
(0.033)
0.022
(0.027)
IndividualExpected ImpactMean
(Std. dev)
0.324
(0.275)
0.297
(0.278)
0.302
(0.280)
0.329
(0.266)
0.346
(0.277)
Treatment Impact (Prop.)
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS26
Why does the training under review have a limited impact on employment and what types of training work better?
17.Thissectionexaminesthereasonsbehind the limited impact of trainingunder review on employment andidentifies the types of training thatwork better. The study investigatesfour channels through which trainingisexpectedtoaffectemployment(seeSection 2): (i) course attendance andcompletion; (ii) thequalityof training;(iii)thevalue-addedoftraining;and(iv)thelabormarketcontext.
18. Course attendance and completion rates are not good indicators of howtrainees value courses. As noted earlier, 77 percent of those selectedfor treatment attended the coursebeyondthesecondday,and72percentcompleted it. The impact estimates inTable 1 already adjust for attendance.The impact of training does not varysignificantly with the percentage oftrainees that attend the course andcompleteit(Table6).Thisresultsuggeststhatcourseattendanceandcompletionrates are not good indicators of theextent to which individuals taking thecourse value it in practice.
TABLE 6
Modestimpactsoftrainingonothermeasuresofindividualandhouseholdwell-being
Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.SeeAnnex1fordetailsontheestimationmethodologyandAnnex2forvariabledefinitions.
ITT Estimate
Control Group Mean
Control Group Std. Dev.
Sample Size
MHI-5 Mental Health(higher better)
-0.060
(0.092)
18.508
3.410
5437
Expected Prob.Of Workingin 2 years
0.923
(0.915)
54.1
33.4
4878
Individual outcomes Household Outcomes
HouseholdIncome in last
year
485.524
(392.553)
21711
14674
5396
Subjective Well- Being in
5 years
0.061
(0.053)
5.822
1.980
5289
CurrentSubjective Well- Being
0.066*
(0.040)
4.436
1.514
5508
DurableAssetIndex
0.106**
(0.044)
-0.082
1.736
5495
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 27
19. The quality of training: privatelyprovided and competitive courses have a large impact on employment.The median course length is 320hours, which does suggest enoughhourstoenablelearningtotakeplace.Surprisingly, longer courses have lessimpact on employment than shortercourses (the difference is small,however; see Table 7). One possibleexplanation is that individuals reducejobsearchwhilsttakingpartintraining,so individuals in longer training courses have had less time to look for jobs.Havingmoreexperiencedoreducatedtrainersdoesnotseemtomatter,whichmayjustindicatethatteachereducationand experience explain little of theactualvariationinteachereffectiveness(HanushekandRivkin2006).
20. Both private provision and
competition yield higher impacts.
Previousnon-experimentalliteraturein
developed countries has found some
evidencetosuggestthattheimpactsof
training arehigher for trainingoffered
by private providers (Jespersen et al.
2008).Possiblereasonsarethatprivate
training providers are more responsive
to private sector employer demand
and/or they potentially face more
competition and thus must increase
quality in response. Table 8 shows
that İŞKUR courses offered by private
providersaswellascoursesofferedby
providersthatfacecompetition(having
two or more competitors) do yield
higherimpactsonemployment.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS28
TABLE 7
Coursecharacteristicsassociatedwithbetterimpacts
Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.SeeAnnex1fordetailsontheestimationmethodologyandAnnex2forvariabledefinitions.Thecompetitionvariable isbasedonthequestiontoprovidersaboutthenumberofcompetitorstheyface:(i)zero;(ii)one;(iii) twotofive;(iv)sixtonine;or(v)10ormorecompetitors.
Treatment Interaction with:
Proportion assigned to course who attended it
Proportion assigned to course who completed it
Proxies for course quality
Course length above 320 hours
Average teacher experience greater than 12 months
Percent of course teachers with tertiary education
Course has two or more competitors
Course offered by private provider
Measures of what trainees thought course did
Proportion who thought course taught new technical skills
Proportion who thought course certified skills they had already
Proportion who thought course taught new job finding strategies
Proportion who thought course made them aware of new jobs
Measure of labor market demand
Provincial unemployment rate is above median
Differential Impact on:
Employed20+ hours
0.114
(0.092)
0.055
(0.083)
-0.042*
(0.024)
0.009
(0.026)
-0.000
(0.000)
0.040
(0.025)
0.044*
(0.023)
-0.087
(0.093)
-0.012
(0.108)
-0.025
(0.090)
-0.052
(0.072)
0.013
(0.023)
Control Mean (Std. Dev.) of Interact. Var.
0.765
(0.140)
0.723
(0.164)
0.423
(0.494)
0.418
(0.493)
65.0
(43.2)
0.674
(0.468)
0.348
(0.477)
0.796
(0.127)
0.842
(0.103)
0.604
(0.147)
0.449
(0.183)
0.463
(0.499)
SampleSize
5497
5497
5494
4833
4833
4833
5494
5497
5497
5497
5497
5497
AggregateEmployment
Index
0.173
(0.172)
0.118
(0.155)
-0.082*
(0.044)
0.019
(0.048)
-0.000
(0.001)
0.083*
(0.048)
0.117*** (a)
(0.043)
-0.209
(0.185)
-0.096
(0.213)
-0.067
(0.166)
-0.153
(0.140)
0.033
(0.044)
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 29
21. But it is really the combinationof private provision and competition thatbringsthehighestpayoff?Table8exploresfurthertheinteractionbetweencompetition and private provision. There is a large and highly significantpositive treatment effect for courses offered by private providers facinglots of competition: 17 percent of thecoursesevaluatedfitthesecriteria,andthe treatment impact is 9 percentagepointsforemployment,and0.2standarddeviations for theoverall employmentindex.Thusprivatecoursesthatdonot
facemuchcompetitionalsodonotoffersignificantimpacts,butneitherdopubliccoursesfacinglotsofcompetition.Thissuggests that it is important to haveboth the performance spur providedbycompetitionandtheincentivesandabilitytorespondtothiscompetitiontogeneratelargeimpactsonemployment.It isworth noting that this result is insharpcontrastwith theperceptionsoftrainees(beforecoursesstart),whoseeprivatecoursesashavinglowerqualitythanpubliccourses.
TABLE 8
Itisreallythecombinationofprivateprovisionandcompetitionthatworksbest?
Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.SeeAnnex1fordetailsontheestimationmethodologyandAnnex2forvariabledefinitions.
ITT Impact for:
Public Course, 0 or 1 competitor
Public Course, 2 to 5 competitors
Public Course, 6+ competitors
Private Course, 0 or 1 competitor
Private Course, 2 to 5 competitors
Private Course, 6+ competitors
p-value for testing equality of impact
Sample Size
Proportion of samplein this group
0.24
0.21
0.23
0.09
0.07
0.17
Employed20+ hours
-0.027
(0.025)
0.014
(0.027)
0.012
(0.024)
0.002
(0.030)
-0.049
(0.043)
0.090***
(0.031)
0.064
4850
AggregateEmployment Index
-0.051
(0.045)
0.025
(0.051)
0.040
(0.048)
0.045
(0.069)
-0.045
(0.054)
0.203***
(0.055)
0.013
4820
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS30
22. Private provision and competition dohaveagenuinemarketvalue. Table A4inAnnex3showsthatthesecoursesdodiffer fromother types in termsofcharacteristics and applicants. One ofthekeydifferencesisthatthesecoursesaremore focused on accounting thanothertypesofcourses,andaccountingcourses have a larger impact onemployment than courses in otherfields. However, privately providedcourses, particularly those offered byproviders that face intensecompetition,still yield a significantly higher impactonemploymentthanothercoursesaftercontrolling for differences in course and trainee characteristics (Table A5in Annex 3). This evidence suggeststhatprivateproviders,especiallythosefacing more competition, have betterincentivestoofferhighqualitycoursesand services more attuned to labor market needs.
23. One year after the trainings,mosttrainees in thestudystillattachvalue-addedto İŞKURtrainingbut thatdoesnot seem to be enough to make adifference in the labor market. One yearaftercompletingthetraining,mostcourse participants believe training providedsomevalue-added,particularlyasasignalingdevicetoemployers:84percentofparticipantsthinkthecoursescertifiedskillstheyalreadyhad,and80percent think they taught new skills,while60percentsaythecoursehelped
with job-finding strategies and 45percentwithmakingthemmoreawareof job opportunities (Table 6). Those“assessments”are lowercompared tothesituationbefore thecoursestarted(Figure8),butstillyieldhighmarksforİŞKURtrainingasperceivedbycourseparticipants.However,therealityisthatthosewho value İŞKUR trainingmoredonotdobetterthanthosewhovalueitless.Theseresultssuggestthatthevalueattachedto İŞKURtrainingmayreflectotherattributesthanjustanincreaseinjobplacementprobabilityandmaybeduetothestrongeffectoflaborsupplyonemploymentoutcomes.
24. Thelabormarketcontextdoesnotseem to matter much. Although atthe time of training there was a widevariationinunemploymentratesacrossthe23evaluationprovinces(between5percentand17percent),traineesfromhighunemploymentprovincesdidnotdo better or worse than those fromlow unemployment provinces. TheevaluationofİŞKURtrainingtookplaceinthecontextofaremarkablesurgeinemployment after the crisis. Althoughtheunemploymentrateisnottheonlyindicatoroflabormarketperformance,this result suggests that the smallimpactoftrainingonemploymentwasnotdrivenbythebuoyantlabormarketat the time the follow-up data werecollected(early2012).
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 31
Cost-benefit analysis of İŞKUR training under review
25. İŞKUR courses offered by privateproviders facing competition have alarge net return. This section looksat how the benefits of İŞKUR trainingcomparetoitscostsandidentifiesthetypes of İŞKUR training that aremostcost-effective.
26. Fromtheprovidersurvey,themeanestimatedcost foran İŞKURcourse is1,574TLperperson(privatelyprovidedcourses have a mean cost of 1,792TL per person compared to 1,455 TLfor publicly provided courses). İŞKURcoursesprovidedbyprivateprovidersfacing six or more competitors havea mean cost of 1,771 TL per person.Participants receive a stipend of 15TL per daywhile they are in training;for an average course length of 57days, this equates to a further cost of855 TL. Thus the average cost to thegovernment of providing a course is 2,429TLperperson,2,671TLforprivateand competitive courses.
27. The LATE estimate of the overallgain in monthly income (Table 2) is
26TL (although this isnot statistically
significant).Assumingthecourseleads
to a permanent increase in income of
this amount, it would take 93months
forthegaininincometooffsetthecost
of the course, and the annual internal
rate of return (IRR) over 30 years of
thesegainswouldbe12percent.Ifthe
gains depreciate at 10 percent a year,
the IRRwouldbe3percent.However,
given that the point estimate of the
gain is insignificant, we cannot rule
out zero returns. Training was more
effectiveforprivatelyprovidedcourses
offered by providers facing significant
competition.TheLATEestimateofthe
overallgaininmonthlyincomeforthis
group is a statistically significant 128
TLpermonth.Basedonthisestimated
gainpersisting,itwouldtake21months
forthegaininincometooffsetthecosts
ofprovision,andtheIRRover30years
would be 58 percent (or 48 percent
with 10 percent per year depreciation
ofthegains).Bywayofcomparison,in
the evaluation conducted inColombia
(Attanasioetal.2011),coursescostan
average of US$750 and hadan IRR of
22-35percentforyoungwomen,andno
measureablebenefitsforyoungmen.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS32
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 33
SECTION 4:STRENGTHENING THE IMPACT OF İŞKURTRAINING AND SERVICES
1. İŞKUR has come a long way since2008, increasing the coverage andquality of vocational training. As part of the2008 labor reform,participationin ALMPs was opened to all registered unemployed regardless of whetherthey are eligible for unemploymentinsurance. The number of vocationaltraineesincreasedfrom30,000in2008to 464,000 in 2012, representing 19.6percentoftheregisteredunemployed.İŞKUR has been working to improvethequalityoftrainingthroughanumberof measures. One such measure hasbeen the introduction of the NationalQualifications System, includingthe development of 294 nationaloccupational standards in 16 sectors (127 are under implementation).Also in 2011 the selection of trainingproviders introduced specific qualityand performance criteria (not justcost).In2012,İŞKURstartedhiringjoband vocational counselors to advise jobseekersonoccupationalchoiceandtraining courses.
2.AnewregulationpublishedinMarch2013 introduces a number of important initiativestoimprovetheeffectiveness
of vocational training and otherservices. In terms of training, newmodules (not exceeding 15 percentof thecurriculacontent)willbeaddedto training programs to improve employability of participants (e.g., jobsearchskills,interviewtechniques,andbasic skills). Also past job placementperformance and accreditation by theVocationalQualificationInstitution(VQI)willbemorerewardedwhenselectingprivate providers. Service providers are alsorequiredtouselicensinginstitutions(accreditedbyVQI)during testingandcertification of participants of training programs. All training programs for whichVQI has approvedoccupationalstandardswillbedevelopedinlinewiththenationaloccupationalstandards.
3. Service providers with a low jobplacementperformancewillberequiredto provide additional services. Providers ofgeneraltrainingarenowalsosubjectto jobplacement requirements,whichincrease to: at least 50 percent oftrainees finding employment for aperiod that cannot be lower than theduration of the course or at least 120days.Asanincentivetohireandkeep
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS34
young and female trainees employedin occupations for which they aretrained, the initial 6-month period inwhich employers’ social contributionsforyoungandfemalehiresarepaidbythegovernmentisnowextendedtoanadditional36monthsiftheyareISKURtraineesandtheyareemployed in theoccupationsforwhichtheyaretrained.The new regulation also links trainingto jobsearch: traineeswhorefuse joboffers suitable to their characteristicsare not allowed to benefit from anyALMP for 24months. Finally, job andvocational counselors will be more activeintheselectionofparticipantsoftraining programs.
4. These important recent reforms,which are not captured by this study,already build on the results of thisevaluation. Thisfinalsectionbuildsonthesegoodinitiativesandachievementsto suggest some options to furtherstrengthen the employment impact ofİŞKURtrainingandservicesbasedon:(i)theresultsoftheevaluation;and(ii)lessons from international experience.Morework isneededto furtherdefinethese(andotherpolicy)options.Reformcanonlyhappengradually,introducinga few new initiatives and evaluating thembeforemovingtothenextsetofreforms.ThisistheapproachİŞKURis
takingandthisevaluationillustratesit.
5. Improving the relevance of skills training. The study find that the lowoverall impactofİŞKURtrainingunderreview on employment is likely to bepartly due to the low value-added ofthesecoursesintermsoftheskillstheyhelp tobuild.As İŞKURdevelopsnewmodules to be added to the trainingprogram, it may consider puttingmore emphasis on behavioral skills,whicharehighlyvaluedbyemployersTurkey (McKinsey 2012). There isincreasinginternationalevidenceoftheemploymentimpactofbehavioralskillstraining (Almeida et al. 2012). Goingforward,itisalsoimportanttocontinuestrengtheningthelinkbetweenİŞKUR,trainingproviders,andlocalemployers,building on the experience of theProvincial Employment and TrainingCouncils.22
6. Incentivizing and supporting more job search. Only half of İŞKUR trainees inthestudywerelookingforajobbeforecourses started and not all of themusedİŞKURservices.Thissuggeststheneed to encourage more job searchandexpandemploymentservices (jobplacement, counseling, job-searchassistance).Thenewregulationlinkingtraining to jobssearch isastep in the
22-Thesecouncilswerecreatedin2008tobringtogetherthetrainingandlabormarketsidesatthelocallevel.Theyarechairedbytheprovincialgovernorsandhaverepresentationfromemployers,educators,andotherlocalactors,withİŞKURandMONEservingasthesecretariat.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 35
right direction. International evidenceshows that it ismore cost-effective tofirst encourage jobseekers to look fora job and to assist them in this taskthrough employment services beforethey get any training than to offertrainingfirst(asinTurkey).Toencouragejob search, it is important to tie thereceipt of unemployment or socialassistance benefits to it—Turkey doesthis,butperhapscompliancecouldbeimproved.Employmentservices,whicharelimitedinTurkey,shouldbecentralto employment activation efforts. Forexample, in the U.K.’s jobcenter plus,all registered unemployed people areoffered some employment services(rangingfromminimumjobplacementservices to job-search assistance forthe hard-to-employ) and are requiredto take individual actions to find a job (i.e., the market test– receipt ofbenefits is conditional on that) beforeit is determined whether they needadditionalservices(includingtraining).
7. Defining the priority groups for training. The low overall impactof İŞKUR training under review onemployment could also be related towho İŞKUR is actually training andwhetherthesejobseekersaretheoneswhocanbenefitthemostfromtrainingrelative to other groups of jobseekers(and are most likely to be activelyseekingemployment).İŞKURistrainingthe most educated jobseekers. Aside
fromtheminimumrequirementtohavebasiceducation,thisismostlytheresultof the courses İŞKUR offers and theselectionofapplicantsbyproviders.Theevaluation does not show differentialimpactsbygenderandage,but theseresults refer to the current profile ofİŞKURtrainees(e.g.,highlyeducated)aswellasthetrainingandservicesİŞKURcurrently offers. Low-skilled workersaccountformostofthelaborforceandface the greatest jobs challenge. Andthey face barriers to productive jobsotherthanskills,includinginformation.Countries with well-developed publicemployment services (e.g., the U.K.,Germany, and Australia) serve alljobseekersthatregister,butthebulkoftheirresources(includingskillstraining)support hard-to-employ jobseekers.İŞKURhasstartedservingonesegmentof the hard-to-employ, namely thosereceiving social assistance benefits and able to work.
8. The focus of İŞKUR trainings onwomen and youth seems a prioriappropriate fromapolicyperspective.The lack of differential impacts oftrainingonemploymentbygenderandageistiedtothecurrentprofileofİŞKURtraineesaswell as to the trainingandservicesİŞKURcurrentlyoffers.Activityrates are especially low for womenand youth. International evidenceshowsthatwell-designedskillstrainingprograms have a higher return for
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS36
youththanothers,mainlybecauseitiseasiertolearnwhenyoung.SuccessfulprogramsforyouthintheU.K.,theU.S.,and several Latin American countries (e.g., the Jovenes program) targetdisadvantaged out-of-school youth(typically 15- to 29-year-oldswith lessthan a secondary education). And incountrieswithlowfemaleemployment,like Colombia (Attanasio et al. 2011)andtheDominicanRepublic(Ibarraran2012), well-designed programs foryouthcanhavealargepayoffforyoungwomen.
9. Better information on jobseekers to adjust services to needs. The newjob and vocational counselors hiredby İŞKURwill most certainly result ina better match of trainees to courses(a possible problem identified in thisevaluation),buttheydonotassesstheemployability of jobseekers and thustheirneedfortraining(orotherservices)to begin with. Countries with well-developedpublicemploymentserviceslike the U.K. and Australia do makean initial employability assessmentof jobseekers, which is then usedto “profile” jobseekers into differentgroupsreceivingdifferentemployment
support packages, with the bulk ofresourcesgoingtothehard-to-employ.
10. More contracting of services to the private sector while ensuring quality. The high return of İŞKURcourses offered by private providersfacing more competition suggests that increasing the share of coursessubcontracted to private providers and increasing competition among them would significantly increase theemployment impactof İŞKURtraining.However,itisalsoimportanttoensurethequalityofprovidersandmakethemaccountable for results. İŞKUR hasalready taken a number of measuresto increase the quality of providersthrough the selection process. İŞKURhas also recently extended jobsplacement requirements to providersof general training.There is still roomtostrengthenthecontractswithprivateproviders to improve the impact oftraining. And to avoid “creaming” offthe easy-to-employ by providers, andtotakeintoaccountthehighercostofhelping the hard-to-employ becomeemployed, the contracting of servicesforthemcouldbedoneseparately,asisdoneintheU.K.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 37
REFERENCES
Abdulkadiroglu, Atila, Joshua Angrist,SusanDynarski,ThomasKaneandParagPathak(2011)“Accountabilityand Flexibility in Public Schools:Evidence from Boston’s Chartersand Pilots.” Quarterly Journal of Economics 126(2):699-748.
Rita Almeida, Jere Behrman, DavidRobalino(Editors).2012.The Right Skills for the Job?: Rethinking Effective Training Policies for Workers,WorldBank,Washington,DC.
Attanasio, Orazio, Adriana Kugler,CostasMeghir (2011) “SubsidizingVocational Training for DisadvantagedYouth inColombia:Evidence from a Randomized Trial”,American Economic Journal: Applied Economics3(3):188-220.
Benjamini, Yoav and Yosef Hochberg(1995) “Controlling the FalseDiscovery Rate: A Practical andPowerful Approach to MultipleTesting”, Journal of the Royal Statistical Society Series B, 57(1):289-300.
Betcherman, Gordon, Karina Olivas,and Amit Dar (2004) “Impacts ofActive Labor Market Programs:New Evidence from Evaluations withParticular Attention to Developing and Transition Countries. ”World
Bank Social Protection Discussion Paper Series 0402.
Betcherman, Gordon, M. Godfrey,S. Puerto, F. Rother and A.Stavreska, 2007, Global Inventory of Interventions to Support Young Workers Synthesis Report, World Bank.
Bruhn, Miriam and David McKenzie(2009), “In Pursuit of balance:Randomization in Practice in Development Field Experiments,”American Economic Journal: Applied Economics1(4):200-232.
Burbidge, John, Lonnie Magee andA. Leslie Robb (1988). “AlternativeTransformationstoHandleExtremeValues of theDependent Variable”Journal of the American Statistical Association83(401):123-127
Card,David,PabloIbarraran,FerdinandoRegalia, David Rosas-Shady, andYuri Soares (2011) “The LaborMarketImpactsofYouthTraininginthe Dominican Republic”, Journal of Labor Economics29(2):267-300.
Fink,Günther,MargaretMcConnellandSebatianVollmer(2012)“TestingforHeterogeneous Treatment Effects in ExperimentalData:FalseDiscoveryRisks and Correction Procedures”,Mimeo. Harvard School of PublicHealth.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS38
Ganzeboom,HarryandDonaldTreiman(1996) “InternationallyComparableMeasures of Occupational Status forthe1988InternationalStandardClassification of Occupations”,Social Science Research 25(1):201-39.
Hanushek, Eric and Steven Ritkin(2006)“TeacherQuality”,pp.1051-78 in E. Hanushek and F. Welch(eds.)Handbook of the Economics of Education Volume 2. Elsevier,Amsterdam.
Heckman,JamesJ.,RobertJ.Lalondeand Jeffrey A. Smith (1999) “TheEconomics and Econometrics of Active Labor Market Programs”,Handbook of Labor Economics Volume 3, Part A,1865-2097
Ibarraran, Pablo, LauraRipani, BibianaTaboada, Juan Miguel Villa,Brigida Garcia (2012) “ Life Skills,Employability and Training forDisadvantaged Youth: Evidencefrom a Randomized Evaluation Design”, IZA discussion paper No. 6617.
Jespersen, Svend, Jakob Munch andLars Skipper (2008) “Costs andBenefits of Danish Active LabourMarket Programmes”, Labour Economics 15(5):859-84.
Kluve,Jochen(2010)“Theeffectivenessof European active labour market policy”, Labour Economics 17(6):904-18.
McKinsey&Company(2012), Education to Employment: Designing and System that Works, McKinsey&Company.
WorldBank (2010)“Turkey: InvestmentClimate Assessment—From Crisisto Private Sector Led Growth.”Report 54123-TR. World Bank,Washington,DC.
World Bank (2013a) “Managing LaborMarkets through the EconomicCycle,” Report 70130-TR. WorldBank,Washington,DC.
World Bank (2013b) “PromotingExcellence in Turkey’s Schools.”Report 77722-TR. World Bank,Washington,DC.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 39
ANNEX 1: ESTIMATION METHODOLOGY
We canmeasure the intent-to-treat (ITT) effect of vocational training on a particularoutcomeofinterestbyestimatingthefollowingequation,analogoustoequation(1):
(1)
where i denotes individuals, s denotes a course*gender*age group (randomizationstrata),and,δi,sisabinaryvariableindicatingwhetherindividuali belongs to group s. Thiscontrolsforrandomizationstrata(BruhnandMcKenzie2009)aswellascontrollingforindividualswhoappliedformorethanonecourse(Abdulkadirogluetal.2011).
Sincewedonothavebaselinedataforallapplicantswhoansweredthefollow-upsurvey,andbecausesomeofthebaselinedatawerecollectedafterthecoursehadbegun,wedonotcontrolforbaselinevaluesoftheoutcomevariableofinterest.βisthentheaverageeffectofbeingselectedforavocationaltrainingcourseonthisoutcome.
WecanalsoestimatetheimpactofactuallycompletingtrainingbyreplacingAssigned to TrainingwithCompleted Trainingin(1),andinstrumentingthiswithtreatmentassignment.Undertheassumptionthatassignmenttotraininghasnoimpactonoutcomesforthosewho do not complete the course, and that there are no individualswhowould takecoursesonly if assigned to thecontrolgroup, thisyields the localaverage treatmenteffect(LATE).Thisistheimpactofcompletingtrainingforanindividualwhotakesuptrainingwhenheisselectedinthecourselottery,anddoesnottakeitupotherwise.Aconcernwith thisestimation is thepossibility thatsimplybeingselected foracoursemayaffectemploymentoutcomes,evenifindividualsdonottakethecourseoriftheydropoutafteronlyafewdays.
Inadditiontoestimatingtheoverall impactof training,weare interested inexploringtheheterogeneityofimpactstohelpunderstandwhethercertaintypesofcoursesofferlargerimpactsorwhethercertaintypesofpeoplebenefitmorefromtraining.ToestimateheterogeneitywithrespecttocharacteristicX,weestimate:
(2)
Coursecharacteristicsonlyvaryat thecourse level,soweclusterstandarderrorsbycourse.Recently,Finketal.(2012)havecriticizedrandomizedexperimentslookingforheterogeneityintreatmenteffectsfornotcontrollingadequatelyformultiplehypothesistesting.TheyrecommendtheuseoftheBenjaminiandHochberg(1995)approachwhichholdsconstantthefalsediscoveryrate(theexpectedproportionoffalselyrejectednullhypotheses).Weusethisapproachtoexaminewhichdimensionsofheterogeneityarerobusttothisconcern.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS40
ANNEX 2:DEFINITIONS OF KEY OUTCOME VARIABLES
Employment outcomes
The following employment outcomes are defined based on data collected in thefollow-upsurvey.
Working at all: An indicator variable thattakes the value one if the individual hasworkedforcashorin-kindincomeatallinthepastfourweeks(andzerootherwise).
Employed 20 hours +: Anindicatorvariablethattakesthevalueoneiftheindividualiscurrentlyworkingfor20hoursaweekormore(andzerootherwise).
Weekly hours:Hoursworkedperweek inthe lastmonthemployed.This iscodedaszeroforindividualscurrentlynotworking,andtop-codedas100hoursperweek(the99thpercentileofthebaselineresponse)toreducetheinfluenceofoutliers.
Monthly income: Totalmonthlyincomefromworkinthelastmonth.Thisiscodedaszeroforindividualsnotworking,andtop-codedasthe99thpercentileofthecontrolgroupearningsdistribution(2500TL)toreducetheinfluenceofoutliers.
Transformed monthly income: Theinversehyperbolicsinetransformationofmonthlyincome fromwork in the lastmonth, log(y+(y2+1)1/2 ).This is intended tobemorerobusttooutliersthanlevelsofincomeandissimilartothelogarithmtransformation,butisalsodefinedwhenincomeiszero(Burbidgeetal.1988).
Occupational status: ThisiscodedbasedonworkoccupationusingtheinternationalmeasuresofsocioeconomicoccupationalstatusofGanzeboomandTreiman(1996).This isacontinuousmeasurerangingfrom16(e.g.,domestichelpers) to90 (e.g.,judges),andiscodedaszeroforindividualsnotworking.
Formal work: Thisisanindicatorvariablecodedasoneiftheindividualiscurrentlyworkinginajobcoveredbysocialsecurity(andzerootherwise).
Formal income: Thisismonthlyincomeearnedinjobscoveredbysocialsecurity.
Aggregate employment index:Astandardizedindexobtainedastheaverageofeachoftheabovevariables,aftereachhasbeenstandardizedbysubtractingthemeananddividingbyitsstandarddeviation.Thismeasureissettomissingforindividualswhoaremissingdataforthe“workingatall”variable;otherwise,itistheaverageoftheoverallemploymentvariableswithnon-missingdata.
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 41
Well-being measures
Expected probability of working in two years: Theexpectedchanceofhavingajobintwoyears’time,codedasmissingifanansweroutsideofthe0to100rangeisgiven.
Mental health index (MHI-5): This isa five-item index;ithasamaximumscoreof25andminimumscoreof 5.Higher scoresaredesirable in that they indicate theexperienceofpsychologicalwell-beingandtheabsenceofpsychologicaldistress.Individualsareaskedhowofteninthepastfourweekstheyhavedoneeachofthefollowing(eachansweredona5pointscale,where1denotesnoneofthetimeand5allofthetime):
•Beenanervousperson(reverse-coded)
•Feltsodowninthedumpsthatnothingcouldcheerthemup(reverse-coded)
•Feltcalmandpeaceful
•Feltdownheartedandblue(reverse-coded)
•Beenahappyperson
TheMHI-5isthesumoftheseresponses.
Current subjective well-being: Individualsareaskedonwhichstepofa10-stepCantrilladder(wherethepoorestpeoplestandonthefirststepandtherichestonthetenthstep)theythinktheirhouseholdstandstoday.
Subjective well-being in five years: ThestepoftheCantrilladderonwhichindividualsthinktheirhouseholdwillbeinfiveyears.
Total household income in past year: Incomefromallsources,topcodedasthe99th percentileofthecontrolgroupdistribution(74,000TL).
Transformed household income: The inverse hyperbolic sine of total householdincome.
Durable asset index: The first principal component of 15 indicators of householddurable asset and infrastructure ownership (own a gas or electric oven; own amicrowaveoven;ownadishwasher;ownaDVD/VCDplayer;ownacamera;haveDigiturk/Satelite;ownanairconditioner;ownaCDplayeroriPod;ownatelephone;ownacomputer;haveaninternetconnection;ownaprivatecar;ownataxi,minibus,orcommercialvehicle;ownabicycle;havefourormoreroomsintheirhouse).
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS42
TABLE A1
Summarystatisticsandtestingofdifferencesbetweentreatmentandcontrolgroups
Notes:Standarderrorsarerobuststandarderrors.*,**,and***indicatesignificanceatthe10,5,and1percentlevels,respectively.Samplesizesforthefollow-updatafortheindividualcharacteristicsmeasuredatbaselineareforthesampleinterviewedatbothbaselineandfollow-up;therearealsoindividualsinterviewedatfollow-upbutnotatbaseline.
Individual Characteristics (administrative data)
Female
Age
At least high school
Individual characteristics (baseline data)
Years of education
Has done previous training course
Household head
Household size
Married
Ever employed
Total years working for pay
Control Mean(S.D.)
0.629
(0.483)
27.1
(7.2)
0.724
(0.447)
11.3
(3.3)
0.264
(0.441)
0.134
(0.340)
4.09
(1.57)
0.346
(0.476)
0.631
(0.483)
3.38
(4.91)
N
5308
5308
5308
5255
5308
5276
5308
5033
5308
5277
Baseline Sample Follow-up Sample
Control Mean(S.D.)
0.623
(0.485)
27.0
(7.2)
0.724
(0.447)
11.3
(3.3)
0.265
(0.441)
0.133
(0.340)
4.10
(1.57)
0.351
(0.478)
0.626
(0.484)
3.33
(4.88)
N
5529
5529
5529
5008
5057
5027
5057
4797
5057
5027
TreatmentDifference(Std. Error)
0.005
(0.013)
-0.389**
(0.198)
0.009
(0.012)
-0.005
(0.069)
-0.007
(0.012)
-0.000
(0.008)
0.024
(0.040)
0.004
(0.011)
-0.021*
(0.012)
0.006
(0.111)
TreatmentDifference(Std. Error)
0.010
(0.013)
-0.306
(0.194)
0.005
(0.012)
0.014
(0.071)
-0.008
(0.012)
-0.003
(0.008)
0.011
(0.040)
0.002
(0.012)
-0.020
(0.013)
0.006
(0.113)
ANNEX 3: ESTIMATION RESULTS
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 43
TABLE A2
Differences in the impact of İŞKUR training by level of education, previous workexperience,andtraining
Notes:Robuststandarderrors(se)inparentheses.*,**,and***indicatesignificanceatthe10,5,and1percentlevels,respectively.“Treatmentb”istheLATEestimateoftheimpactofİŞKURtraining,while“Interactionb”istheLATEestimateoftheinteractionofeducation/previousworkexperience/previoustrainingwithtraining;e.g.,theimpactoftrainingfortraineeswithatleastahighschooleducationcompleted.SeeAnnex1fordetailsonestimationmethodologyandAnnex2 for variable definitions.
At least high school
Years education
Previous training
Ever worked
se
(0.042)
(0.042)
(0.039)
(0.006)
(0.006)
(0.006)
(0.043)
(0.042)
(0.040)
(0.038)
(0.037)
(0.036)
b
-0.032
-0.006
0.013
-0.002
0.001
0.000
0.046
0.093**
0.065
0.027
0.058
0.062*
Interaction Sample sizeTreatment
Working at all
Employed 20 hours +
Formal Emp.
Working at all
Employed 20 hours +
Formal Emp.
Working at all
Employed 20 hours +
Formal Emp.
Working at all
Employed 20 hours +
Formal Emp.
se
(0.036)
(0.035)
(0.032)
(0.070)
(0.069)
(0.065)
(0.022)
(0.022)
(0.021)
(0.028)
(0.028)
(0.025)
b
0.054
0.025
0.022
0.050
0.009
0.025
0.019
-0.004
0.014
0.018
-0.012
-0.003
5497
5529
5508
5497
5529
5508
5497
5529
5508
5497
5529
5508
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS44
TABLE A3
ImpactsofİŞKURtrainingonempowermentandgenderattitudes
Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.Testofequalityteststhenullofequalityoftreatment impactacrossthefourage*genderstrata.Numberofdecisionsmade isoutof7.Themaximumnumberofpro-genderequalitybeliefs is5.SeeAnnex1 fordetailsonestimationmethodologyandAnnex2forvariabledefinitions.
Panel A: Pooled Sample
ITT Estimate
Control Mean DepVar
Sample Size
Panel B: Age*Gender Heterogeneity
ITT for males under 25
ITT for males over 25
ITT for females under 25
ITT for females over 25
p-value for testing equality:
Control Mean: young men
Control Mean: older men
Control Mean: young women
Control Mean: older women
Is householdhead orspouse
0.004
(0.010)
0.188
5472
-0.002
(0.021)
0.002
(0.031)
-0.004
(0.013)
0.016
(0.015)
0.785
0.14
0.54
0.06
0.13
Number ofdecisions
made
0.058
(0.044)
5.601
5515
0.024
(0.071)
0.056
(0.088)
0.105
(0.088)
0.042
(0.086)
0.910
6.3
6.1
5.5
5.1
Number ofpro-gender
equal beliefs
0.057
(0.036)
2.540
5469
0.179**
(0.079)
0.072
(0.086)
0.053
(0.069)
-0.010
(0.060)
0.300
2.0
2.1
2.8
2.8
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 45
TABLE A4
Whatisdifferentaboutcompetitiveandprivatecourses?
Notes:Samplemeans shown. *, **, and*** indicate statisticallydifferent frompublicproviders at the10, 5, and1percentlevels,respectively.‘+,++,and+++indicatestatisticallydifferentfrom0or1competitorcoursesatthe10,5,and1percentlevels,respectively.Publiclyprovidedcoursestestedagainstprivatecourses;morecompetitivecoursesagainsttheleastcompetitivecourses;thelastcolumnsimplyshowsmeansfortheprivatecourseswith6+competitors,butdoesnottesttheseagainstanyothergroup.
Course Characteristics
Course length (days)
Course length (hours)
Accounting course
Professional course
Craftsman course
Technical course
Service course
Course in Istanbul
Average teacher experience greater than 12 months
Percent of course teachers with tertiary education
Individual Characteristics (administrative data)
Female
Age
At least high school
Individual characteristics (baseline data)
Years of education
Has done previous training course
Household head
Household size
Married
Ever employed
Total years working for pay
Raven test score (out of 12)
Numeracy score (out of 4)
Tenacity
2 to 5competitors
59
344+++
0.12+++
0.17+++
0.22
0.15+++
0.16+++
0.24+++
0.39+++
80.1+++
0.55+++
26.7
0.75+++
10.2+++
0.23
0.14+++
3.72++
0.28+
0.62+++
3.60+++
6.41+++
3.45+++
8.42++
0 or 1competitor
60
375
0.19
0.05
0.24
0.11
0.33
0.09
0.24
59.8
0.63
27.1
0.69
9.6
0.21
0.10
3.88
0.31
0.49
2.57
5.44
3.21
8.31
PublicProvider
55
328
0.13
0.11
0.23
0.19
0.24
0.27
0.45
71.9
0.61
27.0
0.66
9.7
0.25
0.12
3.86
0.30
0.57
3.17
6.10
3.31
8.36
6 + competitors
50+++
287+++
0.36+++
0.14+++
0.06+++
0.20+++
0.09+++
0.26+++
0.58+++
58.9
0.64
26.2+++
0.81+++
10.8+++
0.26+++
0.11
3.73+++
0.26+++
0.57+++
2.84+
6.26+++
3.48+++
8.32
6+ competitorsand private
60
352
0.48
0.23
0.00
0.10
0.05
0.29
0.51
60.0
0.68
25.8
0.89
11.2
0.20
0.10
3.58
0.24
0.58
2.64
6.29
3.50
8.39
PrivateProvider
60***
353***
0.45***
0.14***
0.02***
0.07***
0.14***
0.12***
0.34***
51.2***
0.66***
26.5***
0.85***
10.9***
0.20***
0.10**
3.64***
0.27*
0.55
2.64***
5.93*
3.51***
8.35
TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS46
TABLE A5
The“value-added”ofprivateprovisionandcompetitionremainsaftercontrollingfordifferencesincourseandapplicantcharacteristics
Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively. See Annex 1 for details on the estimationmethodology and Annex 2 for variable definitions.We usetwoalternativemethods toestimate thedifferential impactofprivatecoursesandprivatecourseswith6+providerscontrollingfordifferencesinthecoursesandtraineescharacteristicsinTableA4.First,weestimateapropensityscoreforthelikelihoodofbeinginthosetypeofcoursesasafunctionofthecharacteristicsinTableA4.Wethenre-runequation(2)inAnnex1weightingcoursesbythispropensityscore,whichyieldsthedifferentialimpactestimatecalled“weightedbypropensity-score.”Thealternativemethodinvolvesre-runningequation(2)inAnnex1andcontrollingfortheinteractionbetweenthetreatmentvariableandthepropensityscore.
Treatment Interaction with:
Course offered by private provider
Unweighted (as reported in Table 6)
Weighted by propensity-score
Controlling for interaction of treatment with propensity score
Course offered by private provider with 6+ competitors
Unweighted
Weighted by propensity-score
Controlling for interaction of treatment with propensity score
AggregateEmployment
Index
0.117***
(0.043)
0.112*
(0.059)
0.107*
(0.054)
0.190***
(0.058)
0.232***
(0.070)
0.214***
(0.065)
Employed20+ hours
0.044*
(0.023)
0.058*
(0.030)
0.061**
(0.029)
0.095***
(0.034)
0.127***
(0.037)
0.110***
(0.036)
Differential Impact on: