Abbreviations · Web viewThe Interim Evaluation TAT, however, accentuates that the SWOT in this...

1

Specific Contract N° TR07H4.02-01

Providing Technical Assistance for the First Interim Evaluation of Human Resources Development Operational

Programme

INTERIM EVALUATION REPORT

November 2011

Framework Contract

This project is co-financed bythe European Union and the Republic of Turkey

“The content of this publication is the sole responsibility of the Consultant and can in no way be taken to reflect the views of the European Union.”

2

ContentsAbbreviations.......................................................................................................................................... 7

1. Executive Summary............................................................................................................................ 9

2. Introduction....................................................................................................................................... 12

2.1 Overview of the HRD OP priority structure, measures and objectives.......................................12

2.2 Purpose of the Interim Evaluation and evaluation questions......................................................13

2.3 Outline of the Interim Evaluation Report.....................................................................................15

3. Main methods................................................................................................................................... 16

3.1 General approach to the Interim Evaluation...............................................................................16

3.2 The evaluation methodology subject to the evaluation questions..............................................17

The Socio-Economic Study in Annex A1.....................................................................................17

Mapping of the institutional environment and studying current practices.....................................17

The logic of the Report and the way the evaluation question was addressed..............................18

3.3 Challenges and solutions found.................................................................................................25

4. Main findings, conclusions and recommendations...........................................................................26

4.1 Examination of the Management and control structures of the HRD OP and SWOT.................26

4.1.1 Operating Structure (OS) ...................................................................................................27

4.1.2 The operation beneficiaries.................................................................................................30

4.1.3 The Social Partners and the Chambers..............................................................................34

4.2 The Monitoring Arrangements and the Monitoring Information System with respect to the Grant Monitoring........................................................................................................................................ 34

4.2.1 Central Grant Monitoring Teams.........................................................................................34

4.2.2 Regional Grant Monitoring and Technical Assistance Teams (RGMTTs)...........................36

4.2.3 The Monitoring Information System....................................................................................36

4.3 The use and financial allocation of financial assistance and the financial management of the budget.............................................................................................................................................. 37

Overall comments on financial management...............................................................................37

Commitment and disbursement data...........................................................................................37

National co-financing and EU contribution...................................................................................38

Payments to contractors..............................................................................................................38

Shorter implementation periods and decommitment risk.............................................................38

4.4 Review programme outputs and results in relation with the expenditures incurred at priority and measure levels; in the light of financial and physical progress.........................................................40

4.5 Examine the efficiency of Operation Selection Criteria...............................................................41

4.6 Assessment of local absorption and implementation capacities.................................................42

Assessment of local absorption capacities..................................................................................42

Assessment of local implementation capacities...........................................................................44

4.7 Analysis of the accuracy of the budget allocations under measures..........................................44

4.8 The study on Indicators..............................................................................................................45

Conclusions and recommendations on the study of Indicators....................................................46

4.9 Overall assessment of the HRD OP...........................................................................................48

4.10 Recommendations................................................................................................................... 49

3

5. Annexes: Overview........................................................................................................................... 51

Annex A1. Socio-Economic Study........................................................................................................52

1 Introduction................................................................................................................................... 52

2 Demographic Indicators................................................................................................................53

3 Main Economic Indicators.............................................................................................................58

4 Labour Market Indicators...............................................................................................................64

5. Indicators on Education................................................................................................................68

Appendix 1. Background Tables......................................................................................................72

Annex 2. Major developments since 2007............................................................................................92

1. Major Developments since 2007 at National Level......................................................................92

Implementation of Partnership Approach.....................................................................................92

Sectoral Monitoring Committee....................................................................................................92

Services Provided by Different Institutions Related to HRD-OP Target Groups..........................92

Collaborations amongst Related Institutions................................................................................93

Evaluation of Institutional Collaborations.....................................................................................94

Development in Higher Education................................................................................................94

Removing Coefficient System to Promote of Vocational and Technical Secondary Schools’ Students to Universities...............................................................................................................94

Vocational Qualification Authority (VQA).....................................................................................94

Major HRD-Related Projects Implemented since 2007................................................................95

Major Outcomes and Evaluation of the HRD-OP Projects...........................................................96

Other HRD Related Projects........................................................................................................97

Yearly Major Developments in the Implementations of the HRD-OP...........................................98

Major Developments in the Implementation of the HRD-OP in 2007...........................................98

Major Development in the Implementation of the HRD-OP in 2008...........................................100

Major Developments in the Implementation of the HRD-OP in 2009.........................................101

Major Development in the Implementation of the HRD-OP in 2010...........................................102

2. Major developments since 2007 at EU level..............................................................................103

2.1 The Lisbon Strategy.............................................................................................................103

2.2 The Europe 2020 Strategy...................................................................................................104

2.3 The EU Employment Guidelines and the Broad Economic Policy Guidelines......................105

3 Coherence in objectives..............................................................................................................106

3.1 The Strategic Coherence Framework..................................................................................106

3.2 Coherence between HRD OP and the main EU strategy.....................................................107

Annex 3. Review of the results of ex-ante evaluation.........................................................................111

1. Partnership Approach................................................................................................................. 111

2. The study of the Current Situation..............................................................................................111

3. The Effectiveness of the planned strategy.................................................................................113

4. Indicators.................................................................................................................................... 113

5. Sustainability of Results.............................................................................................................113

6. Absorption Capacities................................................................................................................113

7. The ex-ante evaluation’s conclusions and recommendations regarding HRDOP implementation....................................................................................................................................................... 114

4

Annex 4. Study of the HRD OPs Effectiveness and Efficiency...........................................................117

1 The use and financial allocation of financial assistance and the financial management of the budget............................................................................................................................................ 117

Background information.............................................................................................................117

2. Review programme outputs and results in relation with the expenditures incurred at priority and measure levels; in the light of financial and physical progress.......................................................119

Statistical data............................................................................................................................ 119

Analysis..................................................................................................................................... 120

3. Examine the efficiency of Operation Selection Criteria...............................................................121

Background information.............................................................................................................121

4. Assessment of local absorption and implementation capacities.................................................123

Statistical data on local absorption capacities............................................................................123

5. Analysis of the accuracy of the budget allocations under measures..........................................126

Annex A5 The Study of Indicators......................................................................................................128

1. Overview and Introduction: The study on Indicators and the Relevance of the HRD OP...........128

PART I: Methodology and approach to the study on indicators......................................................128

I.1. Methodological note on Evaluation of reform programmes and strategies in Human Resources.................................................................................................................................. 128

I.2 Approach and limitations to the study of indicators...............................................................138

PART II: Output Indicators.............................................................................................................143

II.1 Priority axis 1.......................................................................................................................143

II.2 Priority axis 2.......................................................................................................................161

II.3 Priority axis 3.......................................................................................................................167

II.4 Priority axis 4.......................................................................................................................174

II.5 Priority axis 5: Technical Assistance....................................................................................177

PART III: Result Indicators.............................................................................................................180

III.1 Priority axis 1......................................................................................................................181




III.5 Priority axis 5: Technical Assistance...................................................................................217

PART IV. Reflexions upon the studies in Part I, II and III...............................................................219

IV.1 Findings, main conclusions and recommendations............................................................219

IV.2 Summary of the examination of Output Indicators in Part II................................................226

IV.3 Summary of the examination of Result Indicators in Part III...............................................250

Appendix to the study on Indicators: Well-developed indicators....................................................257

ANNEX B1. List of Meetings held.......................................................................................................267

Regional visits................................................................................................................................ 267

Meetings with central institutions....................................................................................................267

Other meetings and similar appearances.......................................................................................268

List of people met during meetings................................................................................................268

CENTRAL LEVEL......................................................................................................................268

REGIONAL LEVEL....................................................................................................................271

5

Annex B2. Terms of Reference..........................................................................................................273

Annex B3. Generic Question Frame for meetings..............................................................................286

Annex B4. Basic Survey on the functionality of the HRD OP: Questionnaire.....................................289

Annex B5. Survey on Indicators: Questionnaire.................................................................................296

6

Abbreviations

CAO Competent Accrediting OfficerCFCU Central Finance and Contracts Unit CGMTs Central Grant Monitoring Teams CoHE Council of Higher Education ESF European Social Fund EU European UnionEUD Delegation of the European Union to Turkey FAQ Frequently Asked Questions FMU Financial Management Unit GDP/c Gross Domestic Product per capita (inhabitant)GVA Gross Value AddedHCI Harmonised Consumer Prices (index)HDI Human Development IndicatorsHRD Human Resources DevelopmentHRD OP Human Resources Development Operational ProgrammeIMIS Integrated Monitoring Information SystemIPA Instrument for Pre-Accession Assistance IPA IR IPA Implementing Regulation IPA MD IPA Management Department IPTA Information, Publicity and Technical Assistance Unit ISCED International Standard Classification of EducationISCO International Standard Classification of OccupationsİŞKUR Turkish Employment AgencyKOSGEB Small and Medium Sized Enterprises Development OrganizationLE Larger EnterprisesMIPD Multi-annual Indicative Planning Document MIS Management Information System MoD Ministry of DevelopmentMoLSS Ministry of Labour and Social Security MoNE Ministry of National EducationNACE Statistical classification of economic activities in the European

CommunityNAO National Authorising Officer NGOs Non-Governmental Organisations NIPAC National IPA Coordinator NQS National Qualifications SystemNUTS Nomenclature of Units for Territorial Statistics (Regional Classification)OCU Operation Coordination Unit OIS Operation Identification Sheet OP Operational ProgrammeOS Operating Structure PES Public Employment ServiceQA Quality AssuranceQACU Quality Assurance and Control Unit RC OP Regional Competitiveness Operational ProgrammeRGMTT Regional Grant Monitoring and Technical Assistance TeamSMC Sectoral Monitoring CommitteeSME Small and Medium Sized EnterprisesSPO State Planning OrganizationSSI Social Security Institution SWOT Strengths, Weaknesses, Opportunities, and ThreatsTAT Technical Assistance Team

7

TNA Training Needs AnalysisToR Terms of Reference TURKSTAT Turkish Statistical InstituteTVET Technical and Vocational Education and TrainingVET Vocational Education and Training VQA Vocational Qualifications Authorities

8

1. Executive Summary

This evaluation aims to make an independent assessment of the overall relevance, effectiveness and efficiency of the implementation of the HRD OP with a view to provide all the stakeholders with reliable data, analysis and feedback that will help the upcoming OP revision.

During the Interim Evaluation, the Technical Assistance Team has carried out desk studies, has held more than 52 meetings with key stakeholders at both central and regional level, and has conducted surveys of semi-qualitative nature. The Interim Evaluation has been based on a combination of quantitative and qualitative studies with the main focus on the latter.

The examination of the efficiency of the management structure revealed some shortcomings in the functioning of the HRD OP, mostly in terms of level of knowledge about content and practice as well as experience. To some extent this reflects the functioning of the SMC, which has limited room for discussion and merely serves as top-down information.

The examination of the effectiveness and efficiency of the HRD OP indicates that the criteria in the HRD OP and Calls for Proposals are not completely identical, and that set of criteria is rather broad and open, which allows the applicants to propose rather different project ideas. The examination indicates that the chosen approach could thus dissolve the results and outputs in such a way that it would be difficult in the end to measure common impact.

Overall, the study indicates rather normal distribution of grants per region and per population unit though the variations are still significant. In case of some regions, variations are still notable. Some regions receive a lower amount of projects per population unit than others and also the success rate of applications appears to be lower in these regions. This indicates that the awareness of the grant schemes is still at a reasonable level but the quality of applications is lower than the quality of applications in more successful regions. It also appears that due to global competition, the regions with a strong proactive central institution as well as with strong project writing capacity tend to do better in the project selection process.

Local absorption capacity appears influenced by the possibilities of potential applicants at the time the calls for proposals are launched. Although this factor should not cause differences between regions, it could lower the absorption capacity to a certain extent.

The examination of indicators, their efficiency and relevance, revealed a total of 91 Output indicators. The examination of the 25 Result Indicators, however, revealed that approximately half of the named indicators were in fact Output indicators. But even worse, the examination clearly showed that the HRD OP has no Impact indicators which make ex-post evaluation fruitless if not directly impossible.

The Interim Evaluation finds that the current system is:• Insufficient since the current set of indicators lack Impact measures• Inappropriate since many indicators have no achievable baselines, lack data, lack

infrastructure, and in certain cases are not well defined. • Too ambitious due to the number of indicators

The Interim Evaluation has identified 30 main recommendations. Concerning the efficiency and effectiveness of the HRD OP management structure:

1. Ensure far more pragmatism and flexibility based upon know-how and knowledge of “content” instead of narrowly focusing on administrative procedures;

9

2. Ensure stronger commitment and involvement from the Operation Beneficiaries

Concerning the Monitoring Arrangements, notably the Monitoring Information System, MIS:3. Ensure that the MIS system becomes more user-friendly or at least open up for a dialogue on

how to ensure consistent entering of data to the system4. Ensure that the system actually reflects the need (the issue of the number and quality of

indicators, which definitely needs to be redefined)5. Ensure a interlink with basic data, including official statistics and administrative records when

needed for measurement6. Ensure solid training in quality assurance on all levels, notably on micro level where data are

entered7. Clean all records in close liaison with caretakers of administrative records, notably SSI,

İŞKUR, and MoNE8. Make quality controls on a permanent basis.

Concerning the efficiency and effectiveness of the HRD OP contractual, tendering, and financial procedures:

9. Ensure far more readiness and speed in the evaluation procedures and try to avoid conflicts in central level responses to implementation level

10. Consider to establish quotas in order to ensure that all regions have at least a limited number of projects reflecting the needs on the local level.

Concerning the efficiency and effectiveness of the HRD OP Indicators:11. Revise the current set of indicators. The need for a reform is urgent;12. Revise the indictors toughly until a decent and doable target is reached. The Interim

Evaluation proposes to bring the total number of indicators down to 18 indicators: six Output, six Outcome, and six Impact Indicators

13. Understand the basics in evaluation and design a list of indicators which summarizes the main and basic idea and purpose of the overall programme;

14. Establish a multi-level system of indicators, ensuring that the main evaluation reports only contains a very small number of smart indicators;

15. Establish solid training programmes in monitoring and evaluation, data control, and quality assurance, not least on regional level. Place ownership and delegate task where the single stakeholders can see the purpose.

In case the HRD OP intend to stand up to the words of integration of the horizontal issues of sustainable development and environmental protection the Interim Evaluation recommends:

16. to integrate impact indicators in the programme for the ex-post evaluation to ensure that the HRD OP has achieved to ensure sustainable development or least made progress in that direction

17. to redefine and establish outcome indicators and impact indicators which particularly focus on environmental protection

18. to ensure that environmental protection is acknowledged as a future demand in the educational sector notably on VET/TVET level (making it an integral part of new curriculum)

19. to ensure that topics related environmental protection is an integral part of all Lifelong Learning Trainings underneath Priority 3

20. to give priority to environmental protection topics in priorities and measures addressing gender equality as well as social inclusion

21. to establish Training Needs Analysis (anywhere in the complete programme) which addresses the topic of environmental protection both in order to grab a demand in time and partly to increase awareness of the subject among the employers.

10

The Interim Evaluation has performed a solid investigation on participation. The study reveals problems with participation on regional level mainly though some problems with the SMC construction are evident. The study of the participative approach recommends:

22. Support and continue the strengthening of the partnership approach on central level, not least with respect to the structure of the SMC and its meeting procedures

23. Actively strengthen, perhaps through legislation, a far more viable partnership approach on regional level.

And the regional level is subsequently examined where the main recommendations of the Interim Evaluation are:

24. To ensure a solid platform on the regional level with a consolidated and consistent structure, the build of a professional secretariat and governed by a tripartite body perhaps inspired by the way the RDAs are constituted

25. Ensure “promoters” (ambassadors) are based regionally and that the promoters are able to act as support in every phase. The TAT cannot see why the RGMTTs and the promoters are not directly interlinked or perhaps even the same persons

26. Ensure means of commitment from Social Partners and other main regional actors on a regional basis by placing ownership through direct involvement in the programming, monitoring and evaluation phases

27. Ensure far more active involvement from both İŞKUR and RDAs in regional programming, planning, design of indicators and targets;

28. Design objectives, measures, indicators and targets on regional level and establish a system of reporting in accordance to a template designed from central level (which could feed directly in to evaluation reports, like the SAR) to promote and reinforce regional involvement in the HRD OP

29. Ensure that local members are part of the process (and not just representative from the Growth centres) to ensure a far more active spread of the programmes internally in the regions

30. Ensure gender equality, local representation, and representation of target groups in all advisory boards to the HRD OP.

11

2. Introduction

2.1 Overview of the HRD OP priority structure, measures and objectives

The HRD OP priority structure is as follows: Priority axis 1: To attract and retain more people in employment, particularly by increasing labour force participation of women, and decrease unemployment rates especially for young peopleMeasure 1.1 To promote women's participation into the labour market, and increase female employment, including those formerly employed in agricultureMeasure 1.2 To increase employment of young peopleMeasure 1.3 To promote registered employmentMeasure 1.4 To improve the quality of public employment servicesSpecific objective 1: Promote labour force participation and employment of women, including those formerly employed in agricultureSpecific objective 2: To increase youth employmentSpecific objective 3: To promote registered employmentSpecific objective 4: To improve public employment services

Priority axis 2: To enhance investment in human capital by increasing the quality of education, improving the linkage between education and labour market, and raising enrolment rates at all levels of education, especially for girlsMeasure 2.1 To increase enrolment rates particularly for girls with a view to developing female human resources and access to labour marketMeasure 2.2 To improve the content and quality of Vocational and Technical EducationSpecific objective 1: To increase enrolment rates particularly for girls at all levels of education and vocational trainingSpecific objective 2: To increase the quality of education especially in vocational education and trainingSpecific objective 3: To develop innovative approaches to improve linkage between education and labour marketSpecific objective 4: To promote a “partnership approach” to modernize VET system

Priority axis 3: To promote a “partnership approach” to modernize VET system.Measure 3.1 To promote the development and implementation coherent and comprehensive strategies for lifelong learning.Measure 3.2 To increase adaptability of employees and employers by investing more in human capital.Specific objective 1: To promote Life Long Learning (LLL) Opportunities under a LLL Strategy.Specific objective 2: To improve quality of non-formal trainingsSpecific objective 3: To increase adaptability of employeesSpecific objective 4: To increase adaptability of employers in SMEsSpecific objective 5: To promote well-functioning of the National Qualifications System

Priority axis 4: To promote an inclusive labour market with opportunities for disadvantaged people, with a view to their sustainable integration into the labour force and combat all forms of discrimination in the labour market.Measure 4.1 To increase employability of disadvantaged persons, facilitate their access to labour market, and eliminate barriers for their entrance into labour market.

12

Measure 4.2 Better functioning and coordination among the institutions and mechanisms in the field of labour market and social protection particularly in order to facilitate the integration of disadvantaged persons into the labour market.Specific objective 1: To facilitate sustainable integration of the disadvantaged into the labour market.Specific objective 2: To improve the functioning and coordination of institutions and mechanisms in the field of labour market and social protection, particularly in order to facilitate the integration of disadvantaged persons into the labour market.

Priority axis 5 for Technical Assistance:Measure 1 Support for management, implementation, monitoring, control, evaluation and dissemination activities.Measure 2 Support for development of absorption capacity of final beneficiaries.Measure 3 Information and publicity activities.Specific objective 1: Strengthening the necessary capacity at central level to efficiently develop, implement, evaluate, monitor and control the IPA funds in the period 2007-2009 within the framework of HRD OP and improving the information about the ESF structures and best examples, providing assistance in the transition to Decentralised Management without ex-ante control of the EC Delegation.Specific objective 2: Increasing the absorption capacity of the final beneficiaries and administrative capacity of stakeholders which may get more responsibilities in the upcoming period.Specific objective 3: Increasing the information and public awareness with respect to the preparation for and effective use of IPA funds in Turkey in line with the HRD OP.

2.2 Purpose of the Interim Evaluation and evaluation questions

The main objective of this evaluation is to make an independent assessment of the overall relevance, effectiveness and efficiency of the implementation of the HRD OP with a view to provide all the stakeholders with reliable data, analysis and feedback that will help the upcoming OP revision exercise.

The specific objectives of the assignment are: - to assess the relevance (in particular with the strategic documents such as the Strategic

Coherence Framework , Multi-Annual Indicative Planning Document , employment guidelines, strategic community guidelines and the Lisbon Strategy), efficiency, effectiveness of the programme.

- to assess the complementarity between the HRD OP and Regional Competitiveness Operational Programme (RC OP).

- to provide data on the output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring system.

- to assess the achievement of the horizontal principles.- to identify best practices, factors for success and failure and provide recommendations to

improve the remaining programming and implementation of the HRD OP.

13

Table 1 Evaluation Questions

Q1 Relevance, efficiency and effectiveness of the HRD OP Q1.1 The appropriateness of the management structures formed at the levels of Operation Structure

and Operation BeneficiariesQ1.2 The use and financial allocation of financial assistanceQ1.3 The quality of projects designed to achieve the objectives, targets and indicators determined in

the HRD OPQ1.4 Main AnalysesQ1.4.1 A. Analysis of Previous Evaluation’s ResultsQ1.4.1.1 Review the results of the ex-ante evaluation Q1.4.2 B. Analysis of the Relevance of the HRD OPQ1.4.2.1 Analyse and review the major developments since 2007 at national and EU level that

influence the HRD OPQ1.4.2.2 Analyse the HRD OP’s coherence with the objectives of pre-accession assistance,

strategic documents such as the SCF, Lisbon Strategy, MIPD, employment guidelines, strategic community guidelines

Q1.4.2.3 Make a SWOT analysis of the beneficiaries and stakeholdersQ1.4.2.4 Analyse the accuracy, clarity, quality, usefulness and internal consistency of the overall

objectives, purposes, targets, indicators and eligible activities at the priority and measure level of the HRD OP

Q1.4.3 C. Analysis of the HRD OP’s Effectiveness and EfficiencyQ1.4.3.1 Review programme outputs and results in relation with the expenditures incurred at priority

and measure levels; in the light of financial and physical progressQ1.4.3.2 Collect data on indicators not provided through the regular monitoring systemQ1.4.3.3 Make an overall examination of the HRD OP’s programming, monitoring and evaluation

structures and procedures (in particular of the content, timeliness and compliance) to provide an assessment of the public administration systems’ efficiency and effectiveness in terms of constructing the necessary mechanisms for attaining the objectives under HRD OP

Q1.4.3.4 Assess the concrete progress of the HRD OP on the basis of the indicatorsQ1.4.3.5 Examine the efficiency of Operation Selection CriteriaQ1.4.3.6 Analyse the role of national bodies, the relations/co-ordination with local authorities,

institutions, beneficiaries and other stakeholders, the commitment of all the stakeholdersQ1.4.3.7 Assessment of local absorption and implementation capacitiesQ1.4.3.8 Analyse the financial management of the budgetQ1.4.3.9 Analyse the accuracy of the budget allocations under measures

Q2 Complementarity between the HRD OP and RC OP Q2.1 RC OP’s objectives have been achieved through programming and implementation of the HRD

OPQ2.2 RC OP has influenced the management structures of the OS and Operation Beneficiaries of

HRD OP

Q3 Output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring system

Q3.1 Analyse and provide data on output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring system

Q4 What are the results of the programme in achieving the horizontal principles during the design and implementation of Operations?

Q4.1 Equal opportunities for men and womenQ4.2 Sustainable development & environmental protectionQ4.3 Participation of civil societyQ4.4 Geographic, sectoral and thematic concentration

14

Q4.5 Concerns of disadvantaged personsQ4.6 Good governance

Q5 Best practices, factors of success and failure regarding the planning and implementation process of the HRD OP

Q5.1 Propose recommendations to the actors involved in the programming, determining financial allocations, monitoring and evaluation

2.3 Outline of the Interim Evaluation Report

Chapter 3 describes the main methods and the approach to the interim evaluation subject to the evaluation questions just described. The chapter further describes some of the limitations in the study, including the problems encountered and the way the challenges subsequently were handled.

Chapter 4 subsequently contains the main report on findings, conclusions and recommendations. Some parts of the study are however viewed as less important for the main report and are subsequently addressed in annexes. That goes for the Socio-Economic Background Study, the study of Major Developments on National and EU level, the study of the Ex-ante Evaluation Results, and the Study on Indicators. The three parts, which basically is part of the set of evaluation questions raised, can be found in:

Annex A1 Socio-Economic Study Annex A2 Major Developments on National and EU level since 2007 Annex A3 Results of the ex-ante evaluation Annex A4 Annex to the study of Efficiency and Effectiveness Annex A5 The study on Indicators

Chapter 5 provides an overview of the complex of annexes to the report.

Finally, as more or less independent parts of the Interim Evaluation Report, two Thematic Reports have been developed which due to methodological considerations can be difficult to separate from the rest of the studies:

Thematic Report I on Complementarity between RC OP and HRD OP Thematic Report II on Horizontal Issues

15

3. Main methods

3.1 General approach to the Interim Evaluation

The methodology is based on the best practice procedures in critical academic evaluation of programmes and templates to which the ToR refer.

The overall methodology and approach to the assignment is based on a high degree of transparency and stakeholders’ involvement. The approach is based upon international standards of good practices in evaluation of programmes. Moreover, our approach has focused on making sure that every element of the studied practices and outcomes are understood and agreed by the stakeholders.

The Interim Evaluation has focused on four types of indicators following good practices as recommended by EU external programme and as described in Chapter 8. The main purpose of the evaluation is to evaluate the HRD OP in accordance to the following:

Relevance: The relevance of the programme relates primarily to its design and the extent to which the stated objectives correctly address the identified problems or real needs. It includes an assessment of the quality of programme and its design (that is, the logic and its justification in real issues, and the internal logic and coherence of the programme design).

Efficiency: This assesses how well the various activities (as set out in the logical framework, work plans and work) transformed the available resources into intended results in terms of quantity, quality and timeliness. This generally requires comparing alternative approaches to achieving the same results, to see whether the most efficient process has been adopted.

Effectiveness: The effectiveness of the programme relates to how far the results of the programme have achieved the actual programme purpose and how assumptions have affected programme achievements. This will include a specific assessment of the benefits accruing to target groups, including women and men and identified vulnerable groups such as children, the elderly and disabled (where appropriate).

Impact: The impact of the programme involves an assessment of the relationship between the purpose and overall objectives of the programme and the extent to which the benefits received by the target beneficiaries have had a wider overall effect on larger numbers of people in the region and the wider policy or sector objectives (as summarised in the programme’s Overall Objective).

Sustainability: This relates to whether the positive outcomes of the programme are likely to continue after external funding ends, with particular reference to factors of ownership by beneficiaries, policy support, economic and financial factors, socio-cultural aspects, gender equality, appropriate technology, environmental aspects, and institutional and management capacity.

16

3.2 The evaluation methodology subject to the evaluation questions

The Socio-Economic Study in Annex A1

The Report includes a Socio-Economic Analysis based upon a limited number of socio-economic indicators within the areas of economic, demographic, educational, and labour market indicators. The purpose is to understand the relevance and the scope of the HRD OP and to evaluate the Operation Selection Criteria.

The socio-economic background study has drawn on a list of international harmonized indicators, including not least ETF indicators and indicators for the use of monitoring the European Employment Strategy. Additional indicators e.g. HDI-indicators (UN) and KILM (ILO) have been tested for relevance and efficiency in evaluation. However, the Interim Evaluation found that the most obvious indicators are not available on NUTS 2 level. Nevertheless, the Socio-Economic Study provides insight in data which can be used as indictors of the HRD OP.

The desk study was performed including an examination of the management information system, MIS, for the examination of the management structure and effectiveness and efficiency of the HRD OP. The findings of the desk study were subsequently followed up by a long sequence of meeting with various actors on central and regional level.

Mapping of the institutional environment and studying current practices

A mapping of the institutional environment in which the HRD OP operates has been conducted during the very early phase of the implementation. The purpose of the mapping was partly to understand the institutional and organisational structure of stakeholders in the HRD OP, and partly to ensure a solid background for the evaluation of the manage structures in HRD OP.

The mapping was used for the examination of roles and the functioning of the management structure of the HRD OP.

In the evaluation of current practices under evaluation question 5 the Consultant primarily focused on the conception of the various stakeholders. The Consultant has substantial experience in involvement of stakeholders in the process. The Consultant especially focused on evaluation of practices and outcomes of implementing the OP.

In that respect, bilateral meetings with the stakeholders has ensured solid feedback on the perception of practices. Where practices including logistics and organisation of working procedures appear impractical or less feasible the Interim Report has proposed recommendations for improving the system.

Accordingly, the study of current practices is based upon a quantitative approach with a high degree of self-evaluation and self-reflexion. A question frame based upon standardized questions directed to all target institutions, beneficiaries and stakeholders was designed in order to evaluate relative weakness and strengths.

During the complete examination of roles, structure and procedures the Interim Evaluation has explicitly assessed the involvement of external stakeholders, both employers and employees, in the HRD OP. The main findings and observations made are mainly addressed in the Thematic Report on Horizontal Issues.

17

Finally, a survey was conducted in the very end of the project, providing further support to the findings from the examination.

The logic of the Report and the way the evaluation question was addressed

The logic of the Interim Evaluation Report is fundamentally based upon the assessment structure followed by the evaluation questions. The number of evaluation questions is fairly impressive and so is the underlined tasks connected to each of the evaluation questions. This section briefly introduce the way the evaluation questions has been handled, assessed, and approached in the report.

Q1 Relevance, efficiency and effectiveness of the HRD OP Q1.1 The appropriateness of the management structures formed at the levels of Operation Structure and Operation Beneficiaries

The main approach to the study of the is obviously of qualitative nature where the main point is to how well the current structure is designed to cope with the both the administrative tasks related to running a large scale HRD OP as well as how the management structure has the natural ability to cope with the technical and methodological problems connected with actions within the area of Human Resource Development.

In general, the organisation of HRD strategies is a complex matter somehow illustrated by figure 1.

18

Figure 1. The organisation of Human Resource Development Strategies and the compliance with other prime strategic policy areas

The figure shows how HRD strategies involve a number of core policy areas, which all have to be taken into account. And yet again, the figure is only illustrative in terms of displaying the interaction with other main policy areas; obviously it does not illustrate the need for alignment of policies and hence the need for coordination and communication between the core policy areas (and often Ministries and their administrative bodies). But more so, the figure only stipulates how the various geographical levels have to be taken into account: It appears obvious that it tend to be a complete waste of means to implement actions where the need is only marginal. And finally, the ways to incorporate the views and opinions of the stakeholders involved in HRD is hardly reached in the figure, though still being of vital importance for the ability of the strategy to address the needs and to achieve the goals of the strategy.

In the assessment of the relevance of the management structure of the HRD OP in Turkey the Interim Evaluation has focused on the following:

The extent the objectives of the HRD OP are in compliance with the overall national agenda in Annex A2

The extent the objectives of the HRD OP are mirrored in the agenda of line ministries in Annex A2

The extent the objectives of the HRD OP comply with the national agenda as described in Annex A2

The extent the objectives of the HRD OP comply with the overall agenda and strategies in the EU as assessed in Annex A2

The extent the objectives of the HRD OP are mirrored in the Indicators and their targets as assessed in Annex A5

19

The extent the administration of the HRD OP are taken on board by the management structure as assessed in Chapter 4 and Annex A5

The extent the administration of the HRD OP are able to direct the incentives and actions into a HRD Framework as assessed in Chapter 4, which obviously includes the ability to make solid follow through monitoring arrangements (Chapter 4) and evaluation as assessed in Annex A5

The extent the Social Partners and similar private and public stakeholders are taken into account as assessed in Thematic Report II

The extent the Social Partners are ready to take on board the responsibilities in a participative approach as assessed in Thematic Report II

The extent the overall structure within the governmental institutions involved in HRD strategies are ready to take on board the responsibilities to manage, design policies and implement actions within the area of HRD as assessed in Chapter 4

The extent the central administration approach the partnership approach as assessed in Chapter 4 and Thematic Report II

The extent the regional and local level needs and priorities are taken into account in the overall framework, the objectives, the management (on consultancy basis), and in identification of indicators and their targets as assessed in Thematic Report II, Chapter 4 and Annex A5

In addition, the assessment includes a SWOT on the main stakeholders (see below).The methods used are a mix of quantitative and qualitative studies based upon desk studies, studies of data from the central Monitoring System, the national statistical office (TÜİK/Turkstat), and administrative records (unfortunately primarily only from İŞKUR, the Turkish Public Employment Service), and interviews with key persons in both regional and central institutions.

In addition a survey was conducted to collect structure viewpoints and opinions on the overall management structure of the HRD OP as well as its relevance. The results from the survey had only been used to support the findings in the evaluation studies undertaken through other means and methods; the survey was not designed to stand alone insofar the main point was to obtain some very fragmented viewpoints including providing a method about the knowledge of the HRD OP among the stakeholders.

A total of 52 meetings were held during the preparation of the Interim Evaluation Report, including 26 meetings on regional level and 26 meeting with key stakeholder institutions on central level. The meetings were conducted according to a pre-prepared question frame (see Annex B7 as well as the methodological sections in the studies).

Q1.2 The use and financial allocation of financial assistanceQ1.4.3 C. Analysis of the HRD OP’s Effectiveness and EfficiencyQ1.4.3.1 Review programme outputs and results in relation with the expenditures incurred at priority and measure levels; in the light of financial and physical progressQ1.4.3.5 Examine the efficiency of Operation Selection CriteriaQ1.4.3.7 Assessment of local absorption and implementation capacitiesQ1.4.3.8 Analyse the financial management of the budgetQ1.4.3.9 Analyse the accuracy of the budget allocations under measures

All the evaluation questions related to procedures, financial allocation, expenditures, local absorption and implementing capacities, accuracy and financial management are gathered in the examination of the HRD OP’s Effectiveness and Efficiency. The main judgement criteria for these entrances to the

20

study of the effectiveness of the HRD OP follows best practices and should otherwise be evident from the context, since the main point is to assure effectiveness in the programme.

Q1.3 The quality of projects designed to achieve the objectives, targets and indicators determined in the HRD OP

The Interim Evaluation has only limited means to undertake a full study of the quality of the projects within the complete HRD OP for obvious reasons, notably the allocated time. Thus the main focus is to evaluate the relevance of criteria for selection as well as the criteria for follow up in terms of output, outcome, and impact of the operations and measures taken. The evaluation method is further supported by the overall delays in the complete programme, which has had created delays in the implementation of the single projects, and thus created limited means to study the overall impact of the measures taken, and thus the projects assigned to meet the purposes of the objectives.

The Interim Evaluation has subsequently made use of a mix of methods to evaluate the quality of the projects (or perhaps rather the measures and actions taken) based upon the following:

An assessment of regional and local absorption in Chapter 4 An assessment of the regional and local stakeholders’ ability to have impact on the projects

implemented in their region in Thematic Report II An assessment of the selection criteria on an overall level in Chapter 4 An assessment of the Output indicators to monitor the projects in Annex A5 An assessment of the Result indicators to monitor and evaluate the projects and their impact

in Annex A5

Q1.4.1 A. Analysis of Previous Evaluation’s ResultsQ1.4.1.1 Review the results of the ex-ante evaluation

A study of the follow-up on the ex-ante comments and recommendations has been undertaken throughout the Interim Evaluation studies. Basically, the Interim Evaluation has no finer criteria and method underlined the follow-up except from the traditional examination based upon 1) Desk studies; 2) Interviews with prime stakeholders on central and regional level; and 3) the examination of the relevance of priorities, measures, objectives, targets and indicators which follows from the general examination of the relevance of the HRD OP. The main results of the study can be found in Annex A3 (though parts of the study are reported into the studies elsewhere in the Interim Evaluation Report).

Q1.4.2.1 Analyse and review the major developments since 2007 at national and EU level that influence the HRD OPQ1.4.2.2 Analyse the HRD OP’s coherence with the objectives of pre-accession assistance, strategic documents such as the SCF, Lisbon Strategy, MIPD, employment guidelines, strategic community guidelines

The study on Major Developments since 2007 follows no general practical overall method per se. The general approach is plain desk study and can be found in Annex A2 to the Interim Evaluation Report.

Q1.4.3.6 Analyse the role of national bodies, the relations/co-ordination with local authorities, institutions, beneficiaries and other stakeholders, the commitment of all the stakeholdersQ1.4.2.3 Make a SWOT analysis of the beneficiaries and stakeholders

The making of SWOT analysis is definitely not done lightly. All SWOT analyses suffer from a high degree of subjectivity and cannot be used as background for scientific studies without an extremely

21

clear methodology, which basically eliminate the complete structure of the SWOT insofar the SWOT analysis always serves to the purpose of making clear statements based mainly on a high degree of self-assessment. The nature of SWOT is subjective whether performed by an independent observer or done as a self-assessment exercise among the involved stakeholders.

The main approach to the SWOT conducted in this report is based upon a subjective assessment of the criteria mention under Q1.1 above; which means the ability to perform and conduct the management of a HRD strategy based upon observation and reflexions obtained either from own observations through meetings with the examined organisations and from other stakeholders within the programme; normally called a 360 degrees assessment.

The meetings with the stakeholders, including the OS Units, followed more or less the same structure and were based upon question frames which included question within the following brackets:

Function of the unit and job functions Responsibilities Procedures for follow-up Work flows Problem areas, including:

o Understaffingo Poor coordination (which includes not least ability to perform Social Dialogue)o Lack of legal basis (how does that influence your work)o Ownershipo Involvement with external stakeholders

The Interim Evaluation TAT, however, accentuates that the SWOT in this Interim Evaluation Report cannot stand alone since it refers and relates to the studies in other parts of the report.

Q1.4.2.4 Analyse the accuracy, clarity, quality, usefulness and internal consistency of the overall objectives, purposes, targets, indicators and eligible activities at the priority and measure level of the HRD OP

The evaluation question draws upon various activities and not least competences which cannot be gathered in one single approach. In general, the main part follows the approach mentioned under Q1.1 above, but in addition the study related to procedures, financial allocation, expenditures, local absorption and implementing capacities, accuracy and financial management gathered in the examination of the HRD OP’s Effectiveness and Efficiency above as well as (and not least) the study on indicators mentioned below.

As for the Interim Evaluation, the study on the relevance of the chosen indicators in Annex A5 combined with the study on the readiness of the management structure and the study of Effectiveness and Efficiency both in Chapter 4 address the highly complex matters mentioned in the evaluation question.

The study on Indicators also constitutes a study of relevance of the objectives in combination with the Socio-Economic Study in Annex A1.

Q1.4.3.2 Collect data on indicators not provided through the regular monitoring systemQ1.4.3.3 Make an overall examination of the HRD OP’s programming, monitoring and evaluation structures and procedures (in particular of the content, timeliness and compliance)

22

to provide an assessment of the public administration systems’ efficiency and effectiveness in terms of constructing the necessary mechanisms for attaining the objectives under HRD OPQ1.4.3.4 Assess the concrete progress of the HRD OP on the basis of the indicatorsQ3 Output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring systemQ3.1 Analyse and provide data on output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring system

The study of the indicators was based upon a combined quantitative and qualitative approach since the study soon revealed serious shortcomings in the design of indicators. The study of indicators was subsequently focused to explain the fundamental principles in establishing performance indicators and targets based upon a solid examination of each of the indicators proposed by the HRD OP.

With respect to output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring system, the consultant has screened all available data obtained from the operating system as well as the data supplied from the main actors.

Where data on indicators were not available, the Interim Evaluation has made a note on the lack of data / information and proposed ways to obtain the data / information in the future (methodology) in accordance to the methods described in the Inception Report. When it was impossible to obtain the required data / information or in case the data are inconsistent, misleading, or not relevant for the measures new alternative indicators have been proposed.

The study of the Result and Output indicators includes:• Screening of data and information streams, including accessibility and production dates• Study of baselines and their relevance for the time period of HRD OP• Study of consistency and validity of data and information, which includes collection

methods• Irregularities in deliveries of data and information• The relevance of the pre-selected measures and indicators as measures of medium and

long term outcome and impact assessments• And in addition the TAT will obtain information and comments to the measures and

indicators from beneficiaries and stakeholders in the programme• Analysis of data provided mainly in order to evaluate relevance• Propose alternative ways to collect data and information where data and information are

lacking and not available in the nearest future OR proposed new indicators which have relevance and are accessible.

An important part of the evaluation of indicators was to evaluate the process of producing high quality data and information available for future monitoring and evaluation as well as programming process. In that respect, the Interim Evaluation has found it necessary to accentuate the need for ensuring consolidated data.

A separate introductory part of the Annex A5 on the study of indicators within the Interim Evaluation Report is almost exclusively devoted theoretical aspects in the respect, simply because the current system of indicators needs to be improved.

Obviously, the HRD OP needs to be evaluated in accordance to an international set of practices allowing the Turkish stakeholders to make efficient use of international benchmarking. The Interim Evaluation has accordingly tried to give some hints to how to do effective monitoring and evaluation.

23

Q2 Complementarity between the HRD OP and RC OP Q2.1 RC OP’s objectives have been achieved through programming and implementation of the HRD OPQ2.2 RC OP has influenced the management structures of the OS and Operation Beneficiaries of HRD OP

In order to understand how well the two programmes complement each other in Turkey the Interim Evaluation has made a solid desk study of the main reports, including the two programmes, the progress reports, the Strategic Coherence Framework, SCF, and the final report of the ex-ante evaluation. Based upon the desk research the Interim Evaluation has conducted interviews with main stakeholders on central and regional level, including the contractors and the two main ministries involved in the management of the programme. And finally a survey was conducted where some few questions was designed to get viewpoints on the complementarity between the two programmes.

The feedback on potential problems and conflicts were sparse and hardly worth mention. Nevertheless, the Interim Evaluation has gathered and reflected the main viewpoints in the Thematic Report I on Complementarity between the RC OP and the HRD OP.

Q4 What are the results of the programme in achieving the horizontal principles during the design and implementation of Operations?Q4.1 Equal opportunities for men and womenQ4.2 Sustainable development & environmental protectionQ4.3 Participation of civil societyQ4.4 Geographic, sectoral and thematic concentrationQ4.5 Concerns of disadvantaged personsQ4.6 Good governance

The study of the achievements in meeting the horizontal principles are based upon a variety of studies, based upon both quantitative and qualitative methods, notably following the methods mentioned under Q1.1 above, and further heavily supported by interviews with stakeholders, the study on indicators (which basically examine how well the principles are integrated in the programme), desk study on key national documents, and lightly supported by the basic survey mentioned elsewhere above.

The findings, conclusion and recommendations from the intensive studies are gathered and presented as main findings, conclusion and recommendations in the Thematic Report II on Horizontal Issues which further presents a description of the methods used to examine the principles where the methods differ from the overall methodological approach of the evaluation.

Q5 Best practices, factors of success and failure regarding the planning and implementation process of the HRD OPQ5.1 Propose recommendations to the actors involved in the programming, determining financial allocations, monitoring and evaluation

The study of Best practices within the Interim Evaluation Report has followed two main routes: 1) international best practices have been used mainly as an intellectual reference throughout the study and incorporated as some underlined standard where it made sense for the evaluators; and 2) best practices observed from meetings around the country.With respect to international best practices the practices has been used as reference points in the study of Efficiency and Effectiveness and in the study of the Management Structure both in Chapter 4. But more so international best practices have been heavily drawn upon in the study on Indicators in Annex A5 and in the examination of the functionality of the MIS in Chapter 4.

24

The Interim Evaluation Report, fully acknowledging the special conditions of Turkey, the largest country doing HRD OP and experiencing (in EU context) a variety of cultural traditions still completely not familiar to the EU context, procedures and agendas, has accordingly only drawn upon international best practices where there was a solid pragmatic and intellectual reason to do so and yet still in accordance to feasibility given the working habits and working traditions which applies; that is in the theoretical context of evaluation theory which forms the main approach to the study of indicators and in the routes to make electronic MIS-systems work in practice1. Furthermore, international best practices have further been used as a reference for providing recommendations to participation and involvement of civil society in Thematic Report II on horizontal principles.

When it comes to best practices from Turkey, which the Interim Evaluation team perhaps think is slightly more relevant for the Turkish stakeholders as reference points, the Interim Evaluation Report primarily base its recommendation upon observed practices through meetings with key stakeholders on regional and central level. In that respect, the Interim Evaluation report has benefitted to a large extent from the relative high number of meetings conducted2.

Observation of best practices are inserted where appropriate in the context of studies, though not necessarily explicitly (international best practices) unless the Interim Evaluation found reasons to use the best practices observed as a reference point for the Turkish stakeholders (notably in Thematic Report II on Horizontal Issues and almost exclusively with reference to best practices from Turkey).

3.3 Challenges and solutions found

Throughout the Interim Evaluation the Technical Assistance Team (TAT) has faced several unforeseen challenges. The TAT has done its upmost to meet the challenges and through a high degree of flexibility, not least in the timing of crucial events, the TAT has been able to find doable solutions.

Problems experienced Impact The way they were handledLong inception phase Difficulties in starting operation

Postponement of meetings Postponement of surveys

The consultant started operation before approval of the Inception Report

Postponed meetings on central level

Postponement of meetings at regional level

Postponement of surveys

Rescheduling of work plans

Postponed meetings at regional level

Postponed surveys Postponed follow-up

Rescheduling of work plans

Postponed surveys Postponed analysis phase and report phase

The survey was done during the reporting phase

Difficulties in gathering basic data

Hampered Data quality examination Hindered evaluation of relevance and

availability of indicators

Examination of the Indicators was conducted on a theoretical basis mainly with heavy inclusion of best practices and drawing on experience form impact studies in general

1 The team has in particular benefitted from a newly conducted scientific research study for the TVET in Egypt on International Best Practices in establishing MIS, Knowledge Systems for TVET, LMIS (Labour Market Information Systems), and CGS (Carrier Guidance Systems) not published yet conducted by one of the members of the TAT in the evaluation of the MIS. 2 relative to the requirements of the TOR, which requirements the TAT found far too low notably to assess best practices

25

4. Main findings, conclusions and recommendations

4.1 Examination of the Management and control structures of the HRD OP and SWOT

The structures and authorities as well as their functions and responsibilities are described in the HRD OP3. They are in compliance with the relevant provisions of the IPA Implementing Regulation (Articles 21 to 31).

Under the management and control provisions of the Regulation and Article 6 of the Framework Agreement, the following structures and authorities are designated as management structures and authorities:

National IPA Coordinator Strategic Coordinator Competent Accrediting Officer National Authorising Officer National Fund Audit Authority Operating Structures per component or programme to deal with the management and

implementation of assistance under the IPA Regulation;

With the exception of the Operating Structure, these bodies essentially perform tasks that are generally applicable to all IPA components; their functions are specified in the relevant articles of the IPA Implementing Regulation and in Annex A of the Framework Agreement concluded between the Commission and Turkey.

The meetings with the stakeholders, including the OS Units, followed more or less the same structure and were based upon question frames which included question within the following brackets:

Function of the unit and job functions Responsibilities Procedures for follow-up Work flows Problem areas, including:

o Understaffingo Poor coordinationo Lack of legal basis (how does that influence your work)o Ownershipo Involvement with external stakeholders

The SWOT tables mention some of the main observations from a very broad scale.

4.1.1 Operating Structure (OS) 4

3 Human Resource Development Operational Programme, second version4 Please note that the EUD, the CFCU, and to some extent the National Fund, National Authorizing Officer, and the National IPA Coordinator, all form an integral part of the Operating Structure of the HRD OP. However, the two (or four) institutions are not explicitly examined in the section, mainly since their functions and roles for the HRD OP are on a very aggregated level and/or since their functions and roles only are marginally connected to HRD OP alone. Accordingly, a full examination based upon a 360 degree study of their management structure would have only very limited value. The named institutions are accordingly not explicitly examined in the following sections.

26

The HRD OP describes that the operating structure shall be responsible for managing and implementing the IPA programme or programmes concerned in accordance with the principle of sound financial management. The Ministry of Labour and Social Security is designated as Operating Structure for the HRD Component by the Prime Ministry Circular no: 2009/18, and will be managed by the Ministry of Labour and Social Security, which in compliance with the Annex A of the Framework Agreement carries out the following functions:

drafting the annual or multi-annual programmes; monitoring programme implementation and guiding the work of the sectoral monitoring

committee as defined in Article 36(2) of this Framework Agreement and in Article 59 of the IPA Implementing Regulation, notably by providing the documents necessary for monitoring the quality of implementation of the programmes;

drawing up the sectoral annual and final implementation reports defined in Article 38(1) and (2) of this Framework Agreement and in Article 61(1) of the IPA Implementing Regulation and, after their examination by the sectoral monitoring committee, submitting them to the Commission the NIPAC and the NAO;

ensuring that operations are selected for funding and approved in accordance with the criteria and mechanisms applicable to the programmes, and that they comply with the relevant Community and national rules;

setting up procedures to ensure the retention of all documents regarding expenditure and audits required to ensure an adequate audit trail;

arranging for tendering procedures, grant award procedures, the ensuing contracting, and making payments to, and recovery from, the final Beneficiary;

ensuring that all bodies involved in the implementation of operations maintain a separate accounting system or a separate accounting codification;

ensuring that the NF and the NAO receive all necessary information on the procedures and verifications carried out in relation to expenditure;

setting up, maintaining and updating the reporting and information system; carrying out verifications to ensure that the expenditure declared has actually been

incurred in accordance with the applicable rules, the products or services have been delivered in accordance with the approval decision, and the payment requests by the final Beneficiary are correct: These verifications shall cover administrative, financial, technical and physical aspects of operations, as appropriate;

ensuring internal audit of its different constituting bodies; ensuring irregularity reporting; ensuring compliance with the information and publicity requirements.

In addition to the above-mentioned responsibilities resulting from the Annex A of the Framework Agreement, the Operating Structure will also be responsible for:

managing the Secretariat of the HRD OP Monitoring Committee; ensuring the evaluations of the HRD OP; implementing the measures under Technical Assistance of the HRD OP.

The HRD Operating Structure is according to the HRD OP composed of the following bodies: 1. IPA Management Department

1.1. Programming Unit1.2. Monitoring and Evaluation Unit1.3. Information, Publicity and Technical Assistance Unit1.4. Technical Implementation Unit1.5. Quality Assurance and Control Unit

2. Internal Audit

27

3. HRD FCU5

Since the HRD OP was described the composition of the Units has been altered. The structure is now as follows:

1. The Programming Unit, constituting a staff of 11, has been divided into three sections: a. Programme Management Section, b. Procurement Section, and c. Tender Evaluation and Contracting Section.

2. Financial Management Unit (FMU) is composed of 3 sections and 11 staff3. Monitoring and Evaluation Unit (M&E) has conducted its activities with 4 staff4. The Information, Publicity and Technical Assistance Unit (IPTA) have a staff of 5 persons 5. Contract Management Unit has 11 staff6. The Quality Assurance and Control Unit (QACU) is performing its duties with 5 staff members7. The Administrative Office has 8 staff

The strong sides of the OS are the young and talented teams in all units and their seemingly ability to inspire each other. No doubt that they are skilful and well-educated. The Interim Evaluation has made the following comments on the work of the OS inserted directly in a SWOT study.

SWOT assessment of the Operating Structure (all units)Strengths

Young team with will-power and encouragement

Strong will to do everything “by the book” and in accordance to regulations

Professionalism in approach to the tasks Knowledgeable in procedures

Weaknesses Less experience in HRD (content) The young talent and the will power to prove

generates a stiff and rigid discipline Lack of pragmatism makes compromises

difficult6

The handling of the regional dimension appears weak and founded on principles of establishing a unified system (the lack of cooperation between central and regional levels appears visible)

Lack of knowledge in how to ensure dialogue between various stakeholders, including private stakeholders and Social Partners on local, regional and national level in order to

5 In terms of the Cooperation Agreement on Delegation of Certain Tasks of Operating Structure to the Central Finance and Contracts Unit for the Transition Period, all procedural aspects of the tendering process, contracting matters and financial management including payment of project activities are delegated to the CFCU, under the IPA component IV for a transition period of 2007-2010. After the transition period, a Finance and Contracts Unit for the HRD OP in the MoLSS will be established.6 “Pragmatism is a philosophical tradition centred on the linking of practice and theory. It describes a process where theory is extracted from practice, and applied back to practice to form what is called intelligent practice. Pragmatism is the philosophical attitude that the validity of an idea lies in its practical consequences. Pragmatism is the first American philosophy developed independently of European schools of thought. Pragmatists have agreed with traditional empiricists that ideas must be tested against experience. However, they have departed from the empirical stress on the origin of ideas in experience in order to lay a new stress on the effect of ideas in experience.” [Quotes from Wikipedia]Pragmatism is the ability to make things work in practice where scholared books and procedures may insinuate more pure and straightforward measures. A certain degree of Pragmatism is notably necessary to make actions where private stakeholders, notably the Social Partners, simply know better how to create an effect on certain policy measures (or where the lack of involvement from Social Partners may create distortions on the labour market), which is evidently necessary in every measure related to Human Capital formation. It further stipulates that compromises are the only way forward. In this case, the Interim Evaluation insinuate that the OS perhaps tries to be slightly too ambitious in terms of pursuing (and pushing forward) certain goals, objectives, measures and target, making the OS slightly blind for the need for consensus about the measures, the actions, the targets, the target groups, and perhaps notably how to address the target groups. Accordingly, Pragmatism as a weakness and a threat simply reflects the point that the OS constitute of general administrators with perhaps only limited experience from practice and negotiation.

28

ensure “content” Seemingly lack of knowledge on labour

market and educational issues (the OS works as administrators, not experts on the subject)

Opportunities The accreditation and the forthcoming ability

to govern and manage things more independently may form a more flexible platform

With age and not least experience a more pragmatic and flexible attitude can be created

Threats Lack of pragmatism can easily halter the

Social Dialogue and create severe practical problems for the regional and local level actors

The focus on doing everything correct can impose too many restrictions to the local implementation level, where flexibility is needed to obtain results in HRD

Lack of legal basis

In general, the Units lack experience from practice. The hardworking units consist of young highly talented and well-educated persons, perhaps even ambitious, somehow eager to do everything “by the book”. The feature is of course appreciable but not entirely flexible and do hardly stand a change when dealing with HRD issues on a practical level which necessitate involvement of external stakeholders to a high extent, including social partners.

From the Interim Evaluation point of view it appears quite evident from the overall structure of the HRD OP that the OS mainly maintain the duties as general administrators, which means ensuring that the monitoring mechanisms are in place, that the stakeholders, including the external stakeholders, are taken into account, that the procedures, including the PRAG, are followed, that the selection of project are done in compliance with the regulations, and that the monitoring reports, including the MIS, works efficiently. That is obviously very appreciable.

However, and at the same time, the Interim Evaluation had difficulties in being assured that the OS level also possesses the methodological and theoretical expertise and know-how in labour market issues, educational issues, and social issues, to ensure solid feedback to the central key institutions involved in the programme. The last thing obviously also necessitate know-how in labour market analysis, Training Needs Analysis, and similar targeted surveys of labour market demand and training needs, know-how in design of Active Labour Market Measures and Education Actions for demand-driven education and training, as well as know-how in how to organise actions on local, regional and central level. This know-how could easily turn out to be of paramount importance insofar the main bulk of the assessment of the relevance of the content in the programme as well as in single projects appear to be left to the Operation Beneficiaries almost solely7.

In other words, the core competences within the OS are within general administration indicating that the methodological and theoretical expertise and know-how in labour market issues, educational issues, social issues, and similar have been brought in from outside, including the MoLSS, which makes the OS units slightly narrow and vulnerable in their entire focus on procedures and administration. The dialogue is hardly achievable when it comes to content matters since the level of expertise is not founded and not generated from experience. But then again, perhaps it is not needed to administrate a HRD OP.

The assessment of the efficiency and effectiveness of the Operating Structure is thus that the Operating Structure has means to pursuit administrative procedures and ensure a stringent discipline insofar the units obtain the accreditation and get experience with their new roles thereafter.

7 The point was also addressed several by İŞKUR, the Public Employment Service of Turkey, on both central and regional level.

29

The more open question is how the lack of experience and knowledge of the content within the HRD areas will impact on the effects of the programme, not least sustainability. In that respect the lack of knowledge in establishing indicators from theory and practice is definitely an important feature when it comes to on-board the responsibility of defining measures and indicators which are workable in practice. Thus, and again, the study of indicators appears to be an indicator of the ability of the Operating Structure to be front runners in the HRD OP, perhaps in particular on the content.

4.1.2 The operation beneficiaries

The operation beneficiaries have the control of the implementation of the operations within the HRD OP. The operation beneficiaries are Turkish Employment Agency (İŞKUR), Social Security Institution (SSI), MoLSS, Ministry of National Education (MoNE), The Union of Chambers and Commodity Exchanges of Turkey (TOBB), and Ministry of Culture and Tourism.

The following table figures out the institutional and financial overview of the Operations being implemented under HRD OP:

Measure Title of the Operation Budget Operation Beneficiary Date of Signature of the Operational Agreement

M.1.1 Promoting Women’s Employment- I 27 MEUR Turkish Employment Agency (İŞKUR)

8th October 2009

Promoting Women’s Employment- II 30 MEUR Turkish Employment Agency (İŞKUR)

-

M.1.2 Promoting Youth Employment- I 29 MEUR Turkish Employment Agency (İŞKUR)

21st December 2009

Promoting Youth Employment- II 26 MEUR Turkish Employment Agency (İŞKUR)

-

M.1.3 Promoting Registered Employment Through Innovative Measures- I

13 MEUR Social Security Institution(SSI)

8th October 2009

Promoting Registered Employment Through Innovative Measures- II

7 MEUR Social Security Institution(SSI)

-

M.1.4 Improving the Quality of Public Employment Services- I

12 MEUR Turkish Employment Agency (İŞKUR)

21st December 2009

Improving the Quality of Public Employment Services- II

4 MEUR MoLSS -

M.2.1 Increasing Enrolment Rates Especially for Girls- I

16 MEUR Ministry of National Education (MoNE)

21st December 2009

Increasing Enrolment Rates Especially for Girls- II


-

M.2.2 Improving the Quality of Vocational Education and Training in Turkey- I


-

Improving the Quality of Vocational Education and Training in Turkey- II


-

M.3.1 Promoting Lifelong Learning-I 15 MEUR Ministry of National Education (MoNE)

8th October 2009

Promoting Lifelong Learning- II 15 MEUR Ministry of National -

30

Measure Title of the Operation Budget Operation Beneficiary Date of Signature of the Operational Agreement

Education (MoNE)M.3.2 Increasing Adaptability of

Employers and Employees to the Changes in Global Economy- I

5 MEUR The Union of Chambers and Commodity Exchanges of Turkey (TOBB)

-

Increasing Adaptability of Employers and Employees to the Changes in Global Economy- Tourism

10 MEUR Ministry of Culture and Tourism

-

Increasing Adaptability in the Field of Occupational Health and Safety

4 MEUR MoLSS

M.4.1 Promoting Active Inclusion in Turkey

24 MEUR Turkish Employment Agency (İŞKUR)

-

TOTAL 292 MEURSource: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme 2010, pp. 17-19

SWOT assessment of Operation Beneficiary: İŞKUR; the Turkish Public Employment Service

Strengths Solidly founded organisation with strong

commitment on both central and regional level Strong expertise in analysis and handling of

data Strong capacity in handling and management

of projects High degree of professionalism in labour-

market related issues within the context of HRD OP

Commitment but not without flexibility (experience with practical issues and challenges in HRD)

Strong will to listen and become engaged in new processes

Regional level has become much stronger and more active the past five years

Stronger focus on Active measures for a longer period has ensure readiness to deal with HRD issues

Solid data provider Solid partner as CGMT

Weaknesses Heavy organisation and perhaps with some

long communication lines Too little involved in the implementation and in

monitoring the HRD OP on regional level8 Still no uniform approach to barometer

surveys on labour market demand and shortages

Still slightly weak on analytical expertise on regional level (stronger cooperation with RDA appears to be encouraged, though; but still the RDAs expertise on these matters of labour market analyses appears an open question)

Opportunities In relation to HRD OP: Not relevant

Threats In relation to HRD OP: Not relevant

Both İŞKUR and MoNE are mature organisations with solid experience in project making, including donor funded projects. The SWOT analysis clearly illustrates the point. And this is where the similarities end. The nature of the two organisations determines their strengths but also the weaknesses. Not that it’s a bad sign – it is highly common – the issue is just a matter of how well the two organisations are able to make use of each other strengths for the benefit of the programme, and Turkey of course.

SWOT assessment of Operation Beneficiary: MoNE, The Ministry of National Education

8 There are 19 RGMTs. At the beginning, İŞKUR was asked to involve at least two experts in each RGMT but this was not done with the justification of workload of regional directorate.

31

Strengths Solid knowledge of project implementation Strong CGMT Solid knowledge on educational issues Massive organisation with huge possibilities to

actively make a demand driven impact

Weaknesses Heavy and difficult manageable administration

with many different entities, each with their own highly differentiated expertise

Too narrow focus on education in relation to HRD OP, mainly on topics concerning outcome

Far too weak in labour oriented themes, including issues of “demand driven education”

Statistical knowledge is still rather weak; the need for solid data appears not to be recognized

No uniform approach to TNA and surveys on Labour Market Imbalances and Shortages

Opportunities More bridge building between VET/TVET and

HE Potentials for design of new VET/TVET

courses with large know-how based on research

Research founded VET educations makes the inclusion of issues of environmental protection potentially more demand driven

Potentials for innovative educations which stimulates and reinforce the cooperation between the educational sector and private enterprises (inducing larger economic growth potentials)

Threats As in many other countries the stringent focus

on educational issues, including management of education and training institutions, can make the education and training institutions blind for employment effects

The Universities plays an important role in VET/TVET; it might give rise to:

o Less labour market related educations

o Weak sustainability in TVET sector due to the natural focus on HE (HE is the fundament, the platform, including the financial basis, for the universities obviously)

o Sustainability in TVET may be questioned as soon as the donor funded projects has ended (the economy is anchored in HE)

o And sustainability in TVET may be threaten if the economy slows down and goes into recession

From an outside perspective on behalf of the Interim Evaluation, and as mentioned above, the Operating Structure might appear slightly weak in know-how on implementation of Active Labour Market Measures and design of education action underneath the umbrella of Demand-driven Education and Training among other issues (including practical measures to ensure registration of work places, implementation of Occupational Safety and Health standards, establishing occupational and educational standards, labour market demand analyses and similar targeted studies, and presumably even coordination of these initiatives). On the other hand, that does not indicate that the practical know-how on these topics in not present in Turkey; the know-how is obviously anchored in the Operation Beneficiaries.

The Interim Evaluation recognizes and acknowledges the design of the management structure of the HRD OP which appears highly plausible from an organisational point of view, but still the Interim Evaluation fear that conflicts in measures, actions, policies, and perhaps even targets might arise without a solid, technical, cooperation between the main Operation Beneficiaries, hopefully boarded and controlled by a strong chair to avoid the almost global and everywhere present conflict in means notably between the two main policy areas of National Education and Public Employment Service9.

9 The reference to global level, which includes EU Member States as well, is not just theoretic. Most countries experience more or less permanently conflicts in vision, objectives, measures, targets, target groups, between the two main Ministries. Some countries have tried to combat the conflict by merging the two ministries unfortunately

32

The question becomes trickier when it comes to turn studies of demand into something workable for the educational sector, where İŞKUR still lack basic tools and techniques while the educational institutions suffer from relative high autonomy, notably as the VET/TVET sector is primarily founded on university grounds. The universities of Turkey may conduct their own training needs analyses and studies of local and regional demand, but the ability to see future demands as well as the ability to induce knowledge sharing across regions (allowing mobility of skilled labour to become a player in the fulfilment of needs) is hampered by autonomy and the subsequent lack of consensus in methods to display the main features of the current and the future labour market.

Again, we have to recall that studies of labour market and labour market demand and shortages are not the core expertise of the educational sector, and notably not the HE sector. In that respect a high cooperation with İŞKUR is needed in order to ensure sustainability but not least in order to ensure a sustainable economic development in the regions.

In that respect the communication between the two main players are yet to be proven for real (just like in many other countries). And perhaps the TAT was able to spot some sort of mismatch between the two organisations insight in the subject combined with their proven large experience with international projects, and the weak Operating Structure, not least when it comes to establish a viable agenda on other issues that formal and doll procedures and reporting routines.

There is no doubt that both organisations contain efficiency and effectiveness, notably when it comes to content. They are both experienced in procedural issues. Somehow exactly these facts may be a reason for the observed strange holding back towards the Operating Structure, allowing the Operating Structure to set the agenda for the HRD OP.

The Social Security Institution, SSI, involvement is personalized in one single employee. It appears fruitless to make SWOT on him if not directly inappropriate. The Interim Evaluation did not receive data from SSI, as mentioned a couple of times; hence the evaluation of the data handling and management of monitoring and evaluation cannot be examined.

4.1.3 The Social Partners and the Chambers

SWOT assessment of Social Partners and ChambersStrengths

Solid knowledge of project implementation Strong commitment Still stronger involvement and knowledge on

educational issues Strong willingness to take part in the process Flexibility and open, modern attitude towards

counterpart as well as the public sector

Weaknesses The strong commitment is still mainly on

central level (Social Partners; the Chambers work on regional level as well)

Especially the Trade Unions appear to have a weak infrastructure and may not have an exclusive focus HRD OP oriented regions and localities

Basically no tripartite systems, bodies and

with limited success. The Interim Evaluation has accordingly no doable solutions or recommendations to offer from Best Practice experience except from the fact that most EU Member States have experienced still growing integration and interaction (perhaps through technical cooperative bodies) between the two Ministries; a process which has shown to take long to obtain and a process which have never been without conflicts.In Turkey an additional hurdle may arise from the fact that most of the professional VET and TVET educations and courses are provided by strong and more or less autonomous Universities.

33

boards on regional level creates less active involvement in all phases of the implementation and monitoring of the HRD OP

Opportunities Involvement not just in HRD OP but also in

establishment of NQF will gradually increase the demand for still more active involvement of the Social Partners from a professional perspective

Better dialogue between the main actors on the labour market decrease the risk of conflicts

Better dialogue between the main actors on the labour market increase the potentials for inclusion, including:

o Registered employment (decrease the informal sector)

o Social inclusion of disadvantaged persons

Better dialogue between the main actors on the labour market increase the potentials for inclusion of other main themes, not least matters of OSH (which is completely left out the HRD OP)

Threats The main threat is the lack of will power to

build a formal structure for Dialogue and participation on regional level (first) and local level (next). In that respect the main actor will be the public sector which has the obligation to construct the fundament. In here, the legislative fundament should be brought up.

The main players on the labour market and in all matters of HRD in Turkey as well as in the HRD OP are mainly subject for examination in the Thematic Report on Horizontal issues to which the Interim Evaluation Report therefore refer.

4.2 The Monitoring Arrangements and the Monitoring Information System with respect to the Grant Monitoring

4.2.1 Central Grant Monitoring Teams

The Central Grant Monitoring Teams, CGMT, are founded within İŞKUR (11 permanent and full time staff members of the unit), MoNE (approximately the same staff number, but not all are full time), and SSI (one person full time). With respect to SWOT tables and the broader description of the organisation we refer to the previous section above. The section mainly focuses on the Central Grant Monitoring Teams with the three organisations which functions is broadly specified above.

We briefly mentioned the weaknesses of the Operating Structure in terms of knowledge of content within HRD issues which is reflected in the expected taking the lead in the programming of the HRD OP. Similarly, we commented the apparent weakness in the management structure due to the silent parties of İŞKUR and MoNE and their somehow natural different approach to these issues (which just as naturally brings some conflicts on-board). These weaknesses, which appear to be generally acknowledged, should accordingly be met by a strong involvement on behalf of the CGMTs.

That part appears to be fulfilled with solid expertise, knowledge, and experience from all three teams. Nevertheless, the TAT have been unable to see the influence of the three CGMTs in programming of the HRD OP including not least defining the outcome and the content of the programme, even though all three bodies appear to agree with the priorities as well as the execution of the HRD OP. The point is that the TAT would have expected the CGMTs to have a far more active role as key sparring partners for the execution of the projects and their main outcome.

34

Of course, that viewpoint is only accentuated by the study of the indicators, which indicators all three CGMTs were soon to criticise. In that respect, the TAT has measured the influence of the CGMTs (as well as the mother institutions) on the programming of the HRD OP by the study of the outcome and the way the complete programme was anticipated to be evaluated. Since the lack of impact indicators make every evaluation impossible the TAT can only conclude that the CGMTs have had very limited impact on the overall design of the programme10.

In fact, it appears that the main role of the CGMTs is to act as a prolonged monitoring function supervised (?) by the M&E Unit of the OS11. If that is true something is wrong and the sustainability is deeply threatened; obviously the CGMTs should have far more influence on the selection of projects, the content of the projects and their contextual route, as well as the way the outcome of the projects should be assessed12. The TAT, however, have no documentation which should point in that direction, and can only presume that the CGMTs (and their mother organisations) have determining impact on the HRD OP and have the ultimate power to set the agenda for the HRD OP, including the agenda of the SMC Meetings13.

The Interim Evaluation is not in any position to question the structure of the HRD OP management system. Likewise, the Interim Evaluation has no reason to question the willingness to use the HRD OP as a solid platform for structural changes in matters of HRD. The Interim Evaluation can only raise comments based on reflexions and observations from the examination of the system as impartial outsiders.

It appears that the overall management structure put a lot of effort in procedures and financial management on the expense of ensuring discussions of content, outcome and impact and how to address the regional needs. That viewpoint is further reflected, and perhaps even accentuated, by the study on indicators as well as the assessment of the involvement on regional level.

Accordingly, the Interim Evaluation has no means to make firm conclusions; the logic from a content part is missing or at least the TAT sees broken links which appear to contradict the well-proven and ready-designed structure of the HRD OP. And the TAT has no means to prove and document on the observations.

However, the Interim Evaluation strongly recommend the CGMTs to take on board more ownership in terms of identifying the purpose and the means, measures and actions of the programme in order to ensure sustainability and compliance with regional needs. In particular, the Interim Evaluation recommends the CGMTs to take a far more active role in determining indicators which reflects real effect (outcome and impact), in ensuring solid means for monitoring and follow-up, and perhaps even to ensure strong dialogue with stakeholders in each of the single projects under implementation.

As the Interim Evaluation sees the organisation, the CGMTs and their mother organisations have the responsibility to ensure that the programme reflects real needs. The CGMTs should, according to the Interim Evaluation’s view, have the necessary experience, knowledge and know-how to embark on a far more prudent involvement in the programme and its design, perhaps not during the 2007-09 programme, but at least in future HRD OPs.

10 Unless, of course, that the CGMTs lack knowledge of measurement techniques which indeed would appear extremely strange due to their long term experience with initiatives and international projects.11 Indeed that is the way the function of the CGMTs have been presented to the TAT12 The Operation Structure has the following comment to the reflexion of the TAT: “This is not true. The ME Unit is not carrying out monitoring functions at grant level. As there are many monitors at regional and central levels and they make the visits together, the ME unit is just coordinating the preparation of the monitoring visit plans.”13 Which certainly is not the case as far as the TAT has observed, which just raise the question of the main reason for establishing the SMC if the SMC function is procedural exclusively (see the Thematic Report on Horizontal Issues).

35

In that respect it will be of paramount importance that the OS within the MoLSS embark on a dialogue with the CGMTs in finding ways to ensure a stronger commitment and ownership in the programme on behalf of the CGMTs and their mother organisations. The same, by the way, goes for the involvement of the Social Partners, which we noted above.

4.2.2 Regional Grant Monitoring and Technical Assistance Teams (RGMTTs)

The results of the examination of the RGMTTs are covered by the thematic report on Horizontal Issues to which the Interim Evaluation Report hereby refers.

4.2.3 The Monitoring Information System

The Monitoring Information System, MIS for short, has been being developed and updated since 2006. The use and, notably in this context, the commitment of entering data and providing updates and inputs to system is not compulsory. In order to ensure consistent input to the system monitors (the RGMTs) have been assigned indicating that the feed in to the system still necessitate a back-up organisation. Some difficulties appear even on the present structure, though, mainly due to the limitations of the RGMTs members (they do not work full time among other things; see Thematic Report II).

Accordingly, the system suffers from some shortcomings which include pitfalls in quantifying and updating of the agreed indicators, in providing project-specific data and perhaps reliable information on the current state of the projects. The system, still under development, accordingly suffers from incomplete data.

In the examination of the MIS the Interim Evaluation has noted the complexity and feasibility of the system, mainly when it comes to providing information and data on administration and financial progress. The system is solidly used by the Operation Structure and appears to have become an important tool for the overall management of the programme.

However, the Interim Evaluation has also discovered and experienced its shortcomings. In the study of indicators for follow-up on the programme several difficulties appeared, and the overall conclusion from the study on indicators is that data still has to be provided from the main central stakeholders, mainly the CGMTs and their organisations.

There are several reasons for this:

1. Data on some indicators are simply not there and the data, which are there, are either incomplete or not directed towards the indicators14

2. The reporting to the system do not appear to be conducted consistently when progress is made and identified;

3. Feeding in to the system is not compulsory indicating that the prime caretakers have only limited interest in supporting the system

4. Some, if not a large part, of the indicators are not workable in practice for monitoring and evaluation purposes and has to be redefined in order to become quantifiable or in order to ensure the correct measures and data (see the study on indicators)

5. Data and the quality of data appears to vary across operations, projects and regions;

14 Agreed, one of the main problems is that the indicators are defined wrong. Most of them includes words like increase and especially when it comes to Result indicators the indicators includes rates without specifying the denominator (and without having the ability to capture the most plausible denominators within the system).

36

6. Data lacks basic background statistics where background statistics are needed for assessments of indicators; and it appears extremely difficult to see how the MIS data links to other statistics, including official statistics and administrative records15

7. The MIS lacks updated statistics and information from other sources relevant for the indicators

The Interim Evaluation recommends to

Upgrade the system to be of benefit for all stakeholders (project contractors sees limited benefit from entering data on a daily basis)

Consider to make the system compulsory for contractors and other key stakeholders16

Ensure that the system actually reflects the need (the issue of the indicators, which basically has to be redefined)

Ensure interlinks with basic data, including official statistics and administrative records when needed for measurement

Ensure solid training in quality assurance on all levels, notably micro level where data are entered

Clean all records in close liaison with caretakers of administrative records, notably SSI, İŞKUR, and MoNE

Make quality controls on a permanent basis

4.3 The use and financial allocation of financial assistance and the financial management of the budget

Overall comments on financial management

The overall financial management system is functioning well. The flow of funds from the EC level till the final beneficiary level is smooth. While this is not specifically subject to this evaluation, the financial aspects are covered well with relevant legal documents and reporting systems.

Commitment and disbursement data

Commitment rate shows the amount of funds contracted compared to total available budget of a programme and thus provides an indirect indication of the progress of a programme. It is a more valuable indicator for programmes in case of which the deadline for the conclusion of contracts precedes the deadline for corresponding payments.

The commitment rate of 60.33% at less than two years from the beginning of the commitment period i.e. the signature of Financing Agreement should be normally considered satisfactory in the context of the n+3 rule.

Disbursement rate shows the amount of funds paid to contractors compared to total available budget of a programme giving indirect information of the overall physical progress at a certain point of time. However, as according to the contractual rules payments are not always linked to physical outputs due large advance payments and delayed final payments, the disbursement rate does not allow to measure the physical progress of a programme very accurately.

15 At least it has shown to be extremely time-consuming to find links between project data and administrative records; the study thereof ended up inconclusive, though the TAT tends to believe that there is no link whatsoever.16 This point should go in line with a dialogue with the stakeholders on how to make the system both user-friendly (if that is a real issue for the stakeholders) and informative (e.g. cross-regional or cross-project information)

37

Disbursement rates could also give an indication of a possible risk of decommitment, but under the n+3 rules with complex decommitment deadlines the disbursement rate proves less informative.

Similarly with the commitment rate, the disbursement rate of 40.32% at less than two years from the beginning of the commitment period i.e. the signature of Financing Agreement, should be normally considered satisfactory in the context of the n+3 rule.

National co-financing and EU contribution

The national co-financing and the EU financing have been available in a timely manner. Payments are made in accordance with 85/15 ratio between EU and national financing. This also indicated by identical commitment and especially disbursement rates of both EU contribution and national co-financing.

Payments to contractors

While the evaluation team did not have access to specific financial documents, interviews with some contractors indicated that payments to contractors have been made in a timely manner, although mostly closer to the end of the legally allowed deadline for payments. The contracts provide legal sanctions for delayed payments which in turn should be a sufficient guarantee that the payments are made in a timely manner.

Shorter implementation periods and decommitment risk

According to the Financial Regulation and the Financing Agreements each programme should be implemented within a specific period to be eligible for EC financing. In the event that the allocated funds are not used within a prescribed time period, the EC decommits the funds, i.e. the funds can no longer be used for the intended purpose even if the beneficiary country has signed binding contracts to be financed for these funds. The HRD OP is also subject to that “n+3 rule”, which in very broad terms should enable to have an implementation period of more than three years for the implementation of a programme. For multi-annual programmes like the HRD OP the implementation period is even longer although having several interim deadlines.

In case of the HRD OP 2007-2009, the implementation period lasts for three further years from each of the three programming years. Due to different rules applied e.g. to the pre-financed amounts received from the EU and the end of the implementing period, the interim deadlines are slightly different in reality. In broad terms, the minimum amounts of EC funds that have to be spent in accordance with specific requirements are the following:

Year Budget (MEUR)

End of 2010 2.59

End of 2011 52.9

End of 2012 55.6

End of 2018 47.6117

These deadlines for the HRD OP 2007-2009 appear to leave ample time for the implementation of the programme, however, it should be considered that it was possible to first use the funds under the HRD OP only after the signature of the Financing Agreement on September 16, 2009.

17 Maximum amount not taking into account any previous decommitments under the programme

38

Since accreditation process may become one of the contributing factors to the possible decommitment of funds, the evaluation team analysed also the chain of events that caused later signature of the Financing Agreement.

According to the IPA regulations, the funds under IPA Human Resources Development component are accessible only to candidate countries accredited to manage funds in a decentralised manner. Therefore, the candidate countries, including Turkey, had to undergo the process of preparations for the accreditation, a process expected to take up to three years based on the experience of other countries.

Table. Key steps in IPA accreditation process in Turkey

Time Activity/Event

July 2006 Publication of the IPA Framework Regulation (Council Regulation (EC) No 1085/2006 of 17 July 2006 establishing an Instrument for Pre-Accession Assistance (IPA))

March-April 2007 Gap Assessment study performed

June 2007 Publication of the IPA Implementing Regulation that stipulated the criteria required for the accreditation process (Commission Regulation (EC) No 718/2007 of 12 June 2007 implementing Council Regulation (EC) No 1085/2006 establishing an instrument for pre-accession assistance (IPA))

August-October 2007 Gap Plugging

December 2007 – July 2008

Compliance Audit

September 2008 National Accreditation

October 2008 Accreditation packages sent to EC

November-December 2008

Verification Audits by EC

January 2009 Verification Audit Report issued by EC

March-June 2009 Further Verification Audits by EC

August 2009 Conferral of management powers by EC

September 2009 Signature of Financing Agreement

The table indicates that the decision on Conferral of management powers was made approximately two years after the publication of the accreditation criteria that established the basis for the requirements to be fulfilled. Based on the experience from similar accreditations in other countries before, an approximately two-year period for the completion of the accreditation process could have been foreseeable at the time of publishing the accreditation criteria.

At the same time, the first OPs in the candidate countries started with the budget year of 2007. While the complexities and expected duration of the accreditation were foreseeable, no grace period or transitional mechanism was provided. As a result, for the first years of implementation, the principle of n+3 was in reality transformed into n+2 or n+1. This may not be the (single) reason leading to possible decommitments in candidate countries, but obviously the implementation periods for the first years of HRD OP are shorter.

Consequently, any decommitments in the first years of HRD implementation should not lead automatically to long-term conclusions about implementation capacities in the candidate countries as in future years with longer implementation periods decommitments can be more easily avoided.

39

Similarly, the shorter implementation period should not be automatically seen as a single reason for any decommitments in the candidate countries, but other aspects should be equally analysed in this context.

In case of HRD OP in Turkey, a relatively minor amount of 198 591 EUR was decommited at the end of 2010. However, in 2011 the amount to be declared to the EC is 52.9 MEUR and according risk assessments, at least some of the amount would probably be decommitted. However, according to the same risk assessments, there will be no decommitment risk for any part of the amount of 55.6 MEUR at the end of 2012. This may indicate that all other conditions remaining the same there would have been no decommitments if HRD OP could have been implemented following the n+3 rule without time limitations arising from the accreditation process. Despite the limitations coming from the accreditation process during the first years of the HRD OP, the Turkish administration and the EUD have taken measures to minimise the risk of decommitments, by e.g., launching calls for proposals and tender with a suspension clause prior to the signature of the Financing Agreement. These activities clearly indicate the good will, cooperation and foresight by the institutions involved.

At the same time, this does not necessarily mean that there is no room for improvement in the processes. These issues are addressed in other parts of this report.

4.4 Review programme outputs and results in relation with the expenditures incurred at priority and measure levels; in the light of financial and physical progress

The task of reviewing programme outputs and results in relation with the expenditures incurred at priority and measure levels; in the light of financial and physical progress could be best performed after the projects under a measure or priority have been completed and the relevant statistical data is available. By the cut-off date of April 30, 2011 of this evaluation project, none of the projects were completed making this assessment difficult.

The financing model of 80% of advance payment and 20% of final payment that is not linked to physical progress makes it impossible to make such an evaluation before a project is completed. Therefore, this task should be addressed by the following evaluations. However, although the cut-off date for this evaluation has been set for April 30, 2011, the project can still accommodate the partial analysis of the grant scheme for the Promotion of Life Long Learning in this report that was completed in May 2011. Consequently, the data from the beneficiaries was still continuously updated in the MIS in the course of preparing final reports and closing the administrative side of the project after the end of the implementation period. The monitoring data is, thus, incomplete and therefore the analysis can not be completely valid, however a preliminary analysis is included in the Annex.

However, the TAT could make some general observations in the course of desk study and interviews:

- The market of external trainers used in grants projects appears to be quite volatile. In some cases the fees of trainers have exceeded anything that could be considered feasible even by international standards. While this area could be difficult to regulate, it would be advisable to see that any such fees would be not be unnecessarily excessive.

- Since the national vocational education standards are under preparation, it is not ensured at this stage that the funds spent on vocational training would result in a training that would be of sufficient quality. Equally the certificate provided at the end of the trainings might prove thus worthless for the participants.

4.5 Examine the efficiency of Operation Selection Criteria

40

According to article 158 of the IPA Implementing Regulation, all operations which are not major projects and which are implemented by final beneficiaries other than national public bodies shall be selected through calls for proposals. The selection criteria shall be drawn up by the operating structure and shall be published with the call for proposals. In other words, the selection criteria will be used in grant schemes and published with the Call for Proposals.

There have been five grant schemes launched under the HRD OP under five different measures:

Measure Grant Scheme1.1. Promoting Women’s Employment (PWE)1.2. Promoting Youth Employment (PYE)1.3. Promoting Registered Employment through Innovative Measures (PRE)2.1. Increasing School Enrolment Rates Especially for Girls (IER)3.1. Promotion of Life Long Learning (LLL)

The selection criteria have been included in the Calls for Proposals and also at a more general level in the HRD OP. The term “selection criteria” may be used in a wider and more specific sense, but in the context of this evaluation is should include also the criteria published as “eligible actions”, which do set the scope of projects financed under the grant schemes.

Since the selection criteria are mostly similar, the for all measures (obviously with the exception of the main area), the criteria for measure 1.1. have been used as a sample. The criteria in the HRD OP and Calls for Proposals are not completely identical, but fall under the objectives of Measure 1.1. In both cases, the set of criteria is rather broad and open and allows the applicants to propose rather different project ideas. Such a policy may be justified considering that this is the only grant scheme in that area at a given point of time. The method of using rather broad priority themes with a rather long non-binding sample list of activities should be considered justified in such a situation.

The disadvantage of a system is that the selected set of grants projects could become very diverse in the end focussing on different sub-themes in different areas. This approach could thus dissolve the results and outputs in such a way that it would be difficult in the end to measure common impact.

In the future under Structural Funds with larger amount of funds that could have measurable socio-economic impact, it might be advisable to use more targeted grant schemes. However, as the HRD OP is in a way a pilot scheme and a preparation for the management of structural policies with limited funds, a broader set of themes may be justified. However, it should be then considered that it would make final evaluation of the programme more complicated.

Equally, while this would be outside the scope of the current interim evaluation project, studying the all the applications submitted under the call for proposals could reveal how diverse the project proposals were in content and to what extent it complicated the comparison and selection of project proposals. This may be a subject for another study in order to find out if such a broad definition of eligible actions could create to a certain extent a “lottery effect” in the project selection process.

The evaluation team has similar comments to the selection criteria of other calls for proposals and therefore these will not be analysed here. This analysis based on the call for proposal under measure 1.1. to equally valid for other calls for proposals under other measures.

The budget brackets for the grants can in general considered optimal. However, with the current budgets the number of grants is relatively high, which makes monitoring of the grants more complex. At the same time, due to the requirement of own contribution by beneficiaries, increasing the minimum

41

budget requirements may close force some beneficiaries out of the market, as the amount of own contribution is considered to be high already now even by some larger and more capable institutions.

4.6 Assessment of local absorption and implementation capacities HRD OP has a strong regional dimension and therefore assessment of the local absorption and implementation capacities could give valuable inputs for future programmes and implementation practices. It must be still said that since the current assignment is carried out as an interim evaluation at a relatively early stage of the implementation of the programme, the assessment can be conducted on the basis of a rather limited sample. However, certain practices and commonalities can be observed.

Local absorption and implementation capacities can be assessed on the basis of five regional grant schemes being implemented in the regions at the start of the Interim Evaluation.

The grant schemes are the following:

Measure Grant Scheme1.1. Promoting Women’s Employment (PWE)1.2. Promoting Youth Employment (PYE)1.3. Promoting Registered Employment through Innovative Measures (PRE)2.1. Increasing School Enrolment Rates Especially for Girls (IER)3.1. Promotion of Life Long Learning (LLL)

Assessment of local absorption capacities

Based on the experience of the five grant schemes, different aspects of absorption capacity can be assessed by using the following data.

- The number of applications received from a specific region indicates the activity level and awareness of the HRD OP grant schemes by the applicants.

- The percentage of applications funded of the total number of applications submitted from one region indicates the average quality of applications from a specific region.

- The average number of projects selected per one million inhabitants from each region indicates how the funds have been distributed per population unit between the regions.

The statistical data annexed to this report provides a good basis for such analysis. All the data should be still analysed broadly by looking at tendencies rather than at specific figures as there are several external aspects that influence the figures. First of all, although the regions are of comparable size in terms of population, they are not of equal size. In the light of that, the data can be analysed as e.g. the number of projects financed per e.g. million inhabitants in a specific region. While this adds an additional dimension, the limited number of projects per regions and a certain extent of subjectivity that is always involved in grant evaluations would add in too many variables for too specific and detailed conclusions.

Since regional absorption capacities should be analysed in the context of the whole HRD OP, individual grant scheme data will not be focussed on in this part of the report. Instead, the data is used in the evaluation in combination with other information, including observations made by the evaluation team during the visits to different regions.

It must be also emphasised that the evaluation team has not had access to individual grants selection documents nor has it visited all the regions. While this also could not have been within the scope of

42

the HRD OP evaluation, the findings and conclusions in this part of the report are based on available statistical data and observations based on samples. For a detailed analysis taking into account the specificities in each region, a separate study could be considered.

Overall, the data indicates rather normal distribution of grants per region and per population unit. Without a regional quota system there are natural differences in the number of projects per region or per an amount of population. Still, in some cases the variations are still notable. For example, the NUTS II regions of Kars, Batman and Gaziantep receive a lower amount of projects per population unit than other regions.

Also, the percentage of the submitted applications that have been selected for financing appears to be lower in the regions that receive less funding. In other words, while the number of applications is not low, only a relatively small percentage gets selected. This indicates that the awareness of the grant schemes is still at a reasonable level but the quality of applications is lower than the quality of applications in more successful regions.

Another remarkable case appears to be the NUTS II region of Van that has a relatively low percentage of applications that have been selected for financing. However, in that region the number of projects per population unit is one of the highest and this is also reflected by the relatively higher number of applications submitted. This indicates that the awareness and activity level is very high in the region, but the average quality of applications as compared to other regions is lower.

Grant schemes are competitive in nature and without a regional quota system lower rates of selected projects need not necessarily indicate a much lower quality of applications. It may equally indicate that simply projects from some other regions have been of slightly better quality.

It also appeared from the regional visits that the regions with a strong proactive central institution as well as with strong project writing capacity tend to do better in the project selection process.

In some regions, there are institutions (e.g. Chambers of Commerce, Regional Grant Monitoring and Technical Assistance Teams) that are taking strong proactive measures to promote the grant schemes. Awareness raising events, targeted contacts with institutions that could be successful applicants but lack specific procedural knowledge, advice on project drafting and implementation are some of the practices that have contributed to higher regional success rates.

Another success factor appears to be strong project drafting capacity in a region. In some regions universities or other institutions have trained master students or other interested people on project drafting skills. Regions, where these activities were more prevalent tend to have better success rates also in the statistics.

Therefore, one of the reasons for higher concentration of projects in some regions could be linked to the fact of “global competition” between the projects. As a simplified example, if two regions are competing for 10 projects and one region has all the applications only of a marginally better quality then that region could still win all the projects while the other would get none. If such tendencies appear in regional socio-economic programmes in case of which reasonably equal regional distribution of funds should be one of the aims, some moderate regional quota system not having a significant effect on the quality of projects, could be considered. For example, a solution by which the half of the funds will be distributed based on regional quotas, provided that the project meets certain quality standards and the rest of the funds could be distributed based on “global competition”.

A further factor that could influence local absorption capacity is related to the possibilities of potential applicants at the time the calls for proposals are launched. For example, according to grant

43

beneficiaries, they have not applied for grants for time reasons if several grant schemes have been open in parallel. Equally, some have skipped application if they have not had funds for co-financing because of cash-flow issues, e.g. when waiting for the approval of the final report and subsequent payment for a previous project. This factor should not cause however differences between regions, but could lower the absorption capacity in general to a certain extent.

Assessment of local implementation capacities

Local implementation capacities can be measured mostly on the basis of the implementation experience of the five grant schemes in the 12 NUTS II regions. Since at the start of the evaluation, none of the grant schemes had been completed, the data for making fundamental conclusions is not sufficient. Still, some observations could be made.

Overall, the implementation capacities in the regions depend on the experience of specific grant beneficiaries. The grant beneficiaries have a reasonable external support structure that they can also rely on. The grant beneficiaries may contact RGMTTs, CGMTs, the relevant ministries and the CFCU. Equally, they can use the Grant Monitoring Information System for support. In some regions, some local organisations provided further advice.

While the grant beneficiaries should address first the RGMTTs this practice is not always observed. In such cases, other counterparts often do not receive the information and remain unaware of the issues.

Local implementation capacities can be more issues the grant beneficiaries face are of procedural nature and therefore they have developed a practice of contacting directly the thoroughly assessed after a number of the grants schemes have been completed and the respective final reports approved. This may be subject to a further study in the future.

4.7 Analysis of the accuracy of the budget allocations under measures

The wider aim of the HRD OP is both to prepare Turkey for the implementation of ESF type of measures functioning as a pilot scheme and within that frame address more specifically the areas of human resources development that would address the most acute needs of the country. The Annex provides an overview of the division of funds between priorities and measures as well as regional coverage.

Considering the aims of preparing for the ESF implementation and addressing the key priorities in the human resources development sector, such a division of funds is justified. The areas of employment, education and social inclusion are all covered for the purposes of HRD OP serving as a pilot scheme with a relatively larger amount dedicated to the more prioritised employment sector. The budget allocation under measures corresponds to the breakdown of the budget between the priority areas.

The territorial breakdown had been also reasonably planned. With a clear need of assistance at this pilot phase on the central level, the majority of funds are still targeted to the NUTS II regions. The breakdown between the NUTS II growth centres and hinterlands reflects broadly the division of population between the two in the regions. For the purposes of a pilot scheme such a division is justified, however, for future programmes a different balance could be considered, if targeting specific population groups would bring better results.

The division between the growth centres and hinterlands has been planned as a range rather than fixed percentages. Provided that the ranges are not too broad, which they are not in case of this HRD OP, this budgeting method should be encouraged for practical reasons. Namely, the grants schemes

44

under HRD OP are addressed jointly to the growth centres and hinterlands and in the project selection process a certain level of flexibility on regional level is beneficial in order to finance the most relevant projects. Fixed percentage could unnecessarily complicate the evaluation process.

The experience with five grant scheme launched under the HRD OP shows that using no quotas between the regions could lead to relatively uneven distribution of projects between the NUTS II regions. Therefore, it would be worth considering introducing setting minimum quotas for each of future grants schemes while allowing still sufficient global competition and maintaining minimum quality levels of the projects. It would not be advisable to use fixed divisions between the NUTS II regions on the level of the HRD OP, as this could unnecessarily complicate planning and implementation of some projects.

4.8 The study on Indicators

The Interim Evaluation has been able to recognize some 95 Output indicators from the HRD. In addition we further investigated 28 Result indicators, which can be divided into the following:

• 9 indicators are Output indicators• The remaining 19 indicators are all Outcome Indicators of which:

o 10 Outcome Indicators are workable in practice o 4 Outcome Indicators could work with some light adjustmentso 5 Outcome Indicators cannot work in its present form and needs to be adjustedo We found not even one single Impact Indicator

In total we have identified 104 Output Indicators, 19 Outcome Indicators and 0 (zero) Impact Indictors. Of the 19 Outcome Indicators, ten of them passed the Evaluation without remarks, 9 of the indicators can be workable with some adjustments. 5 Result Indicators cannot work in practice.

Obviously, we are talking about a very high number of indicators, which makes it close to impossible to undertake evaluation reports; each indicator will have to examined in time series analysis throughout the implementation and the following years, and each of them has to be examined in various cross-sections studies, not just on regional levels but toward target groups and control groups. Using a conservative estimate based upon experience we could easily suggest that each indicator would require close to 5 tables, with presumably another 5 tables in annex, and close to 6 pages description. Due to the number of indicators it would generate evaluation reports of 700 pages, which obviously would take months to produce. And yet the HRD OP does not even have any Impact Indicators making it impossible to evaluate the effect of the HRD OP.

Keeping track of a record of 104 output indicators is very appreciable, but no one bother read the lengthy report the evaluation will create in that case, basically indicating that the system creates a sequence of reports where only a fraction of the conclusions will be read, understood and acted upon.

In that respect we have to recall that the burden of monitoring and evaluation does not fall on single M&E experts on central level; the burden is transferred to the complete system, perhaps in some cases even to final beneficiaries; I.e. finding and creating data, controlling data, ensuring timely input, ensuring quality procedures in order to produce data of sufficient quality, follow-up, and making comments.

The programme lacks Impact Indicators. Without impact indicators it becomes impossible to see the real effects of the programme. By real effects we mean the impact on the local / regional economy, the labour market and the welfare.

45

Impact Indicators measures the consequences of the outcomes in terms of wider objectives (for example literacy rates, health improvement). The definition covers the wider effects of the outcomes but there might also be higher level impacts, related to broader objectives – growth and income poverty, for example. In order to turn it into a net impact indicator the effect of other economic, educational, and employment programmes have to be assessed and accordingly subtracted.

The Impact indicators will become vital for the final evaluation of the programme. The final evaluation will require means and tools to do a fair and solid evaluation of the impact of implementing so far 431 projects in 12 NUTS 2 Regions. In order to make the impact assessment data should be secured for a long time period ensuring solid means to do impact studies across time. And the indicators should be agreed upon long before the evaluators arrive; it would be impossible for the evaluators to establish the evaluation criteria and the indicators on top of the impact assessment.

The complete lack of impact indicators tells us that the difference between output, outcome and impact has never been understood by the stakeholders.

Conclusions and recommendations on the study of Indicators

In the Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010, the following comments to the functionality of the indicators were found:

“…some difficulties were encountered in determining the actual situation in terms of achievement of indicators. Although there were some problems stemming mainly from the fact that practice of monitoring the indicators was at its initial stage, these kinds of preliminary studies were beneficial from many aspects; for example, importance of entering the “performance indicators” were emphasized by the MEU and the RGMTT Experts, and the grant beneficiaries were requested to enter the targets and status of achievement in terms of indicators determined within their project. In addition, some revisions were made in the indicators in terms of expression and wording, also some indicators were divided in different parts for converting the indicator into a measurable format.”[SAR, p. 38]

Somehow, the Interim Evaluation finds the adjustments insufficient. At least the first and perhaps prime recommendation is…:

… to revise the current set of indicators, which revision should bring the number of indicators significantly down. The need for a reform is urgent since the current system of indicators are insufficient, inappropriate and yet far too ambitious. The Interim Evaluation proposes to bring the total number of indicators down to 18 indicators: six Output, six Outcome, and six Impact Indicators18.

I.e. the current system is:• Insufficient since the current set of indicators lack Impact measures• Inappropriate since many indicators have no doable baselines, lack data, lack

infrastructure, or are just not measuring the right (defined wrongly)• Too ambitious due to the number of indicators

The Interim Evaluation has of course suggestions to how ease the administrative burden of a far too ambitious monitoring and evaluation programme. The suggestions fall in three main categories:

18 The mix may vary slightly, of course

46

1. Revise the indictors toughly until a decent and doable target is reached19; Understand the basics in evaluation and design a list of indicators which summarizes the main and basic idea and purpose of the overall programme;

2. Establish a multi-level system of indicators, ensuring that the main evaluation reports only contains a very small number of smart indicators. The multi-level system does not decrease the overall workload of the complete system, but at least the main work falls on the central level;


The first suggestion is surely the hardest one. On the other hand the TAT has been informed that many central level key stakeholders have never seen the present set of indicators20. And besides, it is not just a matter of a far too long list of indicators; the complete programme has no means to make evaluations due to the complete lack of impact indicators.

The second solution, which draws upon the introduction in Chapter 7, is further explained in a section below.

The third suggestion leads to open up for a far more flexible administration where it perhaps would be fair to delegate monitoring and even parts of the evaluation to other stakeholders than the central level Operation Structure. Agreed, it is exactly what the stakeholders try to achieve, but many of the stakeholders cannot find the logic yet notably on regional level. And exactly that fact makes the “burden” a real burden and difficult to create flexible solutions.

The three suggestions can easily be implemented simultaneously.

Two detailed tables of conclusions and recommendations are annexed to the study of Indicators in Annex A4. The two lengthy tables (due to the number of Indicators in the present programme) could be consulted in case the HRD OP intends to keep the present far too ambitious number of indicators. Otherwise, the main recommendations from the Interim Evaluation are:

Revise the current set of indicators, which revision should bring the number of indicators significantly down. The need for a reform is urgent since the current system of indicators are insufficient, inappropriate and yet far too ambitious

Revise the indictors toughly until a decent and doable target is reached; The Interim Evaluation proposes to bring the total number of indicators down to 18 indicators: six Output, six Outcome, and six Impact Indicators

Understand the basics in evaluation and design a list of indicators which summarizes the main and basic idea and purpose of the overall programme;

Establish a multi-level system of indicators, ensuring that the main evaluation reports only contains a very small number of smart indicators. The multi-level system does not decrease the overall workload of the complete system, but at least the main work falls on the central level

Establish solid training programmes in monitoring and evaluation, data control, and quality assurance, not least on regional level. Place ownership and delegate tasks where the single stakeholders can see the purpose.

19 The study of indicators calls for a complete reform of the indicators. Perhaps it would be better to start from scratch.20 That goes, strangely, notably for the CGMTs by the way, which might be another point to add to our study of the management structure. It is a complete puzzle why the central and key stakeholders were caught by surprise when presented to the Result Indicators of the HRD OP. The Interim Evaluation did not follow up on the point except from this very light note.

47

4.9 Overall assessment of the HRD OP

In general, the Interim Evaluation acknowledges the structure of the HRD OP which appear to work and in good progress. In particular, the Interim Evaluation acknowledges:

The first time with multiple projects The very new institutional and administrative setup The difficulties in getting the setup to manage the programme (it is Turkey; the largest country

doing HRD OP) The delays in the implementation of the HRD OP The rigid EU system and the not lesser EU procedures which will impose an increase in

bureaucracy in Turkey and will create challenges due to the very different administrative culture in Turkey

The wait for the accreditation of the OS The will power, the enthusiasm and the overall professionalism we have met

None of these points have been used in the assessments conducted, since that would not have been fair.

The overall assessment of the HRD OP of Turkey is subsequently that the operations within the programme are running and appear to be both relevant and in line with the intensions of the programme as well as the national agenda and the national priorities. The management structure to follow up appears geared and capable to handle a large scale programme, even though various delays in the initial phase of the programme and perhaps not least the long wait for accreditation has created significant challenges for the management.

In other words, the HRD OP has started and it works; the Interim Evaluation has found no “disqualifying” features. Having said that, adjustments on a more detailed level appears necessary in order to make a large scale HRD programme work efficiently in practice in the future and not least in order not to over-burden the administration and henceforth create inflexible bureaucratic structures which will only make the practical implementation of the single projects far more difficult. Professional delegation to mainly regional stakeholders is the main weakness of the present system and another main barrier for the administrative procedures is the almost intolerable number of indicators, which will necessitate an overwhelming administrative burden both in terms of international follow-up and in terms of the overall follow-up through monitoring and evaluation reports on behalf of the complete programme and its operations. Some, if not directly many, of the indicators cannot be used in practice and are not consolidated with neither the national statistical production nor the current system of administrative records.

Accordingly, and in short, some elements or functions within are weakening the system to a critical extent which appears to need attention notably in the longer run:

Partnership approach, mainly on regional level RGMT setup, function and “ambassadorship” The MIS Too many indicators makes reporting impractical, if not directly impossible No impact indicators defined makes it impossible to measure effect

4.10 Recommendations

48

The main and core recommendations are as listed in the executive summary above and repeated here. In addition, recommendations connected to the study on indicators follows from the study.

Concerning the efficiency and effectiveness of the HRD OP management structure:1. Ensure far more pragmatism and flexibility based upon know-how and knowledge of “content”

instead of narrowly focusing on administrative procedures, which includes ensuring a far stronger Social Dialogue, commitment, participation and involvement by all Social Partners on all four levels of execution

2. Ensure stronger commitment and involvement from the Operation Beneficiaries.

Concerning the Monitoring Arrangements, notably the Monitoring Information System, MIS:3. Ensure that the MIS system becomes more user-friendly or at least open up for a dialogue on

how to ensure consistent entering of data to the system4. Ensure that the system actually reflects the need (the issue of the number and quality of

indicators, which definitely needs to be redefined)5. Ensure a interlink with basic data, including official statistics and administrative records when

needed for measurement6. Ensure solid training in quality assurance on all levels, notably micro level where data are

entered7. Clean all records in close liaison with caretakers of administrative records, notably SSI,

İŞKUR, and MoNE8. Make quality controls on a permanent basis.

Concerning the efficiency and effectiveness of the HRD OP contractual, tendering, and financial procedures:

9. Ensure far more readiness and speed in the evaluation procedures and try to avoid conflicts in central level responses to implementation level

10. Consider to establish quotas in order to ensure that all regions have at least a limited number of projects reflecting the needs on the local level.

Concerning the efficiency and effectiveness of the HRD OP Indicators:11. Revise the current set of indicators, which revision should bring the number of indicators

significantly down. The need for a reform is urgent since the current system of indicators are insufficient, inappropriate and yet far too ambitious

12. Revise the indictors toughly until a decent and doable target is reached. The Interim Evaluation proposes to bring the total number of indicators down to 18 indicators: six Output, six Outcome, and six Impact Indicators

13. Understand the basics in evaluation and design a list of indicators which summarizes the main and basic idea and purpose of the overall programme;

14. Establish a multi-level system of indicators, ensuring that the main evaluation reports only contains a very small number of smart indicators. The multi-level system does not decrease the overall workload of the complete system, but at least the main work falls on the central level


In case the HRD OP intend to stand up to the words of integration of the horizontal issues of sustainable development and environmental protection the Interim Evaluation recommends

49

16. to integrate impact indicators in the programme for the ex-post evaluation to ensure that the HRD OP has achieved to ensure sustainable development or least made progress in that direction

17. to redefine and establish outcome indicators and impact indicators which particularly focus on environmental protection

18. to ensure that environmental protection is acknowledged as a future demand in the educational sector, notably on VET/TVET level (that could be part of new curriculum, for instance)

19. to ensure that topics related environmental protection is an integral part of all Lifelong Learning Trainings underneath Priority 3

20. to give priority to environmental protection topics in priorities and measures addressing gender equality as well as social inclusion

21. to establish Training Needs Analysis (anywhere in the complete programme) which addresses the topic of environmental protection both in order to grab a demand in time and partly to increase awareness of the subject among the employers.

The Interim Evaluation has performed a solid investigation on participation. The study reveals problems with participation on regional level mainly though some problems with the SMC construction are evident. The long study of the participative approach ends with the following recommendations:

22. Support and continue the strengthening of the partnership approach on central level, not least with respect to the structure of the SMC and its meeting procedures

23. Actively strengthen, perhaps through legislation, a far more viable partnership approach on regional level.

And the regional level is subsequently examined where the main recommendations of the Interim Evaluation are:

24. To ensure a solid platform on the regional level with a consolidated and consistent structure, the build of a professional secretariat and governed by a tripartite body perhaps inspired by the way the RDAs are constituted

25. Ensure “promoters” (ambassadors) are based regionally and that the promoters are able to act as support in every phase. The TAT cannot see why the RGMTTs and the promoters are not directly interlinked or perhaps even the same persons

26. Ensure means of commitment from Social Partners and other main regional actors on a regional basis by placing ownership through direct involvement in the programming, monitoring and evaluation phases

27. Ensure far more active involvement from both İŞKUR and RDAs in regional programming, planning, design of indicators and targets;

28. Design objectives, measures, indicators and targets on regional level and establish a system of reporting in accordance to a template designed from central level (which could feed directly in to evaluation reports, like the SAR) to promote and reinforce regional involvement in the HRD OP

29. Ensure that local members are part of the process (and not just representative from the Growth centres) to ensure a far more active spread of the programmes internally in the regions

30. Ensure gender equality, local representation, and representation of target groups in all advisory boards to the HRD OP.

5. Annexes: Overview

50

The sequence of annexes to the Interim Evaluation Report falls in two main brackets or here Parts: Part A consisting of underlying studies and Part B of background information connected to the conducted evaluation. In addition, two Thematic Reports are annexed to the Interim Evaluation Report. The idea of separating the background studies in Part A from the main report should be fairly evident; the main point is that the background studies do not directly feed in to the main conclusions and recommendations or are otherwise too detailed for main conclusion, albeit they all constitute an integral part of the main findings. The humble hope is to increase the readability of the report.

PART A: Studies: Annex A1 Socio-Economic Study Annex A2 Major Developments on National and EU level since 2007 Annex A3 Results of the ex-ante evaluation Annex A4 Annex to the study of Efficiency and Effectiveness Annex A5 The study on Indicators

PART B: Background Information Annex B1 Meetings held Annex B2 Terms of Reference Annex B3 Generic Question Frame for meetings Annex B4 Basic questionnaire on the functionality of the HRD OP Annex B5 Questionnaire on Indicators Annex B6 Survey Report on Basic questionnaire on the functionality of the HRD OP (in

Turkish) Annex B7 Survey Report on Indicators (in Turkish)

Thematic Report I on Complementarity between RC OP and HRD OPThematic Report II on Horizontal Issues

51

Annex A1. Socio-Economic Study

1. Introduction

The main purpose of the Socio-Economic Study is to provide references to the current situation in Turkey and not least in its NUTS 2 regions. In that respect, the study is brief and kept to use a minimum of indicators. Nevertheless, the study sets a scope for future evaluation of Impact of the HRD OP. Hence the study serves as reference for the study of Indicators for the HRD OP in later chapters of the Interim Evaluation Report.

The inclusion of data from EUROSTAT serves at least two purposes:1. To provide a reference and ensure transparent statistics2. To give reasons for the HRD OP mainly in terms of providing the ability to make Benchmark

studies

In accordance to best practice in studies on Human Resource Development the Socio-Economic Analysis focuses on four main areas:

1. Demographic Indicators2. Main Economic Indicators3. Labour Market Indicators4. Indicators on Educational enrolment and attainment

The Socio-Economic background analysis is based upon official statistics derived from TÜİK/TURKSTAT and EUROSTAT.

Not all statistics are available on the NUTS 2 Regions of Turkey. In particular, this study, which further had the purpose of investigating potential impact indicators, could in particular have gained from statistics on the following indicators on regional (NUTS 2) level:

1. Number of persons above the age of 23 (or perhaps 24) with at least ISCED level 3 by gender 2. Educational attainment levels (% of the population aged 25 and above having attained: a. less

than upper secondary; b. upper secondary or postsecondary non-tertiary; and c. tertiary) by gender and age group

3. Number of students by major domains and gender 4. Number of graduates by major domains and gender5. Trade (import and export) statistics 6. Gross Capital Formation 7. Foreign Direct Investment 8. Gini index 9. Number of households with computers / with internet access (Information Society Statistics in

general) 10. Number of persons suffering from chronic deceases by type of decease, by gender 11. Life expectance at birth (in years) by gender12. Number of children per household (or rather Total Fertility Rate)13. Legislators, senior officials and managers by gender 14. GVA on NACE Rev. 2 or NACE Rev 1.1 (Main Economic Sectors; e.g. Section A to U in

NACE Rev. 2)15. Production statistics on SITC16. Number of registered unemployed by NUTS 2 Region by gender and sex on an annual bases

from 2005 and until at least 2010;

52

17. Number of registered vacancies by Economic Sector (NACE) and Occupation (ISCO) by NUTS 2 Region on an annual bases from 2005 and until at least 2010

18. Number of recent (e.g. past 12 months) placements by Economic Sector (NACE) and Occupation (ISCO) by NUTS 2 Region on an annual bases from 2005 and until at least 2010

19. Expenditure on Active Labour Market Measures and Passive Measures

In general, the HDI indicators developed by United Nations serve as a platform for desirable indicators for evaluation of HRD programmes. However, the UN HDI indicators are only available on national level, indicating general severe limitations to its use since HRD measures always should be taken the regional and local needs into account.

A list of general indicators for comprehensive studies of HRD programmes can be found in Annex 2 to this report. The main part of the indicators is normally available on regional level. Please note, that the list does not include Health Indicators and Information Society Indicators, though.

2 Demographic Indicators

The study of demographic indicators serves basically as a study of the composition of population and population changes.

Table 2.1 shows the population by NUTS 2 Regions in Turkey in 2010 according to the Address Based Population Registration System. The table indicates the large difference in population size across regions of Turkey, but otherwise it is difficult to make comments to a table which shows absolute numbers. The coloured regions of the table indicate the 12 NUTS 2 Regions being part of the Human Resource Development Operational Programme, HRD OP.

Table 2.1 Population by NUTS 2 regions 2010CODE GEO FEMALE MALE TOTALTR10 İstanbul 6,600,591 6,655,094 13,255,685TR21 Tekirdağ 749,613 771,715 1,521,328TR22 Balıkesir 809,478 833,242 1,642,720TR31 İzmir 1,963,480 1,985,368 3,948,848TR32 Aydın 1,363,692 1,375,496 2,739,188TR33 Manisa 1,472,974 1,532,584 3,005,558TR41 Bursa 1,788,340 1,807,120 3,595,460TR42 Kocaeli 1,617,691 1,628,456 3,246,147TR51 Ankara 2,392,490 2,379,226 4,771,716TR52 Konya 1,134,889 1,111,589 2,246,478TR61 Antalya 1,308,782 1,376,717 2,685,499TR62 Adana 1,875,034 1,858,090 3,733,124TR63 Hatay 1,479,406 1,525,202 3,004,608TR71 Kırıkkale 754,854 741,442 1,496,296TR72 Kayseri 1,165,208 1,187,763 2,352,971TR81 Zonguldak 517,368 517,703 1,035,071TR82 Kastamonu 374,726 368,303 743,029TR83 Samsun 1,374,337 1,366,349 2,740,686TR90 Trabzon 1,269,479 1,246,688 2,516,167TRA1 Erzurum 527,061 541,385 1,068,446TRA2 Ağrı 549,859 583,801 1,133,660TRB1 Malatya 814,220 810,938 1,625,158TRB2 Van 992,233 1,030,140 2,022,373TRC1 Gaziantep 1,203,683 1,211,150 2,414,833TRC2 Şanlıurfa 1,593,444 1,598,885 3,192,329TRC3 Mardin 986,874 998,736 1,985,610TURKEY TOTAL 36,679,806 37,043,182 73,722,988

Source: TÜİK (TURKSTAT) latest data available

53

Table 2.2 and table 2.3 show the average annual growth rates in population by gender (table 2.2) and by age groups (table 2.3). The tables show no clear patterns, though some regions experience higher than average growth rates, notably Antalya. In general, average annual growth rates are complex measures since they contain both natural growth (stemming from births exceeding deaths) as well as migration.

Both Gaziantep and Şanlıurfa experience above average growth rates, and both regions have an above average growth of infant children. Otherwise, the above average growth rates of the two regions appear to general across age and gender.

Erzurum and Ağrı have experience negative growth rates in population, which for both regions comes from negative growth in schooling age population between 5 and 24. The rate of decrease in population is however by no means significant.

Table 2.2 Average annual growth rates in Population by NUTS 2 regions 2007-2010CODE GEO FEMALE MALE TOTALTR10 İstanbul 1.7% 1.9% 1.8%TR21 Tekirdağ 2.0% 0.9% 1.4%TR22 Balıkesir 0.8% 1.2% 1.0%TR31 İzmir 1.7% 2.0% 1.9%TR32 Aydın 1.6% 1.4% 1.5%TR33 Manisa 0.2% 1.3% 0.7%TR41 Bursa 2.1% 2.4% 2.2%TR42 Kocaeli 2.2% 2.1% 2.2%TR51 Ankara 2.2% 2.3% 2.3%TR52 Konya 0.9% 1.0% 0.9%TR61 Antalya 2.8% 3.3% 3.1%TR62 Adana 1.3% 1.2% 1.2%TR63 Hatay 1.5% 2.3% 1.9%TR71 Kırıkkale 0.3% 0.4% 0.3%TR72 Kayseri 0.6% 1.1% 0.8%TR81 Zonguldak 0.1% 1.1% 0.6%TR82 Kastamonu 0.2% 0.7% 0.5%TR83 Samsun 0.0% 0.3% 0.2%TR90 Trabzon 0.3% 0.4% 0.4%TRA1 Erzurum -0.3% -0.1% -0.2%TRA2 Ağrı 0.0% -0.2% -0.1%TRB1 Malatya 0.7% 0.4% 0.5%TRB2 Van 1.3% 0.9% 1.1%TRC1 Gaziantep 2.0% 2.5% 2.3%TRC2 Şanlıurfa 2.3% 2.4% 2.3%TRC3 Mardin 1.3% 0.7% 1.0%Grand Total 1.4% 1.6% 1.5%

Source: Own calculations based upon TÜİK (TURKSTAT) latest data available

Table 2.3 Average annual growth rates in Population by NUTS 2 regions by broad age groups 2007-2010

CODE GEO 0-4 5-14 15-24 25-49 50-64 64+TR10 İstanbul 3.3% 0.4% 0.2% 2.4% 3.9% -0.2%TR21 Tekirdağ 2.7% 0.0% -1.7% 1.4% 5.6% 2.3%TR22 Balıkesir 1.0% -1.6% 0.7% 0.3% 4.0% 2.6%TR31 İzmir 3.2% -0.5% 1.2% 1.6% 4.2% 3.5%TR32 Aydın 1.7% -0.6% -0.3% 1.4% 5.0% 3.2%TR33 Manisa 0.2% -1.5% 0.8% 0.4% 3.7% 1.7%TR41 Bursa 3.0% 0.3% 1.7% 2.1% 4.6% 2.8%TR42 Kocaeli 3.1% 0.2% 0.4% 2.4% 5.0% 3.2%TR51 Ankara 3.9% 0.8% 0.0% 2.4% 5.1% 3.5%TR52 Konya 0.7% -0.5% -1.1% 1.5% 3.6% 2.8%TR61 Antalya 3.4% 1.0% 2.9% 2.9% 5.8% 3.8%

54

CODE GEO 0-4 5-14 15-24 25-49 50-64 64+TR62 Adana 3.0% -0.7% -0.8% 1.5% 4.5% 2.7%TR63 Hatay 2.6% 0.4% 0.8% 2.2% 4.9% 2.2%TR71 Kırıkkale 0.4% -1.9% -1.2% 0.6% 3.6% 2.8%TR72 Kayseri 0.6% -1.3% 0.0% 1.1% 3.7% 2.8%TR81 Zonguldak 0.9% -2.0% -0.5% 0.3% 4.0% 2.4%TR82 Kastamonu -0.3% -2.1% -1.6% 0.6% 3.4% 2.1%TR83 Samsun -0.6% -2.8% -0.4% 0.2% 2.9% 3.0%TR90 Trabzon -0.2% -2.7% -0.9% 0.3% 4.3% 2.9%TRA1 Erzurum -1.6% -2.4% 0.2% 0.3% 1.7% 2.0%TRA2 Ağrı 1.5% -1.3% -0.8% 0.4% 0.5% 1.2%TRB1 Malatya 0.8% -1.5% -1.2% 1.4% 3.6% 1.6%TRB2 Van 0.3% -0.4% 1.8% 2.0% 2.7% 1.8%TRC1 Gaziantep 3.6% 0.9% 1.3% 3.0% 3.7% 1.7%TRC2 Şanlıurfa 4.3% 0.7% 2.5% 3.0% 3.1% 0.5%TRC3 Mardin 1.0% -0.3% 1.7% 2.1% 1.9% -0.9%TR TURKEY 2.2% -0.4% 0.4% 1.8% 4.1% 2.2%

Source: Own calculations based upon TÜİK (TURKSTAT) latest data available

Table 2.4 shows cross-regional internal migration in Turkey in percentage of total population. The table stipulates some volume, perhaps even some high volumes here and there, but not volumes which appear frightening in an international context. However, the table makes the reader blind in purpose; it is obvious to look at net-migration.

Table 2.4 Cross-regional in- and out migration by NUTS 2 Regions, 2010 in pct. of total population

CODE GEOIn-

migrationOut-

migrationNet-

migrationTR10 İstanbul 3.3% 2.5% 0.8%TR21 Tekirdağ 3.9% 3.1% 0.8%TR22 Balıkesir 3.1% 2.9% 0.2%TR31 Izmir 2.8% 2.5% 0.3%TR32 Aydın 2.7% 2.6% 0.2%TR33 Manisa 2.2% 2.7% -0.5%TR41 Bursa 2.9% 2.4% 0.6%TR42 Kocaeli 3.3% 2.8% 0.5%TR51 Ankara 3.8% 2.8% 1.0%TR52 Konya 2.3% 2.7% -0.5%TR61 Antalya 3.6% 2.8% 0.8%TR62 Adana 2.5% 2.6% -0.2%TR63 Hatay 2.0% 2.4% -0.4%TR71 Kırıkkale 3.3% 4.7% -1.4%TR72 Kayseri 2.8% 3.4% -0.6%TR81 Zonguldak 2.7% 3.5% -0.8%TR82 Kastamonu 4.7% 5.7% -0.9%TR83 Samsun 2.9% 4.1% -1.3%TR90 Trabzon 3.6% 4.5% -0.9%TRA1 Erzurum 3.7% 4.7% -1.1%TRA2 Ağrı 2.9% 4.6% -1.7%TRB1 Malatya 3.0% 3.7% -0.7%TRB2 Van 2.3% 3.3% -1.1%TRC1 Gaziantep 2.1% 2.2% -0.1%TRC2 Şanlıurfa 2.0% 2.5% -0.5%TRC3 Mardin 3.0% 3.5% -0.6%TR TURKEY 3.0% 3.0% 0.0%Source: Own calculations based upon TÜİK (TURKSTAT) latest data available

It serves a clear purpose to show net-migration rates (which normally are in pro mille) since net-migration is closely linked to population increase and decrease. But net-migration is not always the only measure of relevance in economic studies.Table 2.5 shows basically the same, this time added internal migration within the NUTS 2 regions, and then, as the final point, a quite interesting figure of Total mobility pressure.

55

Table 2.5 Total internal mobility pressure, provisional and cross-regional in- and out-migration, by NUTS 2 Regions during three years (2008-2010) related to initial population (2008) in pct.

CODE GEO In-migration Out-migration

TOTAL Mobility Pressure

TR10 İstanbul 9.5% 8.1% 17.6%TR21 Tekirdağ 13.4% 10.5% 23.8%TR22 Balıkesir 9.7% 8.9% 18.6%TR31 Izmir 9.1% 7.4% 16.4%TR32 Aydın 9.6% 8.6% 18.2%TR33 Manisa 7.8% 9.1% 16.9%TR41 Bursa 10.0% 7.5% 17.5%TR42 Kocaeli 11.9% 9.4% 21.3%TR51 Ankara 11.2% 8.6% 19.8%TR52 Konya 7.4% 8.6% 16.1%TR61 Antalya 13.3% 10.1% 23.4%TR62 Adana 8.2% 8.8% 17.0%TR63 Hatay 7.0% 7.9% 14.9%TR71 Kırıkkale 11.4% 14.0% 25.3%TR72 Kayseri 9.3% 11.5% 20.8%TR81 Zonguldak 9.6% 10.9% 20.6%TR82 Kastamonu 15.9% 15.5% 31.4%TR83 Samsun 10.2% 12.8% 22.9%TR90 Trabzon 12.7% 13.7% 26.4%TRA1 Erzurum 10.8% 15.3% 26.1%TRA2 Ağrı 8.5% 14.9% 23.5%TRB1 Malatya 10.2% 11.9% 22.1%TRB2 Van 7.6% 11.5% 19.1%TRC1 Gaziantep 7.2% 7.8% 15.0%TRC2 Şanlıurfa 6.0% 8.0% 14.0%TRC3 Mardin 9.6% 12.6% 22.2%TR TURKEY 9.6% 9.6% 19.2%

Source: Own calculations based upon statistics from TÜİK (TURKSTAT) latest data availableNote: The measure is based upon data on provisional mobility on NUTS 3 level. The measure does not include external

migration (cross-national migration)

Total migration pressure is not as common as net-migration. But the point in this socio-economic study which is explicitly addressed to HRD strategies is to show the total volume of migration within a single as a rough estimate of the pressure on the local and regional labour market. The pressure on the local / regional labour market is of course that a total of 19.2 per cent of total population (which must include labour force as well) has change location within three years21.

And that is quite impressive even on international scale indicating that the local labour market has to cope with fast job-shifts in a quite impressive number. And that, of course, only adds to the problems of sustainability of the local economies, and induces a significant pressure for flexibility on the work places; the employers are faced with quite astonishing demand for continuously obtaining work experience.

The main explanation must rely on easy access to jobs, indicating that we are talking of unskilled labour for unskilled work. Accordingly, the indictor is another measure of the need to reinforce process industry and advanced production. Only by increasing the demand for skilled labour and through upgrading the educational level of the population the mobility pressure can be decreased. The out-migration regions / locations needs job opportunities and not least good and lasting jobs.

21 We further have to note that it is mostly people in the working age that moves, which makes the pressure on the labour market even higher of course.

56

Table 2.5 shows the elderly dependency ratio while table 2.6 shows the youth dependency ratio. The elderly dependency ratio is not something to worry about the coming years for Turkey – contrary to many mature EU Member States. The same cannot be said about the youth dependency ratio.

Table 2.5 Elderly dependency ratio (Age 65 and over)CODE GEO 2007 2008 2009 2010TR10 İstanbul 8.4 7.5 7.7 7.9TR21 Tekirdağ 12.7 12.3 12.5 12.9TR22 Balıkesir 16.5 16.5 16.8 17.1TR31 İzmir 11.5 11.3 11.6 11.9TR32 Aydın 13.3 13.2 13.4 13.9TR33 Manisa 13.4 13.5 13.6 13.6TR41 Bursa 11.3 11.0 11.2 11.4TR42 Kocaeli 10.5 10.1 10.3 10.7TR51 Ankara 9.2 9.0 9.2 9.6TR52 Konya 11.0 10.9 11.2 11.5TR61 Antalya 10.7 10.7 10.8 10.8TR62 Adana 8.9 8.6 8.8 9.2TR63 Hatay 9.2 8.9 9.0 9.2TR71 Kırıkkale 11.9 11.8 12.1 12.7TR72 Kayseri 11.6 11.5 11.7 12.2TR81 Zonguldak 13.1 13.1 13.4 13.7TR82 Kastamonu 22.3 22.0 22.5 23.2TR83 Samsun 14.0 14.0 14.4 15.0TR90 Trabzon 16.0 15.8 16.1 16.9TRA1 Erzurum 11.8 11.6 11.9 12.4TRA2 Ağrı 8.7 8.1 8.5 9.0TRB1 Malatya 11.6 10.9 11.3 11.8TRB2 Van 6.0 5.5 5.7 5.9TRC1 Gaziantep 8.5 7.9 8.0 8.2TRC2 Şanlıurfa 6.9 6.1 6.3 6.5TRC3 Mardin 7.9 6.8 7.0 7.3


Table 2.6 Youth dependency ratio (Age 0-14)CODE GEO 2007 2008 2009 2010TR10 İstanbul 34.7 34.7 34.3 33.8TR21 Tekirdağ 28.2 27.7 27.6 27.6TR22 Balıkesir 27.6 27.2 26.7 25.9TR31 İzmir 28.8 28.7 28.3 27.6TR32 Aydın 32.3 31.8 31.2 30.8TR33 Manisa 33.4 33.5 32.7 31.3TR41 Bursa 31.9 31.9 31.3 30.7TR42 Kocaeli 35.7 35.2 34.8 34.4TR51 Ankara 31.9 32.1 31.8 31.4TR52 Konya 42.2 41.8 41.2 40.6TR61 Antalya 33.8 34.0 33.3 32.3TR62 Adana 40.4 40.3 39.5 39.2TR63 Hatay 48.4 48.1 47.7 46.9TR71 Kırıkkale 41.1 40.1 39.3 38.8TR72 Kayseri 41.8 41.3 40.5 39.5TR81 Zonguldak 30.5 30.2 29.7 28.7TR82 Kastamonu 32.6 31.5 30.9 30.4TR83 Samsun 37.7 36.7 35.8 34.7TR90 Trabzon 36.4 34.9 34.1 33.4TRA1 Erzurum 48.2 47.0 45.8 44.4TRA2 Ağrı 62.2 60.8 61.4 61.4TRB1 Malatya 42.3 41.0 40.4 40.1TRB2 Van 76.0 72.8 72.0 71.2TRC1 Gaziantep 57.3 57.0 56.8 56.1TRC2 Şanlıurfa 71.9 70.5 70.6 70.0TRC3 Mardin 78.3 74.6 74.8 74.2


57

The youth dependency ratio shows the population between 0 and 14 years of age relative to working age population.

The remarkable feature is the very high youth dependency ratio in Agri, Van and Mardin of which the last has a youth dependency ratio of 74 per cent. It is evident that the region cannot support itself (a total pressure on the working age population of being able to feed and provide means to close to half the population of the region22).

3 Main Economic Indicators

Table 3.1 shows GDP per capita (inhabitant) in Turkey and selected other countries in Europe indexed towards the average of EU27.

Table 3.1 GDP per capita in Turkey, European Union Member States and selected other countries selected years, indexed (Index EU27=100)

GEO COUNTRY 2000 2005 2006 2007 2008EU27 European Union (27 countries) 100 100 100 100 100AT Austria 136 132 131 131 136BE Belgium 129 129 127 126 129BG Bulgaria 9 13 15 16 19CH Switzerland 198 179 176 168 179CY Cyprus 76 80 80 81 87CZ Czech Republic 31 44 47 49 57DE Germany 132 121 119 118 121DK Denmark 171 170 170 167 170EE Estonia 24 37 42 47 48ES Spain 82 93 94 94 95FI Finland 134 133 133 136 139FR France 124 122 120 119 121GR Greece 66 78 80 81 84HR Croatia 0 36 38 39 43HU Hungary 26 39 38 40 42IE Ireland 145 173 176 174 162IT Italy 176 197 185 192 128LT Lithuania 110 108 106 104 105LU Luxembourg 19 27 30 34 38LV Latvia 264 290 303 313 324MK FYROM 19 25 30 37 41MT Malta 10 11 11 12 13NL Netherlands 57 53 54 54 56NO Norway 138 140 140 140 145PL Poland 25 28 30 33 38PT Portugal 65 65 64 64 65RO Romania 10 16 19 23 26SE Sweden 159 147 148 147 144SI Slovenia 57 64 65 69 74SK Slovakia 21 32 35 41 48TR Turkey 24 25 26 27 28UK United Kingdom 143 135 136 135 118

Source: EUROSTAT and own calculations

The table clearly indicates the challenges facing Turkey in order to become a modern welfare state; Turkey experiences a GDP/c of 28 per cent of EU average.

Table 3.2 GDP/c TURKEY=100

22 Presumably far more, since working age population consists of unemployed, students and inactive as well as employed. The employment rate of Mardin is as low as 33 per cent as we shall see later.

58

CODE GEO 2000 2005 2006 2007 2008EU27 European Union (27 countries) 424.4 401.8 395.0 373.1 358.6TR Turkey 100.0 100.0 100.0 100.0 100.0TR10 İstanbul 157.8 175.0 165.0 165.7 164.3TR21 Tekirdag 124.4 133.9 133.3 131.3 135.7TR22 Balıkesir 100.0 94.6 88.3 92.5 98.6TR31 İzmir 144.4 132.1 128.3 126.9 125.7TR32 Aydın 108.9 101.8 103.3 98.5 97.1TR33 Manisa 84.4 82.1 85.0 83.6 85.7TR41 Bursa 115.6 142.9 143.3 144.8 142.9TR42 Kocaeli 173.3 150.0 148.3 144.8 148.6TR51 Ankara 151.1 139.3 138.3 138.8 138.6TR52 Konya 77.8 66.1 70.0 70.1 70.0TR61 Antalya 91.1 103.6 108.3 107.5 104.3TR62 Adana 111.1 78.6 78.3 79.1 77.1TR63 Hatay 68.9 62.5 60.0 61.2 64.3TR71 Kırıkkale 82.2 64.3 66.7 67.2 67.1TR72 Kayseri 60.0 67.9 68.3 70.1 70.0TR81 Zonguldak 100.0 121.4 115.0 104.5 98.6TR82 Kastamonu 66.7 73.2 71.7 70.1 68.6TR83 Samsun 71.1 67.9 70.0 68.7 71.4TR90 Trabzon 62.2 60.7 66.7 65.7 67.1TRA1 Erzurum 48.9 48.2 51.7 52.2 51.4TRA2 Ağrı 33.3 42.9 41.7 38.8 38.6TRB1 Malatya 62.2 55.4 55.0 55.2 55.7TRB2 Van 33.3 35.7 33.3 34.3 35.7TRC1 Gaziantep 62.2 57.1 53.3 52.2 51.4TRC2 Şanlıurfa 48.9 42.9 41.7 40.3 40.0TRC3 Mardin 40.0 41.1 43.3 41.8 41.4


Table 3.2 the goes the level deeper and relate the GDP/c in the various NUTS 2 regions of Turkey to relative to Turkey average. The table the again clearly indicates the divergences in Turkey. Since the commencement of the HRD OP Samsun has catch up and taken over Konya which region is not part of the HRD OP. Otherwise all NUTS 2 regions of the HRD OP are clearly behind the rest of the regions of Turkey.

Table 3.3 shows share of nation Gross Value Added, GVA, by broad economic sectors across Turkish NUTS 2 regions. Gross Value Added is a slightly more pure economic measure relative to GDP, since it does not include taxes and subsidies.

The table does not uncover surprises though we again are able to spot some differences across regions. Perhaps the most interesting feature of the table is the relative larger share of national GVA in Trabzon, Hatay, Kayseri and Samsun than for instance Agri; all of these being part of the HRD OP.

Table 3.3 Share of National GVA 2008 by broad economic sectors. 2008

2008 Agriculture Industry Services TOTAL

TR Turkey 100.0 100.0 100.0 100.0TR10 İstanbul 0.6 26.6 31.2 27.4TR21 Tekirdağ 3.7 3.7 2.3 2.8TR22 Balıkesir 5.2 1.9 2.0 2.2TR31 İzmir 3.8 6.4 7.0 6.6TR32 Aydın 6.1 2.8 3.6 3.6TR33 Manisa 8.8 4.4 2.8 3.7TR41 Bursa 4.1 10.2 5.4 6.6TR42 Kocaeli 4.3 8.6 5.3 6.1TR51 Ankara 2.5 7.7 9.3 8.3TR52 Konya 5.6 1.9 2.2 2.4

59

2008 Agriculture Industry Services TOTAL

TR61 Antalya 6.6 2.1 4.2 3.9TR62 Adana 7.2 3.5 3.8 4.0TR63 Hatay 4.8 2.7 2.4 2.7TR71 Kırıkkale 3.5 1.4 1.3 1.5TR72 Kayseri 3.4 2.6 2.1 2.3TR81 Zonguldak 0.9 1.8 1.2 1.3

TR82Kastamonu 2.0 0.5 0.7 0.8

TR83 Samsun 6.3 2.3 2.6 2.8TR90 Trabzon 4.5 2.1 2.7 2.7TRA1 Erzurum 1.7 0.6 0.9 0.9TRA2 Ağrı 1.8 0.3 0.6 0.6TRB1 Malatya 2.2 1.0 1.4 1.4TRB2 Van 2.6 0.6 1.0 1.0TRC1 Gaziantep 1.9 1.8 1.5 1.6TRC2 Şanlıurfa 4.2 1.1 1.7 1.7TRC3 Mardin 1.7 1.3 1.0 1.2

Source: TÜİK (TURKSTAT) latest data available and own calculations

Table 3.4 Growth rates in GVA 2005-2008 corrected for HCI2008 Agriculture Industry Services TOTAL

TR Turkey -8.3% 14.1% 25.2% 18.5%TR10 İstanbul -42.3% 5.5% 22.9% 17.6%TR21 Tekirdağ -5.7% 37.7% 27.4% 25.8%TR22 Balıkesir -3.1% 44.0% 34.2% 26.7%TR31 İzmir 3.3% 0.0% 23.7% 15.5%TR32 Aydın -8.3% -2.8% 26.2% 13.1%TR33 Manisa 9.7% 24.4% 28.1% 22.8%TR41 Bursa -22.6% 19.5% 25.7% 19.3%TR42 Kocaeli -11.6% 14.5% 30.3% 20.7%TR51 Ankara -14.6% 18.2% 19.2% 17.8%TR52 Konya -4.0% 1.5% 35.0% 17.2%TR61 Antalya 1.8% 5.7% 18.1% 13.6%TR62 Adana -8.2% 16.8% 19.4% 13.7%TR63 Hatay -6.5% 42.6% 35.0% 28.3%TR71 Kırıkkale -18.9% 30.3% 31.7% 17.6%TR72 Kayseri -17.1% 26.2% 23.4% 17.3%TR81 Zonguldak -12.5% -9.7% 6.1% -1.5%TR82 Kastamonu -2.2% 7.4% 17.7% 10.8%TR83 Samsun 0.3% 25.0% 24.1% 19.1%TR90 Trabzon -8.4% 12.8% 29.1% 18.8%TRA1 Erzurum -13.9% 15.9% 28.2% 16.8%TRA2 Ağrı -18.9% -4.7% 24.3% 6.9%TRB1 Malatya -18.7% 8.7% 29.3% 16.0%TRB2 Van -4.0% 2.6% 30.8% 17.0%TRC1 Gaziantep -20.8% 19.2% 17.5% 12.6%TRC2 Şanlıurfa -19.5% 12.6% 22.8% 9.3%TRC3 Mardin -14.9% 34.3% 31.4% 23.9%

Source: Own calculations based upon latest data available from TÜİK (TURKSTAT)

Table 3.4 then shows growth rates in GVA 2005 to 2008 corrected for inflation23. Please note that the measure covers a three years period and not as normal average annual growth rates. The point is of course to indicate a potential base line for the HRD OP which also covers a three year period; we would of course see some impact after the commencement of the HRD OP in GVA.

23 Table A1.1 in Annex shows Average annual growth rates in GDP/c. selected European countries for reference.

60

Otherwise, the table shows no major surprises though the differences between the regions are more pronounced. Perhaps it is interesting that Mardin has experienced one of the highest growth rates noting the low employment rate. Table 3.5 then enters the grey zone between labour market indicators and economic indicators as it illustrates a measure of productivity in European countries relatively to EU average. The important note in the table is the very low productivity of Turkey, which however is increasing.

Table 3.5 Measures of Productivity: GVA per Employee, EU27=1002005 2008

AT Austria 121.8 127.4BE Belgium 135.1 139.4BG Bulgaria 14.0 17.8CY Cyprus 76.4 82.2CZ Czech Republic 40.1 53.9DE Germany 117.8 116.9DK Denmark 136.5 141.4EE Estonia 35.5 45.1ES Spain 90.8 98.9FI Finland 120.9 129.1FR France 128.4 132.7GR Greece 85.6 93.5HR Croatia 42.5 51.5HU Hungary 41.1 47.1IE Ireland 156.1 155.2IT Italy 121.6 NALT Lithuania 27.3 38.7LU Luxembourg 294.6 354.6LV Latvia 24.2 38.3MT Malta 59.0 63.3NL Netherlands 119.7 124.8PL Poland 32.7 40.8PT Portugal 58.2 61.2RO Romania 17.1 28.0SE Sweden 128.7 129.8SI Slovenia 57.3 67.0SK Slovakia 32.6 48.3TR Turkey 35.7 43.4UK United Kingdom 122.2 113.3

Note: EstimatesSource: EUROSTAT and own calculations

Table 3.6 than turn to NUTS 2 regions in Turkey; this time by main economic sectors in 2008 relative to Turkish average. The table shows large and significant divergences in productivity across regions where we again find Agri in the bottom.

Measures of productivity is a highly important indicator since it measure how labour intensive the production is which is one of the best measures of how technological advanced the production is. Obviously, the better detail in the data the better the measure; we would of course have liked if we could have produced the measure on main NACE Rev 2.1 sectors.

Table 3.7 the shows the growth rates in productivity by main sectors between 2005 and 2008; again on a three years scale.

61

Table 3.6 Measures of Productivity: GVA per Employee, by main economic sector, 2008, Turkey=100 Agriculture Industry Services TOTALTR Turkey 100.0 100.0 100.0 100.0TR10 İstanbul 330.7 95.3 143.6 148.4TR21 Tekirdağ 141.4 107.3 86.2 98.4TR22 Balıkesir 101.3 84.8 85.2 75.1TR31 İzmir 152.4 98.4 120.3 126.2TR32 Aydın 93.9 86.0 90.1 82.2TR33 Manisa 97.8 101.8 76.1 76.1TR41 Bursa 147.3 104.0 113.2 117.0TR42 Kocaeli 213.7 156.4 117.5 147.7TR51 Ankara 306.9 131.4 107.1 137.3TR52 Konya 172.1 94.3 68.6 85.7TR61 Antalya 154.4 84.4 91.1 99.0TR62 Adana 190.2 83.5 77.1 91.9TR63 Hatay 104.9 73.2 66.2 69.5TR71 Kırıkkale 107.6 112.5 72.7 74.7TR72 Kayseri 107.0 85.5 75.0 78.9TR81 Zonguldak 38.7 129.2 90.3 87.9TR82 Kastamonu 117.1 94.4 88.5 83.6TR83 Samsun 57.2 85.7 75.3 57.1TR90 Trabzon 37.3 106.1 68.9 49.0TRA1 Erzurum 39.5 168.3 79.7 49.1TRA2 Ağrı 53.8 103.4 57.1 43.9TRB1 Malatya 95.0 101.9 78.4 78.4TRB2 Van 65.9 91.8 72.1 57.8TRC1 Gaziantep 127.1 56.7 68.3 71.5TRC2 Şanlıurfa 155.8 79.3 65.8 77.6TRC3 Mardin 112.5 133.3 55.7 74.8


Table 3.7 Measures of Productivity: Growth rates in GVA per Employee, by main economic sector, 2005 to 2008, corrected for price inflation (HCI)

Agriculture Industry Services TOTALTR Turkey -5.8 6.1 14.9 12.2TR10 İstanbul -38.2 6.0 11.1 11.1TR21 Tekirdağ 14.0 9.9 27.9 22.6TR22 Balıkesir 15.8 15.5 22.9 26.5TR31 İzmir 32.6 4.6 -2.9 4.4TR32 Aydın 24.7 6.7 2.4 15.4TR33 Manisa 38.4 12.5 29.6 31.7TR41 Bursa -14.0 19.8 11.6 15.3TR42 Kocaeli -42.1 -4.0 16.7 0.7TR51 Ankara 42.4 1.2 6.4 5.9TR52 Konya -36.4 -26.2 16.5 -10.2TR61 Antalya -36.6 3.5 14.9 -2.8TR62 Adana -28.2 8.6 11.6 1.8TR63 Hatay 3.8 12.9 33.7 24.6TR71 Kırıkkale 55.5 11.4 13.2 35.0TR72 Kayseri -7.9 28.9 23.9 21.8TR81 Zonguldak -42.3 3.6 19.3 -10.1TR82 Kastamonu -42.2 -22.6 -4.1 -23.6TR83 Samsun 4.5 1.6 18.8 16.4TR90 Trabzon 4.0 -19.3 23.7 20.7TRA1 Erzurum 18.0 -38.4 -0.6 23.3TRA2 Ağrı -29.9 1.3 64.1 5.5TRB1 Malatya -19.9 -23.2 8.7 0.3TRB2 Van 49.8 -22.0 -11.6 15.7TRC1 Gaziantep -58.0 11.0 39.6 0.6TRC2 Şanlıurfa -16.9 2.5 30.8 12.4

62

TRC3 Mardin 7.7 18.5 50.6 39.4Source: Own calculations based upon latest data available from TÜİK (TURKSTAT)

Finally, table 3.8 provides an overview of the main features of the main economic indicators directly linked to labour market features as it shows the average annual growth rates in Real GVA, in Productivity, in Real Wages, and in consumer prices.

Table 3.8 OVERVIEW: Table of average annual growth rates in Real GVA, Productivity, Real wages, and consumer prices 2005-2008

GROWTH RATES GVA Productivity Wages HCITR Turkey 6.2% 4.1% 4.8% 10.5%TR10 İstanbul 5.9% 3.7% 5.9% 11.3%TR21 Tekirdağ 8.6% 7.5% 3.1% 9.6%TR22 Balıkesir 8.9% 8.8% -5.3% 10.8%TR31 İzmir 5.2% 1.5% -1.5% 9.2%TR32 Aydın 4.4% 5.1% 1.5% 9.8%TR33 Manisa 7.6% 10.6% 4.7% 9.9%TR41 Bursa 6.4% 5.1% 5.7% 10.6%TR42 Kocaeli 6.9% 0.2% 4.0% 11.9%TR51 Ankara 5.9% 2.0% 6.7% 11.0%TR52 Konya 5.7% -3.4% 6.6% 10.2%TR61 Antalya 4.5% -0.9% 6.7% 9.8%TR62 Adana 4.6% 0.6% 2.4% 10.8%TR63 Hatay 9.4% 8.2% 9.4% 10.2%TR71 Kırıkkale 5.9% 11.7% 16.9% 11.0%TR72 Kayseri 5.8% 7.3% 8.0% 11.4%TR81 Zonguldak -0.5% -3.4% 2.9% 10.9%TR82 Kastamonu 3.6% -7.9% -3.6% 10.3%TR83 Samsun 6.4% 5.5% -1.8% 10.8%TR90 Trabzon 6.3% 6.9% 1.4% 11.1%TRA1 Erzurum 5.6% 7.8% 4.1% 11.3%TRA2 Ağrı 2.3% 1.8% 43.3% 11.0%TRB1 Malatya 5.3% 0.1% -4.5% 10.4%TRB2 Van 5.7% 5.2% -0.3% 11.4%TRC1 Gaziantep 4.2% 0.2% 13.3% 10.8%TRC2 Şanlıurfa 3.1% 4.1% 9.7% 10.8%TRC3 Mardin 8.0% 13.1% 29.3% 9.8%


The table tells a story by itself, comprehensive and somehow complex as it is. Showing these indictors next to each other is not arbitrary. The main is of course to follow a causal sequence where the end point, wage increases, never should be pursued without solid economical reason.

For instance, if GVA is growing fast we would expect a wage increase. The growth in GVA, however, could stem from a lot of factors not least inflation. We have corrected the GVA according to very rough measures here, but for the sake of the story we have shown the inflation rates in the last column in any case.

The next, and most important point, is that no wage increase can be justified with increase in productivity, which means that increase in wages has to be smaller than productivity growth in order to ensure a sustainable development and perhaps not least a profitable economy. Recall, that GVA or simply overhead has to finance labour costs and remuneration to business owners, but also physical technological change and investments.

Unfortunately, no indicator which clearly can show capital formation by economic sector and region (the more technical title of investments in physical capital) is available, recalling that investments in

63

Human Capital (i.e. education and training, including Lifelong Learning) is of paramount importance if Turkey, and the Turkish regions, will catch up with the fastest growing economies, if not in Asia then in Europe.

And then this section ends by a very clear link and indicator on HRD OP and its overall purpose. The reason for the massive investment in Human Resource Development in Turkish regions actually has a very serious purpose to ensure that the children have a decent opportunity to get a job.4. Labour Market Indicators

Table 4.1 shows the share of population in activity according to labour force survey data. The most interesting part is perhaps the fraction which is not in labour force. However, the share covers the complete population which includes elderly citizens, children, and students, so perhaps the table is not that interesting after all.

Table 4.1 Share of population in activity, total, 2010

CODE Region Population

LABOUR FORCE Not inLabour ForceEmployed Unemployed

TR Türkiye 100% 44% 6% 49%TR10 İstanbul 100% 42% 7% 51%TR21 Tekirdağ 100% 52% 6% 42%TR22 Balıkesir 100% 48% 4% 48%TR31 İzmir 100% 45% 8% 46%TR32 Aydın 100% 52% 7% 41%TR33 Manisa 100% 45% 4% 51%TR41 Bursa 100% 44% 5% 51%TR42 Kocaeli 100% 46% 7% 47%TR51 Ankara 100% 42% 6% 52%TR52 Konya 100% 49% 5% 47%TR61 Antalya 100% 51% 6% 43%TR62 Adana 100% 46% 9% 45%TR63 Hatay 100% 43% 7% 51%TR71 Kırıkkale 100% 45% 5% 50%TR72 Kayseri 100% 40% 6% 54%TR81 Zonguldak 100% 50% 6% 44%TR82 Kastamonu 100% 54% 5% 40%TR83 Samsun 100% 49% 4% 47%TR90 Trabzon 100% 58% 4% 38%TRA1 Erzurum 100% 49% 4% 47%TRA2 Ağrı 100% 44% 5% 50%TRB1 Malatya 100% 43% 6% 50%TRB2 Van 100% 36% 7% 57%TRC1 Gaziantep 100% 41% 6% 53%TRC2 Şanlıurfa 100% 31% 5% 65%TRC3 Mardin 100% 33% 4% 62%

Source: Own calculations based upon LFS, TÜİK/TURKSTAT

The table however leads directly to table 4.2 which shows growth rates for the same population groups and where we find an indicator closely linked to one of the measures of the HRD OP, namely the growth rate of inactive24. In general, the table shows some remarkable high growth rates, and even though the growth rates cover a three years’ span we could imagine that we, so soon in our study of labour market indicators, have started to discover some of the fundamental problems in using LFS data25.

24 Please note that we again, very contrary to common practice, show growth rates covering three years and not just annual averages.25 Basically not the purpose of the story of this study, we have note the nature of LFS Household survey which not just are based upon self-reporting among a sample of the population, but also uses a rolling panel principle (the

64

Table 4.2 Growth in population by activity, 2007 to 2010, total

CODE Region Population

LABOUR FORCE Not inLabour ForceEmployed Unemployed

TR Türkiye 5% 9% 28% 0%TR10 İstanbul 6% 3% 47% 6%TR21 Tekirdağ 5% 12% 45% -7%TR22 Balıkesir 4% -4% 34% 10%TR31 İzmir 6% 9% 64% -2%TR32 Aydın 5% 15% 43% -9%TR33 Manisa 4% 5% 13% 2%TR41 Bursa 8% 0% 27% 13%TR42 Kocaeli 7% 18% 35% -4%TR51 Ankara 7% 8% 14% 5%TR52 Konya 4% 19% -9% -8%TR61 Antalya 10% 6% 69% 10%TR62 Adana 4% 15% 23% -7%TR63 Hatay 7% 19% 32% -4%TR71 Kırıkkale 2% 11% 19% -6%TR72 Kayseri 4% 10% 32% -3%TR81 Zonguldak 3% 6% 39% -4%TR82 Kastamonu 2% 10% 117% -12%TR83 Samsun 2% -5% -24% 14%TR90 Trabzon 2% 3% -6% 3%TRA1 Erzurum 1% 18% 33% -13%TRA2 Ağrı 0% 4% 89% -8%TRB1 Malatya 3% 19% 3% -8%TRB2 Van 6% 12% 63% -2%TRC1 Gaziantep 8% 33% -17% -3%TRC2 Şanlıurfa 8% 26% 20% 1%TRC3 Mardin 6% 47% -22% -6%

Source: Own calculations based upon LFS, TÜİK/TURKSTAT

Table 4.3 then shows Economic activity rates for female across NUTS 2 regions in 2006 and 2010. Tables for total male and female as well as other EU countries are listed in Annex 1 as tables A1.2-7. Activity rates represent the labour force as a percentage of the population of working age (15-64 years).

Table 4.3 shows some remarkable low activity rates mainly in some the NUTS 2 regions covered by the HRD OP. Even though Turkey on average experiences activity rates for females less than half the EU27 average, a region like Mardin has female activity rates of 37 per cent of Turkish average; only 11 per cent of the women in Mardin are active.

Table 4.3 Economic activity rates (in per cent), Female, Turkish NUTS 2 Regions 2006 and 2010EU27=100 TURKEY=100

CODE GEO 2006 2010 2006 2010 2006 2010

EU27European Union (27 countries) 67.1 64.4 100.0 100.0 267.8 217.5


TR Turkey 25.1 29.6 37.3 46.0 100.0 100.0

respondents change every time instead of using the same). Furthermore, even though we call it Labour Force Survey, it is basically a household survey. In general, LFS provide basic data, which is not precise enough for creating solid interventions. However, we have no alternative in Turkey, yet

65

EU27=100 TURKEY=100CODE GEO 2006 2010 2006 2010 2006 2010TR10 İstanbul 22.8 25.8 33.9 40.0 90.9 87.1TR21 Tekirdağ 33.8 37.7 50.3 58.5 134.7 127.2TR22 Balıkesir 32.3 34.3 48.2 53.2 129.1 115.8TR31 İzmir 23.6 34.8 35.1 54.0 94.1 117.4TR32 Aydın 32.6 40.0 48.5 62.1 130.0 135.0TR33 Manisa 27.2 30.6 40.5 47.5 108.4 103.3TR41 Bursa 29.4 27.7 43.8 43.0 117.3 93.6TR42 Kocaeli 22.1 29.3 32.9 45.4 88.0 98.8TR51 Ankara 24.6 27.0 36.6 41.9 98.1 91.2TR52 Konya 16.5 31.3 24.6 48.5 65.8 105.5TR61 Antalya 37.0 42.3 55.1 65.6 147.6 142.8TR62 Adana 22.9 33.5 34.2 51.9 91.5 113.0TR63 Hatay 23.9 29.2 35.6 45.3 95.5 98.4TR71 Kırıkkale 27.6 27.3 41.1 42.4 110.2 92.2TR72 Kayseri 13.9 24.0 20.7 37.2 55.5 81.0TR81 Zonguldak 34.6 41.4 51.6 64.2 138.1 139.6TR82 Kastamonu 37.7 47.4 56.2 73.6 150.4 160.1TR83 Samsun 36.1 36.4 53.8 56.4 144.1 122.8TR90 Trabzon 52.0 49.1 77.5 76.2 207.4 165.8TRA1 Erzurum 29.1 36.2 43.3 56.2 116.0 122.3TRA2 Ağrı 31.5 30.0 47.0 46.6 125.8 101.3TRB1 Malatya 21.9 27.3 32.6 42.3 87.4 92.1TRB2 Van 19.6 17.8 29.2 27.7 78.1 60.2TRC1 Gaziantep 8.6 18.7 12.9 29.1 34.5 63.2TRC2 Şanlıurfa 5.8 10.2 8.6 15.8 23.1 34.4TRC3 Mardin 4.9 10.9 7.3 16.9 19.6 36.7

Source: EurostatActivity rates represent the labour force as a percentage of the population of working age (15-64 years).

If we then turn to youngest part of the working age population table 4.4 shows the activity rates for female between 15 and 24 years of age. Again, the activity rates are low, though not as low as for total women. However, this indicator bears a two-aged sword, since we would expect that a larger fraction of the age group being heavily occupied doing studies for the purpose of obtaining a solid job on the future labour market. And that indicates, that though Turkey appear to close in on EU27 average it might be a sign of EU27 being better a prolonging the stay at the educational centres for female.

In any case, Turkey differentiates from the main EU Member States, but more so, the topic is of vital interest for the HRD OP; both the topic of female / girl’s education but also the economic activity.

Table 4.4 Economic activity rates 2006 and 2010, Turkish NUTS 2 regions, Female, Age 15-24

EU27=100 TURKEY=100CODE GEO 2006 2010 2006 2010 2006 2010



TR Turkey 23.4 25.5 53.8 64.2 100.0 100.0TR10 İstanbul 28.7 30.2 66.0 76.1 122.6 118.4TR21 Tekirdağ 37.1 32.5 85.4 81.8 158.6 127.3TR22 Balıkesir 30.6 28.5 70.4 71.6 130.8 111.5TR31 İzmir 25.7 34.0 59.2 85.7 109.9 133.3TR32 Aydın 32.8 37.6 75.4 94.5 140.0 147.1TR33 Manisa 26.5 25.3 61.0 63.7 113.3 99.2TR41 Bursa 30.0 27.8 68.9 70.0 127.9 108.9TR42 Kocaeli 26.3 30.1 60.4 75.8 112.2 118.0TR51 Ankara 19.5 19.3 44.9 48.6 83.4 75.6TR52 Konya 17.7 25.5 40.6 64.1 75.4 99.8

66

EU27=100 TURKEY=100CODE GEO 2006 2010 2006 2010 2006 2010TR61 Antalya 35.5 36.9 81.7 92.9 151.8 144.5TR62 Adana 22.9 29.7 52.6 74.6 97.8 116.1TR63 Hatay 18.6 24.1 42.8 60.7 79.5 94.6TR71 Kırıkkale 25.1 18.7 57.6 47.0 107.0 73.1TR72 Kayseri 13.7 19.6 31.4 49.4 58.3 76.9TR81 Zonguldak 31.3 34.6 72.0 87.1 133.8 135.6TR82 Kastamonu 30.2 46.2 69.4 116.3 128.9 181.0TR83 Samsun 33.7 30.6 77.5 76.9 144.0 119.7TR90 Trabzon 38.4 29.5 88.3 74.3 164.0 115.7TRA1 Erzurum 20.9 25.5 48.1 64.1 89.3 99.8TRA2 Ağrı 17.2 16.0 39.6 40.4 73.6 62.8TRB1 Malatya 15.4 17.1 35.4 43.0 65.7 67.0TRB2 Van 12.8 11.7 29.3 29.4 54.5 45.7TRC1 Gaziantep 8.5 17.3 19.6 43.4 36.5 67.6TRC2 Şanlıurfa 5.9 10.8 13.5 27.2 25.2 42.3TRC3 Mardin 4.4 12.1 10.0 30.3 18.6 47.2


Table 4.5 shows employment rates by age group by Turkish NUTS 2 regions in 2010. Employment rates are of crucial importance for the HRD OP and address issues of several Priorities. Thus table 3.4.6 shows growth rates in employment rates for male and female.

Table 4.5 indicates some differences across regions, which to some extent is a lesser surprise now after the above studies. And notably Mardin and Sanliurfa appear to have almost criminal low employment rates. Fortunately these two regions appear to have a high growth rate in employment rates.

Table 4.5 Employment rates by age group 2010, male and femaleCODE GEO 15-24 25-54 55-64 TOTALTR Türkiye 28% 54% 28% 44%TR10 İstanbul 31% 50% 13% 42%TR21 Tekirdağ 34% 62% 34% 52%TR22 Balıkesir 27% 60% 30% 48%TR31 İzmir 29% 56% 20% 45%TR32 Aydın 36% 61% 33% 52%TR33 Manisa 24% 57% 32% 45%TR41 Bursa 28% 55% 19% 44%TR42 Kocaeli 32% 54% 28% 46%TR51 Ankara 22% 54% 17% 42%TR52 Konya 34% 58% 36% 49%TR61 Antalya 30% 61% 36% 51%TR62 Adana 31% 55% 28% 46%TR63 Hatay 26% 51% 34% 43%TR71 Kırıkkale 26% 56% 29% 45%TR72 Kayseri 21% 51% 27% 40%TR81 Zonguldak 26% 59% 46% 50%

TR82Kastamonu 41% 65% 37% 54%

TR83 Samsun 30% 60% 36% 49%TR90 Trabzon 25% 70% 58% 58%TRA1 Erzurum 25% 64% 40% 49%TRA2 Ağrı 25% 58% 35% 44%TRB1 Malatya 23% 54% 38% 43%TRB2 Van 20% 47% 30% 36%TRC1 Gaziantep 28% 50% 26% 41%TRC2 Şanlıurfa 21% 37% 23% 31%TRC3 Mardin 21% 42% 25% 33%

67

Source: Own calculation based upon LFS series, TÜİK/TURKSTAT

Table 4.6 Growth in Employment rates by age group 2007 to 2010 (percentage points)CODE GEO 15-24 25-54 55-64 TOTALTR Türkiye -0.6% 2.3% 1.2% 1.4%TR10 İstanbul -3.2% -1.1% -0.4% -1.6%TR21 Tekirdağ 0.4% 4.3% 4.3% 3.4%TR22 Balıkesir -5.0% -2.0% -7.2% -3.8%TR31 İzmir -4.6% 3.3% 2.3% 1.0%TR32 Aydın 0.6% 5.5% 6.8% 4.4%TR33 Manisa -6.4% 3.2% 4.8% 0.6%TR41 Bursa -7.7% -1.3% -2.4% -3.3%TR42 Kocaeli 4.1% 3.6% 8.5% 4.2%TR51 Ankara -2.2% 1.2% 1.4% 0.4%TR52 Konya 4.2% 6.8% 9.3% 6.5%TR61 Antalya -8.5% 0.3% 1.1% -2.0%TR62 Adana 4.6% 4.3% 3.7% 4.2%TR63 Hatay 4.0% 4.6% 5.4% 4.5%TR71 Kırıkkale 1.0% 5.2% 0.2% 3.5%TR72 Kayseri -1.7% 3.5% 6.6% 2.4%TR81 Zonguldak -6.1% 4.4% 1.1% 1.5%TR82 Kastamonu 4.5% 6.9% -7.6% 3.8%TR83 Samsun -2.8% -2.1% -11.1% -3.6%TR90 Trabzon -5.5% 1.1% 4.0% 0.1%TRA1 Erzurum 2.7% 9.8% 4.1% 6.9%TRA2 Ağrı 3.2% 3.2% -15.9% 1.7%TRB1 Malatya 7.0% 5.2% 2.7% 5.8%TRB2 Van 1.6% 3.6% -11.4% 1.8%TRC1 Gaziantep 2.0% 11.2% 2.7% 7.7%TRC2 Şanlıurfa 6.7% 2.9% 2.2% 4.3%TRC3 Mardin 10.1% 9.5% 3.8% 9.3%


5. Indicators on Education

Table 5.1 shows population by literacy status in 2010. Literacy is still an issue in Turkey as just 90 per cent of the population is literate and in some of the regions as low as 82 per cent. The HRD OP has captured that in order to increase the educational attainment in the 12 NUTS 2 regions.

Table 5.1 Population by literacy status (6 years of age and over) 2010, both genderCODE GEO Illiterate Literate Unknown TOTALTR10 İstanbul 3.1% 91.5% 5.3% 100.0%TR21 Tekirdağ 3.8% 93.2% 3.0% 100.0%TR22 Balıkesir 5.0% 91.7% 3.3% 100.0%TR31 İzmir 3.8% 92.1% 4.2% 100.0%TR32 Aydın 5.0% 91.2% 3.8% 100.0%TR33 Manisa 6.4% 91.4% 2.2% 100.0%TR41 Bursa 3.7% 92.8% 3.5% 100.0%TR42 Kocaeli 4.2% 92.1% 3.8% 100.0%TR51 Ankara 3.0% 92.9% 4.1% 100.0%TR52 Konya 5.2% 92.2% 2.5% 100.0%TR61 Antalya 3.0% 92.9% 4.1% 100.0%TR62 Adana 5.9% 90.3% 3.7% 100.0%TR63 Hatay 7.2% 89.0% 3.8% 100.0%TR71 Kırıkkale 6.7% 90.2% 3.1% 100.0%TR72 Kayseri 6.3% 88.4% 5.3% 100.0%TR81 Zonguldak 7.4% 90.0% 2.7% 100.0%TR82 Kastamonu 9.4% 87.9% 2.6% 100.0%TR83 Samsun 7.6% 88.7% 3.8% 100.0%TR90 Trabzon 8.4% 87.7% 3.9% 100.0%

68

TRA1 Erzurum 8.3% 87.1% 4.6% 100.0%TRA2 Ağrı 10.2% 81.7% 8.1% 100.0%TRB1 Malatya 9.2% 86.4% 4.4% 100.0%TRB2 Van 11.7% 83.1% 5.2% 100.0%TRC1 Gaziantep 7.9% 87.8% 4.2% 100.0%TRC2 Şanlıurfa 12.2% 82.3% 5.5% 100.0%TRC3 Mardin 12.2% 83.3% 4.5% 100.0%TR TURKEY 5.8% 90.0% 4.2% 100.0%

Source: TÜİK/TURKSTAT and own calculationsTable 5.2 shows the same for the female population, which issue is of particular focus of one the five Priorities of the HRD OP. Again, Turkey has a low literacy level and in some regions even as low as 75 per cent of the total female population cannot read and write.

Table 5.2 Population by literacy status (6 years of age and over) 2010, femaleCODE GEO Illiterate Literate UnknownTR10 İstanbul 5.2% 89.9% 4.9%TR21 Tekirdağ 6.1% 91.3% 2.6%TR22 Balıkesir 7.8% 89.3% 2.9%TR31 İzmir 6.1% 90.2% 3.7%TR32 Aydın 8.3% 88.3% 3.3%TR33 Manisa 10.6% 87.7% 1.8%TR41 Bursa 6.1% 90.6% 3.3%TR42 Kocaeli 7.0% 89.3% 3.7%TR51 Ankara 5.1% 91.0% 3.9%TR52 Konya 8.8% 89.0% 2.3%TR61 Antalya 5.2% 91.3% 3.5%TR62 Adana 9.6% 87.0% 3.4%TR63 Hatay 11.8% 84.9% 3.3%TR71 Kırıkkale 11.3% 85.8% 2.9%TR72 Kayseri 10.4% 84.3% 5.3%TR81 Zonguldak 12.2% 85.2% 2.6%TR82 Kastamonu 14.6% 83.0% 2.4%TR83 Samsun 11.7% 84.7% 3.6%TR90 Trabzon 13.9% 82.2% 3.9%TRA1 Erzurum 13.8% 81.6% 4.6%TRA2 Ağrı 16.8% 75.2% 8.0%TRB1 Malatya 14.6% 80.9% 4.4%TRB2 Van 19.0% 75.4% 5.6%TRC1 Gaziantep 12.9% 83.1% 4.0%TRC2 Şanlıurfa 19.7% 74.2% 6.2%TRC3 Mardin 19.6% 75.5% 4.9%TR TURKEY 9.5% 86.5% 4.0%

Source: TÜİK/TURKSTAT and own calculations

Table 5.3 are able to split literacy up according to educational attainment for male and female in 2010. The main regional divergences can be found in column 1 and 2 on illiteracy (which we have dealt with just above) and “Literate but no school completed”, where some of the regions of the HRD OP show very high rates.

Table 5.3 Share of Population aged 15 and over by educational attainment, 2010, Male and Female

CODE GEO Illiterate

Literate but no school completed

Junior high school or vocational school at

junior high school level

graduate

High school or

vocational school at

high school level

graduate

Higher education graduate and more

TR10 İstanbul 3.7% 4.3% 6.7% 23.4% 11.8%TR21 Tekirdağ 4.4% 4.3% 6.5% 22.6% 8.5%TR22 Balıkesir 5.7% 5.1% 5.3% 19.4% 8.9%

69

CODE GEO Illiterate


Junior high school or vocational school at

junior high school level

graduate

High school or

vocational school at

high school level

graduate

Higher education graduate and more

TR31 İzmir 4.3% 4.1% 6.4% 23.2% 12.1%TR32 Aydın 5.9% 4.5% 5.1% 18.9% 9.1%TR33 Manisa 7.4% 5.2% 4.9% 17.9% 6.8%TR41 Bursa 4.3% 3.8% 7.2% 23.9% 9.7%TR42 Kocaeli 5.0% 4.3% 6.3% 22.7% 8.8%TR51 Ankara 3.5% 2.9% 7.2% 27.4% 17.0%TR52 Konya 6.4% 3.9% 4.8% 17.2% 7.8%TR61 Antalya 3.6% 5.2% 5.5% 21.9% 10.2%TR62 Adana 7.2% 5.8% 6.0% 21.8% 8.8%TR63 Hatay 9.1% 6.3% 5.4% 18.6% 6.5%TR71 Kırıkkale 8.1% 4.9% 6.1% 19.3% 7.1%TR72 Kayseri 7.6% 4.9% 5.5% 20.7% 7.8%TR81 Zonguldak 8.5% 5.6% 5.3% 20.0% 7.4%TR82 Kastamonu 10.9% 7.1% 4.9% 16.3% 6.5%TR83 Samsun 9.0% 6.3% 4.7% 17.5% 7.2%TR90 Trabzon 9.9% 6.2% 5.7% 21.0% 7.8%TRA1 Erzurum 10.3% 7.6% 4.9% 20.2% 7.5%TRA2 Ağrı 13.7% 13.6% 3.4% 13.5% 4.2%TRB1 Malatya 11.1% 7.2% 5.7% 21.3% 7.8%TRB2 Van 16.2% 16.2% 2.8% 13.7% 4.0%TRC1 Gaziantep 10.3% 7.9% 4.8% 16.0% 5.6%TRC2 Şanlıurfa 16.7% 16.4% 2.8% 13.2% 4.3%TRC3 Mardin 17.1% 13.6% 3.0% 15.0% 4.2%TR TURKEY 7.0% 5.9% 5.7% 20.8% 9.2%

Source: TÜİK/TURKSTAT and own calculationsSee more extensive table in Annex 1

Table 5.4, showing Share of population age 15 and above with at least High school or vocational school at high school level, tries to come closer to a doable international measure (we often tries to obtain a measure of share of persons aged 25 with at least ISCED 3). The table are able to indicate a significant gender difference, which difference becomes more accentuate when we look at the NUTS 2 regions taking part of the HRD OP.Table 5.4 Share of population age 15 and above with at least High school or vocational school at high

school level, male and female, 2008 and 20102008 2010

CODE GEO MALE FEMALE TOTAL MALE FEMALE TOTALTR10 İstanbul 33.7% 28.4% 31.0% 38.1% 32.2% 35.2%TR21 Tekirdağ 31.2% 23.2% 27.4% 35.2% 26.8% 31.1%TR22 Balıkesir 29.1% 20.0% 24.5% 33.3% 23.2% 28.3%TR31 İzmir 34.0% 28.7% 31.3% 38.3% 32.3% 35.3%TR32 Aydın 27.1% 21.2% 24.2% 31.1% 24.9% 28.0%TR33 Manisa 25.8% 16.0% 20.8% 30.4% 18.8% 24.7%TR41 Bursa 34.7% 24.0% 29.3% 39.4% 27.8% 33.6%TR42 Kocaeli 32.7% 21.3% 27.0% 37.7% 25.3% 31.5%TR51 Ankara 45.5% 36.4% 40.9% 49.2% 39.7% 44.4%TR52 Konya 28.5% 16.3% 22.3% 31.6% 18.7% 25.0%TR61 Antalya 31.4% 24.0% 27.7% 36.1% 27.9% 32.1%TR62 Adana 30.4% 23.4% 26.8% 34.6% 26.5% 30.5%TR63 Hatay 26.1% 17.0% 21.5% 30.2% 19.8% 25.1%TR71 Kırıkkale 28.5% 16.8% 22.6% 32.7% 20.2% 26.3%TR72 Kayseri 30.5% 18.7% 24.6% 34.9% 22.1% 28.5%TR81 Zonguldak 29.1% 17.9% 23.4% 33.9% 21.0% 27.4%TR82 Kastamonu 25.2% 14.1% 19.5% 29.2% 16.8% 22.9%TR83 Samsun 25.7% 16.4% 21.0% 30.1% 19.5% 24.8%TR90 Trabzon 30.4% 19.2% 24.8% 35.1% 22.6% 28.7%

70

TRA1 Erzurum 31.4% 15.5% 23.5% 35.9% 19.4% 27.7%TRA2 Ağrı 20.0% 9.9% 15.2% 22.8% 12.3% 17.7%TRB1 Malatya 32.8% 18.4% 25.6% 36.7% 21.7% 29.1%TRB2 Van 21.7% 8.1% 15.1% 25.0% 10.2% 17.7%TRC1 Gaziantep 23.1% 13.5% 18.2% 26.9% 16.3% 21.6%TRC2 Şanlıurfa 20.3% 9.2% 14.7% 23.8% 11.5% 17.5%TRC3 Mardin 24.8% 9.0% 17.1% 27.7% 10.9% 19.3%TR TURKEY 30.7% 21.7% 26.2% 35.0% 25.1% 30.0%

Source: TUIK/TURKSTAT and own calculationsTable 5.5 finally shows the growth in the share of population with at least High school or vocational school at high school level where the most remarkable feature is the regions experiencing a lower than average growth rate. That is the feature of most of the NUTS 2 Regions of HRD OP.

Table 5.5 Share of population age 15 and above with at least High school or vocational school at high school level, male and female, 2008 and 2010: Growth from 2008 to 2010 in percentage points

CODE GEO MALE FEMALE TOTALTR10 Istanbul 4.4% 3.9% 4.2%TR21 Tekirdag 4.0% 3.6% 3.7%TR22 Balikesir 4.2% 3.2% 3.7%TR31 Izmir 4.3% 3.6% 4.0%TR32 Aydin 4.0% 3.7% 3.8%TR33 Manisa 4.6% 2.7% 3.8%TR41 Bursa 4.7% 3.8% 4.3%TR42 Kocaeli 5.0% 4.0% 4.5%TR51 Ankara 3.7% 3.4% 3.5%TR52 Konya 3.1% 2.4% 2.8%TR61 Antalya 4.7% 3.9% 4.4%TR62 Adana 4.2% 3.1% 3.7%TR63 Hatay 4.1% 2.8% 3.6%TR71 Kirikkale 4.2% 3.4% 3.7%TR72 Kayseri 4.4% 3.4% 3.9%TR81 Zonguldak 4.7% 3.0% 4.0%TR82 Kastamonu 4.0% 2.7% 3.4%TR83 Samsun 4.4% 3.1% 3.8%TR90 Trabzon 4.7% 3.4% 4.0%TRA1 Erzurum 4.5% 3.9% 4.2%TRA2 Agri 2.8% 2.4% 2.5%TRB1 Malatya 3.9% 3.3% 3.5%TRB2 Van 3.3% 2.1% 2.6%TRC1 Gaziantep 3.9% 2.8% 3.3%TRC2 Sanliurfa 3.4% 2.3% 2.8%TRC3 Mardin 2.9% 1.9% 2.2%TR TURKEY 4.2% 3.4% 3.8%

Source: TÜİK/TURKSTAT and own calculations

Something appears odd, however; in just three years the share of population with at least High School level has increased significantly in all regions by 3.8 per cent on average. The significant increase does call for a quality check of the data, which however was not possible during this study.

The Socio-Economic Study is not designated to provide basic new knowledge inasmuch as the Interim Evaluation would anticipate that the table and the results shown are well-known to all stakeholders in the HRD OP. The study is solely meant as a reference for the remainder of the Interim Evaluation report and does accordingly not call for conclusions.

71

Appendix 1. Background Tables

Table A1.1 Average annual growth rates in GDP/c. selected European countriesGEO COUNTRY 2005 2006 2007 2008EU27 European Union (27 countries) 3.6 5.3 5.5 0.4AT Austria 2.9 5.1 5.5 3.7BE Belgium 3.5 4.5 4.3 2.2BG Bulgaria 15.3 13.3 17.6 17.5CH Switzerland 1.4 3.5 0.7 6.7CY Cyprus 4.8 5.6 6.8 7.4CZ Czech Republic 12.7 13.3 10.8 15.4DE Germany 1.7 3.7 5.0 2.0DK Denmark 3.6 5.0 3.7 1.9EE Estonia 16.9 20.5 18.0 1.7ES Spain 6.6 6.7 5.4 1.7FI Finland 3.5 5.0 7.9 2.4FR France 3.1 4.0 4.2 2.4GR Greece 7.8 8.6 6.8 3.9HR Croatia NA 9.9 10.1 9.2HU Hungary 15.2 1.1 12.4 6.0IE Ireland 8.3 6.7 4.3 -6.7IT Italy 6.5 -1.4 9.6 -32.9LT Lithuania 3.3 3.3 3.2 0.8LU Luxembourg NA NA NA NALV Latvia 14.9 16.4 19.7 12.9MK FYROM 5.9 10.1 8.8 4.0MT Malta 11.7 22.8 32.9 9.7NL Netherlands 5.3 8.3 11.5 13.8NO Norway 2.0 6.7 6.3 4.4PL Poland 4.0 5.1 5.4 4.0PT Portugal 5.8 9.5 4.5 6.6RO Romania 6.1 10.9 15.5 15.9SE Sweden 3.5 3.4 5.3 1.9SI Slovenia 21.1 21.6 28.9 12.1SK Slovakia 1.9 6.1 5.1 -2.2TR Turkey 6.7 7.6 10.3 7.6UK United Kingdom 14.6 16.9 22.9 16.7


72

Table A1.2 Economic activity rates (in per cent), both gender

EU27 AVERAGETURKEY

AVERAGECODE GEO 2006 2010 2006 2010 2006 2010EU27 European Union (27 countries) 71.0 75.0 100.0 100.0 136.7 153.1EU25 European Union (25 countries) 71.4 71.9 100.6 96.0 137.5 146.9AT Austria 75.1 73.7 105.7 98.3 144.6 150.5BE Belgium 67.7 66.5 95.3 88.7 130.3 135.8BG Bulgaria 66.5 64.5 93.7 86.0 128.1 131.6CH Switzerland 82.4 81.3 116.1 108.4 158.7 165.9CY Cyprus 74.4 73.0 104.8 97.4 143.3 149.1CZ Czech Republic 70.2 70.3 98.8 93.8 135.1 143.6DE Germany 76.6 75.3 107.9 100.4 147.5 153.7DK Denmark 79.5 0.0 111.9 0.0 153.0 0.0EE Estonia 73.8 72.4 103.9 96.7 142.0 147.9ES Spain 73.4 70.9 103.3 94.5 141.2 144.7FI Finland 74.5 75.2 104.9 100.3 143.4 153.6FR France 70.3 69.6 99.1 92.9 135.4 142.1GR Greece 68.2 67.0 96.1 89.4 131.4 136.9HR Croatia 61.5 0.0 86.6 0.0 118.4 0.0HU Hungary 62.4 62.0 87.9 82.7 120.2 126.5IE Ireland 69.5 71.8 97.9 95.8 133.8 146.6IS Iceland 84.7 87.1 119.3 116.2 163.1 177.8IT Italy 62.2 62.7 87.6 83.7 119.7 128.1LT Lithuania 70.5 67.4 99.3 90.0 135.8 137.7LU Luxembourg 68.2 66.7 96.1 89.0 131.3 136.2LV Latvia 73.2 71.3 103.1 95.1 140.9 145.5MT Malta 60.2 57.6 84.8 76.9 116.0 117.6NL Netherlands 78.2 77.4 110.1 103.2 150.5 158.0NO Norway 78.1 78.0 110.0 104.1 150.4 159.4PL Poland 65.6 63.3 92.4 84.4 126.4 129.2PT Portugal 74.0 73.9 104.2 98.6 142.5 150.9RO Romania 63.7 0.0 89.6 0.0 122.5 0.0SE Sweden 79.5 0.0 112.0 0.0 153.1 0.0SI Slovenia 71.5 70.9 100.7 94.6 137.7 144.8SK Slovakia 68.7 68.6 96.7 91.6 132.2 140.1TR Turkey 51.9 49.0 73.1 65.3 100.0 100.0UK United Kingdom 75.5 75.7 106.3 101.0 145.3 154.6


73

Table A1.3 Economic activity rates (in per cent), female

EU27 AVERAGETURKEY

AVERAGECODE GEO 2006 2010 2006 2010 2006 2010EU27 European Union (27 countries) 67.1 64.43 100.0 100.0 267.8 217.5EU25 European Union (25 countries) 64.35 64.88 95.9 100.7 256.8 219.0AT Austria 66.99 69.33 99.8 107.6 267.3 234.1BE Belgium 59.53 61.82 88.7 95.9 237.5 208.7BG Bulgaria 60.23 62.32 89.8 96.7 240.3 210.4CH Switzerland 74.71 76.43 111.3 118.6 298.1 258.0CY Cyprus 63.8 67.39 95.1 104.6 254.6 227.5CZ Czech Republic 62.34 61.52 92.9 95.5 248.8 207.7DE Germany 69.1 70.82 103.0 109.9 275.7 239.1DK Denmark 0 76.12 0.0 118.1 0.0 257.0EE Estonia 69.31 70.99 103.3 110.2 276.6 239.7ES Spain 60.17 65.9 89.7 102.3 240.1 222.5FI Finland 73.26 72.5 109.2 112.5 292.3 244.8FR France 64.62 66.02 96.3 102.5 257.9 222.9GR Greece 55.01 57.55 82.0 89.3 219.5 194.3HR Croatia 0 55.89 0.0 86.7 0.0 188.7HU Hungary 55.51 56.7 82.7 88.0 221.5 191.4IE Ireland 61.9 61.97 92.3 96.2 247.0 209.2IS Iceland 83.37 81.81 124.2 127.0 332.7 276.2IT Italy 50.81 51.13 75.7 79.4 202.8 172.6LT Lithuania 64.57 68.8 96.2 106.8 257.7 232.3LU Luxembourg 58.23 60.28 86.8 93.6 232.4 203.5LV Latvia 66.65 70.72 99.3 109.8 266.0 238.8MT Malta 36.52 42.19 54.4 65.5 145.7 142.4NL Netherlands 70.73 72.61 105.4 112.7 282.2 245.1NO Norway 74.75 75.52 111.4 117.2 298.3 255.0PL Poland 56.7 59.01 84.5 91.6 226.3 199.2PT Portugal 68.43 69.86 102.0 108.4 273.1 235.9RO Romania 0 55.79 0.0 86.6 0.0 188.4SE Sweden 0 76.69 0.0 119.0 0.0 258.9SI Slovenia 66.74 67.4 99.5 104.6 266.3 227.5SK Slovakia 60.93 61.31 90.8 95.2 243.1 207.0TR Turkey 25.06 29.62 37.3 46.0 100.0 100.0UK United Kingdom 69.24 69.38 103.2 107.7 276.3 234.2


74

Table A1.4 Economic activity rates at NUTS levels 2 (in per cent), male

EU27 AVERAGE TURKEY AVERAGECODE GEO 2006 2010 2006 2010 2006 2010



TR Turkey 49.0 51.9 65.3 73.1 100.0 100.0TR10 Istanbul 49.4 50.8 65.9 71.5 100.9 97.7TR21 Tekirdag 58.4 58.7 77.9 82.7 119.3 113.1TR22 Balikesir 53.5 53.6 71.4 75.5 109.3 103.2TR31 Izmir 47.8 54.6 63.7 76.9 97.6 105.2TR32 Aydin 53.7 58.9 71.6 82.9 109.6 113.4TR33 Manisa 50.7 53.1 67.7 74.8 103.6 102.2TR41 Bursa 52.7 50.6 70.4 71.2 107.7 97.4TR42 Kocaeli 48.3 52.4 64.4 73.8 98.6 101.0TR51 Ankara 48.3 49.5 64.5 69.7 98.7 95.2TR52 Konya 44.2 54.7 58.9 77.0 90.2 105.2TR61 Antalya 59.3 61.6 79.1 86.7 121.0 118.5TR62 Adana 47.7 55.3 63.6 77.9 97.4 106.5TR63 Hatay 45.3 51.4 60.4 72.3 92.4 98.9TR71 Kirikkale 49.7 50.0 66.2 70.5 101.4 96.3TR72 Kayseri 41.4 48.1 55.2 67.7 84.5 92.6TR81 Zonguldak 53.4 57.3 71.2 80.6 109.0 110.2TR82 Kastamonu 58.6 62.2 78.2 87.6 119.7 119.7TR83 Samsun 54.5 54.6 72.8 76.9 111.4 105.1TR90 Trabzon 63.9 61.8 85.3 87.0 130.5 118.9TRA1 Erzurum 50.3 56.3 67.1 79.3 102.7 108.4TRA2 Agri 52.6 52.9 70.1 74.5 107.3 101.9TRB1 Malatya 43.5 49.6 58.0 69.9 88.8 95.6TRB2 Van 42.6 45.1 56.9 63.5 87.0 86.8TRC1 Gaziantep 40.6 47.3 54.1 66.6 82.8 91.0TRC2 Sanliurfa 32.6 34.5 43.5 48.6 66.5 66.5TRC3 Mardin 30.4 37.8 40.5 53.3 62.0 72.8

Source: EUROSTAT, Labour Force Statistics

75

Table A1.5 Lowest 15 Economic activity rates at NUTS levels 2 (in per cent) among 319 identified NUTS 2 Regions in EUROSTAT, Sorted by EU27 Average 2006

EU27 AVERAGECODE GEO 2006 2010 2006a 2010bTRA1 Erzurum 50.28 56.29 67.1 79.3TR71 Kirikkale 49.65 50.04 66.2 70.5TR10 Istanbul 49.39 50.76 65.9 71.5TR51 Ankara 48.34 49.46 64.5 69.7TR42 Kocaeli 48.29 52.44 64.4 73.8TR31 Izmir 47.78 54.63 63.7 76.9TR62 Adana 47.69 55.34 63.6 77.9TR63 Hatay 45.26 51.37 60.4 72.3TR52 Konya 44.15 54.65 58.9 77.0TRB1 Malatya 43.48 49.64 58.0 69.9TRB2 Van 42.62 45.09 56.9 63.5TR72 Kayseri 41.37 48.08 55.2 67.7TRC1 Gaziantep 40.57 47.27 54.1 66.6TRC2 Sanliurfa 32.57 34.54 43.5 48.6TRC3 Mardin 30.35 37.83 40.5 53.3

Sorted by EU27 Average 2010ITF4 Puglia 52.48 51.41 70.0 72.4TR63 Hatay 45.26 51.37 60.4 72.3TR10 Istanbul 49.39 50.76 65.9 71.5TR41 Bursa 52.73 50.59 70.4 71.2ITG1 Sicilia 52.12 50.06 69.5 70.5TR71 Kirikkale 49.65 50.04 66.2 70.5TRB1 Malatya 43.48 49.64 58.0 69.9TR51 Ankara 48.34 49.46 64.5 69.7TR72 Kayseri 41.37 48.08 55.2 67.7ITF6 Calabria 52.38 47.94 69.9 67.5TRC1 Gaziantep 40.57 47.27 54.1 66.6ITF3 Campania 50.7 46.44 67.6 65.4TRB2 Van 42.62 45.09 56.9 63.5TRC3 Mardin 30.35 37.83 40.5 53.3TRC2 Sanliurfa 32.57 34.54 43.5 48.6


76

Table A1.6 Economic activity rates 2006 and 2010, both gender, Age 15-24

EU27=100 TURKEY=100CODE GEO 2006 2010 2006 2010 2006 2010EU27 European Union (27 countries) 47.3 43.0 100.0 100.0 130.4 115.0EU25 European Union (25 countries) 45.8 43.9 96.9 102.0 126.4 117.4AT Austria 59.4 58.8 125.6 136.9 163.9 157.5BE Belgium 34.7 32.5 73.4 75.6 95.7 87.0BG Bulgaria 28.9 28.9 61.0 67.3 79.6 77.4CH Switzerland 68.6 67.9 145.1 157.9 189.2 181.6CY Cyprus 41.5 40.6 87.8 94.4 114.4 108.6CZ Czech Republic 33.5 30.9 70.9 71.8 92.5 82.6DE Germany 50.2 51.3 106.1 119.4 138.4 137.4DK Denmark 0.0 67.4 0.0 156.7 0.0 180.3EE Estonia 35.9 38.3 76.0 89.1 99.1 102.5ES Spain 48.2 42.7 101.8 99.3 132.8 114.2FI Finland 51.8 49.4 109.5 114.9 142.8 132.1FR France 38.2 39.4 80.7 91.6 105.2 105.4GR Greece 32.4 30.3 68.5 70.6 89.3 81.2HR Croatia 0.0 37.7 0.0 87.7 0.0 100.9HU Hungary 26.8 24.9 56.6 58.0 73.8 66.7IE Ireland 54.7 42.0 115.6 97.8 150.7 112.5IS Iceland 78.6 73.7 166.2 171.4 216.8 197.1IT Italy 32.5 28.4 68.7 66.0 89.6 76.0LT Lithuania 26.3 29.6 55.5 68.9 72.4 79.3LU Luxembourg 27.8 24.7 58.8 57.5 76.7 66.2LV Latvia 40.8 40.4 86.3 94.0 112.6 108.1MT Malta 52.6 51.5 111.1 119.9 144.9 137.9NL Netherlands 70.8 69.0 149.7 160.5 195.2 184.7NO Norway 57.4 56.7 121.4 131.9 158.4 151.8PL Poland 34.0 34.5 71.8 80.2 93.7 92.2PT Portugal 42.7 36.7 90.3 85.4 117.8 98.3RO Romania 0.0 31.2 0.0 72.7 0.0 83.6SE Sweden 0.0 51.7 0.0 120.3 0.0 138.4SI Slovenia 40.6 39.9 85.9 92.9 112.0 106.9SK Slovakia 35.4 31.1 74.7 72.3 97.5 83.1TR Turkey 36.3 37.4 76.7 86.9 100.0 100.0UK United Kingdom 62.5 59.2 132.1 137.6 172.3 158.3


77

Table A1.7 Economic activity rates 2006 and 2010, Female, Age 15-24

EU27=100 TURKEY=100CODE GEO 2006 2010 2006 2010 2006 2010EU27 European Union (27 countries) 43.5 39.7 100.0 100.0 185.7 155.7EU25 European Union (25 countries) 42.3 40.7 97.2 102.5 180.6 159.5AT Austria 55.1 54.1 126.7 136.2 235.2 212.0BE Belgium 31.9 29.8 73.4 74.9 136.4 116.6BG Bulgaria 26.4 24.2 60.6 60.8 112.6 94.7CH Switzerland 67.0 66.5 154.2 167.4 286.3 260.6CY Cyprus 38.4 40.7 88.2 102.5 163.8 159.6CZ Czech Republic 29.2 25.3 67.1 63.6 124.6 98.9DE Germany 47.6 48.9 109.4 123.0 203.1 191.4DK Denmark 0.0 67.2 0.0 169.2 0.0 263.3EE Estonia 30.6 34.3 70.4 86.2 130.7 134.2ES Spain 43.9 40.1 101.0 101.0 187.7 157.2FI Finland 51.0 49.3 117.4 124.1 218.0 193.2FR France 34.4 35.8 79.0 90.0 146.7 140.0GR Greece 28.7 27.2 65.9 68.5 122.4 106.7HR Croatia 0.0 31.2 0.0 78.5 0.0 122.2HU Hungary 23.4 22.1 53.8 55.6 99.9 86.5IE Ireland 50.2 41.2 115.5 103.7 214.6 161.5IS Iceland 80.3 76.1 184.7 191.5 343.1 298.0IT Italy 27.0 23.4 62.0 58.8 115.1 91.5LT Lithuania 23.1 26.3 53.1 66.2 98.5 103.1LU Luxembourg 25.0 22.7 57.5 57.0 106.8 88.7LV Latvia 33.6 37.7 77.4 94.8 143.7 147.6MT Malta 48.3 47.4 111.0 119.3 206.2 185.7NL Netherlands 70.1 69.4 161.1 174.7 299.2 272.0NO Norway 58.0 56.9 133.3 143.1 247.6 222.7PL Poland 30.5 29.7 70.1 74.7 130.2 116.3PT Portugal 38.7 34.8 89.0 87.5 165.4 136.2RO Romania 0.0 26.1 0.0 65.7 0.0 102.3SE Sweden 0.0 51.4 0.0 129.2 0.0 201.2SI Slovenia 36.4 34.8 83.7 87.6 155.5 136.4SK Slovakia 30.9 25.5 71.0 64.2 131.8 99.9TR Turkey 23.4 25.5 53.8 64.2 100.0 100.0UK United Kingdom 59.7 56.4 137.4 141.9 255.2 220.9


78

Table A1.8 Share of population in activity, 2010, total population

MALE AND FEMALE MALE FEMALE

LABOUR FORCE LABOUR FORCE LABOUR FORCE

CODE GEO POP EMP UEMP NILF POP EMP UEMP NILF POP EMP UEMP NILFTR Türkiye 100% 44% 6% 49% 100% 63% 8% 29% 100% 26% 4% 71%TR10 Istanbul 100% 42% 7% 51% 100% 63% 10% 27% 100% 21% 4% 75%TR21 Tekirdag 100% 52% 6% 42% 100% 71% 7% 23% 100% 33% 5% 62%TR22 Balikesir 100% 48% 4% 48% 100% 63% 5% 32% 100% 31% 3% 66%TR31 Izmir 100% 45% 8% 46% 100% 62% 9% 29% 100% 28% 7% 65%TR32 Aydin 100% 52% 7% 41% 100% 68% 8% 24% 100% 35% 6% 59%TR33 Manisa 100% 45% 4% 51% 100% 62% 5% 33% 100% 27% 2% 70%TR41 Bursa 100% 44% 5% 51% 100% 64% 7% 29% 100% 24% 3% 73%TR42 Kocaeli 100% 46% 7% 47% 100% 66% 9% 25% 100% 25% 5% 70%TR51 Ankara 100% 42% 6% 52% 100% 62% 7% 31% 100% 22% 4% 73%TR52 Konya 100% 49% 5% 47% 100% 70% 6% 23% 100% 28% 3% 69%TR61 Antalya 100% 51% 6% 43% 100% 65% 7% 28% 100% 36% 5% 59%TR62 Adana 100% 46% 9% 45% 100% 65% 12% 24% 100% 27% 7% 66%TR63 Hatay 100% 43% 7% 51% 100% 58% 10% 32% 100% 26% 4% 70%TR71 Kirikkale 100% 45% 5% 50% 100% 64% 8% 28% 100% 25% 2% 72%TR72 Kayseri 100% 40% 6% 54% 100% 59% 8% 32% 100% 19% 4% 76%TR81 Zonguldak 100% 50% 6% 44% 100% 62% 9% 29% 100% 37% 4% 59%TR82 Kastamonu 100% 54% 5% 40% 100% 68% 5% 27% 100% 41% 5% 53%TR83 Samsun 100% 49% 4% 47% 100% 64% 5% 31% 100% 34% 3% 63%TR90 Trabzon 100% 58% 4% 38% 100% 69% 6% 26% 100% 46% 2% 51%TRA1 Erzurum 100% 49% 4% 47% 100% 64% 5% 30% 100% 34% 1% 65%TRA2 Agri 100% 44% 5% 50% 100% 58% 9% 32% 100% 29% 1% 70%TRB1 Malatya 100% 43% 6% 50% 100% 62% 9% 29% 100% 25% 4% 72%TRB2 Van 100% 36% 7% 57% 100% 55% 12% 32% 100% 15% 3% 82%TRC1 Gaziantep 100% 41% 6% 53% 100% 65% 10% 25% 100% 17% 2% 82%TRC2 Sanliurfa 100% 31% 5% 65% 100% 51% 9% 40% 100% 10% 1% 89%TRC3 Mardin 100% 33% 4% 62% 100% 56% 8% 36% 100% 10% 1% 89%

Source: Own calculations based upon LFS, TÜİK/TURKSTATAbbreviations: POP – Population; EMP – Employed; UEMP – Unemployed; NILF – Not in labour force

79

Table A1.9 Share of population in activity, 2010, Age 15 to 24





80






81






82

Table A1.12 Growth in population by activity, 2007 to 2010, Total



CODE GEO POP EMP UEMP NILF POP EMP UEMP NILF POP EMP UEMP NILFTR Türkiye 5% 9% 28% 0% 6% 5% 21% 4% 5% 20% 45% -1%TR10 Istanbul 6% 3% 47% 6% 7% 0% 45% 14% 6% 13% 53% 3%TR21 Tekirdag 5% 12% 45% -7% 2% 9% 41% -19% 7% 20% 53% -1%TR22 Balikesir 4% -4% 34% 10% 5% -4% 67% 19% 3% -3% -6% 7%TR31 Izmir 6% 9% 64% -2% 7% 3% 45% 7% 6% 26% 96% -6%TR32 Aydin 5% 15% 43% -9% 5% 10% 26% -11% 6% 27% 79% -8%TR33 Manisa 4% 5% 13% 2% 6% 0% 0% 20% 1% 20% 57% -5%TR41 Bursa 8% 0% 27% 13% 8% 0% 19% 28% 7% 1% 48% 8%TR42 Kocaeli 7% 18% 35% -4% 7% 11% 26% -6% 8% 46% 53% -3%TR51 Ankara 7% 8% 14% 5% 7% 4% 15% 12% 7% 22% 9% 3%TR52 Konya 4% 19% -9% -8% 4% 6% -8% 0% 3% 69% -8% -10%TR61 Antalya 10% 6% 69% 10% 11% 4% 46% 25% 9% 11% 114% 3%TR62 Adana 4% 15% 23% -7% 4% 6% 22% -9% 4% 41% 25% -7%TR63 Hatay 7% 19% 32% -4% 8% 13% 20% -2% 5% 38% 84% -6%TR71 Kirikkale 2% 11% 19% -6% 2% 6% 15% -7% 2% 26% 33% -5%TR72 Kayseri 4% 10% 32% -3% 5% 3% 16% 6% 3% 46% 74% -6%TR81 Zonguldak 3% 6% 39% -4% 5% 3% 23% 3% 1% 11% 100% -8%TR82 Kastamonu 2% 10% 117% -12% 3% 2% 63% -1% 1% 28% 225% -18%TR83 Samsun 2% -5% -24% 14% 3% -6% -26% 35% 1% -5% -19% 6%TR90 Trabzon 2% 3% -6% 3% 3% 5% -2% -3% 2% -1% -14% 7%TRA1 Erzurum 1% 18% 33% -13% 2% 6% 19% -8% 1% 54% 100% -15%TRA2 Agri 0% 4% 89% -8% -1% 13% 100% -27% 1% -12% 50% 7%TRB1 Malatya 3% 19% 3% -8% 2% 15% 2% -17% 4% 28% 19% -3%TRB2 Van 6% 12% 63% -2% 5% 15% 54% -17% 7% 2% 150% 6%TRC1 Gaziantep 8% 33% -17% -3% 9% 24% -23% -7% 7% 79% 44% -2%TRC2 Sanliurfa 8% 26% 20% 1% 9% 18% 21% -3% 8% 98% 0% 3%TRC3 Mardin 6% 47% -22% -6% 4% 31% -22% -16% 8% 338% -20% -1%


83

Table A1.13 Growth in population by activity, 2007 to 2010, Age 15-24



CODE GEO POP EMP UEMP NILF POP EMP UEMP NILF POP EMP UEMP NILFTR Türkiye 1% -1% 10% 1% 1% -3% 6% -37% 1% 4% 19% -1%TR10 Istanbul 1% -9% 32% 2% 0% -13% 24% -53% 1% -1% 45% -2%TR21 Tekirdag -5% -4% 18% -8% -11% 4% 63% -46% 3% -16% -22% 15%TR22 Balikesir 2% -14% 0% 10% 6% -6% 17% -6% -2% -24% -17% 8%TR31 Izmir 4% -11% 49% 6% 6% -18% 38% -42% 1% 6% 57% -7%TR32 Aydin -1% 1% 35% -7% -3% -5% 22% -50% 1% 10% 67% -8%TR33 Manisa 2% -19% 12% 12% 9% -23% -5% -15% -5% -12% 100% -6%TR41 Bursa 5% -18% -3% 20% 7% -16% -15% -34% 3% -20% 25% 11%TR42 Kocaeli 1% 16% 20% -8% -1% 13% 3% -46% 3% 26% 44% -6%TR51 Ankara 0% -9% -14% 5% 0% -15% -13% -33% 0% 9% -19% 1%TR52 Konya -3% 10% -23% -7% -5% -3% -11% -56% -2% 52% -31% -9%TR61 Antalya 9% -15% 57% 21% 12% -21% 57% -21% 5% -7% 56% 6%TR62 Adana -2% 15% 3% -11% -4% 19% 8% -55% 0% 7% -3% -2%TR63 Hatay 2% 21% 18% -5% 5% 16% 8% -27% 0% 33% 33% -8%TR71 Kirikkale -4% 0% -20% -3% -6% -4% -21% -41% -2% 12% 0% -4%TR72 Kayseri 0% -7% -6% 3% 2% -10% -10% -22% -2% 0% 0% -2%TR81 Zonguldak -1% -20% 13% 7% 3% -17% 0% -29% -6% -24% 50% -3%TR82 Kastamonu -5% 7% 60% -16% -4% -4% 0% -45% -5% 24% 400% -28%TR83 Samsun -1% -10% -41% 8% 0% -14% -45% -27% -3% -2% -38% 0%TR90 Trabzon -3% -20% -26% 9% -4% -14% -14% -34% -1% -26% -46% 16%TRA1 Erzurum 1% 13% 50% -5% 1% 0% 40% -13% 0% 38% 100% -8%TRA2 Agri -2% 12% 83% -9% -6% 34% 100% -2% 1% -24% 100% 7%TRB1 Malatya -4% 38% -8% -12% -7% 24% -6% -36% -1% 69% 0% -8%TRB2 Van 5% 14% 65% -1% 2% 23% 59% -18% 9% -10% 133% 9%TRC1 Gaziantep 4% 12% -23% 4% 5% 10% -27% -45% 3% 19% 0% 1%TRC2 Sanliurfa 7% 57% 4% -1% 7% 42% 9% -25% 8% 125% -33% 2%TRC3 Mardin 5% 102% -10% -7% 0% 68% -6% -25% 11% 360% -33% 2%


84

Table A1.14 Growth in population by activity, 2007 to 2010, Age 25 to 54



CODE GEO POP EMP UEMP NILF POP EMP UEMP NILF POP EMP UEMP NILFTR Türkiye 5% 10% 37% -4% 6% 5% 27% -86% 5% 24% 67% -4%TR10 Istanbul 7% 5% 53% 5% 7% 2% 50% -85% 7% 18% 60% 2%TR21 Tekirdag 5% 12% 64% -13% 4% 7% 29% -89% 5% 29% 100% -11%TR22 Balikesir 2% -2% 57% 3% 2% -3% 110% -88% 2% 2% 10% 1%TR31 Izmir 5% 11% 68% -11% 5% 5% 42% -87% 5% 29% 138% -11%TR32 Aydin 5% 15% 41% -15% 5% 9% 28% -87% 5% 30% 86% -13%TR33 Manisa 1% 7% 9% -7% 2% 1% -3% -86% 1% 27% 30% -9%TR41 Bursa 6% 4% 47% 6% 7% 3% 34% -88% 6% 7% 79% 4%TR42 Kocaeli 7% 15% 46% -6% 8% 7% 39% -87% 7% 48% 65% -6%TR51 Ankara 7% 10% 30% 2% 7% 6% 25% -86% 7% 23% 39% 0%TR52 Konya 4% 18% -2% -11% 5% 5% -3% -88% 3% 68% 40% -14%TR61 Antalya 8% 9% 69% 1% 9% 8% 34% -86% 8% 13% 146% -1%TR62 Adana 4% 13% 36% -12% 4% 3% 28% -89% 4% 49% 49% -13%TR63 Hatay 6% 17% 43% -8% 7% 10% 25% -79% 5% 37% 109% -8%TR71 Kirikkale 2% 13% 50% -13% 3% 7% 53% -87% 1% 32% 60% -10%TR72 Kayseri 3% 11% 53% -9% 4% 2% 26% -86% 2% 59% 214% -12%TR81 Zonguldak 1% 9% 65% -16% 2% 3% 43% -87% 0% 22% 100% -15%TR82 Kastamonu 1% 13% 167% -25% 2% 5% 100% -89% 0% 35% 250% -24%TR83 Samsun 0% -3% -12% 9% 1% -3% -13% -87% 0% -2% -6% 2%TR90 Trabzon 1% 3% 19% -5% 2% 4% 11% -91% 0% 2% 33% -3%TRA1 Erzurum 0% 18% 8% -24% 1% 7% 10% -93% -1% 53% 100% -20%TRA2 Agri 0% 6% 83% -14% 1% 12% 91% -81% -1% -5% 0% 2%TRB1 Malatya 4% 15% 8% -9% 4% 11% 12% -86% 3% 23% 10% -4%TRB2 Van 5% 14% 61% -8% 6% 16% 57% -82% 4% 5% 250% 2%TRC1 Gaziantep 9% 40% -14% -11% 9% 29% -23% -88% 8% 122% 167% -5%TRC2 Sanliurfa 8% 18% 35% 1% 9% 12% 29% -71% 8% 85% 50% 3%TRC3 Mardin 5% 35% -29% -7% 5% 23% -28% -80% 5% 300% 100% -4%


85

Table A1.15 Growth in population by activity, 2007 to 2010, Age 55 to 64

MALE AND FEMALE MALE FEMALE LABOUR FORCE LABOUR FORCE LABOUR FORCE CODE GEO POP EMP UEMP NILF POP EMP UEMP NILF POP EMP UEMP NILFTR Türkiye 19% 24% 72% 16% 19% 21% 74% -30% 18% 31% 75% 15%TR10 Istanbul 17% 14% 122% 16% 17% 10% 138% -14% 17% 45% 0% 16%TR21 Tekirdag 24% 42% 50% 15% 24% 37% 50% -35% 23% 44% - 16%TR22 Balikesir 16% -7% 0% 30% 16% -10% 0% -35% 15% 0% -100% 22%TR31 Izmir 18% 33% 150% 13% 18% 26% 167% -25% 19% 80% -100% 16%TR32 Aydin 20% 51% 400% 7% 22% 51% 33% -35% 19% 60% - 12%TR33 Manisa 18% 38% - 9% 19% 38% - -35% 17% 42% - 12%TR41 Bursa 19% 5% 0% 23% 20% 4% - -8% 18% 0% -100% 21%TR42 Kocaeli 21% 73% 25% 8% 21% 63% 33% -29% 21% 100% - 14%TR51 Ankara 22% 33% 133% 18% 22% 24% 200% -17% 21% 86% -100% 19%TR52 Konya 18% 59% 33% 2% 19% 42% -33% -46% 17% 138% -100% 6%TR61 Antalya 25% 29% 200% 20% 25% 19% 150% -34% 24% 43% - 17%TR62 Adana 22% 41% 29% 16% 24% 24% 50% -25% 21% 100% 0% 12%TR63 Hatay 23% 46% 0% 14% 24% 30% 33% -42% 22% 57% - 13%TR71 Kirikkale 15% 16% 100% 13% 17% 19% -50% -33% 13% 9% - 14%TR72 Kayseri 16% 55% 300% 4% 19% 46% 200% -25% 14% 86% -100% 9%TR81 Zonguldak 20% 23% 0% 18% 20% 33% 0% -46% 19% 11% - 21%TR82 Kastamonu 17% -3% 100% 31% 18% -10% - -40% 16% 8% 0% 20%TR83 Samsun 15% -12% -67% 42% 16% -8% -67% -40% 15% -20% - 35%TR90 Trabzon 20% 29% -100% 12% 21% 42% -50% -55% 19% 13% - 24%TRA1 Erzurum 12% 25% - 0% 13% 6% 0% -48% 11% 100% - -11%TRA2 Agri 10% -25% - 41% 11% -17% - -49% 8% -40% - 33%TRB1 Malatya 19% 29% 100% 13% 21% 40% -33% -42% 18% 14% - 15%TRB2 Van 18% -14% 100% 40% 17% -22% -100% -48% 20% 20% 0% 20%TRC1 Gaziantep 16% 30% 0% 13% 18% 17% 50% -28% 14% 100% -100% 10%TRC2 Sanliurfa 14% 26% -67% 13% 18% 19% 0% -39% 10% 67% - 7%TRC3 Mardin 15% 36% 0% 9% 18% 31% -100% -41% 12% #DIV/0! -100% 10%


86

Table A1.16 Employment rates by age group 2010, Male and Female

2010 MALE FEMALECODE GEO 15-24 25-54 55-64 TOTAL 15-24 25-54 55-64 TOTALTR Türkiye 37% 78% 39% 63% 19% 25% 14% 22%TR10 Istanbul 46% 81% 25% 67% 23% 20% 3% 19%TR21 Tekirdag 37% 82% 44% 67% 29% 32% 15% 29%TR22 Balikesir 35% 86% 51% 69% 29% 37% 24% 33%TR31 Izmir 44% 78% 30% 64% 22% 28% 6% 24%TR32 Aydin 45% 77% 39% 65% 26% 33% 14% 29%TR33 Manisa 39% 81% 41% 65% 21% 25% 15% 23%TR41 Bursa 46% 84% 35% 69% 25% 28% 9% 25%TR42 Kocaeli 36% 80% 32% 64% 18% 20% 9% 18%TR51 Ankara 36% 81% 27% 64% 11% 25% 4% 19%TR52 Konya 47% 83% 45% 69% 14% 20% 10% 17%TR61 Antalya 46% 82% 49% 70% 29% 39% 23% 35%TR62 Adana 35% 79% 38% 63% 19% 22% 10% 20%TR63 Hatay 30% 70% 46% 56% 14% 23% 17% 20%TR71 Kirikkale 37% 77% 41% 62% 13% 24% 19% 20%TR72 Kayseri 32% 79% 33% 61% 13% 15% 8% 14%TR81 Zonguldak 33% 78% 48% 63% 30% 33% 42% 34%TR82 Kastamonu 43% 82% 57% 68% 29% 34% 33% 33%TR83 Samsun 39% 84% 59% 69% 27% 39% 37% 36%TR90 Trabzon 30% 84% 57% 67% 31% 55% 51% 48%TRA1 Erzurum 28% 83% 52% 62% 16% 26% 20% 22%TRA2 Agri 23% 67% 69% 51% 20% 42% 34% 33%TRB1 Malatya 23% 73% 42% 55% 9% 24% 26% 20%TRB2 Van 27% 65% 73% 51% 10% 20% 14% 16%TRC1 Gaziantep 40% 67% 42% 57% 12% 9% 7% 10%TRC2 Sanliurfa 24% 63% 40% 47% 5% 6% 5% 5%TRC3 Mardin 18% 63% 44% 44% 3% 3% 0% 3%


87

Table A1.17 Growth in employment rates by age group 2007 to 2010, Male and Female (percentage points)MALE FEMALE

CODE GEO 15-24 25-54 55-64 TOTAL 15-24 25-54 55-64 TOTALTR Türkiye -1.6% -0.2% 0.6% -0.5% 0.6% 4.7% 1.6% 3.2%TR10 Istanbul -5.9% -4.4% -1.4% -4.4% -0.4% 2.1% 0.7% 1.2%TR21 Tekirdag 6.5% 2.0% 4.6% 4.0% -5.4% 7.2% 2.5% 3.6%TR22 Balikesir -4.0% -3.6% -11.6% -5.6% -6.5% -0.1% -3.1% -2.0%TR31 Izmir -10.0% 0.2% 2.0% -2.5% 1.2% 6.4% 3.2% 4.6%TR32 Aydin -1.1% 3.2% 9.5% 2.9% 2.2% 7.9% 4.7% 6.0%TR33 Manisa -11.2% -0.6% 6.6% -3.5% -1.5% 6.7% 3.4% 4.2%TR41 Bursa -9.9% -2.9% -4.4% -5.2% -5.8% 0.3% -1.4% -1.5%TR42 Kocaeli 4.9% -0.5% 11.0% 2.2% 4.0% 7.7% 5.8% 6.5%TR51 Ankara -5.2% -1.2% 0.4% -2.0% 1.0% 3.7% 2.3% 2.8%TR52 Konya 0.7% 0.2% 9.1% 1.6% 7.4% 12.5% 10.2% 10.8%TR61 Antalya -13.7% -0.9% -2.7% -4.8% -3.3% 1.8% 3.5% 0.7%TR62 Adana 8.4% -1.1% 0.2% 1.5% 1.3% 9.5% 6.8% 7.0%TR63 Hatay 3.2% 2.0% 2.1% 2.3% 4.8% 7.1% 4.8% 6.2%TR71 Kirikkale 0.6% 2.3% 0.8% 2.0% 1.8% 7.4% -0.7% 4.9%TR72 Kayseri -3.7% -1.5% 7.7% -1.3% 0.2% 8.3% 5.1% 5.7%TR81 Zonguldak -6.7% 0.6% 5.4% -0.9% -5.6% 7.5% -3.1% 3.4%TR82 Kastamonu 0.1% 2.2% -13.8% -0.9% 9.0% 11.9% -2.3% 8.7%TR83 Samsun -5.5% -3.7% -11.9% -5.5% 0.3% -0.8% -11.2% -2.0%TR90 Trabzon -3.2% 1.4% 9.9% 1.8% -7.9% 0.8% -2.2% -1.6%TRA1 Erzurum -0.2% 4.8% -3.2% 2.3% 5.9% 14.1% 15.9% 11.8%TRA2 Agri 9.8% 7.4% -17.3% 7.0% -4.9% -1.8% -15.0% -4.3%TRB1 Malatya 7.6% 5.1% 6.8% 6.8% 6.2% 4.8% -0.9% 4.7%TRB2 Van 5.5% 6.5% -24.2% 4.7% -1.8% 0.2% 0.0% -0.7%TRC1 Gaziantep 1.8% 12.3% -0.4% 8.2% 1.9% 9.6% 4.9% 6.7%TRC2 Sanliurfa 7.8% 1.8% 0.5% 3.9% 5.7% 4.0% 2.5% 4.5%TRC3 Mardin 12.4% 10.5% 4.8% 11.4% 8.6% 7.9% 4.8% 7.9%


88

Table A1.18 Share of Population aged 15 and over by educational attainment, 2010, Male and Female

CODE GEO Illiterate


Junior high school or vocational school at junior high school level graduate

High school or vocational school at high school level graduate

Higher education graduate and more Unknown TOTAL

TR10 Istanbul 3.7% 4.3% 6.7% 23.4% 11.8% 6.3% 100.0%TR21 Tekirdag 4.4% 4.3% 6.5% 22.6% 8.5% 3.4% 100.0%TR22 Balikesir 5.7% 5.1% 5.3% 19.4% 8.9% 3.7% 100.0%TR31 Izmir 4.3% 4.1% 6.4% 23.2% 12.1% 4.7% 100.0%TR32 Aydin 5.9% 4.5% 5.1% 18.9% 9.1% 4.4% 100.0%TR33 Manisa 7.4% 5.2% 4.9% 17.9% 6.8% 2.5% 100.0%TR41 Bursa 4.3% 3.8% 7.2% 23.9% 9.7% 4.0% 100.0%TR42 Kocaeli 5.0% 4.3% 6.3% 22.7% 8.8% 4.4% 100.0%TR51 Ankara 3.5% 2.9% 7.2% 27.4% 17.0% 4.7% 100.0%TR52 Konya 6.4% 3.9% 4.8% 17.2% 7.8% 3.0% 100.0%TR61 Antalya 3.6% 5.2% 5.5% 21.9% 10.2% 4.8% 100.0%TR62 Adana 7.2% 5.8% 6.0% 21.8% 8.8% 4.5% 100.0%TR63 Hatay 9.1% 6.3% 5.4% 18.6% 6.5% 4.7% 100.0%TR71 Kirikkale 8.1% 4.9% 6.1% 19.3% 7.1% 3.7% 100.0%TR72 Kayseri 7.6% 4.9% 5.5% 20.7% 7.8% 6.2% 100.0%TR81 Zonguldak 8.5% 5.6% 5.3% 20.0% 7.4% 3.0% 100.0%TR82 Kastamonu 10.9% 7.1% 4.9% 16.3% 6.5% 3.0% 100.0%TR83 Samsun 9.0% 6.3% 4.7% 17.5% 7.2% 4.4% 100.0%TR90 Trabzon 9.9% 6.2% 5.7% 21.0% 7.8% 4.6% 100.0%TRA1 Erzurum 10.3% 7.6% 4.9% 20.2% 7.5% 5.6% 100.0%TRA2 Agri 13.7% 13.6% 3.4% 13.5% 4.2% 10.7% 100.0%TRB1 Malatya 11.1% 7.2% 5.7% 21.3% 7.8% 5.3% 100.0%TRB2 Van 16.2% 16.2% 2.8% 13.7% 4.0% 6.9% 100.0%TRC1 Gaziantep 10.3% 7.9% 4.8% 16.0% 5.6% 5.4% 100.0%TRC2 Sanliurfa 16.7% 16.4% 2.8% 13.2% 4.3% 7.4% 100.0%TRC3 Mardin 17.1% 13.6% 3.0% 15.0% 4.2% 6.0% 100.0%TR TURKEY 7.0% 5.9% 5.7% 20.8% 9.2% 5.0% 100.0%

Source: Own calculation based upon statistics by TÜİK/TURKSTAT

89

Table A1.19 Share of Population aged 15 and over by educational attainment, 2010, Male

CODE GEO Illiterate




Higher education graduate Unknown TOTAL



90

Table A1.20 Share of Population aged 15 and over by educational attainment, 2010, Female

CODE GEO Illiterate




Higher education graduate Unknown TOTAL



91

Annex 2. Major developments since 2007

1. Major Developments since 2007 at National LevelThe HRD-OP (2007-2009) aims to address the main challenges Turkey faces in the fields of employment, education and training, and social inclusion. The Programme has been prepared in compliance with Turkey's 9th National Development Plan (NDP), Turkey's Multi-annual Indicative Planning Document (MIPD) and the Strategic Coherence Framework (SCF). The priorities and measures set out in HRD OP reflect the key problems and challenges identified in the NDP, the MIPD, the SCF, and the joint strategy papers. The SCF, the JIM and the JAP have been jointly elaborated by Turkey and the European Commission. The major developments since HRD-OP (2007-2009) produced are summarised below.

Implementation of Partnership Approach

We observed that during the implementation of the HRD-OP strong collaborations have established with relevant public institutions, non-governmental organisations, social partners, and universities during different phases of implementations of HRD-OP, although some members of organisations did not fount the collaboration sufficient. This collaborative approach helped to improve HRD-OP mechanism, enabled stakeholders share their experience and knowledge, built up opportunities for more creative ways to address problems and allowed a more effective management.

Sectoral Monitoring Committee

The Sectoral Monitoring Committee, a permanently acting body of HRD-OP established under the provisions of Article 36 of the Framework Agreement between the government of the Republic of Turkey and the European Commission, had its first meeting on 13 December 2007 and hence after it held meetings twice a year at the initiative of the Operating Structure or the Commission. These meetings contributed greatly to the development and implementation of HRD-OP. The dates of the meetings are presented in Table 3.

Table 3: The date of Sectoral Committee Meetings

Meetings Date Location

1st Sectoral Committee Meeting 13.12.2007 Ankara

2nd Sectoral Committee Meeting 17.06.2008 Ankara

3rd Sectoral Committee Meeting 13.11.2008 Ankara

4th Sectoral Committee Meeting 29.07.2009 Ankara

5th Sectoral Committee Meeting 23.11.2009 Ankara

6th Sectoral Committee Meeting 24.06.2010 Van

7th Sectoral Committee Meeting 30.11.2010 Hatay

8th Sectoral Committee Meeting 07.07.2011 Trabzon

Services Provided by Different Institutions Related to HRD-OP Target Groups

Several activities in different institutions have been carried out in order to improve human resources of the particular groups related to the priority axis of HRD-OP. The Ministry of Labour and Social Security has been working on the preparation of a national employment strategy. One of the main aims of this strategy is to determine measures that will increase employment of disadvantaged developed during a series of workshops with the participation of over 40 relevant bodies and stakeholder s. The Ministry has also prepared an action plan to strengthen relationships between vocational education and employment of these groups prioritised in HRD-OP.

92

IŞKUR, with UMEM Project has determined the demand of the workforce market through need analyses in each province. The vocational and technical teachers of 100 vocational high schools have been trained. Students and non-students could both attend these trainings.

The General Directorate of Technical Education for Boys and the General Directorate of Technical Education for Girls provided various services or contributed to different projects at institutional level to improve education and employment of young people. During 2009, in accordance with the request of the Ministry of Labour and Social Security, the General Directorate of Technical Education for Boys worked with the Ministry of National Education and Ministry of Industry and Commerce in order to discuss the measures needed to overcome the supply and demand discrepancy between technical education and workforce demand. TOBB also contributed to this study. The General Directorate of Technical Education for Boys also contributed to the Beceri’10 Project of Specialized Career Planning Centers (UMEM) which is primarily focused on training and employment. The Turkish Union of Chambers and Stock Markets (TOBB), the Ministry of Labour and Social Security, the Ministry of National Education and the TOBB University of Economics and Technology (TOBB-ETÜ) have signed a protocol in order to implement the pilot project in 19 provinces. The executive board of the project consists of Deputy Secretary of Ministry of Labour and Social Security, General Manager of İŞKUR, Director of General Directorate of Technical Education for Boys, General Secretary of TOBB and TOBB-ETÜ President. On-the-job trainings for teachers were opened under UMEM. 2,000 teachers already attended these courses and 4,000 more will be trained.

The Ministry of Development (formerly SPO) has been implementing SODES (Social Support Program) in Eastern and Southeastern Anatolia. This program supports vocational trainings, sports activities and various courses for youth, fragile, disadvantaged groups. This is one of the most comprehensive and successful projects on social inclusion in Turkey. It encourages the development of projects created by local initiatives and NGO partnerships.

Collaborations amongst Related Institutions

Several institutions collaborate at certain level with each other in order to improve employment, education, and social inclusion. The Ministry of Labour and Social Security implements their action plan with the coordination of the Ministry of Industry and Commerce and the Ministry of National Education. They hold annual meetings at the end of each year with the participation of the three ministers in order to secure political support at the top management level.

İŞKUR have collaborations with various public and private bodies such as the Ministry of National Education, KOSGEB, Turkcell, Turk Telecom, General Directorate of Social Support and Solidarity and Ministry of Forests and Environment.

The General Directorate of Technical Education for Boys has been collaborating with different institutions for different projects setting. It has close collaboration with the Ministry of Labour and Social Security, Ministry of National Education, Turkish Union of Chambers and Stock Markets (TOBB), and TOBB University of Economics and Technology (TOBB-ETÜ) for UMEM Project, and with the Ministry of National Education and Ankara Chamber of Industry (ASO) for OSEP –School Industry Education Program.

The General Directorate of Commerce and Tourism Education have intensive collaboration with the Ministry of Culture and Tourism. The Directorate also collaborates with İŞKUR within UMEM project, and collaborates with TÜRSAB in order to meet the qualified workforce demand of the sector.

The General Directorate of Technical Education for Girls also collaborates with other public, private and NGO partners in order to strengthen the relationship between vocational education and employment, to contribute to the development of vocational and technical education, to solve the problems in education system and to create awareness.

93

The Ministry of Development has partnerships with public institutions, youth and other NGOs. It also provides guidance to other public institutions.

Evaluation of Institutional Collaborations

Although institutions collaborate with each other, we noticed that this is usually based on project-based partnership. If they are not conducting the same project, they are not usually having sufficient information on the services they provide. This has naturally a negative effect on the creation of an integrated policy with common targets. At the same time it reduces the quality and effectiveness of measures taken by individual Institutions. We believe that common plans in a strategic framework would have much enhanced the application of such policies and would have contributed to them having a lasting effect in the Turkish society rather than “project-duration” effect. Also, such a deeper and more permanent collaboration would have helped to direct better the efforts spent and balance the distribution of projects according to the whole spectrum of needs rather than randomly covering specific areas that happened at a certain time to arise.

Also we noticed that Institutions are not always able to keep consistent, detailed records of their services and information relevant to beneficiaries and all their stakeholders. There is also lack of a well working database system that will allow different bodies to access the relevant information. Therefore there is lack of effective and continuous communication and this result to often having projects that duplicate previously run activities.

Development in Higher Education The Turkish Higher Education system has seen a big increase both in numbers of Universities and consequently in numbers of students. Since 2007, 50 new Universities have been established bringing their total University number from 115 to 165. From these, 30 are Foundation (private) Universities and 18 are State Universities. This makes for a large and imbalanced increase in favour of the Private sector: The number of students enrolled at Higher Education establishments has also increased over 30% from 2007 to 2011. Comparing the actual numbers, 1.075.670 more students were registered in 2011 than in 2007 (Özcan, 2011)26. Most of the State Universities are established in the NUTS-2 region which has a high proportion of people belonging to disadvantaged groups. We believe that this will increase the opportunities for members of such groups to access higher education.

Removing Coefficient System to Promote of Vocational and Technical Secondary Schools’ Students to Universities

A change, in effect from 2012, will be implemented in order to equalise the opportunities of the students of Vocational and Technical High Schools to enter Universities. This was to abolish the coefficient system which was applied to University entrance exam scores and lowered the marks of the students from such High Schools. This system was in effect since 1998. The chance is expected to increase the enrolment to Vocational and Technical High Schools. Nevertheless, more measures need to be taken to also increase the quality of education in these schools.

Vocational Qualification Authority (VQA)

Under the coordination of the Vocational Qualifications Authority (VQA), centres for sectoral occupational standards development, skill and knowledge testing and certification’ were established and started to develop occupational standards and infrastructure for knowledge and skills tests in cooperation with relevant NGOs and private-sector organisations. Turkey needs to continue its efforts to establish a national qualifications system applying to all types of education, training and qualifications and linked to the European Qualification Framework.

26 Reference: Özcan, Y. Z. (2011). Challenges to the Turkish Higher Education System. Paper presented in the 22nd International Conference on Higher Education. Bilkent University, Ankara (June 17-19, 2011).

94

During the implementation of grant scheme projects, a variety of vocational training activities were conducted and some of them are still being conducted. However, at present, there is no standardization of these trainings. In order to increase the quality of the project activities, the use and compliance of these standards should be mandatory for the vocational trainings to be provided under OP projects.

Major HRD-Related Projects Implemented since 2007

The Major HRD-related projects implemented since 2007 are outlined below.

Table 1: Major HRD Related Projects

Project Name Objectives Starting Date Beneficiaries

Promoting Women's Employment Grant Scheme

To increase women employment, to help women to enter more jobs and better jobs, and to reduce barriers preventing women employment

30 July 2010 İŞKUR

Promoting Youth Employment Grant Scheme

To increase youth employment, to organize entrepreneurship training at local level and deliver counseling and guidance services for establishing their own jobs, to increase the opportunities of internship in order students to transit from schools to jobs easily, to increase vocational qualifications of youth people, and increase the number of people who benefitted from apprenticeship programme.

23 Nov. 2010 İŞKUR

Increasing School Enrolment Rates Especially for Girls Grant Scheme

To enhance investment in human capital by increasing the quality of education, improving the linkage between education and the labour market, and raising enrolment rates at all levels of education, especially for girls within the scope of positive discrimination for women in the EU adaptation process.

Jan. 2011 MoNE

Promoting of Life Long Learning Grant Scheme

To transfer dynamic labour force, employment, continuous learning and culture of learning and qualitative education access compliant with different age groups, improved technology and the demands of labour force into a lifelong learning strategy for adaptation in global changes of today's world in which social and economic power depends on knowledge.

15 May 2010 MoNE

Promoting Registered Employment through Innovative Measures Grant Scheme

To build institutional capacity of the Social Security Institution, other relevant institutions, social partners and relevant NGOs in order to promote registered employment and to support those mentioned authorities to prepare and implement their strategies in related areas.

15 July 2010 SSI

95

Major Outcomes and Evaluation of the HRD-OP Projects

An immense number of institutions have benefitted from grant projects of HRD-OP. Among them trade associations, chambers, federations-confederations, foundations, unions, NGO’s, municipalities, universities, vocational secondary schools, public training centres, primary schools, industrial zones and farmer groups.

The data indicate that 44.792 people benefitted or will benefit from the various training activities. The number of trainees is 13.002 in “Promoting Women's Employment”, 11.490 in “Promoting Youth Employment”, 8238 in “Increasing School Enrolment Rates Especially for Girls”, 7811 in “Promoting Registered Employment through Innovative Measures”, and 4251 in “Promoting of Life Long Learning”.

While the data were collected from MIS, training activities of the some projects were still going on. At the data collection time, the percentage of trainees who completed their training was 41% in “Promoting Registered Employment through Innovative Measures”, 36% in “Promoting Women's Employment”, 28% in “Promoting of Life Long Learning”, 26% in “Promoting Youth Employment”, and 10% in “Increasing School Enrolment Rates Especially For Girls”. The ratio of unsuccessful trainees varies from 2.2% (Promoting Registered Employment through Innovative Measures) to 4.9% (Promoting Youth Employment). The unsuccessful students failed or simply left the training.

The trainings had all very high rates of success. The percentages of successful trainees varied from 98% (Promoting Registered Employment through Innovative Measures) to 88% (Increasing School Enrolment Rates Especially for Girls). About half of the unsuccessful trainees failed because of absentees.

Most of the training topics in these projects are categorised as “general/occupational skills”,” child and/or elderly care training”, “entrepreneurship training” (Promoting Women's Employment), “informative meetings or training for families or parents on promotion of girls’ education”, “informative meetings or training for social partners on promotion of girls’ education”, “basic skills (reading, writing etc.)” and “general occupational skills” (Increasing School Enrolment Rates Especially For Girls), “general occupational skills”, “employment guarantee vocational training”, and “entrepreneurship training” (Promoting Youth Employment), “general occupational skills”, “employment guarantee vocational training”, “general skill training”, and “basic skills” (Promoting of Life Long Learning), and above 90% of the trainees trained on general occupational skills, “informatory meeting or training for social partners to foster registered employment” and “general skill training” (Promoting Registered Employment through Innovative Measures).

The gender distribution of the trainees naturally varied according to the projects. For example, the percentage of females is 100% in “Promoting Women's Employment”, it is 86% in “ Increasing School Enrolment Rates Especially For Girls” and slightly less than 50% in “Promoting of Life Long Learning”. However, the percentage of females is 38.9% in “Promoting Youth Employment” and 37% in “Promoting Registered Employment through Innovative Measures”.

Most of the trainees are either primary or secondary school graduates. The percentages of trainees graduated from primary and secondary schools vary from 65% (Promoting Youth Employment and Promoting of Life Long Learning) to 85% (Promoting Women's Employment).

The age of the trainees vary from one grant project to another. For example, the youth and adult were nearly evenly represented both in “Promoting Women's Employment” and “Promoting Registered Employment through Innovative Measures”. However, only 1/3 of the trainees were youth in “Promoting of Life Long Learning”, nearly 80% of them were youth in “Increasing School Enrolment Rates Especially for Girls” and about 99% in “Promoting Youth Employment”.

96

http://www.ikg.gov.tr/openform/Projeler.aspx?ProgramKOD=TR08H2.01-2


http://www.ikg.gov.tr/openform/Projeler.aspx?ProgramKOD=LLL

http://www.ikg.gov.tr/openform/Projeler.aspx?ProgramKOD=PRE














In the case of social exclusion groups, the percentages varied from 1% to 9%. In the “Increasing School Enrolment Rates Especially for Girls” about 9% of the participants were from disadvantaged groups with most of them originating from two groups: 1) “Poor or under risk of poorness” and 2) “Other people in need of special attention (substance addiction, victimised women in family, family/parents of working children)” were the majority.

In the case of “Promoting Youth Employment”, the percentage of the trainees whose handicapped level above 25%, terror victims, Romany people who needs special attention, Ex convicted, Poor or under risk of poorness, including one who live in shanty towns (Gecekondu areas), and other people in need of special attention (substance addiction, victimised women in family, family/parents of working children) was about 5%. When Poor or under risk of poorness were eliminated the percentage of the remaining groups made for less than 1% of the participants. In the remaining projects the percentage of disadvantaged groups was less than 1%.

Since HRD-OP is expected to develop final beneficiaries, the data given above revealed that participation rates in employment, particularly for women and young people have been increased to a certain level, after the training activities conducted through project activities. However, since the project activities are still going on, the rate of increase in enrolment and decrease of dropouts, particularly of girls in secondary/VET education will give a clearer picture later, after completions of the projects.

Nevertheless, the data collected from MIS indicates that the participations of handicapped people, terror victims, Romany people who needs special attention, ex convicted, poor or under risk of poorness and other people in need of special attention (substance addiction, victimized women in family, family/parents of working children were limited.

Other HRD Related Projects

In addition to the above projects, other Projects played a positive role on human resources development in less developed regions. These are outlined below.

Table 2: Other HRD-related projects implemented since 2007

Project Name Objectives Duration Partners

GAP-2 Project To develop human resources and reduce unemployment rate in the provinces of GAP region (Gaziantep, Adıyaman, Diyarbakır, Şırnak, Siirt, Batman, Şanlıurfa, Kilis and Mardin).

2008-2012 İŞKUR

Improving Labour Market Intervention to the Benefit of Employability of Long Term Unemployed

To develop and implement labour market intervention instruments for decreasing long term unemployment.

2008 –2011 İŞKUR, Ministry of Social Services and Employment (Netherland)

Development with Human Caring Job: Youth Employment Programme in Antalya

To promote employability of fragile groups of youth, youth women and youth immigrant families. İŞKUR is expected to develop a national youth employment action plan for Turkey.

2008-2001 İŞKUR, UNDP, ILO, FAO, IOM

Gender Equality by Providing Humanistic Employment Opportunities for Women

Formulating proposals for Strategic Plan and Action Plan for Increasing Women Employment and developing of “Handbook of Gender Equality”.

2009-2010 İŞKUR and ILO

Strategy Development Project for Promoting Women Employment

To strengthen gender equality within labour market and raise awareness on it.

2009-2010 İŞKUR and British Council

Support to Human Resources Development through VET

To improve labour force for the future and contribute to the development of SMEs in the region and their competition by increasing the quality of vocational education in 8 provinces of

2008-2010 MoNE

97

Project Name Objectives Duration Partners

Eastern and South-eastern Anatolia Region.Strengthening the Administrative Capacity of MoNE (MEBGEP)

To evolve an action plan to facilitate improvement of the capacity of MoNE in the fields of administration, management and organization, handling financial resources and monitoring/evaluating in order to make the system more effective and productive in the process of restructuring.

2008-2009 MoNE

Active Employment Measures and Support to Turkish Employment Organization at Local Level

To assist İŞKUR to increase its capacity to design and implement measures in order to boost employment and employability of specific target groups of women and young people.

2007-2009 İŞKUR

Support Women Entrepreneurship Through Training Centres and Relays in Turkey

To address the low participation of women in Turkey’s economic life, to promote entrepreneurship among them and enhance their role in society. The Project aimed to establish 9 training centres and 16 training relays in different parts of Turkey as well as train and provide consultancy to more than 5000 women.

2007-2009 TESKKOSGEB

Yearly Major Developments in the Implementations of the HRD-OP

Major developments in the implementation of the HRD-OP in the years of 2007, 2008, 2009, and 2010 are summarised separately below.

Major Developments in the Implementation of the HRD-OP in 2007

The MoLSS, as designated Operating Structure (OS), established an HRD Technical Committee for the preparation of the HRD OP. On 1 May 2007, the MoLSS provided the first draft HRD OP to the European Commission. Between May and October 2007, discussions took place between the MoLSS and the Commission in view of finalizing the HRD OP. The drafts were submitted to the Commission for comments.

The following relevant institutions and social partners were consulted throughout HRD-OP preparations: 1) Ministry of Finance, 2) Ministry of National Education, 3) Ministry of Health, 4) Ministry of Transport, 5) Ministry of Food, Agriculture and Livestock, 6) Ministry of Science, Industry and Technology, 7) Ministry of Environment and Urban Affairs, 8) State Planning Organization, 9) Under secretariat of Treasury, 10) Ministry for European Union Affairs, 11) DG on Social Services and Child Protection, 12) DG on Social Assistance and Solidarity, 13) DG on Foundations, 14) DG on Women Status, 15) DG on Family and Social Research, 16) Council of Higher Education, 17) Housing Development Administration, 18) Administration for the Disabled People, 19) Small and Medium Sized Business Development Organization (KOSGEB), 20) National Productivity Centre, 21) Confederation of Turkish Employer Organizations (TISK), 22) Confederation of Turkish Employees (TURK-İŞ), 23) HAK-İŞ Confederation (HAK-İŞ), 24) DISK Confederation (DİSK), 25) KAMUSEN Confederation (KAMUSEN), 26) KESK Confederation (KESK), 27) MEMURSEN Confederation (MEMURSEN), 28) BASK Confederation (BASK), 29) The Union of Chambers and Commodity Exchanges in Turkey (TOBB), 30) Confederation of Artisans and Craftsmen (TESK), 31) 15) TURKSTAT.

In order to ensure ownership of the HRD OP, the 43 Provincial Governorships in the 12 eligible NUTS II regions were consulted.

An ex-ante evaluation on the HRD OP was carried out under the responsibility of the MoLSS. The results of the ex-ante evaluation were taken into account in the drafting process of the HRD OP.

98

The Operation Identification Sheet (OIS) was elaborated.

The following institutions were designated for the implementations of the 2007 allocation of the HRD OP: 1) Measure 1.1: Public Employment Agency, 2) Measure 1.3: Social Security Institution – Guidance and Inspection Board, 3) Measure 3.1: Ministry of National Education, 4) Measure 4.2: Social Security Institution – DG on Non Contributory Payments.

A Technical Committee has been established in collaboration with the Ministry of Science, Industry and Technology to ensure regular dialogue and exchange of information on the interventions of the HRD-OP and the RC-OP. This Committee has met regularly during the programming exercise in 2007 to ensure coordination during the implementation phase of the two OPs.

The management structure was set in place in 2007 in order to prepare for the implementation of the HRD OP, once all the necessary requirements for accreditation and conferral of management were met.

To ensure an efficient and effective implementation of the HRD OP and fulfil the tasks, the Ministry of Labour and Social Security has established the following five units: 1) Programming Unit, 2) Technical Implementation Unit, 3) Monitoring and Evaluation Unit, 4) Quality Assurance and Control Unit, 5) Information, Publicity and Technical Assistance Unit.

The functions and responsibilities of each unit have been defined and the coordinators of each unit have been appointed. Furthermore, job descriptions for each staff were prepared.

To manage and control issues related to risks, a Risk Management Coordinator for the IPA Management Department was assigned, and a Risk Management Committee was established.

The first draft Implementation Manual of the HRD OP was prepared with the external assistance of the “Support to SPO” contract.

Internal Audit Unit Department has been established in MoLSS as the HRD OS.

A training need analysis was conducted, and several internal and external trainings were conducted in order to increase the administrative capacity of the OS.

In accordance with Article 59 of the IPA IR, a Sectoral Monitoring Committee (SMC) for the HRD OP in Turkey was set up in 2007. The committee was composed of 44 members. 24 members were from national public institutions, 7 members from economic and social partners, and 13 members from regional partners.

The first committee meeting for the Human Resources Development Operational Programme was held in Ankara, on 13 December 2007, at the premises of the Secretariat General for European Union Affairs.

Involvement of the civil society in the implementation of the OP was to be ensured by the following activities: 1) The SMC should include members from the social partners and other representatives of the civil society, 2) Involvement and participation of the civil society and local administrations in the 12 NUTS II regions, including provinces and municipalities, should be ensured. To that purpose, technical assistance may be used by the MoLSS as OS to establish coordination mechanisms and advisory forums, 3) NGO’s and Social Partners will be grant scheme beneficiaries on the implementation of the measures.

99

Major Development in the Implementation of the HRD-OP in 2008

Drafting of the procurement and call for proposals documents, namely terms of reference, technical specifications for service and supply tenders, as well as guidelines for grant applicants were prepared.

The Human Resources Development Operational Programme Implementation Manual (HRD OP PIM) was completed.

The following institutions were designated for the implementation of the 2007 allocations of the HRD OP: Supporting Women to Enter Labour Market (İŞKUR), Promoting Registered Employment through Innovative Measures (SSI), Promotion of Life Long Learning (MoNE), Promoting Social Inclusion in Turkey (SSI).

The following institutions were designated for the implementation of the 2008 allocations of the HRD OP: Increasing Young Employment (İŞKUR), Improving the Quality of Public Employment Services (İŞKUR), Increasing Enrolment Rates Especially for Girls in Secondary Education (MoNE).

The preparation of the OIS for each operation was launched. Relevant organizations involved in this process include the Ministry of National Education, the Social Security Institution, the Turkish Employment Agency, the General Directorate on Women Status, the General Directorate on Social Services and Child Protection, the DG on Disabilities, the State Planning Organization, representatives of ECD and other relevant institutions, social partners and NGOs.

The IPA Management Department and İŞKUR carried out a stakeholder analysis on employment operations through several consultation meetings with KOSGEB, Ministry of Science, Industry and Technology, Ministry of Agriculture and Rural Affairs, Ministry of Education and the State Planning Organization in which what kind of activities can be implemented in the operations were discussed. The Ministry of National Education also held a consultation meeting with the local education directorates at provincial level. After stakeholder analysis, for each operation, an expert and a substitute expert were appointed.

The Operation Identification Sheets prepared for 2008 allocations were updated. Complementarity of HRD-OP components linked with other instrument. The second SMC meeting for the HRD OP was held on 17 th June 2008 in its final composition.

This composition was made permanent for the first two years (2008 – 2010). There were 24 members from the National Public Institutions and 7 members from Economic and Social Partners, namely trade unions confederations and employer organizations. There were also 13 members from the 12 NUTSII regions and several NGOs working at the national level in the fields related to the HRD OP. The Committee is co-chaired by the Deputy Undersecretary of the Ministry of Labour and Social Security, as Head of the Operating Structure for the HRD OP or Head of IPA Management Department, and a representative of the European Commission. All committee permanent titular members and their substitutes were nominated by their respective authorities through official letters and they were appointed by the Head of the Operating Structure on 16 June 2008 just before the second SMC meeting.

The third SMC meeting was held on 13 November 2008. At this meeting the operations to be funded using the 2007 and 2008 financial allocations were presented to the committee. The minutes were finalized, taking into account comments from members, and published on the website of the Ministry on 29 December 2008.

In terms of monitoring of grant schemes, the Central Grant Monitoring Teams (CGMT) was to be composed of minimum 10 experts; minimum 5 experts were from the Operation Beneficiary and when requested, the Operating Structure would provide the CGMT a maximum of 5 short term experts assigned under the Technical Assistance Priority.

Regional Monitoring and Technical Assistance Teams were to be established in 12 growth centres.

100

A Framework Contract under Support for European Integration (SEI) was implemented to draft tender documents for the implementation of technical assistance priority axis.

In February 2008 two regional conferences were organized in Malatya and Gaziantep by the SPO under the EU funded project named “Capacity Improvement in the field of Economic and Social Cohesion” for the promotion and consultation of the SCF and OPs for all IPA components during the programming process for the period before the start of the implementation.

Between 11 March and 30 April 2008, the EC Delegation organised 4 regional information and awareness raising conferences solely for the HRD OP in Trabzon, Kastamonu, Diyarbakır and Kayseri.

A dedicated HRD OP web site was designed in Turkish and English. The domain for the website (http://ikg.gov.tr) was received, but was not uploaded to a server by the end of the reporting period.

Major Developments in the Implementation of the HRD-OP in 2009

The operational agreement of “Promoting Women’s Employment” was signed on the 8th October 2009, of “Promoting Youth Employment” on the 21st December 2009, of “Promoting Registered Employment Through Innovative Measures” on the 8th October 2009, of “Improving the Quality of Public Employment Services” on the 21st December 2009, of “Increasing Enrolment Rates Especially for Girls” on the 21st December 2009, of “Promoting Lifelong Learning” on the 8th October 2009.

The implementation of the HRD OP started when the Financing Agreement was signed on 16th September 2009.

In order to ensure the complementarity between the HRD OP and other IPA Components, the priority axes related with the ones of HRD OP were tried to be integrated to the programming period of the Operations.

Subsequently, the fourth and fifth SMC meetings for the HRD OP were held on 29th June 2009 and 23rd November 2009. According to conclusions of the 4th SMC meeting, the establishment and the mandate of the ad-hoc sub-committee for interim evaluation of the HRD OP were approved.

At the fifth SMC meeting held on 23rd October 2009, the establishment of Central Grant Monitoring Teams (CGMTs), Regional Grant Monitoring and Technical Assistance Teams (RGMTTs), Monitoring Information System (MIS) was shared and discussed with the committee members.

In order to monitor the projects under the grant component of the Operations (where applicable), the process for establishing the CGMTs was initiated by the OS in 2009.

For the year 2009, overall risks for the IPA Management Department were summarized as “follow-up the accreditation process, lack of motivation and career opportunities, work overload for the staff, lack of administrative capacity of all relevant parties and conflict between tasks of IPA units”.

The OS presented the Communication Action Plan to the SMC in the Fourth SMC meeting held on 29th June 2009.

In 2009, nearly 5500 potential grant applicants were informed about the operations under the 2007 and 2008 Programming Periods. As regards the 2007 Programming Period, approximately 3000 potential grant applicants participated in 15 Grant Information Days organized in 15 provinces; 13 growth centers and 2 hinterlands between February-March 2009. Likewise, under the 2008 Programming Period, 13 Grant Information Days were organized and approximately 2500 potential grant applicants were informed in July 2009.

RGMTTs were established with the aim of ensuring the publicity of the OP in the local level and supporting the beneficiaries who would conduct projects in the regions. RGMTTs would

101

http://ikg.gov.tr/

also be responsible, with respect to information and publicity, for establishing a database of the potential applicants and organizing training programmes on the project application process.

OS designed a dedicated HRD OP web site in Turkish and English. The domain for the website (www.ikg.gov.tr.) was received, and uploaded to the server as of 31st December 2009.

Major Development in the Implementation of the HRD-OP in 2010

Grant contracts under Lifelong Learning Operation (LLL- I) were signed in May 2010 and followed by the signature of the grant contacts under Promoting Registered Employment through Innovative Measures (PRE- I), Promoting Women’s Employment (PWE- I), Promoting Youth Employment- I (PYE- I) and Increasing Enrolment Rates Especially for Girls (IER- I).

By the end of 2010, grant projects under 5 Operations started to be implemented. Besides, four service contracts namely Technical Assistance for Implementation of HRD OP and TA Human Resources Development Operational Programme (TA 5.1- 28 th September 2010), Technical Assistance for Potential Operation and Grant Beneficiaries, Information& and Publicity. (TA 5.2 & 5.3-1st November 2010), PES (22nd December 2010) and PRE (14th

September 2010) have been signed. Within the framework of 5 grant schemes, 435 grant contracts were signed with a total budget

of 81.372.983 Euros including the co-financing contribution of the grant beneficiaries. In 2010 two Sectoral Monitoring Committee (SMC) Meetings were held. The 6 th SMC was held

on 24 June 2010 in Van and the 7th SMC on 30 November 2010 in Hatay. Central and regional monitoring teams (RGMTTs and CGMTs) were established under the

coordination of MoLSS. A service contract was signed for supporting the teams. Monitoring and procurement manuals

have been prepared in compliance with the new IPA rules as well as up-to-date practical guide and legislation.

A total of 106 experts have been assigned in the HRD OP implementation regions, especially in the 12 provinces called growth centres. In the Centre, from the technical assistance project, 1 key grant expert and 4 short term experts have been working for supporting the monitoring experts. For each grant scheme one CGMT was composed of at least 5 experts provided by the Operation Beneficiary. Each CGMT is co-chaired by two representatives; one from the Operation Beneficiary and the other from the OS. A total of 34 experts have been assigned to central teams from the institutions such as MoNE, SSI and İŞKUR that are implementing a grant scheme under an operation. 140 monitoring experts as state officials and 5 experts from the technical Assistance Teams have been conducting their tasks.

Before the appointment of these monitoring experts, the CVs of all candidates were submitted to the OS. Following the examination and scoring of the CVs the OS approved the composition of the CGMTs and RGMTTs.

Monitoring experts assigned from public institutions were provided with a 5- day training programme. Representatives from CFCU also gave their support within the trainings. In addition to the trainings, a manual on the monitoring activities was given to the monitoring experts and they have been provided with MIS accounts. A variety of works have been conducted in order to improve the quality of monitoring activities of the monitoring experts and increase the reliability of monitoring data. For instance, each monitoring visit report is checked and, if needed, the monitors are asked to repeat the visit to obtain more accurate and detailed monitoring data.

In 2010 two technical assistance projects have been started in order to provide support to OS as well as the potential and actual operation and grant beneficiaries. The Technical Assistance activities required for implementing the HRD OP are defined under three measures: 1)

102

Support for Programming, Management, Implementation, Monitoring, Control, Evaluation and Dissemination Activities. 2) Support for development of absorption capacity of final beneficiaries. 3) Information and publicity activities.

2. Major developments since 2007 at EU level

2.1 The Lisbon Strategy

Since March 2000, the EU has formulated its policies in line with the objectives of the Lisbon Strategy, which set a framework for actions until 2010. The Lisbon Strategy aimed specifically for economic as well as social and environmental renewal. The Strategy aimed to increase European competitiveness by investing in a knowledge-based and highly productive society.

Noting the challenges Europe was facing from globalisation, an ageing population, and the emergence of a worldwide information society the political leaders of EU resolved that economic and social reforms had to take place in the context of “a positive strategy which combines competitiveness and social cohesion”, and reaffirmed that the European social model, with its developed systems of social protection, must underpin the strategy.

The Union “set itself a new strategic goal for the next decade: to become the most competitive and dynamic knowledge-based economy in the world, capable of sustainable economic growth with more and better jobs and greater social cohesion.”

This was to be achieved through a range of policies including a sound macroeconomic policy mix that is conducive to high growth, completing the internal market, investing in people and combating social exclusion. EU leaders pledged to aim for full employment in Europe, in a society accommodating the personal choices of women and men.

The Lisbon Strategy intended to deal with the low productivity and stagnation of economic growth in the EU, through the formulation of various policy initiatives to be taken by all EU member states. The broader objectives set out by the Lisbon strategy was to be attained by 2010.

It was adopted for a ten-year period in 2000 in Lisbon, Portugal by the European Council. It broadly aimed to "make Europe, by 2010, the most competitive and the most dynamic knowledge-based economy in the world".

The Lisbon Strategy sets specific targets:- An overall employment rate of 70% by 2010- An employment rate for women of over 60%- An employment rate of 50% among older workers- Annual economic growth around 3%

The Council also adopted the open method of coordination between Member States, at different levels of decision-making, as a means to achieve these ends.

In May 2004, as the Lisbon Strategy neared its halfway point, the European Commission (acting on the conclusions of the March 2004 European Council) set up a High-Level Group of Independent Experts chaired by former Netherlands Prime Minister Wim Kok, to make an independent assessment of progress. The 13 experts represented the different stakeholders in the strategy. The group met six

103

times and submitted its report, “Facing the Challenge”, to the Commission and Council on November 4 2004. Its aim was to prepare for the European Council’s mid-term review in March 2005.

It concluded that the EU was very unlikely to meet its 2010 goals, chiefly due to a lack of determined political action. It highlighted an overloaded agenda, poor coordination, and conflicting priorities. It also pointed out that structural reform has become a codeword for deregulation and weakening workers’ rights, and noted that policies should, instead, help workers to address structural change (investing in skills and productivity instead of deregulating labour markets). It also underlined the vital importance of aggregate demand management to exploit fully Europe’s growth potential, and argued that the Stability Pact was an obstacle to this.

Furthermore, the Kok Report stressed the importance of sustaining the European social model, and advised against copying the US system of minimal social welfare. The High Level Group’s proposals focused on the need to communicate better with EU citizens, and obtain their support for the reform process - something the ETUC has repeatedly emphasised. Importantly, the report stressed the role that social dialogue and the social partners, both at national and European level, can play in delivering the Lisbon objectives of high non- inflationary growth, more and better jobs and strong social cohesion.

In November 2009 the Deutsche Bank wrote: “The Lisbon Agenda of the past decade disappointed expectations that it would drive reform, partly because some of its targets were inconsistent but mainly due to the member states’ lack of reform commitment. Under the Open Method of Coordination the only sanctions available to the EU were “soft” options such as peer pressure. Whilst this did strengthen the hand of governments willing to reform, it was often an inadequate spur to those who had tired of the process.”

[Deutsche Bank Research, EU Monitor 70, November 2009, p. 1]

In June 2010 “The Europe 2020 Strategy” plan for economic renewal was launched, replacing the Lisbon strategy (2000-10). By 2020, the new strategy aims to guide Europe's economy out of the economic recession (which began in 2008) and to enable a high quality of life whilst preserving Europe's Social Model, raising employment, productivity and social cohesion.

2.2 The Europe 2020 Strategy

The Europe 2020 Strategy has established the following global targets: 1. Employment

75% of the 20-64 year-olds to be employed2. R&D / innovation

3% of the EU's GDP (public and private combined) to be invested in R&D/innovation3. Climate change / energy

greenhouse gas emissions 20% (or even 30%, if the conditions are right) lower than 1990

20% of energy from renewables 20% increase in energy efficiency

4. Education Reducing school drop-out rates below 10% at least 40% of 30-34–year-olds completing third level education

5. Poverty / social exclusion

104

at least 20 million fewer people in or at risk of poverty and social exclusion

The Commission has further defined seven flagship initiatives to catalyse progress under each priority theme:

1. "Innovation Union" to improve framework conditions and access to finance for research and innovation so as to ensure that innovative ideas can be turned into products and services that create growth and jobs;

2. "Youth on the move" to enhance the performance of education systems and to facilitate the entry of young people to the labour market;

3. "A digital agenda for Europe" to speed up the roll-out of high-speed internet and reap the benefits of a digital single market for households and firms.

4. "Resource efficient Europe" to help decouple economic growth from the use of resources, support the shift towards a low carbon economy, increase the use of renewable energy sources, modernise our transport sector and promote energy efficiency.

5. "An industrial policy for the globalisation era" to improve the business environment, notably for SMEs, and to support the development of a strong and sustainable industrial base able to compete globally.

6. "An agenda for new skills and jobs" to modernise labour markets and empower people by developing their of skills throughout the lifecycle with a view to increase labour participation and better match labour supply and demand, including through labour mobility.

7. "European platform against poverty" to ensure social and territorial cohesion such that the benefits of growth and jobs are widely shared and people experiencing poverty and social exclusion are enabled to live in dignity and take an active part in society.

Similar to the Lisbon Strategy the Europe 2020 Strategy relies on two pillars: the thematic approach outlined above, combining priorities and headline targets across EU combined with country reporting in which the Member States develop their strategies to return to sustainable growth and public finances all principles given by so-called Integrated guidelines to cover the scope of EU priorities and targets.

The two-level system appears important. The Supra-national vision, supported by guidelines as well as a unified reporting system, facilitates and drives Country-specific recommendations within the Member States. In addition “policy warnings” could be issued in case of inadequate response. We shall return to the system below in this report.

2.3 The EU Employment Guidelines and the Broad Economic Policy Guidelines

The employment guidelines proposed by the Commission and approved by the Council, present common priorities and targets for the national employment policies. They have been in an integrated package with the Broad Economic Policy Guidelines since 2005.

The Broad Economic Policy Guidelines underneath the Europe 2020 Strategy describe an overall economic policy guidance to take the form of guidelines addressed to Member States. These guidelines enable the economic policies of the Member States to be coordinated in order to achieve joint objectives.

The Broad Economic Policy Guidelines are described in Council Recommendation 2010/410/EU of 13 July 2010 on broad guidelines for the economic policies of the Member States and of the Union [Official Journal L 191 of 23.7.2010]. The Guidelines should be taken into account by Member States in the implementation of their economic policies and the development of national reform programmes.

105

The Broad Economic Policy Guidelines comes under four main pillars:1. Sustainability of public finances and macroeconomic stability2. Smart growth3. Sustainable growth4. Inclusive growth

While the Broad Economic Policy Guidelines set the scope for recommendation to national policies in broad sense, the Employment guidelines of the Europe 2020 Strategy “should enable the EU to achieve smart, sustainable and inclusive growth in the next ten years. In this Recommendation, the Council also notes the importance of the sustainability of Member States’ public finances and their macroeconomic stability.” [see: The EU Commission Home Pages]

Since October 2010 the current EU Employment guidelines are as follows: Increasing labour market participation of women and men, reducing structural unemployment

and promoting job quality Developing a skilled workforce responding to labour market needs and promoting lifelong

learning Improving the quality and performance of education and training systems at all levels and

increasing participation in tertiary or equivalent education Promoting social inclusion and combating poverty.

3 Coherence in objectives

3.1 The Strategic Coherence Framework

In order to provide a frame of reference for RC OP and HRD OP, the Strategic Coherence Framework (SCF) has been prepared under the coordination of the Strategic Coordinator, SPO, [now Ministry of Development] by including the contributions of relevant authorities, priorities defined in main strategy documents and contributions of the Commission.

As a major strategic document, the SCF takes into account the priorities of the Republic of Turkey and the European Union as stated in major policy documents, especially in the Multi-annual Indicative Planning Document (MIPD). It aims to provide coherence and consistency between the two operational programmes for the 2007 – 2013 period. HRD OP has been prepared in line with the SCF. In order to provide coherence with the SCF, several consultation meetings have taken place between the Operating Structure (OS) and the Strategic Coordinator.

The key challenges defined in SCF are: Low labour participation rate for women High unemployment especially for young Problems stemming from decrease of agriculture sector Unregistered employment Low enrolment rates of girls Late enrolments, dropouts and problems of girls’ access to education Low quality of education Mismatches of the labour market and education system Inefficient social assistances and social services Problems of access to employment, education and social services for disadvantaged persons

106

Accordingly, the SCF sets out four main priority axes that also constitute the basis for HRD OP. These axes are,

Attract and retain more people in employment, particularly by increasing labour force participation of women, and decrease unemployment rates, especially for young people.

Enhance investment in human capital by increasing the quality of education improving the linkage between education and labour market and raising and enrolment rates at all levels of education especially for girls.

Increase adaptability of workers, enterprises and entrepreneurs, in particular by promoting lifelong learning and encouraging investment in human resources by enterprises and workers.

Promote an inclusive labour market with opportunities for disadvantaged persons in terms of their sustainable integration into the labour force and combat all forms of discrimination in the labour market.

The main priority axes established by the SCF have been incorporated in and adapted by the HRD OP, which again has establish specific objectives, measures, indicators and targets in alignment thereof. It appears accordingly fruitless to make further comments to the coherence between the SCF and the HRD OP, since the HRD OP is based upon the SCF platform.

Throughout the Interim Evaluation the Interim Evaluation TAT has been assured about the communication and control systems between the stakeholders involved in the two central documents, which communication lines appear both solid and comprehensive. The TAT has no further comments to add to the correspondence between the central planning documents and the HRD OP. For further reference on links between the two documents we kindly refer to the ex-ante evaluation.

3.2 Coherence between HRD OP and the main EU strategy

Since the Lisbon Strategy has been replaced by the Europe 2020 Strategy it appears less fruitful to analyse the HRD OP’s coherence with the Lisbon Strategy. Accordingly, let us turn to the Europe 2020 Strategy.

In this context of evaluating the coherence between the Turkish HRD OP and the Europe 2020 Strategy described above, the strategic framework of the Europe 2020 Strategy in which only global targets have been set allowing the Member States to define the specific national objectives, measures, indicators, and targets, the matter of coherence becomes slightly blurred; It would be far more consistent to compare the HRD OP objectives and targets with the national agendas of the Member States. Also since the Europe 2020 Strategy does not exclusively address Human Resource issues.

In return to the issue of coherence between the Turkish HRD OP and the Europe 2020 Strategy the very global and cross-sectoral issues established as targets which not necessarily focus exclusively on Human Resource Development the issue of coherence becomes vague. However, and by noting that all target areas necessarily must include elements which relates to Human Resources, though only indirectly, the Europe 2020 Strategy targets mentioned above at least provides some targets which relates directly to the Human Resources Development; that is:

1. Employmenta. 75% of the 20-64 year-olds to be employed

2. Education

107

b. Reducing school drop-out rates below 10%c. at least 40% of 30-34–year-olds completing third level education

3. Poverty / social exclusiond. at least 20 million fewer people in or at risk of poverty and social exclusion

And similar the Europe 2020 Strategy sets flagships which more or less directly relate to the Human Resources Development; that is:

1. "Innovation Union" to improve framework conditions and access to finance for research and innovation so as to ensure that innovative ideas can be turned into products and services that create growth and jobs;

2. "Youth on the move" to enhance the performance of education systems and to facilitate the entry of young people to the labour market;

3. "An agenda for new skills and jobs" to modernise labour markets and empower people by developing their of skills throughout the lifecycle with a view to increase labour participation and better match labour supply and demand, including through labour mobility.

4. "European platform against poverty" to ensure social and territorial cohesion such that the benefits of growth and jobs are widely shared and people experiencing poverty and social exclusion are enabled to live in dignity and take an active part in society.

Table 3 summarizes the main focus areas of the two. It is worth to note that the Europe 2020 Strategy is highly generic leaving the measures and the national targets to each single Member States; it is accordingly up to the single Member States to comply with the EU targets. In some (if not most) EU Member States the targets is further transposed to regional targets and actions where the national or central level is left only to assure a generic national target. In other words, it tend to be quite impossible to describe the often various different ways the Member States have chosen to comply with the supra-national guidelines and targets.

However, even that feature could be seen as a doable future way for Turkey as well; after all the situation in the various regions of Turkey differ to some extent, which could argue for more regional flexibility and a stronger regional, perhaps even local, involvement in target setting and monitoring thereof in compliance with regional objectives. The central level should then just assure that the overall national targets are met, subject to the situation and the incentives taken in NUTS 2 regions not participating in the HRD OP.

The last comment on changing the overall structure of the HRD OP, including its management, however, appear to be slightly out of scope for the Interim Evaluation of the HRD OP 2007-2009 programme, mainly since the Interim Evaluation has found weaknesses on regional level (see Thematic Report II).

Table 3. Coherence between the EU Framework and HRD OPEU Framework HRD OP

Employment target from Europe 2020 Strategy: 75% of the 20-64 year-olds to be employed

Employment Guidelines: Increasing labour market participation of women

and men, reducing structural unemployment and promoting job quality

Developing a skilled workforce responding to labour market needs and promoting lifelong

Employment:PRIORITY AXIS -1: To attract and retain more people in employment, particularly by increasing labour force participation of women, and decrease unemployment rates, especially for young people

Specific Objectives: Promote labour force participation and

employment of women, including those formerly employed in agriculture.

108

EU Framework HRD OPlearning Increase youth employment.

Promote registered employment. Improve public employment services

PRIORITY AXIS – 3: To increase adaptability of workers, enterprises and entrepreneurs, in particular by promoting lifelong learning and encouraging investment in human resources by enterprises and workers

Specific Objectives: To promote Life Long Learning (LLL) Opportunities

under a LLL Strategy To improve quality of non-formal trainings To increase adaptability of employees To increase adaptability of employers in SMEs To promote well-functioning of the National

Qualifications System

Education targets from Europe 2020 Strategy: Reducing school drop-out rates below 10% at least 40% of 30-34–year-olds completing third

level education

Employment Guidelines: Improving the quality and performance of

education and training systems at all levels and increasing participation in tertiary or equivalent education

Education: PRIORITY AXIS-2: To enhance investment in human capital by increasing the quality of education, improving the linkage between education and the labour market, and raising enrolment rates at all levels of education, especially for girls

Specific Objectives: To increase enrolment rates particularly for girls at

all levels of education and vocational training To increase the quality of education especially in

vocational education and training To develop innovative approaches to improve

linkage between education and labour market To promote a “partnership approach” to modernize

VET system

Poverty reduction / social exclusion targets from Europe 2020 Strategy: at least 20 million fewer people in or at risk of

poverty and social exclusion

Employment Guidelines: Promoting social inclusion and combating poverty

Poverty reduction and Social Inclusion:PRIORITY AXIS – 4: To promote an inclusive labour market with opportunities for disadvantaged people, with a view to their sustainable integration into the labour force and combat all forms of discrimination in the labour market

Specific Objectives: To facilitate sustainable integration of the

disadvantaged into the labour market To improve the functioning and coordination of

institutions and mechanisms in the field of labour market and social protection, particularly in order to facilitate the integration of disadvantaged persons into the labour market.

It appears obvious that the HRD OP which covers objectives within the Human Resource areas, which includes Employment, Education and Lifelong Learning, as well as Social Inclusion tangents the objectives of the EU Framework for a European Strategy as laid down in the Europe 2020 Strategy and further accentuated towards the Human Resource issues in the Employment Guidelines. After all, the subject is the same and to some extent the challenges are identical, since EU faces major

109

challenges partly from the economic turmoil of the global economic crisis and partly faces increasing challenge from other more rapidly expanded countries mainly in Asia and notably China and India.

Similarly it appears obvious that the target frames for the two frameworks might differ just as the intra-EU targets and objectives differ to a certain degree; after all the specific ways to combat the challenges facing the individual Member States as well as Turkey cannot be identical as the starting points differs significantly.

Accordingly, when it comes to evaluation of the coherence between the Europe 2020 Strategy, which by its very nature is a global EU strategy where it is up to the individual Member States to fill the strategy out with specific objectives and targets, and the Turkish HRD OP the question appear more to be: Why be concerned about coherence if the broad perspectives are the same? However, that question is not part of the Report’s target.

The question of coherence between the EU Framework for HRD and the Turkish HRD OP leaves only one answer: Yes, there is a large extent of coherence, since the objectives of EU and Turkey are highly similar and so are the target areas to meet the challenges, and since both EU and Turkey has understood the agenda and taking it into account.

We could perhaps as a final word note that the HRD OP does not specifically mention Poverty Reduction as a main focus area contrary to the Europe 2020 Strategy, which with reference to e.g. the Socio-Economic Analysis in a Chapter above may appear quite strange; after all Turkey faces far more basic and important problems than EU and has not the same means to pursue more income equality between regions.

However, the lack of focus on the topic of Poverty Reduction in the HRD OP is a matter of the degree explicitly formulating an objective which basically works as the fundamental principle of the complete Operational Programme; the issue of Poverty Reduction is definitely included both in the selection of Regions to take part in the HRD OP as well as deeply described more or less explicitly in the rationale of each of the priority axes. And to some extent we have to give credit to the Turkish: It appears trivial to state the obvious; rather focus on the main source for change.

The overall conclusion is accordingly that there appear to be a fairly obvious coherence between the EU Framework and the HRD OP.

110

Annex 3. Review of the results of ex-ante evaluation

The findings of the ex-ante evaluation are summarized in the updated second version of the HRD Operational Programme under a set of headlines. The Interim Evaluation has followed up on the main findings and created some single comments. Of course, time has elapsed and not all of the recommendations given by the ex-ante evaluation are equally relevant anymore.

1. Partnership Approach

See Thematic Report on Horizontal Issues.

2. The study of the Current Situation

The ex-ante evaluators note a valid point of providing reasons to undertake solid measures in Human Resource Development within the less developed regions, not least with respect to Lifelong Learning, which provision should be given by analyses of the current situation. However, some very important arguments appear to be missing in the ex-ante evaluation, and the same goes for theoretical points of methodology.

The point, as the Interim Evaluation understands the ex-ante evaluation, is to provide reasons to undertake solid measures in Human Resource Development within the less developed regions of Turkey. In more plain words the question is: “Why do we do all that effort to support the Human Resource Development in the regions in question?” Hence the background study, which not necessarily is the same as indicators of the performance of the programme (though often tightly related).And in that respect we often runs into methodological challenges: How to prove that we need to invest in Human Resources? From a scientific perspective the answer to that question often blows in the wind, though we are very clear on the impact of our effort.

The challenge in understanding why becomes even more severe and visible when it comes to Lifelong Learning; how to provide reasons for continuously upgrading the existing labour force? Exactly where can we see that the current situation is inefficient, perhaps inappropriate, and needs to be upgrade or perhaps even call for complete reforms of the educational system?

It ought to be obvious that these questions can be difficult to provide solid answers to not least from a scientific perspective. And often, notably in advanced mature EU Member States we hardly ever address the questions in the basic reports, simply because our experience knows actions and measures are needed; it has become a common understanding, a common knowledge27. Most countries embark on systems based upon Barometer Surveys of labour shortages combined with follow-up measures such as Training Needs Analysis targeted to specific sectors and specific occupation within specific locations, but in general these tools hardly ever provide the fundamental reason of “why?”28.

27 This is basically not correct; the point is merely that it takes some solid scientific studies to prove the solid reason for the investment in Lifelong Learning, due to the immense number of other factors which are involved in the basic questions raised here.28 Agreed, the discussion tends to be slightly theoretical and advanced here. Training Needs Analysis, TNA, has some very good features and serves a purpose if done in accordance to the theoretical limits within the method. However, and as recognized by all major scientist within this area, employers do not necessarily respond in accordance to the findings of the TNA; it tends to be obvious at least to understand that some employers rather prefer to hire new, qualified labour than invest in training of the existing employees.

111

Accordingly, the background studies of the current situation should be able to make sure that all readers of the reports understand why we invest so much effort (and means) in continuously developing human capital, which otherwise, in case the free market forces worked perfectly, should be solved by default. We need reasons, that is, and hence the issue raised by the ex-ante evaluators on at least findings some sort of proof for the need for solid measures.

But from here some link appears to be broken from a theoretical perspective. True, Priority Axis 3 and its focus on Lifelong Learning could need some arguments, but basically it appears far too limiting just to “add data regarding the productivity of the Turkish economy, notably economic performance data for Turkish SMEs” as noted in the recommendation.

The reason for the broken link in terms of adding data on SME should be very obvious; though presumably constituting somewhere between 85 and 95 of total enterprises (as in almost all other countries) and presumably a significant fraction of total employment, SMEs are by no means the total labour market. In many cases we could easily argue that establishing in Human Resource Development plans is far more crucial in larger enterprises for various highly justified reasons29.

True, SMEs may not have the same capacity to invest in Lifelong Learning as large enterprises, and similarly it appears obvious that today’s SMEs may rightfully become tomorrow’s LE (large and main enterprise). But still the labour market is (hopefully) the complete labour market which accrues to the question of Lifelong Learning as well.

In other words, it appears less reasonable to incorporate studies of performance among SMEs alone in the background studies to support public investments in Lifelong Learning; we need to do studies of the complete labour market for obvious reasons. And that goes with respect to Priority Axis 3 as well.

However, and this is where the real clarification enters, Priority Axis 3 has a Specific Objective 4 “To increase adaptability of employers in SMEs” as well as a defined target group of “Employees in SMEs in specified sectors in RC-OP” under Specific Objective 3 “To increase adaptability of employees” which obviously necessitate arguments, and hence the recommendation made by the ex-ante evaluators.

It appears from the above that the designers of the HRD OP did not understand the basic rationale; of course, the background studies of the HRD OP should provide solid arguments for the targets in question, which arguments are not done by referring to another OP. In particular, and to refer to the rationale provided in this context, the stakeholders need to understand why the exclusive focus on the SMEs, which do not appear evident from the HRD OP.

Accordingly, the recommendation remains in this very Interim Evaluation Report, and due to a need to allow the stakeholders to understand the rationale behind the exclusive focus on SMEs added the note that:

The background study of the current situation should contain studies of production output on economic sectors of both SMEs and large enterprises

However, the phrase does not provide the rationale for investment in Lifelong Learning. In that respect we need to use measures of productivity.

29 In many countries large enterprises still constitute the backbone if not the complete bearing platform of the economy in question which makes it obvious to ensure that these enterprises understand and adapt to global trends and technological changes in time. Furthermore, it appears similarly obvious that the core well-established enterprises, notably in less-developed, perhaps even poor, regions, should act as driving motors / role models for investments in business plans and accordingly HRD on micro level.

112

Productivity is not production but production (output or value added, rather) per employee. The idea of the measure, which agreed is complex, is that it contains vital elements related to issues of technological input (both physical and human technology (education, that is)) and thus the level of advance. Turning the measure into growth rates is measures further progress and accordingly development towards a more advanced economy. The presentation of the measure will further be described in the evaluation of the indicators below.

In the Socio-Economic Study above the measure is used on a general scale for all types of enterprises. The reason is that it is not possible to obtain the raw data needed for the measure on sizes of enterprises.

Accordingly, the idea of retaining the recommendation from the ex-ante evaluation appears to be weak from a first view: Production provides no rationale for investment in Human Resource and productivity cannot be measured on size of enterprise. However, the logic comes of course in two steps: First, we have to understand the size (and growth) of the output from SMEs relative to total enterprises, and then subsequently we have to understand the growth in productivity. The two should obviously be connected in the background study.

We shall, as mentioned, return to these points in the study of current indicators below.

With respect to the inclusion of target groups for social inclusion (recommendation 2 above) the adjustment of the HRD OP appears appropriate and the TAT found no need to add further comments.

3. The Effectiveness of the planned strategy

The Interim Evaluation has no comments to the steps taken in the revised HRD OP except for the evaluation of the indicators and their relevance in Chapter 4 and Annex A4.

4. Indicators

The ex-ante evaluation has a long range of recommendations to the selected indicators. The Interim Evaluation have made an extensive study on the indicators as well to which we refer. With respect to the findings of the ex-ante evaluation as well as the steps taken to incorporate the recommendations, the interim evaluation has found no reason to make additional comments to the general study of Indicators.

5. Sustainability of Results

The Interim Evaluation has investigated the function of the project promoters and found no reason to make further comments. With respect to Mainstreaming of activities and the function of the Programme Management Unit of the Operating Structure it is the general view of the Interim Evaluation that the process has to be slightly more mature in order to assess the effect30.

6. Absorption Capacities

Absorption capacities are explicitly dealt with in Chapter 4.

30 Most of the project was just started at the cut-off date of April 30. In addition the monitoring of the progress of the projects is subject to time-lags in reporting making is almost impossible partly to evaluate the mainstreaming activities and their effect and partly to obtain a solid platform for documentation.

113

7. The ex-ante evaluation’s conclusions and recommendations regarding HRD OP implementation

In the ex-ante evaluation report the overall recommendations are summarized in the following table 6.1.7. The table further describes the response in the revised HRD OP.

Table 6.1.7 Summarizing table of the overall recommendation of the ex-ante evaluation and the subsequent response

Ex Ante Evaluation Report HRD OP revision and response

The foreseen division of units within the HRDOP OS should be revised and streamlined in the view of the recommendations on IPA structures formulated by the ESC project. Thus, instead of five units there should only be three (Programming, Monitoring & Evaluation, Technical Assistance & Implementation, and Quality Assurance & Control).

The foreseen structure has been developed in line with the consultation process with the NAO which will be responsible for the national accreditation. Regarding the functions and responsibilities of each unit the OS does not agree that Programming, Monitoring and Evaluating should gathered under one unit as well as technical assistance and Implementation Units.

Functions and staffing of the units’ employees have to be described in detail at least in the Implementation Manuals

In line with the ongoing training and work load analysis such information will be integrated on the manual and OP.

The institutions invited for membership to the Sectoral Monitoring Committee should be listed in detail.

The institutions invited for membership are already mentioned under the partnership section. In order to avoid duplication, in the implementation section a different approach has been followed and they have been stated as groups.

The setup of the Monitoring Committee should ensure the representation of regional and local actors from the targeted 12 NUTS II regions and 15 growth centres. This does not mean that every target area should be represented, but the programme should avoid focussing only on central government and Ankara-based stakeholders.

The regional and local actors will be able to take part in the Monitoring Committee meetings with the new set up

The gender balance within the Monitoring Committee should be ensured by the OS by setting up clear regulations. Rules of procedure have to be set up as soon as possible.

Gender balance issues will be dealt in the Implementation manuals.

The steering functions of the MIS for the achievement of programme objectives should be reflected within the OP. It is also needed to describe the cooperation with the other OSs in general and SPO in particular.

Such details will be dealt after their finalisation in the Implementation manuals.

The establishment of a functional The establishment of a functional independent evaluation

114

Ex Ante Evaluation Report HRD OP revision and response

independent evaluation team should be avoided in order not to produce overlapping tasks with the Evaluation (sub-) Committee.

team was a recommendation of the Commission. In order to

The roles and functions of institutions supporting the OS in the implementation of HRDOP should be formulated in detail with a view to the transfer to accredited Implementing Bodies.


The Selection Committee should be organised as a sub-committee of the Sectoral Monitoring Committee.


SPO should urge MoLSS to develop a transparent delivery mechanism and describe it in detail in chapter 5 of HRDOP.


MoLSS should enhance the communication strategy with elements directed at the final beneficiaries’ active participation and develop a mainstreaming strategy in order to ensure the impact of HRDOP.


The Interim Evaluation has noted that the Operating Structure has formed seven (7) units (see points 1 and 2). The observations and assessment of the Operating Structure follows from our study of the management structure elsewhere in the Interim Evaluation Report. With respect to the listing of SMC Members the TAT has no comments.

With respect to the recommendation from the ex-ante evaluation on the set-up of the SMC, it appears less obvious to the Interim Evaluation that the “Monitoring Committee should ensure the representation of regional and local actors from the targeted 12 NUTS II regions and 15 growth centres”. In general, we should expect a decent regional distribution of members and subsequently involvement and ownership, but it is a general illusion that representation alone (e.g. quotas-like solutions) generate any additional value to the quality of the committee.

Running the risk of sending a political incorrect message, the Interim Evaluation would rather take a far more pragmatic view on the question of representation; basically, and from a scientific or statistical viewpoint, the main factor in the HRD OP is not the regions in question, but the target groups within the regions. The pragmatic view would therefore rather favour the following priorities in the constitution of the SMC:

1) A solid Partnership Approach based upon a tripartite representation31

2) Some solid representation from regional level based upon objective grounds of quality, knowledge and perhaps even objectivity (rather than subjectivity)

3) A high degree of representation from both gendersThe illusion is that representation alone is able to outweigh competence, experience and will-power in terms of quality of the work being done. And that illusion is repeated in point 5 on gender balance. There is simply no scientific proof to support that equal representation of both gender works better 31 I.e. one-third from representing Employer’s, one-third representing employees (Trade Unions), and one-third representing public administration and governmental / public interests. The last would include educational institutions as well.

115

than a representation of the most competent person irrespectively the gender in decision-making bodies. We might form idealistic hypothetical illusions on ethical reasons based upon arguments of humanity, fairness and equality, but that does not necessarily indicate that things work towards a more equal, and a more democratic, fair world; if those taking part in the process have limited knowledge and interest in the topics discussed the process might well have no impact.

Human Resource Development is the core interest of the real actors of the labour market, including not least those representatives who can establish changes and ensure solid reforms. Especially when it comes to representatives of employees it appears slightly obvious to the interim evaluators that the representatives of the employees, the Trade Unions of Turkey, have their strongholds in Ankara and Istanbul. In that case it tend to be an illusion, a token of a non-realistic dream world, to recommend actors from regional level to be selected prior to the real actors; that would only decrease the functionality of the SMC, though it might look better on paper for ethical reasons.

It goes without saying that participation of regional and perhaps even local stakeholders will be essential for obtaining viewpoints of the regional and local agendas (which soon might differ from the national agendas in Ankara), but in that respect the focus should be on engaging representatives which have experience with working with HRD issues from local/regional level. In other words, the focus should be on quality more than representativeness; the recommendation should rather be in the direction of ensuring knowledge and know-how from a practical implementation point of view.

The political incorrect message should, however, not be exaggerated and dramatized; of course the stakeholders should ensure solid representation from regional and local level and of course that should be incorporate as a working principle within the constitution of the SMC. And similarly with the issue of the gender balance. The main point raised by the Interim Evaluation is rather to embark on objectives towards obtaining the goal of representation, rather than force it through without solidity.

Of course women should have their seat in the SMC since the gender issue is a real topic in HRD. And of course representatives from regional level should be ensured to provide knowledge of regional and local agendas and provide ideas to what works in terms of implementation and what would never work.

In that respect, the interim evaluation would prefer not to set rigid quotas which would distort and disfavour the work of the SMC but rather pursue something far more pragmatic in its recommendations:

1) Ensure a pragmatic and doable approach to the issue of representation which has its outset in a realistic and objective background based upon knowledge of the Turkish environment

2) Ensure objectives and perhaps even targets in representation. That could be: a. e.g. a target of 50/50 representation between regional and central level in x years; and b. e.g. a target of xx/xx representation between the two gender in x years

The Interim Evaluation has found no reason to comment on the remaining recommendations noted by the ex-ante evaluation.

116

Annex 4. Study of the HRD OPs Effectiveness and Efficiency

1 The use and financial allocation of financial assistance and the financial management of the budget

Background information

Financial Allocation for HRD OP from EU

The Multi Annual Financial Framework (MIFF) provides 158.7 million Euros to the HRD OP for the period 2007-2009 as follows:

Year Budget (MEUR)

2007 50.2

2008 52.9

2009 55.6

Total 158.7

National Co-financing for HRD OP

The Community contribution may not exceed the ceiling of 85 % of the eligible expenditure at the level of the priority axis. No operation shall benefit from a higher co-financing rate than the one relating to the priority axis concerned. A national co-financing rate of 15 % and a Community co-financing rate of 85 % will be applied for all priority axes of the HRD OP.

The HRD OP budget with national co-financing is as follows:Year Budget (EUR)

2007 59 058 824

2008 62 235 295

2009 65 411 766

Total 186 705 885

Commitment and Disbursement Data

The commitment and disbursement date of the HRD OP on July 31, 2011 is presented below:

Commitment data in EURCommitment of EU contribution 91.824.245

Commitment of national co-financing 16.204.278

Total commitment 108.028.523

Commitment ratesCommitment rate of EU contribution 66.84%

Commitment rate of national co-financing 66.84%

Total commitment rate 66.84%

117

Disbursement data in EURDisbursement of EU contribution 55.394.668

Disbursement of national co-financing 9.775.529

Total disbursement 65.170.197

Disbursement rates as percentage of commitmentsDisbursement rate of EU contribution 60.33%

Disbursement rate of national co-financing 60.33%

Total disbursement rate 60.33%

Disbursement rates as percentage of total budgetDisbursement rate of EU contribution 40.32%

Disbursement rate of national co-financing 40.32%

Total disbursement rate 40.32%

118

2. Review programme outputs and results in relation with the expenditures incurred at priority and measure levels; in the light of financial and physical progress

Statistical data 32

Available statistical data for the grant scheme Promotion of Life Long Learning:

General dataTotal budget in EUR 5.620.504

Total commitment of EU contribution and national co-financing in EUR

4.803.264

Total number of grants 45

Distribution of traineesSituation No of Trainees %

1. In progress 2.898 68.192. Completed 1194 28.093. Failed by absenteeism 54 1.274. Left by his/her own will 52 1.225. Failed / unsuccessful 51 1.206. Forced to leave by administration staff 1 0.02

TOTAL 4.250 99.99

Distribution of trainees by duration of training Duration of Training N %

1. 0 - 25 days 2.601 61.202. 25 - 50 days 593 13.953. 50 – 75 days 443 10.424. 75 – 100 days 63 1.485. More than 100 days 550 12.94

TOTAL 4.250 99.99

Distribution of trainees by work experience before trainingWorked Period N %

1. Never worked 2.017 47.462. Worked 0 – 90 days (SSI) 825 19.413. Worked 90 – 180 days (SSI) 230 5.414. Worked 180 – 360 days (SSI) 383 9.015. Worked more than 360 days 795 18.71

TOTAL 4.250 100.00

Distribution of trainees by employment situationEmployment Situation N %

1. Unemployed 2.359 70.332. Working 995 29.67

TOTAL 3.354 100.00

32 Please note that the following should not be considered as a final analysis, but just as a sample. While LLL is the first grant scheme that has been completed under the HRD OP, the beneficiaries were in the 3 months phase of preparing the final report and finalising administrative side of the projects (including MIS updates) at the time of drafting the report.

119

Comments to the statistical data:- The total budget is based on the data from the MIS- The total commitment of EU contribution and national co-financing follows the 85/15 ratio and

the data is obtained from the financial reporting system for IPA. The final disbursement data is not yet available, but it may not exceed the contracting amount. All calculations should be based on the disbursement data, but for provisional and therefore inaccurate results, the commitment data is used.

- The number of grants is based on the data from Europeaid website and from MIS- The data on the number of trainees is obtained from MIS. However, the data has not been

updated yet as the majority of trainees are still identified as “in progress”. The assumption in the following calculations is that the success rate will remain identical with rest of the trainees.

- The rest of the data is obtained from MIS

Based on the information above, it is still premature to perform an assessment programme outputs and results in relation with the expenditures incurred at priority and measure levels; in the light of financial and physical progress even in case of the grant scheme that has been completed first. However, some conclusions based on provisional incomplete data is given below.

Analysis

One of the possibilities to measure outputs and results in relation with the expenditures incurred at priority and measure levels; in the light of financial and physical progress is to analyse the total budget spent on a trainee. In case of the Life Long Learning project the data is the following:

Provisional total budget per trainee in EUR 1.322

Provisional total EU contribution and national co-financing per trainee in EUR

1.130

Estimated provisional total budget per successful trainee in EUR

1.497

Estimated provisional total EU contribution and national co-financing per trainee in EUR

1.280

The calculations above are based on provisional statistical data and are not thus accurate. It was assumed that the success rate of the trainees that have not completed the studies remains the same as the success rate of the trainees that have finished the studies. In reality, the success rate may be lower. Also, in budget calculations the use of the contracted amount means that the final funds spent will not be higher, but most probably will be 5-20% lower. The duration of the courses varies, but the majority of the courses have a duration of less than 25 days. At this point of time, there can not be any data available to which extent the trainings contributed to the employability of the trainees, i.e, how many of the trainees will be employed and as a result of the knowledge and skills obtained in the trainings under this grant scheme.

The amount of provisional data and variables makes it impossible to make a conclusive assessment at this point of time. However, considering that the grant projects have several secondary objectives, the average amount spent on trainees on average is reasonable, considering that almost 40% of the trainings have duration of more than 25 days.

Since it is difficult to assess even a measure that involves a grant scheme that has been completed first under the HRD OP, assessment of other measures would not be feasible at this point of time.

120

3. Examine the efficiency of Operation Selection Criteria

Background information.

Selection criteria for Measure 1.1 are listed below.

The HRD OP includes the following selection criteria and eligible actions:

According to the HRD OP, actions which may be funded are:• Labour market analyses in required regions and/or sectors with a view to guide the operations to be implemented under this measure.• Activities to diminish obstacles hindering target group of women’s participation in the labour market.• Active labour market measures for target group of women out of labour force, as well as those formerly employed in agriculture.• Support to childcare and elderly care facilities, particularly for the target group of women taking part in above mentioned actions.• Actions to provide child and elderly care facilities by training target group of women on such care services• Actions to provide carrier guidance service to target group of women• Supporting target group of women entrepreneurship through consultancy services and training

The HRD OP refers to the following selection criteria:• Demonstration of linkage between activities of the projects and partly guaranteed increase in female employment and/or women’s labour force participation.• Demonstration of how the priority for entrepreneurship under RC OP has been taken into account• Demonstration on how the project will focus on the targeted group in regards to educational attainment

The Call for Proposals list three wider priority themes:• Enhancing women’s employability• Developing practical services facilitating access to labour market and/or to employment services • Raising awareness for tackling obstacles to women’s participation in labour forceThe priority themes are supplemented by a non-exhaustive, non-compulsory and indicative list of actions that the applicants could consider:• Development and delivery of career guidance, advisory services including job clubs, • Training and retraining in professions/skills that local labour market needs • On-the-job training within employers, short term placements within companies i.e. internship programmes• Development and upgrade of technical skills to improve efficiency• Activities promoting women to start their own business and including establishing network• Determining external finance opportunities for establishing business and supporting applications to be made to these sources • Matching services gathering participants and employers • Promoting self-employment and entrepreneurship (consultancy, career guidance)• Trainings for motivation and reinforcement for supporting women to access labour market• Providing one to one guidance services for women (starting from entrepreneurship training until setting up the business) who want to set-up a business.

121

• Developing flexible forms of work and approaches ensuring reconciliation of family and working life • Actions enabling beneficiaries to better access the labour market and/or the employment services through provision of child and elderly care services and/or other facilities • Development of community based child/elderly care services (including training, assessment needs, model developments) • Other pilot services such as community services, customised transport facilities or other initiatives facilitating access to employment services or to the labour market• Field studies related with the above issues including assessment of needs and skills forecasting, new ways and mechanisms for services financing• Awareness and sensitivity raising, information and publicity on tackling cultural obstacles hindering women’s participation in the labour force • Pilot awareness actions at local level (to women, their partners, opinion leaders, employers etc.)• Local partnership approaches including local social dialogue, capacity building for community development. Effective dialogue between community leaders, employers and those providing services to women job-seekers• Field studies, training tools and methodology developments related with the above issues

122

4. Assessment of local absorption and implementation capacities

Statistical data on local absorption capacities

Table. Number of Applications Selected from Each Region

NUTS II Regions LLL PWE PRE PYE IER Total1 TR A2 Kars 3 1 0 2 2 82 TR B2 Van 3 10 5 10 17 453 TR C3 Batman 0 4 0 4 8 164 TR A1 Erzurum 2 6 3 3 4 18

5 TR C2DiyarbakırŞanlıurfa 3 4 4 9 12 32

6 TR C1 Gaziantep 2 4 1 4 4 15

7 TR 72 KayseriSivas 8 22 4 21 10 65

8 TR 90 Trabzon 7 21 2 14 7 51

9 TR B1ElazığMalatya 6 18 1 8 8 41

10 TR 82 Kastamonu 2 3 2 2 4 1311 TR 83 Samsun 4 27 9 22 8 7012 TR 63 K.Maraş 3 11 1 16 3 34

Other provinces 2 4 9 13 2 30

TOTAL 45 135 41 128 89 438

Table. Number of Applications Not Selected from Each Region



6 TR C1 Gaziantep 24 46 10 66 39 185

7 TR 72 KayseriSivas 60 101 15 90 19 285

8 TR 90 Trabzon 47 84 29 81 51 292




TOTAL 443 921 197 994 567 3122

123

Table. Total Number of Applications Received from Each Region



6 TR C1 Gaziantep 26 50 11 70 43 200

7 TR 72 KayseriSivas 68 123 19 111 29 350

8 TR 90 Trabzon 54 105 31 95 58 343




TOTAL 488 1056 238 1122 656 3560

Table. Percentage of Submitted Applications that Have Been Selected from Each Region

NUTS II Regions LLL PWE PRE PYE IER Total1 TR A2 Kars 16,67% 2,70% 0,00% 5,26% 4,76% 5,59%2 TR B2 Van 3,90% 5,68% 11,11% 5,65% 11,72% 7,26%3 TR C3 Batman 0,00% 9,76% 0,00% 6,78% 15,38% 9,47%4 TR A1 Erzurum 5,88% 8,00% 18,75% 4,62% 9,76% 7,79%

5 TR C2DiyarbakırŞanlıurfa 13,64% 6,25% 22,22% 10,23% 16,00% 11,99%

6 TR C1 Gaziantep 7,69% 8,00% 9,09% 5,71% 9,30% 7,50%

7 TR 72 KayseriSivas 11,76% 17,89% 21,05% 18,92% 34,48% 18,57%

8 TR 90 Trabzon 12,96% 20,00% 6,45% 14,74% 12,07% 14,87%

9 TR B1ElazığMalatya 17,14% 18,18% 5,56% 12,12% 20,51% 15,95%

10 TR 82 Kastamonu 9,09% 6,67% 25,00% 3,64% 21,05% 8,72%11 TR 83 Samsun 6,15% 21,95% 45,00% 18,03% 10,53% 17,24%12 TR 63 K.Maraş 12,50% 18,97% 7,69% 20,51% 13,64% 17,44%

Other provinces 6,67% 6,67% 33,33% 13,27% 13,33% 13,04%

AVERAGE 9,54% 11,59% 15,79% 10,73% 14,81% 12,30%

124

Table. Percentage of Projects Selected from Each Region as Compared to Total Number of Selected Projects

NUTS II Regions Percentage1 TR A2 Kars 1,83%2 TR B2 Van 10,27%3 TR C3 Batman 3,65%4 TR A1 Erzurum 4,11%

5 TR C2DiyarbakırŞanlıurfa 7,31%

6 TR C1 Gaziantep 3,42%

7 TR 72 KayseriSivas 14,84%

8 TR 90 Trabzon 11,64%

9 TR B1ElazığMalatya 9,36%

10 TR 82 Kastamonu 2,97%11 TR 83 Samsun 15,98%12 TR 63 K.Maraş 7,76% Other provinces 6,85%

TOTAL 100%

Table. Average Number of Projects Selected per One Million Inhabitants from Each Region

NUTS II Regions

Number of projects per million inhabitants

1 TR A2 Kars 72 TR B2 Van 233 TR C3 Batman 94 TR A1 Erzurum 13

5 TR C2DiyarbakırŞanlıurfa 12

6 TR C1 Gaziantep 7

7 TR 72 KayseriSivas 26

8 TR 90 Trabzon 16

9 TR B1ElazığMalatya 23

10 TR 82 Kastamonu 1511 TR 83 Samsun 2312 TR 63 K.Maraş 13 Other provinces n/a

125

5. Analysis of the accuracy of the budget allocations under measures

Background information

The funds under IPA component IV are limited and therefore the MIPD focuses a limited number of priorities that also adapt the ESF regulation's priorities to the particular needs and challenges of Turkey:

- Attract and retain more people in employment, particularly by increasing labour force participation of women, and decrease unemployment rates, especially for young people.

- Enhance investment in human capital by increasing the quality of education, improving the linkage between education and the labour market, and raising enrolment rates at all levels of education, especially for girls.

- Increase adaptability of workers, enterprises and entrepreneurs, in particular by promoting lifelong learning and encouraging investment in human resources by enterprises and workers.

- Promote an inclusive labour market with opportunities for disadvantaged people, with a view to their sustainable integration

Based on these targets, the MIPD proposes the following indicative division of the budget of IPA component IV in Turkey

Employment 40-50%

Education 30-40%

Social Inclusion 20-25%

The HRD OP elaborates this budget breakdown further and includes funds for Technical Assistance

Employment 44%

Education 30%

Social Inclusion 20%

Technical Assistance 6%

The funds are also divided regionally

National level 20%

NUTS II growth centres 45-55%

NUTS II hinterlands 25-35%

The HRD OP includes the following breakdown of the budget between priorities and measures:

Priority Axis 1 82.150.589

Measure 1.1 27.150.589

Measure 1.2 23.000.000

Measure 1.3 15.000.000

Measure 1.4 17.000.000

126


Measure 2.1 16.011.764

Measure 2.2 20.000.000


Measure 3.1 15.000.000

Measure 3.2 5.000.000


Measure 4.1 30.341.177

Measure 4.2 7.000.000


Measure 5.1 6.202.355

Measure 5.2 3.000.000

Measure 5.3 2.000.000

TOTAL 2007 -2009 186.705.885

127

Annex A5 The Study of Indicators

1. Overview and Introduction: The study on Indicators and the Relevance of the HRD OP

Part I, II and III of the study are designated to perform an evaluation of the HRD Indicators and is, according to the TOR, intended to address the following evaluation questions:

Q1.4.3.2 Collect data on indicators not provided through the regular monitoring systemWhich evaluation question is repeated as Evaluation Question 3:

Q3 Output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring systemQ3.1 Analyse and provide data on output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring system

Q1.4.2.4 Analyse the accuracy, clarity, quality, usefulness and internal consistency of the overall objectives, purposes, targets, indicators and eligible activities at the priority and measure level of the HRD OPQ1.4.3.4 Assess the concrete progress of the HRD OP on the basis of the indicatorsQ1.3 The quality of projects designed to achieve the objectives, targets and indicators determined in the HRD OP

The study falls in four Parts.

The first Part describes the fundamental methodology of which parts are previously described in the Inception Report. Referring to the description of challenges described in the Introduction to the Interim Evaluation Report, the part further describes the impact of the challenges imposed on the study of indicators and the limitations to which these challenges impose. The part finally describes the chosen approach to the study of indicators conditional to the limitations.

Part II makes an evaluation of the Output Indicators, while Part III makes an assessment of the Result Indicators. Part IV then draws up the main conclusions on the Study of the Indicators of the HRD OP ending with a summary of the recommendations derived from the study.

PART I: Methodology and approach to the study on indicators

I.1. Methodological note on Evaluation of reform programmes and strategies in Human Resources

1.1 The rationale for Monitoring and Evaluation

With recent years’ growing public expenditures attention to Monitoring and Evaluation has increased as a still more important, integral tool in modern management. In management terms Monitoring and evaluation is of paramount importance, a necessity, primarily due to the combination of scarce resources and increasing responsibilities. However, and to some extent even more important, the knowledge economy has added to the importance of the management tools dramatically in three very interlinked ways:

1. As information within the information society increases or at least become easier to obtain and more available (transparent systems) the focus on efficiency with public means has increased, not least from mass media. Accordingly, the focus on the decision makers’ ability to do an effort has increased;

128

2. As information becomes highly more accessible the emphasis on knowledge and knowledge sharing in practical term increases. This necessitates a high demand for specialist to manage and take care of highly specialist functions at all levels in the economy. The need of specialists has accentuated the subsequent need for more direct and sharper measures in Human Resource Development and in that respect the public sector (including both the educational sector and the Labour Authorities / Ministry of Labour) often becomes too general. The highly specialised measures are accordingly often outsourced to specialists within the private sector due to their specialized competences in e.g. educating people to specialized function or in creating job solutions and carrier guidance to specialists. Outsourcing to private sector caretakers has obviously accentuated the need for efficient monitoring;

3. As global trade markets and financial sector becomes increasingly globalized the competition between countries and systems become accentuated, mainly due to the pressure for specialists. In order to at least maintain the current welfare state a sharper and more committed approach is called for, not least due to increased competition from transition countries and developing countries. This again accentuates the need for sharper solutions and accordingly increases the need for efficient measures.

Monitoring and Evaluation are basically two different sides of the same process, since the evaluation report is supposed to be based on the progress reports which form the feedback from the implementation phase (i.e. monitoring).

Monitoring is accordingly defined as an on-going process of information collection primarily for the purpose of programme management. Monitoring focuses on activities and outputs based on an agreed template for certain agreed periods of time, sometimes even defined as milestones.

Milestones are predefined values of the indicators typically set for each year leading to the end target value. Accordingly, the set of milestones describes accordingly an expected adjustment path on our way to meet our established target. For obvious reason we would expect to produce an evaluation report based on indicators and their measurement against the milestones, allowing us to make adjustments to our efforts and plan.

Progress reports on the on-going process of implementation might, however, are typically produced on a more frequent basis in mature boards, councils and ministries.

Evaluation is considered an ex-post assessment of subsequent progress reports in some instances further explored through targeted intelligence means (e.g. surveys) in order to process the information collected. Evaluation tends to focus more on outcomes and impacts.

1.2 Methods of Measurement of effect of strategies and programmes

Indicators and target setting is based on measurement and so is the basic fundament for setting up the complete strategy, see Textbox 1.

129

Textbox 1. Basic principles in all Human Resource StrategiesThe planning phase

1. A solid Analysis of the current statusA solid description of the economic conditions which apply at national, regional and local level by use of Official Statistics and International Indicators;

2. A solid Analysis of current trends and tendenciesA study of the most evident trends and tendencies on the labour market in terms of new profiles, imbalances and employers’ expectations to the nearest future based on Labour Market Intelligence elements (Survey, Administrative Records, Partial Information);

3. A consolidated and confirmed strategyA consolidated and confirmed strategy based upon clear tasks for all partners involved, including Social Partners, key employers, the Chambers, Schools and Universities, local authorities, County Employment Agencies, line Ministries (e.g. the Ministry of Finance), and similar key stakeholders;

4. Implementation through adequate measures solidly managed and supervisedImplementation through adequate measures supported by management and supervision with specific focus on “Value for Money”;

5. Continuous monitoring through agreed milestonesContinuous monitoring through agreed milestones and continuous progress reports on on-going measures in accordance with the agreed performance indicators;

6. EvaluationEvaluation on actions in subsequent Strategies mainly based upon subsequent progress reports (evaluation sheets, budget, targets, etc.) evaluation of effect and efficiency (including quality) provides experience and generates scope for new actions

According to textbox 1 there appear to be six discrete steps in each planning circle. Step 3 and 4 is for obvious reasons outside the scope for this report on indicators of which step 3 concerns the establishment of the strategy. The 4 remaining steps, however, all concern the methodology of measurement.

As all four steps, i.e. step 1, 2, 5 and 6, all are important for the establishment of indicators as well as the measurement of the impact of our actions, it serves a purpose to describe them in this context.

It serves a purpose, however, to note the words used in step 3: Consolidated and confirmed. No single agent or body (Ministries, Social Partners, etc.) bears the sole responsibility when it comes to ensure a balanced and progressive labour market, sustainable economic development, and ensuring a welfare state. The responsibility is borne by all partners, including single economic agents (schools, enterprises and individual jobseekers) for obvious reasons.

As soon as the Human Resource Strategy is implemented and actions carried out, feedback from monitoring the process provides the stakeholders with crucial information about how the market (the schools, the students, the enterprises, the local authorities, etc.) responds to our interventions. This information is important as we might need to adjust our actions; i.e. we might need to reinforce actions as they appear to have less effect than that expected or we might be able to slow down on some of our action as they appear to have a far stronger impact than expected. Performance indicators and Targets become important as we will describe in more detail below.

130

The next time we have to update our Human Resource Strategy (e.g. next year) the information obtained from the adjustment process (called monitoring) should be evaluated as an integral part of the assessment. The information becomes ‘alive’ since it provides us with information about how actions work through time and in a local dimension. Please recall here that we never know how actions will work before they are tested in a real market. Accordingly, our experience becomes knowledge, which knowledge should be an integral part of the next action plan.

Accordingly, in order to create a knowledge-based updated Human Resource Strategy we need to make an assessment of the process (evaluation). The logical process of the update can accordingly be seen as a four-step process with reference to the design of Human Resource Strategies above:

Textbox 2. Basic principles in all Human Resource StrategiesThe subsequent evaluation and adjustment phases

A. Assessment of the current situation based on official statistics (Step 1 above)

Key Questions: Where are we? How did we get here? Where are our relative Strengths, Weaknesses, Opportunities, and Threats?

The study should be based on official statistics only, made comparative at least in terms of

a. Geography (county level and averages for regional levels and national and international levels when appropriate);

b. Time; and c. Other strategic focus areas (e.g. gender, age, economic sectors, etc.).

B. Assessment of the most recent trends (Step 2 above)

Key Questions: How does the market respond? Where are we heading? What are the potentials, asymmetries, imbalances?

The study should be based on official statistics supported by information and data obtained through intelligence means (Surveys, administrative records, partial information, other studies, etc.). As in Step 1 the study should be comparative to ensure a high degree of transparency.

C. Assessment of current actions (Step 6 above)

Key Questions: What did we do? What were the outcomes (successes and failures)? How long did it take for the market to respond to our actions (both supply and demand)?

The study should be based on approved performance indicators (re step 3 above) and made comparative to ensure a high degree of transparency.

D. Proposal for a new confirmed and consolidated strategy

Including: a. Adjustments based on assessment of steps (A), B and Cb. New actions based on assessment of steps A and B

Accordingly, the mature strategy does not start from scratch, but encompasses an assessment of the actions in terms of Input, Output, Outcome, and Impact.

131

1.3 The rationale of indicators

The establishment of performance indicators serves at least two general and main purposes:1. The indicators are a necessity for continuing Monitoring and Evaluation;2. Transparent indicators can be used for benchmarking purposes; International transparent

indicators for international benchmarking, Institutional national indicators can be used for cross-regional and/or cross-institutional purposes.

The benchmarking element comes in as a tool for strategic knowledge sharing (mainly on institutional level) or as a platform for strategies. The latter is of course closely related to transparency and knowledge sharing as it makes good sense to ask “why is this country progressing in a faster pace than us?”

In that respect and very brief, benchmarking allows us to obtain knowledge of both possible progress paths and not least knowledge of what works. Practically speaking, we compare our present situation to our countries facing similar problems and cultural environment which allow us to compete with the comparable countries on progress. Of course, the competition is done in a friendly environment where we share our knowledge of successes and failures. That is the whole point in benchmarking, in fact: knowledge sharing.

In benchmarking indicators plays a crucial role both in our task to find a representative pool of comparable countries which face the same conditions as our country, and not least in our benchmarking task to monitor progress.

In monitoring and evaluation as well as in benchmarking the indicators do not answer our basic questions on “how, why, where, and what”, but they are supposed to be our eyes and ears in our actions and effort. Small and tiny as they are, they allow us to make a difference.

1.4 Levels of Execution

In order to define indicators in a practical environment it appears natural to distinguish between the level of performance; e.g. political and institutional contrary to practical implementation, or just central level contrary to local level. In some specific cases we can identify an additional intermediate level between in case the objectives, purposes and target groups differ significantly. That could subsequently be termed multilevel execution of strategies or programmes.

Accordingly, throughout the report we shall implicitly distinguish between different types of processes which mainly refer to the different purposes of the processes, and accordingly the different types of evaluations.

The multilevel mechanism does not indicate lack of link between the levels. Rather on the contrary. In case a multilevel system of indicators and targets is desirable and chosen the links should be described and justified methodologically.

In case the different levels demands different approach and means in terms of indicators for measurement of effectiveness slightly different tools in terms of monitoring and evaluation may apply. However, the basic nature of the content of the levels including the purpose and the objectives and goals they perceive ought to remain the same.

132

In other words, in some cases it appears necessary to establish different sets of performance indicators which refer to:

• First level indicators are meant to measure the overall performance of the overall programme; i.e. the political / institutional framework

• Second level indicators are meant to measure the effect of single measures and actions within the strategy; i.e. practical implementation on local and/ or micro level

This differentiation is even more important during reform processes as the first level of indicators is supposed to be related to institutional capacity building, adaption of new procedures, rules, and regimes, build-up of the overall infrastructure, etc. Accordingly, during reform processes the first level of indicators should reflect (or measure) the impacts of the reform programme itself further to the single measures and actions undertaken within the new framework. As the institutional build up becomes more settled and mature we would expect a gradually higher degree of reflection between the two levels of indicators.

I.e. in the strategy set-up performance indicators should be related to two different levels of the complete monitoring manoeuvre. Obviously, the two levels are interrelated through the strategy, and accordingly we would require some connection between the two levels of indicators; i.e. we establish indicators on the aggregate level which reflects the output / outcome / impact of each single action. However, to each action several different indicators might apply which do not necessarily have relevance on the impact of the overall programme / strategy.

As we move on from basic input and basic output to impact the Performance Indicators should reflect a still higher degree of the basic purposes, goals, and objectives of the complete strategy, the complete reform, or the overall basic institutional purpose. In that sense as we move from output through outcome to impact we distinguish between:

1. The effectiveness of the institutional setup, including accountability and “Value for Money”2. The overall national agenda; i.e. in terms of impact in economic and employment terms.

For a complex programme containing multiple operations and projects with each single measure it accordingly appears logic to investigate the effectiveness of single elements based upon indicators of input, output and to some extent outcome in terms of “Value for the Money” (input) used to increase the employability among the target groups.

And then when it comes to the overall programme we identify a set of indicators which measure the overall relevance and effectiveness of the programme, which mainly include outcome and impact indicators.

The purpose of using dual systems or multiple levels of performance indicators is obviously to make the evaluation easier and far more practical since it appears almost impossible to make a solid assessment of impact in complex programmes or in assessments of a complex national agenda which easily involves multiple objectives, priorities and actions in multiple dimensions.

1.5 Types of Indicators

In general there are four types of indicators:

1. Input Indicators: Input indicators measure the financial resources provided and the administrative and regulatory measures taken (for example, resources allocated, resources used, measures taken, laws passed). The definition of inputs can be treated as very broad covering in some cases what is often called "process indicators".

2. Output Indicators: Output indicators measures the immediate and concrete consequences of the resources used and measures taken (for example, schools built, teachers employed,

133

nurses trained) The definition of output covers those goods and services "supplied", “produced” or "provided" by the public sector with the inputs.

3. Outcome Indicators: Outcome Indicators measures the results at the level of beneficiaries (for example, gross enrolment rates in primary schools, vaccination). The definition covers the outcomes (or results) from the use and satisfaction of the goods and services produced by the public sector - it is where supply comes face-to-face with demand.

4. (Gross) Impact Indicators: Impact Indicators measures the consequences of the outcomes in terms of wider objectives (for example literacy rates, health improvement). The definition covers the wider effects of the outcomes but there might also be higher level impacts, related to broader objectives – growth and income poverty, for example. In order to turn it into a net impact indicator the effect of other economic, educational, and employment programmes have to be assessed and accordingly subtracted.

Evaluations based upon these four types of indicators provide a description of the effectiveness of the programme in terms of impact on the environment and the local society and the overall effect on employment. But effect and impact are not the only parts within a solid evaluation.

The strategic evaluation of impact has to be closely followed by a financial audit in order to ensure a sort of “value for money”. In that respect, it is important to understand the need for prioritized measures and actions.

Accordingly, evaluation has not just to measure an effect in terms of impact but also to evaluate the financial input to generate the effect; and in case the initial financial input was high, indicating a costly project, it might be worth reconsider continuing the project – also in case the project appears to generate a positive impact.

The point is accordingly to measure impact in relation to effort. The consultant has substantial international experience in evaluating projects, measures, and actions within the Human Resource sectors of Labour and Employment, within Education, and within Social Affairs.

1.6 Target setting and time frames

1.6.1 The SMART principle

Performance Indicators as well as their targets should be SMART; i.e. • S is Specific; indicating that the indicators (as well as the single measure and the single

action) precisely should measure the anchor points in our strategies, measures and actions;

• M is Measurable; indicating that the indicators should provide the evaluator and the executioner with sharp and precise feedback to the actions implemented in a simple, straightforward and accountable way;

• A is Affordable or Achievable; indicating that the target as well as the single measure and the single action, though ambitious and progressive, should be within reach with some effort;

• R is Relevant; indicating that the indicator as well as the single measure and the single action should be directed to create the solutions called for;

• T is Time-bound; indicating that the indicators and targets as well as the single measure and the single action should follow a predefined and agreed timeframe with clear and logic milestones and deadlines.

134

In all matters the SMART principle suggests a pathway to formulating indicators and targets which are easy to interpret and generally accepted and recognized. As will be clear below it appears less fruitful to develop a long set of indicators – say, 100 hundred – which generate long report which none are able to read from an instant, including the analyst/evaluator.

In other words, the indicators should accordingly capture the very essence of the strategy and its main purpose, and through a sharp and precise list of measurable indicators indicate success or failure from a general perspective. The precise, concise and short list should facilitate short progress reports which all are able to read and comprehend.

Further indicators, and especially when we turn to Second Level indicators, can be more distinct and more specific and detailed, as well as more from a quantitative perspective (more indicators). Again this aspect only accentuate the focal point that First Level Indicators should encompass or comprehend Second Level Indicators and/or specialized in-depth indicators as raised above.

The very point is that it detail ensure precise knowledge of the root of the problem, while too much detail will leave a blurred overall picture. Only if something is terribly wrong and needs to be corrected we need to turn to the details. Otherwise, we lose focus on the main objectives of the reform process.

Accordingly, the formulating of First Level Indicators becomes important, not just in matters of findings indicators that can be agreed upon but more so as the First Level Indicators be able to capture the essence of important variations all operational indicators (Second Level Indicators) without accentuating single and non-important disturbances in Second Level Indicators.

Finally, the SMART principle accentuates the necessity to establish a timeframe for indicator measurement (monitoring and evaluation) aligned with targets. Milestones have to be established as well indicating the expected pathway to achieve the end goal and objectives as described by the objectives of the strategy. For obvious reason targets should accordingly be assigned to each milestone within the timeframe not least in order to allow the stakeholder to adjust the actions.

1.6.2 General good practices in Target setting

It is probably impossible to avoid political issues in target setting. It is after all an intrinsically political process to declare that certain goals will be given prominence and that certain targets will be met by the incumbent government. However, when all the key issues are represented in a balanced way it become more difficult for politicians to argue for rapid expansion of access to a system without accepting the need to commit commensurate resources to maintain quality.

Key principles in target setting are to base targets on operational assumptions, to take local capacities and resources enough into account when setting targets, to look at past trends in performance as a basis, drawing upon a sufficiently long time span, and to avoid introducing perverse incentives for performance. Targets should be set so as to get the balance right between being over-ambitious or excessively prudent. They should be challenging but achievable. In other words they should be the product of active engagement by the managers and stakeholders with realities of the system and its environment.

Good practices on these issues include:• Clarity of indicators: if indicators are clear and simple it is easier to assess the target

setting objectively.

135

• Debate assumptions not forecasts: in other words force the target setting group to think about the assumptions such as productivity increases, resource availability, which must occur if the target is to be achieved. Involvement of the Ministry of Finance is important in that respect.

• Force them to model those numbers against real data and actual trends.• Discuss resources early: Targets depend crucially on resource commitment so resources

need to be added to the equation early.• Develop execution skills: Targets are meant to stimulate all those involved in the system

to greater efforts. Targets should be accompanied by clear implementation strategies and plans which are linked to clear assumptions and ideally show how improved performance can be achieved at a stable or falling unit cost. Sector managers need to demonstrate how the targets are to be translated into operational procedures at different levels of the service or how incentives will be structured to ensure that both public and private sector service providers move in the desired direction.

Who should be involved in setting targets?• Involve all major stakeholders at some point in the process so that the assumptions are

thoroughly debated and so that there is social buy-in from key actors like unions and private service providers.

• Give the target setters some independence to increase objectivity but don’t divorce target setting from the managers who have to deliver the achievement of those targets.

• Involve a professional statistician in the target-setting group. As illustrated above target setting is not purely a technical task but it has many technical complexities. Without professional involvement targets are likely to be formulated without clarity and possibly without validity.

Target setting should not be seen as a single event occurring once a year or once every few years. It should be part of an on-going process of operational research around key aspects of systems performance informed by thorough analysis and regular research.

The basic principle of performance monitoring is that indicators are meant to measure performance and to provide important input into management decision making and resource allocation. This means that information from indicators needs to be synchronised with the planning, budgeting and reporting cycles.

Preliminary targets for the coming year should be proposed by government during the MDA (Ministry, Department, and Agency) reviews so that what is presented to the annual review involving a wider stakeholder group is already a well-considered proposal based on tentative resource allocations and a thorough understanding of the causes of indicator trends.

Ideally the following steps of a well-organised review should be undertaken:• Management review: Analysis of performance indicators is presented to responsible

managers in sector ministry for review and comment. Where possible disaggregated data should be prepared and sent down to the responsible sub-national levels. Managers are asked to explain significant deviations from targets.

• Management analysis: Where deviations are major it may be appropriate to analyse the source of deviation more deeply. (Were resources available as planned? Are there possible exogenous reasons such as a natural disaster? Is the deviation derived from unexpected results in certain parts of the country?)

• Investigation: It may be appropriate to commission some rapid field assessment to understand the deviation better. Investigation should also be used to look at factors behind success

136

stories and develop good practice cases studies for sharing with relevant managers across the whole system.

• Peer Review: Once managers have been allowed an opportunity to analyse, investigate and understand the performance data there should be an opportunity for them to present this information and its implications for future action to their peers. Such peer reviews are the best environment to encourage professional challenge and to ensure that explanations of sector performance are internally consistent. Such a review should bring in managers from the sub-national level. Often they will be in a better position to explain or challenge explanations.

• Strategy review: Senior managers of the sector need to absorb this information and develop initial proposals for strategy change, increased efficiency, resource reallocation, adjustment of future targets. This would ideally constitute a second forum with more limited membership following up the work of the one described in the last paragraph.

• Inter-ministerial coordination: Most governments are organised in a manner where most service Ministries cannot implement their strategies without substantial support from other Ministries such as Finance (financial resource), Public Service (human resources and government-wide structural reforms), Local Government (coordinator of the local authorities who actually deliver the services), and Legal Affairs (coordinators of any legislative programme). So once the sector Ministry has developed a robust analysis of its own performance and a set of responsive strategies to enhance future performance, it has to go out and sell its proposals to the rest of government both in Cabinet but also at a more technical level with its sister Ministries.

• Public accountability: The final step in the internal review process is the presentation of performance data to the appropriate public forum specified by the law or constitution. This could be a specific Senate committee or an annual parliamentary review of public expenditure.

All of the foregoing should take place before the annual review with the key stakeholders, so that the annual review can reflect on the domestic debate and enrich it where possible but not become a substitute for the constitutional organs and channels of debate. The role of the annual review should then ideally be one of discussion and endorsement.

1.6.3 Progress reports

Progress reports sounds to be a lot more than it actually needs to be, and it appears to impose a lot of hard work on a continuing basis, both from the side of OBs and the OS. Basically, however, in most cases the progress reports do not need to be much more than a selection of predefined and agreed tables (based on the indicators) presumably supplied with some very few words.

The point in progress reports is to provide the members of the council with a management tool. In fact progress reports are supposed to be a product of a management information system. Normally, if the progress works as planned, the presentation of progress reports leads to only few comments from the members. However, in case some of the actions appear to be out of control or if the market adjustment appears to be lacking, members of the council might call for adjustments to the actions.

In most cases, the council will leave it to the secretariat to make smaller adjustments. Obviously, the secretariat will then inform the council members on the next coming meeting. However, if more effect full adjustments are called for the secretariat might need an agreement from the council members before a new action is taken. In all cases, progress reports constitute the main information channel for the management (it actually forms an integral part of management information system) and should be seen as an early warning system.

137

I.2 Approach and limitations to the study of indicators

The basic methodology and approach to the study of the indicators are fairly straightforward and follows best practices in evaluation of indicators. The outset is based upon an understanding of the importance of performance indicators, which is lightly described above.

While the above section 8.2 described the purpose and the context of performance indicators to which we shall refer to in the study of the relevance of the indicators, another main part of the evaluation is to screen the current selected indicators for quality and availability. In order to understand the study some comments to evaluation approach is needed, which necessitates a description of the main risks in establishing indicators. That part will be presented in the context below.

With respect to the approach to evaluate the indicators selected for the HRD OP the following main steps are needed:

1. Define the context within the selected indicators are placed2. Organise the indicators in accordance to purpose3. Obtain data from the monitoring mechanisms in place, and obtain raw data for controlling

purposes as well as to ensure the necessary tools for the evaluation and update of the current indicators within the system

4. Screen the data based upon a number of factors of validity and control5. Mirror the data across sections towards control groups (e.g. target groups, regions, etc.,

towards alternate target groups, regions etc.) to assess their relevance in relation to purpose as well as in order to assess the basic quality of the data

6. Made basic use of the data for analysing purposes, which includes studies of progress etc., in order to see how well the data responds to changes in environment (including economic environment not least)

7. Make a qualitative assessment of the data in relation to the objectives, measures, and targets.

For obvious reasons the steps described only reflex the basic overall approach; experience tells us that multiple problems appear as soon as the evaluation starts which necessitate other, often stronger means to evaluate the basic quality of the data. Of course that mainly accrues to data and statistics not provided by official statistical offices since the official statistics has already undergone the basic validation procedures. Thus dependent on the importance and not least the purpose of the indicators chosen the basic recommendation is to select well-developed indicators.

The HRD OP operates with two types of indicators only: Output and Result Indicators. Well, that is basically not entirely true since it only reflex that the HRD OP does not explicitly describe the Input Indicators. The lack of Input indicators is somehow strange since the HRD OP is using public funding from EU and the Turkish Government. Nevertheless, the main financial input indicators are described elsewhere in the HRD OP while it tends to be rather difficult to find the description physical input indicators. Nonetheless, when it comes to the study of the Input Indicators the Interim Evaluation refers to the evaluation in a separate chapter above.

The Interim Evaluation then turns the focus on Output and Results Indicators within this chapter.

With respect to the evaluation of the quality of data the Interim Evaluation kindly refer to the limitations of the study described in section 1.3 in the Introduction, mainly on that part which refers to lack of basic data. Repeating and accentuating the challenges in obtaining access to the data required and requested repeatedly by the TAT the above mention 7-steps process of evaluation is seriously hampered by lack of the raw data; basically the TAT was not provided with any relevant data other

138

than data provided by the OS, TURKSTAT, İŞKUR and the data the TAT could derive from the MIS. Referring to the above mentioned 7-steps of evaluation the data provided is obviously far from the anticipated as well as far from enough to make a comprehensive evaluation of the quality of the data.

The TAT was accordingly forced to change strategy and approach. The method used in this study is to omit step 4 and 5 from the evaluation. In addition, and due to the delays in the overall programme of the HRD OP also noted in the introduction step 6 can only be done within severe limitations, simply because the programme cannot be expected to have a real effect so soon after it is commenced.

The limitations impose obviously restrictions to the evaluation as well as its conclusions and recommendations.

However, the results of the evaluation of the indicators should not be underestimated. In addition to the technical approach to evaluation of the quality of the indicators, the TAT has launched a designated survey on the relevance of the Result indicators which in addition to qualitative interviews with prime stakeholders have added significantly to the outcome of the evaluation.

In particular, the TAT conducted a long half-day meeting with İŞKURstaff in which the relevance of carefully selected indicators was discussed. Further invitations to undertake the same sessions with notably the Central Grant Monitoring Teams (CGMTs) were however not doable neither through follow-up meetings nor through mail correspondence.

This study on indicators is divided into two main parts. The first part in Chapter 9 evaluates the Output indicators of which the TAT has counted 57 selected indicators from the HRD OP. The second part in Chapter 10 tries to understand the logic in the 27 selected Result Indicators from a theoretical perspective in an evaluation of the relevance of the chosen Result Indicators. The importance of the selection of Result Indicators is thoroughly described.

2.1 The explicit focus on Result Indicators

The stringent focus on Result Indicators in this study, which focus further is accentuated by the designated survey, needs to be explained.

Turning to the basic definition of output indicators (see section 8.2) the Output indicators measures the immediate and concrete consequences of the resources used and measures taken. The purpose of the output indicators is accordingly to measure “how many trainees have participated in a given project?” and “when we embarked upon the new law framework, did it come through?” In other words the output indicators specifically address issues of “where did we spend the resources and efforts” which indicates a very stringent focus on target groups (i.e. those who participated or in more technical terms; the final beneficiaries).

Output indicators are in general not that difficult to define since they appear almost natural from the definition of the measures, actions and target groups. Furthermore, output indicators are by nature almost always of quantitative nature, since the main purpose of output indicators is to measure the direct effect of the effort; that is numbers.

That means, evaluating the relevance of output indicators soon becomes trivial insofar the output indicators are closely linked to the target group and measure numbers benefitted. After all, relevance is a matter of how well the indicator makes sense subject to the measure, the objective and the priority, and since output indicators almost always make sense, assessing their relevance becomes a pretty trivial procedure.

139

Result Indicators are of quite different nature. Referring to the section 8.2, we need to define Result Indicators as a mix of two types of indicators; Outcome Indicators and Impact Indicators.

While the outcome indicators measures the results at the level of beneficiaries (for example, gross enrolment rates in primary schools, vaccination) and cover the use and satisfaction of the goods and services produced by the public sector, the Impact Indicators tries to measure the consequences of the outcomes in terms of wider objectives, and where the difference between output and outcome describes participants (beneficiaries) of the action in relation to the complete target group including those who did not participate.

The difference between Outcome and Impact is not arbitrary; while the outcome indicators try to describe the result for the complete target group in question, the impact indicators focus on the wider perspective for instance for the economy, the region, the health, etc.

Now, describing the difference between output and result using the mentioned definitions uncover at least one vital feature: Output measures the programme / project / action undertaken by the programme itself, while outcome and impact the importance of the programme / project / action from a broader perspective. It is the difference between local and global. And more so it tends to be the difference between simple monitoring through progress reports and evaluation of effect.

Further it appears evident that we need to know how our effort and actions, and perhaps not least the funding, have effect for the conditions of and within the society; it is of paramount importance in a modern democratic welfare state to make sure that our effort pays off. And in that respect there is a solid distinct difference in the degree of importance between output and result; we might find it amusing to invest in some few participants in a given training programme, but the crucial question will obviously be: What does the investment mean for our society, our economy, our welfare?

And hence the logic in this study’s clear focus on measures which effectively measure outcome and impact: Perhaps we can live with the fact that our monitoring of the number of participants provides a slightly incomplete picture of the resources invested, but we cannot live without knowing that what we do serves a purpose which reaches beyond the limited number of beneficiaries taking part. Obviously, it is of vital importance to ensure that the selected Result Indicators (to stay in the odd wordings of the HRD OP) are relevant to the objectives of the effort especially if we, for once, leave our bookkeeper’s glasses for a moment.

But it does not stop here. In addition to the measuring of the effect to a broader audience, so to speak, perhaps even including the whole community, as we move from Output via Outcome to Impact, we achieve to provide some deeper insight to the REAL effect; the effect which lasts (at least longer than the project period). In other words: we measure sustainability.

And exactly that is the core topic of this study.

140

2.2 The data received from İŞKUR

As noted several times, notably in the Introduction, the TAT has received data from İŞKUR33. The request for data followed a template in which the TAT noted the data needed which template allowed the main data providers to insert their comments to the availability of data. The response from İŞKUR is inserted below:

Table 1 Reply from İŞKUR on request for data on indicators of August 17, 2011Indicator Comments on availability

Number of registered employed by region (NUTS 2), gender and age

Registered Number of Employees by age group does not exist.

Number of registered unemployed by region (NUTS 2), gender and age

available

Number of employed through the courses and counselling, by region (NUTS 2) and gender

As a result of drilling through courses and work placements are available.

Number of entrepreneurs by region (NUTS 2) and gender

Public-private distinction is the number of establishments.

Number of participants entrepreneurship courses by region (NUTS 2) and gender

Years prior to 2009 are not available

Number of women having participated in entrepreneurship courses for women by region (NUTS 2)

Not Available

Number of women having participated in child/elderly care trainings by region (NUTS 2)

Not Available

Number of participants in employment oriented courses by region (NUTS 2) and gender

Available

Number of women having participated in employment oriented courses designated for women exclusively

Not Available

Number of persons having benefited from guidance and counselling services by region (NUTS 2) and gender

Available

Rate of young people employed following the courses, internship and counselling activities by region (NUTS 2) and gender

As a result of drilling through courses and work placements are available.

Rate of young entrepreneurs by region (NUTS 2) and gender

Not Available

Number of young people who participated in entrepreneurship courses

Not Available

Number of young people who participated in employment oriented courses

Not Available

Number of young people who participated in internship and apprenticeship programmes

Not Available

Number of young people having benefited from guidance and counselling

Not Available

Number of unemployed people counselled by region (NUTS 2) and gender

Guidance and Counselling Services are available the number of beneficiaries.

Placement rates of İŞKUR by region (NUTS 2) and gender

Employment Placement on the employment rate was calculated during the year. Work placements during the year occurred in previous years an open circuit is applied to the works of over 100% placement rate in some cases seem to work. In addition, Public Works since 2006 by sex derlenememektedir statistical information in question was removed.

Region (NUTS 2) and sex-specific increase in the Total job placements Ratio is calculated in the frame

33 Besides official statistics supplied from YUIK/Turkstat and data generated from the MIS, İŞKUR was the only provider of data for the studies in this report. We did not receive data from the other Operation Beneficiaries despite our continuous requests.

141

Indicator Comments on availabilityratio of people who work completed courses in employability

of the course. No information was available before 2007.

Percentage of participants who obtained a certificate in basic skills by region (NUTS 2) and gender

No information was available before 2008.

Number of people who participated in employability courses provided by networks, by region (NUTS 2) and gender

Not Available

Number of training activities on basic skills and further development of basic skills for the adults particularly for women

See Courses

Number of training activities on basic skills for adults by gender and by region

See Courses

Number of consultancy services on the improvement of the adaptability of employers and employees for social partners, universities, public and private VET institutions, NGOs, public bodies and enterprises by region

Numerical information is provided by the seminar activities at workplaces. The information compiled since 2011 January.

Increased rate of disadvantaged persons entered into labour market (having the qualification and motivation to seek a job) following the rehabilitation programmes, courses and counselling.

Not Available

Increased rate of disadvantaged persons employed following the courses and counselling services

Placement into employment rate of disabled people in the course of total was calculated.

Number of people having participated in guidance and counselling services


Number of people having benefited from rehabilitation programmes


Number of people having participated in courses in scope of ALMPs

Available

Number of people who are reached through the awareness raising activities

?

Region (NUTS 2) according to gender and the central database, monitoring and analysis system recorded the number of disadvantaged people

Available

142

PART II: Output Indicators

II.1 Priority axis 1

To attract and retain more people in employment, particularly by increasing labour force participation of women, and decrease unemployment rates especially for young people

1.1 Measure 1.1

To promote women’s participation into the labour market, and increase female employment, including those formerly employed in agriculture

Specific Objectives:• To improve employability of women• To support women entrepreneurship• To diminish cultural and other obstacles that hinder women’s participation in labour market

Table 1.1 Output indicators on Measure 1.1, Priority axis 1

Count Indicator Baseline (2006) Target (2009)

Target (2011) Source of Verification

1Number of women participated in entrepreneurship courses for women

2751 women (1781 people by İŞKUR and 970 people by KOSGEB) in NUTS II regions

4000 8000MoLSS Programme Reports, Final Beneficiary Reports

2Number of women participated in child / elderly care trainings

N.A. (Data will be collected when the measure implemented)


3Number of women who participated in employment oriented courses for women

Total 25813 women (for these 2 indicators there is just one integrated baseline. Women participated in employment guaranteed courses, guidance and counselling in NUTS II region for 3 years)


4

Number of women benefited from guidance and counselling services

18000 24000İŞKUR Records, MoLSS Programme Reports, Final Beneficiary Reports

Source: Human Resource Development Operational Programme, second version

As mentioned above the output indicators are basically designed to measure progress of the actions. In that respect the TAT has found and received the following data from the two different sources available for the TAT as displayed in the following two tables.

143

Table 1.2 Output indicators on Measure 1.1, Priority axis 1 as found in SAR34

Count Indicators Baseline Target Expected contributionHRD OP OIS

1

Number of women participated in entrepreneurship courses for women

Number of women participated in entrepreneurship courses

2751 women ( 1781 people by İŞKUR and 970 people by KOSGEB) in NUTS II regions

4000(appr. 2600 in OIS)

2485

PYE: 1803

2Number of women participated in child / elderly care trainings

Number of women participated in child/elderly care training

N.A. (Data will be collected when the measure implemented)

1200 (approx. 780in OIS)

1030

3

Number of women who participated in employment oriented courses for women

Number of women participated in employment guaranteed courses for women

Total 25813 women (for these 2 indicators there is just one integrated baseline. Women participated in employment guaranteed courses, guidance and counselling in NUTS II region for 3 years)

12000 (approx. 7,800 in OIS)

1947

PYE: 3068

4

Number of women benefited from guidance and counselling services

Number of women benefited from guidance and counselling service

18000 (approx. 11,700 in OIS)

2612

PYE: 11864

Revised: Number of analyses activities of the grant scheme results including the main lessons learned

0 1

Revised: Number of compendiums published for successful granted projects and results

0 1

Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010, Operating Structure, MoLSS, dated June 2011

Table 1.3 Output indicators on Measure 1.1, Priority axis 1 as found in MISRef. Indicator Target Achieved

1 Number of women participated in entrepreneurship courses for women 4.352 2.739

2 Number of women participated in child / elderly care trainings 1.686 1.184

3 Number of women who participated in employment oriented courses for women 3.958 2.209

4 Number of women benefited from guidance and counselling services 4.877 2.526

Source: Monitoring Information System, Operating Structure, MoLSS

34 The SAR is dated June 2011 indicating the situation by end 2010. More updated data was received unfortunately far too late to be incorporated in this study (data of September 2011).

144

1.1.1 Output Indicator no. 1

Table 1.1.1 shows data provided by İŞKUR on Indicator 1.

Table 1.1.1 Output indicators on Indicator 1, Measure 1.1, Priority axis 1 as recorded by İŞKUR: Number of participants in entrepreneurship courses

2011 (January-July)

2010

CODE GEO MALE FEMALE MALE FEMALETR10 İstanbul 1258 926 857 657TR21 Tekirdağ, Edirne, Kırklareli 206 168 161 165TR22 Balıkesir, Çanakkale 189 224 62 112TR31 İzmir 60 75 26 24TR32 Aydın, Denizli, Muğla 834 845 324 272TR33 Manisa, Afyon, Kütahya, Uşak 362 316 115 84TR41 Bursa, Eskişehir, Bilecik 268 246 245 246TR42 Kocaeli, Sakarya, Düzce, Bolu, Yalova 504 412 182 132TR51 Ankara 601 545 66 59TR52 Konya, Karaman 207 183 49 51TR61 Antalya, Isparta, Burdur 315 300 109 116TR62 Adana, Mersin 327 245 178 122TR63 Hatay, Kahramanmaraş, Osmaniye 385 281 367 192TR71 Kırıkkale, Aksaray, Niğde, Nevşehir, Kırşehir 235 196 159 112TR72 Kayseri, Sivas, Yozgat 420 334 197 122TR81 Zonguldak, Karabük, Bartın 194 172 93 57TR82 Kastamonu, Çankırı,Sinop 133 84 29 21TR83 Samsun, Tokat, Çorum, Amasya 536 429 423 344TR90 Trabzon, Ordu, Giresun, Rize, Artvin, Gümüşhane 534 356 257 211TRA1 Erzurum, Erzincan, Bayburt 143 98 71 49TRA2 Ağrı, Kars, Iğdır, Ardahan 328 186 167 80TRB1 Malatya, Elazığ, Bingöl, Tunceli 378 210 120 103TRB2 Van, Muş, Bitlis, Hakkari 883 217 177 72TRC1 Gaziantep, Adıyaman, Kilis 62 63 119 100TRC2 Şanlıurfa, Diyarbakır 0 0 0 0TRC3 Mardin, Batman, Şırnak, Siirt 54 43 154 96TR Total TURKEY 9416 7154 4707 3599Source: Data provided by İŞKUR

As noted by İŞKUR (see table 1 above) data are only available from 2009 and onwards, which generally makes the indicators highly questionable, since we have no baseline. Where the baseline of 2751 women (1781 people by İŞKUR and 970 people by KOSGEB) in NUTS II regions as noted in table 1.1.1 comes from appears to be a mystery if İŞKUR does not have the data. But again, it indicate that entrepreneurship courses may have been run before 2009, perhaps even before 2006.

Plausible as it may be the point is of course, that without a trustworthy baseline it appears non-sense to establish a mechanism for monitoring since we cannot investigate progress.

Further, we note that the İŞKUR data shows participants on entrepreneurship courses for both male and female (and not female alone) which tends to indicate that entrepreneurship courses are not necessarily designated to women. Table 1.1.2 further elaborates on the data as the table shows the Female participants in entrepreneurship courses in percentage of total participants in entrepreneurship courses by region and for total Turkey.

145

Table 1.1.2 Female participants in entrepreneurship courses in percentage of total participants in entrepreneurship courses by region and for total Turkey, 2010 and 2011 (January to July)

Female participation in pct of total

participants by region

Female participation in pct of total

participants in Turkey

CODE GEO 2011 2010 2011 2010TR10 İstanbul 42.4% 43.4% 5.6% 7.9%TR21 Tekirdağ, Edirne, Kırklareli 44.9% 50.6% 1.0% 2.0%TR22 Balıkesir, Çanakkale 54.2% 64.4% 1.4% 1.3%TR31 İzmir 55.6% 48.0% 0.5% 0.3%TR32 Aydın, Denizli, Muğla 50.3% 45.6% 5.1% 3.3%TR33 Manisa, Afyon, Kütahya, Uşak 46.6% 42.2% 1.9% 1.0%TR41 Bursa, Eskişehir, Bilecik 47.9% 50.1% 1.5% 3.0%TR42 Kocaeli, Sakarya, Düzce, Bolu, Yalova 45.0% 42.0% 2.5% 1.6%TR51 Ankara 47.6% 47.2% 3.3% 0.7%TR52 Konya, Karaman 46.9% 51.0% 1.1% 0.6%TR61 Antalya, Isparta, Burdur 48.8% 51.6% 1.8% 1.4%TR62 Adana, Mersin 42.8% 40.7% 1.5% 1.5%TR63 Hatay, Kahramanmaraş, Osmaniye 42.2% 34.3% 1.7% 2.3%TR71 Kırıkkale, Aksaray, Niğde, Nevşehir, Kırşehir 45.5% 41.3% 1.2% 1.3%TR72 Kayseri, Sivas, Yozgat 44.3% 38.2% 2.0% 1.5%TR81 Zonguldak, Karabük, Bartın 47.0% 38.0% 1.0% 0.7%TR82 Kastamonu, Çankırı,Sinop 38.7% 42.0% 0.5% 0.3%TR83 Samsun, Tokat, Çorum, Amasya 44.5% 44.9% 2.6% 4.1%TR90 Trabzon, Ordu, Giresun, Rize, Artvin, Gümüşhane 40.0% 45.1% 2.1% 2.5%TRA1 Erzurum, Erzincan, Bayburt 40.7% 40.8% 0.6% 0.6%TRA2 Ağrı, Kars, Iğdır, Ardahan 36.2% 32.4% 1.1% 1.0%TRB1 Malatya, Elazığ, Bingöl, Tunceli 35.7% 46.2% 1.3% 1.2%TRB2 Van, Muş, Bitlis, Hakkari 19.7% 28.9% 1.3% 0.9%TRC1 Gaziantep, Adıyaman, Kilis 50.4% 45.7% 0.4% 1.2%TRC2 Şanlıurfa, Diyarbakır - - 0.0% 0.0%TRC3 Mardin, Batman, Şırnak, Siirt 44.3% 38.4% 0.3% 1.2%TR TURKEY 43.2% 43.3% 43.2% 43.3%Source: Own calculation based upon data provided by İŞKUR

Again, we are stunned by doubt; the total number of female participants in total Turkey only accounts for 43 per cent of total participants. Furthermore, while participants in entrepreneurship course in the 12 NUTS 2 regions participating in the HRD OP only constitute 41.8 per cent of total participants in these courses, the fraction of women taken part in the courses in the 12 NUTS 2 regions is 40 per cent (decreasing to 37 per cent in 2011). One can only wonder what the HRD OP was designed to do, since it appears not clear from looking at the indicator.

Of course, the data from İŞKUR shows all entrepreneurship courses, including those courses which are run by İŞKUR. In that respect we turn to the Monitoring Information System, which is designed to follow up on the activities within the programme in table 1.1.2 which clearly states that the HRD OP has achieved to make entrepreneurship courses for a total of 2.739 women so far (the target be 4.352). Returning then to the data provided by İŞKUR we note that a total of 3691 women have participated in entrepreneurship courses from January 2010.

The HRD OP should accordingly have achieved to produce a total of 74 per cent of total women in training courses on entrepreneurship. Does that appear to be sound? Again, the problem is that we have no data from 2006. And yet, we have; the HRD OP mentions that a baseline in 2006 before the HRD OP indicates that a total of 2751 women has been trained in 2006 alone, of which 1781 was trained by İŞKUR.

146

Thus, the facts are: The HRD OP has achieved to train 2.739 women İŞKUR has recorded that 3691 women has been trained so far in 2010 and 2011 Accordingly, the HRD OP has achieved to do 74 per cent of all trained women in

entrepreneurship However, İŞKUR and KOSGEB, before the HRD OP started, trained 2751 women in 2006

alone (of which İŞKUR trained 1781) Assuming that 1781 trained women is the annual average from İŞKUR we would assume that

İŞKUR could train 2671 women in 1.5 years

And the questions to the fact are subsequently: Does that imply that due to the HRD OP İŞKUR has achieved to increase the number of

female participants since 2006 by 1020 participants in 1.5 years (or 680 a year)? Is it plausible to suggest that the increase in number of female participants in entrepreneurship

courses would never have happened in case HRD OP was never launched? Why does the number of participants in the HRD OP selected and HRD OP supported NUTS

2 Regions only constitute 41 per cent of total participants in entrepreneurship courses in Turkey in 2010-201135?

Why does the number of female participants in entrepreneurship courses within the 12 NUTS 2 Regions only constitute 40 per cent in 2010 (seeming to decrease to 37 per cent in 2011) after the implementation of the HRD OP?

The answers to these questions “blow in the wind” for the Interim Evaluation TAT indicating a major challenge to convince the stakeholders about the effect of the programme.

The basic conclusion is that it is extremely difficult to operate with indicators where the baselines can be doubted. The Interim Evaluation, however, understands the logic of the output indicator and its direct link to the purpose (the priority, the measures, and the specific objective (which has not been subject to further theoretical investigation in this context by the way for the same reason)). Thus, the Interim Evaluation has no further comments to the indicator per se, though the Interim Evaluation strongly recommend to find ways and means to make the indicator more forceful and clean through a strategy of QA in raw data; we simply need to know what we are measuring, and as noted in the, agreed, critical and sceptical evaluation presented here, the indicators just raise questions which normally indicates a poor indicator.

Preliminary recommendation36: Clean the indicator through measure of QA in data collection. Define its focus (what it appear to measure) and create a doable baseline somewhere which appears convincing (the present baseline is no way convincing as mentioned).


With respect to Output Indicator 2 İŞKUR announces that no data is available (see table 1) which announcement is repeated in the HRD OP. Accordingly, the indicator cannot be evaluated from a quantitative perspective.

35 To be fair, the population of the 12 NUTS 2 Regions only constitute 33.6 per cent of total population in Turkey in 2010. But still we do not know the fraction of participants in training courses in 2006.36 Throughout this section we use preliminary recommendations only since additional studies of the indicators from a more overall viewpoint follows below. The Chapter will conclude by stating final recommendations.

147

Nevertheless, it appears slightly difficult to understand the indicator and from where it originates as the priority, the measures, and the specific objective never address a certain political focus on driving women into childcare and elderly care. It further sounds strange that the labour markets of the 12 NUTS II Regions should have address a specific or rather exclusive demand for women in childcare and elderly care; why not men? And by the way, how does that indicator align with the overall horizontal issue of equal opportunities?

It is not just from a sociological and ethical point of view the indicator sounds strange; from a theoretical and methodological point it appears very strange that the HRD OP suddenly addresses a specific sector and links that specific sector to a certain target group. None of the other indicators address a specific economic sector.

The point is: It might be justified to start projects within childcare and elderly care, and perhaps it is even just and fair if the majority of participants are women; after all the priority specifically mentions the gender issue and take note on the employability of women in particular. However, when we establish indicators, even output indicators, we would prefer to take a certain objective view not allowing the indicator to be driving force for pushing the actions, priorities, objectives and measures into a certain direction which otherwise was not intended from an overall perspective. In case we have a political agenda the political agenda should be clear and explicitly be addressed by the priority, that is.

Otherwise, the interim evaluation has no further comments at this point; the basics will be subject to further comments below.

Preliminary recommendation: The Interim Evaluation recommends removing indicators which appear to contradict the horizontal principles, and tend to distort or create biases onto the labour market without any reason. It is highly recommended to assure objective Indicators.


Tables 1.1.6-8 shows Participants in employment oriented training courses by gender by region and for total Turkey for selected years between 2006 and 2010, and 2011 (January to July) in accordance to data provided by İŞKUR. Table 8.1.1.6 clearly indicates the growth in the number of employment oriented training courses from 2006 to present.

148

Table 1.1.6 Participants in employment oriented training courses by gender by region and for total Turkey, selected years between 2006 and 2010 and 2011 (January to July)

2011(Ocak-Temmuz)

2010 2009 2006

NUTS CODE GEO MALE FEMALE MALE FEMALE MALE FEMALE MALE FEMALEN2 TR10 İstanbul 2426 2209 5031 5144 2471 2447N2 TR21 Tekirdağ, Edirne, Kırklareli 204 366 661 839 276 335N2 TR22 Balıkesir, Çanakkale 214 423 366 472 110 100N2 TR31 İzmir 284 479 1124 1008 348 320N2 TR32 Aydın, Denizli, Muğla 483 836 1023 825 174 157 52 29N2 TR33 Manisa, Afyon, Kütahya, Uşak 456 839 422 1349 503 699 1 11N2 TR41 Bursa, Eskişehir, Bilecik 1466 539 803 986 189 201 272 364N2 TR42 Kocaeli, Sakarya, Düzce, Bolu, Yalova 968 1147 616 791 192 152N2 TR51 Ankara 419 626 532 915 453 534N2 TR52 Konya, Karaman 188 537 604 1153 271 296N2 TR61 Antalya, Isparta, Burdur 496 512 452 285 1471 876 0 30N2 TR62 Adana, Mersin 1654 1449 1347 537 457 216 0 25N2 TR63 Hatay, Kahramanmaraş, Osmaniye 564 308 571 660 388 290N2 TR71 Kırıkkale, Aksaray, Niğde, Nevşehir,

Kırşehir432 820 691 886 504 883

N2 TR72 Kayseri, Sivas, Yozgat 354 258 121 178N2 TR81 Zonguldak, Karabük, Bartın 124 161 432 313 32 72N2 TR82 Kastamonu, Çankırı,Sinop 89 134 163 185 37 42N2 TR83 Samsun, Tokat, Çorum, Amasya 645 1588 1144 2279 376 725 23 0N2 TR90 Trabzon, Ordu, Giresun, Rize, Artvin,

Gümüşhane407 845 548 904 388 634

N2 TRA1 Erzurum, Erzincan, Bayburt 289 348 590 604 117 153N2 TRA2 Ağrı, Kars, Iğdır, Ardahan 77 74 356 371 82 208N2 TRB1 Malatya, Elazığ, Bingöl, Tunceli 337 224 873 479 906 883N2 TRB2 Van, Muş, Bitlis, Hakkari 651 724 359 336 225 254N2 TRC1 Gaziantep, Adıyaman, Kilis 842 796 681 138 460 52N2 TRC2 Şanlıurfa, Diyarbakır 427 655 210 191 208 102N2 TRC3 Mardin, Batman, Şırnak, Siirt 402 416 450 449 145 194N0 TR TURKEY 14898 17313 20170 22277 10783 10825 348 459Source: İŞKUR

149

Table 1.1.7 Participation in employment oriented training courses by gender by region and for total Turkey, selected years between 2006 and 2010 and 2011 (January to July): male and female fractions in percentage of total

2011 (Ocak-Temmuz)

2010 2009 2006

CODE GEO MALE FEMALE MALE FEMALE MALE FEMALE MALE FEMALETR10 İstanbul 52.3% 47.7% 49.4% 50.6% 50.2% 49.8% - -TR21 Tekirdağ, Edirne, Kırklareli 35.8% 64.2% 44.1% 55.9% 45.2% 54.8% - -TR22 Balıkesir, Çanakkale 33.6% 66.4% 43.7% 56.3% 52.4% 47.6% - -TR31 İzmir 37.2% 62.8% 52.7% 47.3% 52.1% 47.9% - -TR32 Aydın, Denizli, Muğla 36.6% 63.4% 55.4% 44.6% 52.6% 47.4% 64.2% 35.8%TR33 Manisa, Afyon, Kütahya, Uşak 35.2% 64.8% 23.8% 76.2% 41.8% 58.2% 8.3% 91.7%TR41 Bursa, Eskişehir, Bilecik 73.1% 26.9% 44.9% 55.1% 48.5% 51.5% 42.8% 57.2%TR42 Kocaeli, Sakarya, Düzce, Bolu, Yalova 45.8% 54.2% 43.8% 56.2% 55.8% 44.2% - -TR51 Ankara 40.1% 59.9% 36.8% 63.2% 45.9% 54.1% - -TR52 Konya, Karaman 25.9% 74.1% 34.4% 65.6% 47.8% 52.2% - -TR61 Antalya, Isparta, Burdur 49.2% 50.8% 61.3% 38.7% 62.7% 37.3% 0.0% 100.0%TR62 Adana, Mersin 53.3% 46.7% 71.5% 28.5% 67.9% 32.1% 0.0% 100.0%TR63 Hatay, Kahramanmaraş, Osmaniye 64.7% 35.3% 46.4% 53.6% 57.2% 42.8% - -TR71 Kırıkkale, Aksaray, Niğde, Nevşehir, Kırşehir 34.5% 65.5% 43.8% 56.2% 36.3% 63.7% - -TR72 Kayseri, Sivas, Yozgat 57.8% 42.2% 40.5% 59.5% - - - -TR81 Zonguldak, Karabük, Bartın 43.5% 56.5% 58.0% 42.0% 30.8% 69.2% - -TR82 Kastamonu, Çankırı,Sinop 39.9% 60.1% 46.8% 53.2% 46.8% 53.2% - -TR83 Samsun, Tokat, Çorum, Amasya 28.9% 71.1% 33.4% 66.6% 34.2% 65.8% 100.0% 0.0%TR90 Trabzon, Ordu, Giresun, Rize, Artvin, Gümüşhane 32.5% 67.5% 37.7% 62.3% 38.0% 62.0% - -TRA1 Erzurum, Erzincan, Bayburt 45.4% 54.6% 49.4% 50.6% 43.3% 56.7% - -TRA2 Ağrı, Kars, Iğdır, Ardahan 51.0% 49.0% 49.0% 51.0% 28.3% 71.7% - -TRB1 Malatya, Elazığ, Bingöl, Tunceli 60.1% 39.9% 64.6% 35.4% 50.6% 49.4% - -TRB2 Van, Muş, Bitlis, Hakkari 47.3% 52.7% 51.7% 48.3% 47.0% 53.0% - -TRC1 Gaziantep, Adıyaman, Kilis 51.4% 48.6% 83.2% 16.8% 89.8% 10.2% - -TRC2 Şanlıurfa, Diyarbakır 39.5% 60.5% 52.4% 47.6% 67.1% 32.9% - -TRC3 Mardin, Batman, Şırnak, Siirt 49.1% 50.9% 50.1% 49.9% 42.8% 57.2% - -TR TURKEY 46.3% 53.7% 47.5% 52.5% 49.9% 50.1% 43.1% 56.9%Source: İŞKUR and own calculations

150

Table 1.1.8 Participation in employment oriented training courses by gender by region and for total Turkey, selected years between 2006 and 2010 and 2011 (January to July): fractions of regional distribution of total training course (pct.)

2011 (Ocak-Temmuz)

2010 2009 2006

CODE GEO MALE FEMALE MALE FEMALE MALE FEMALE MALE FEMALETR10 İstanbul 16.3% 12.8% 24.9% 23.1% 22.9% 22.6% 0.0% 0.0%TR21 Tekirdağ, Edirne, Kırklareli 1.4% 2.1% 3.3% 3.8% 2.6% 3.1% 0.0% 0.0%TR22 Balıkesir, Çanakkale 1.4% 2.4% 1.8% 2.1% 1.0% 0.9% 0.0% 0.0%TR31 İzmir 1.9% 2.8% 5.6% 4.5% 3.2% 3.0% 0.0% 0.0%TR32 Aydın, Denizli, Muğla 3.2% 4.8% 5.1% 3.7% 1.6% 1.5% 14.9% 6.3%TR33 Manisa, Afyon, Kütahya, Uşak 3.1% 4.8% 2.1% 6.1% 4.7% 6.5% 0.3% 2.4%TR41 Bursa, Eskişehir, Bilecik 9.8% 3.1% 4.0% 4.4% 1.8% 1.9% 78.2% 79.3%TR42 Kocaeli, Sakarya, Düzce, Bolu, Yalova 6.5% 6.6% 3.1% 3.6% 1.8% 1.4% 0.0% 0.0%TR51 Ankara 2.8% 3.6% 2.6% 4.1% 4.2% 4.9% 0.0% 0.0%TR52 Konya, Karaman 1.3% 3.1% 3.0% 5.2% 2.5% 2.7% 0.0% 0.0%TR61 Antalya, Isparta, Burdur 3.3% 3.0% 2.2% 1.3% 13.6% 8.1% 0.0% 6.5%TR62 Adana, Mersin 11.1% 8.4% 6.7% 2.4% 4.2% 2.0% 0.0% 5.4%TR63 Hatay, Kahramanmaraş, Osmaniye 3.8% 1.8% 2.8% 3.0% 3.6% 2.7% 0.0% 0.0%TR71 Kırıkkale, Aksaray, Niğde, Nevşehir, Kırşehir 2.9% 4.7% 3.4% 4.0% 4.7% 8.2% 0.0% 0.0%TR72 Kayseri, Sivas, Yozgat 2.4% 1.5% 0.6% 0.8% 0.0% 0.0% 0.0% 0.0%TR81 Zonguldak, Karabük, Bartın 0.8% 0.9% 2.1% 1.4% 0.3% 0.7% 0.0% 0.0%TR82 Kastamonu, Çankırı,Sinop 0.6% 0.8% 0.8% 0.8% 0.3% 0.4% 0.0% 0.0%TR83 Samsun, Tokat, Çorum, Amasya 4.3% 9.2% 5.7% 10.2% 3.5% 6.7% 6.6% 0.0%TR90 Trabzon, Ordu, Giresun, Rize, Artvin, Gümüşhane 2.7% 4.9% 2.7% 4.1% 3.6% 5.9% 0.0% 0.0%TRA1 Erzurum, Erzincan, Bayburt 1.9% 2.0% 2.9% 2.7% 1.1% 1.4% 0.0% 0.0%TRA2 Ağrı, Kars, Iğdır, Ardahan 0.5% 0.4% 1.8% 1.7% 0.8% 1.9% 0.0% 0.0%TRB1 Malatya, Elazığ, Bingöl, Tunceli 2.3% 1.3% 4.3% 2.2% 8.4% 8.2% 0.0% 0.0%TRB2 Van, Muş, Bitlis, Hakkari 4.4% 4.2% 1.8% 1.5% 2.1% 2.3% 0.0% 0.0%TRC1 Gaziantep, Adıyaman, Kilis 5.7% 4.6% 3.4% 0.6% 4.3% 0.5% 0.0% 0.0%TRC2 Şanlıurfa, Diyarbakır 2.9% 3.8% 1.0% 0.9% 1.9% 0.9% 0.0% 0.0%TRC3 Mardin, Batman, Şırnak, Siirt 2.7% 2.4% 2.2% 2.0% 1.3% 1.8% 0.0% 0.0%TR TURKEY 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%Source: İŞKUR and own calculations

151

Table 1.1.9 Share and fraction on Participation in employment oriented training courses by gender by region and for total Turkey, between 2006 and 2010 and 2011 (January to July): Share of total participants in HRD OP regions (total and female), fraction of female in HRD OP Regions, Share of total participants in regions not taking part in HRD OP and the fraction of female participants in these non-HRD OP Regions (pct.)

SHARE SHARE FRACTION SHARE FRACTIONTotal HRD/total Fem/total NHRD/total Fem/total

TOTAL TR Female HRD OP R Female Non-HRD R2006 2.9% 0.0% 0.0% 100.0% 58.5%2007 9.1% 15.7% 50.8% 84.3% 27.3%2008 36.4% 43.9% 44.9% 56.1% 32.9%2009 31.8% 32.7% 51.5% 67.3% 49.4%2010 30.2% 30.4% 52.8% 69.6% 52.4%2011 35.6% 36.8% 55.6% 63.2% 52.7%

Source: İŞKUR and own calculations

Table 1.1.9 summarizes the study which can be derived from the three tables before. The first column shows the share of reported participants in employment oriented training courses from HRD OP NUTS 2 regions relative to total Turkey. The second column shows the same for all female participants. The third column presents the fraction of female participated in employment oriented training courses from HRD OP NUTS 2 regions in percentage of total participants (same regions, of course). The two last columns repeat the same as the two middle columns; just this time for regions not taking part in the HRD OP.

The table provide reason to do some thoughts. We have already noted the sharp increase in the number of participants in employment oriented training courses, but somehow the fraction of participants does not really show to be directed to the regions taking part in the HRD OP. The share of total participants in all employment oriented training courses of Turkey appears to be in the neighbourhood of 30-35 per cent.

If we look at the number of female participants of HRD OP oriented NUTS 2 regions relative to total female participants in employment oriented training courses there appear to be no difference. The exception is 2008 which is before the start of the HRD OP projects. Accordingly, it appears slightly difficult to see the direct effect of the HRD OP, which notably focuses not just on female participation in employment oriented training courses, but also tries to stimulate employment oriented training courses specially designated for women. If the HRD OP should reinforced employment oriented training courses specially designated for women we would have expected to see a higher fraction of female participants in HRD OP oriented regions.

The only real positive direct effect is seen in column 3 compared to column 5; it appears that the fraction of females taking part in employment oriented training courses are slightly higher in HRD OP oriented regions. However, that effect stops by the commencement of the HRD OP.

These observations served a purpose, and the purpose was not the observations derived from studying the tables. The point was to show how easy it is to make good analysis when presented for reliable data as well as an indicative indicator. Of course we could question the effect of the HRD OP on this basis, but first of all the data was not based upon HRD OP induced projects but both HRD OP projects and initiatives originated by İŞKUR, and furthermore we have to admit that the HRD OP needs time to provide effect, including the direct effect which output indicators are designed to measure.

152

Accordingly, the conclusion is that the indicator, provided that the monitoring unit finds ways to ensure similarly solid data as provided by İŞKUR in this case, is an effective indicator for monitoring purposes. Again, however, relating the data from İŞKUR with the data from MIS presented in table 1.1.2 creates multiple questions which we do not need to repeat here. And true, the data from İŞKUR do NOT just show Number of women having participated in employment oriented training courses for women, but Number of women having participated in employment oriented training courses for both men and women. However, and still, the data provided by İŞKUR are useful, indicating that in case the Operating Structure with help from İŞKUR finds ways to clean the data from 2006 and onwards, a solid indicator is created.

Preliminary recommendation: The Interim Evaluation recommends keeping the indicator on female participation on employment oriented training courses provided that the data are validated, consistent, and based upon solid QA measures of control.


Output indicator no. 4 concerns “Number of women benefited from guidance and counselling services“. Again data is provided by İŞKUR which institution is the true bearer of the responsibility of providing documentation. The study of the indicator is based upon the same technique as on indicator 3 and the conclusion from indicator 3 remains the same. Accordingly, the Interim Evaluation Report finds no reason to repeat the long documentation on this indicator as well37.

Preliminary recommendation: The Interim Evaluation recommends keeping the indicator on Number of women benefited from guidance and counselling services provided that the data are validated, consistent, and based upon solid QA measures of control. Again the additional would be to ensure data on both genders as well as obtain data from other Turkish regions in order to ensure control groups.

1.2 Measure 1.2To increase youth employment

Specific Objective:• To promote integration of young people into the labour market

Table 1.2.1 Output indicators on Measure 1.2, Priority axis 1



5Number of young people who participated in entrepreneurship courses

970 people in NUTS II regions


6Number of young people who participated in employment oriented courses

Total 8562 young people (for these 3 indicators there is just one integrated baseline. young people participated in employment


7 Number of young people who participated in internship and apprenticeship programs

9000 12000 MoLSS Programme Reports, Final Beneficiary Reports

8 Number of young people 9000 12000 MoLSS Programme

37 Basically, it is not the job of the Interim Evaluation Report to make the progress reports. In this case, and not least due to save both time and paper, the Interim Evaluation Report refers to the techniques used in sub-section 8.2.1.3. The data obtained from İŞKUR should already be in the hands of the Monitoring and Evaluation Unit of MoLSS; otherwise the author of this report will happily submit the data in electronic form.

153



having benefited from guidance and counselling guaranteed

courses, internship programmes and

Reports, Final Beneficiary Reports


Table 1.2.2 Output indicators on Measure 1.2, Priority axis 1 as found in SARCount Indicators Baseline Target Expected

contributionHRD OP OIS

5



970 people in NUTS II regions

At least 1500 1803

PWE: 2489

6

Number of young people who participated in employment guaranteed courses

Number of young people who participated in internship, employment guaranteed courses and apprenticeship programs

Total 8562 young people (for these 3 indicators there is just one integrated baseline, young people participated in employment guaranteed courses, internship programmes and guidance and counseling)

6000 3068

PWE: 1947

7

Number of young people who participated in internship and apprenticeship programs

9000 2410

8



9000 11864

PWE: 2612


Table 1.2.3 Output indicators on Measure 1.2, Priority axis 1 as found in MIS Ref. Indicator Target Achieved

5 Number of young people who participated in entrepreneurship courses 4.944 2.311

6 Number of young people who participated in employment guaranteed courses 5.950 2.310

7 Number of young people who participated in internship and apprenticeship programs 5.023 1.426

8 Number of young people having benefited from guidance and counselling 17.130 2.480



The first output indicator under Priority Axis 1, specific objective 2 is Number of young people who participated in entrepreneurship courses. As noted above no data are available for this indicator. Nonetheless, the HRD OP stipulates a target of 1500 in 2009 increasing to 2500 in 2011 from a baseline of 900 in 2006. Where the HRD OP is able to obtain the data from remains a mystery38.

38 In general the HRD OP never names its source which is a disqualifying feature in all public reports. Similarly the use of self-invented acronyms without a glossary list is very common in the supporting documents, which does not make the documents easier to read; sometimes the reader simply has to make an attempt to guess. The best guess of the meaning of PWE as mentioned in table 1.2.3 is “Promoting Women’s Employment” as far as the TAT can make it.

154

The MIS on the other hand announces a target of 4944 and stipulates that so far 2311 young people has benefitted from entrepreneurship courses. We have to assume that the data from MIS is based upon data reported directly from the participating projects.

Again something appears slightly odd; while the HRD OP announces a target of 2500 in 2011 the SAR reports a target of at least 1500 and yet the MIS announce that so far 2311 has participated under the programme in the pursuit of a target of 4944. Somehow these figures do not match and it tends to be difficult to see what the HRD OP actually is heading for.

Preliminary recommendation: There is basically no theoretical or logical objection towards the indicator insofar consistent data is available. It is highly recommended to find ways to agree on the target (as well as the achievements). An additional note could be that the indicator clearly matches indicator 1 under specific objective 1 studied above; it makes sense to have indicators which are fruitful for multiple purposes. Further, it would again be recommendable to ensure valuable control groups to see the net effect of the measure.

1.2.2 Output Indicator no. 6, 7, and 8

Again, we have no raw data on: Number of young people who participated in employment guaranteed courses (indicator no. 6) Number of young people who participated in internship and apprenticeship programs

(indicator no. 7) Number of young people having benefited from guidance and counselling (indicator no. 8)

With respect to targets and baselines the documents appear to agree with each other and with MIS. However, it is generally not recommendable to use complex or aggregated measures as baselines; here the baseline aggregates the three named indicators.

The point is that aggregated baselines make it impossible to see progress since the commencement of the HRD OP, which basically disqualifies the baseline since it serves no point.

Another point from a practically perspective is of course, that it appears to fairly difficult to separate the three actions from each other in the process; normally we would invite potential participants to employment guaranteed courses or, and notably, internship and apprenticeship programs, into an interview before we assign them to the measures as part of the visitation process. And often that interview has close links to what we would define as “guidance and counselling”, since that is simply what it is when we try to persuade clients into something which they hopefully would benefit from. And sometimes it could even easily be difficult to distinguish clearly between employment guaranteed courses and internship and apprenticeship programmes; most times both are present in the same action.Accordingly, we cannot even be sure that the same participants are not counted twice; at least it would be difficult for İŞKUR to separate the three in their daily work, notably if they experience drop-out from one of the two former actions39. But more so, it is generally not particular fair and just to separate the counting into three brackets since they are mutually related to an obvious extent.

Preliminary recommendations:

39 Please note here, that we assume that the Monitoring and Evaluation Unit understand the importance to have control groups in order to investigate the net effect of the HRD OP. What İŞKUR do is mainly addressed to the control group. Well, basically it is the same for the project beneficiaries, since they experience the same difficulties in separating between the three where at least the third appears to form a natural part of the two former. And yet still, all effort has to be reported upon, which notably becomes an issue in case of drop-outs.

155

There is basically no theoretical or logical objection towards the indicators insofar consistent data is available and the indicators can be tracked and recorded practically and with ease for the stakeholders

However, the close relation between the three indicators can never be avoided, and it is doubtful that conflicts and difficulties in double counting can be completely eliminated

Therefore we recommend to do the following:o Aggregate the indicators into one indicator but keep the three indicators

on an internal operational level, providing strict guidelines to coding and quality assurance (QA)

o Eliminate the third indicator on guidance when related to the other two indicators but keep the indicator as measuring separate actions only related to guidance and counselling (that would presumably necessitate a far lower target)

o Ensure solid guidelines to separate the two former actions from each other, and make explicit notes to what to be reported and measured in case of drop-outs

1.3 Measure 1.3To promote registered employment

Specific Objective:• This measure aims to promote registered employment through strengthening the capacity

of social security institution, provision of awareness raising and ensuring coordination among relevant public institutions and social partners.




9Number of established local partnerships

N.A. (There was no such holistic partnership approach before)

15(in Growth

Centres)30

MoLSS Programme Reports, Final Beneficiary Reports

10Number of SSI staff at central and local level who participated in trainings

827 1200 2400MoLSS Programme Reports, Final Beneficiary Reports

11Number of staff from social partners who participated in trainings on National Strategy

N.A. (The strategy is a new one)





9 Number of established local partnerships

Number of permanent local partnerships to promote registered work established and operating

N.A. (There was no such holistic partnership approach before)

15 (in Growth Centres)

336

10

Number of SSI staff at central and local level who participated in trainings

Revised: Number of SSI, relevant institutions' staff and social partners who participated in

827 1200 94

11 Number of staff from N.A. (The 600 673

156

social partners who participated in trainings on National Strategy

trainings to promote registered employment improved.

strategy is a new one)


Table 1.3.3 Output indicators on Measure 1.3, Priority axis 1 as found in MIS Ref. Indicator Target Achieved

9 Number of established local partnerships 379 163

10 Number of SSI staff at central and local level who participated in trainings 3.228 151

11 Number of staff from social partners who participated in trainings on National Strategy 4.122 1.125


Interim Evaluation’ comments to Output Indicators underneath Measure 1.3

The TAT has received no raw background data and is accordingly not able to make an assessment of the quality of the data used in the tables above.

The main comment to the three indicators must be that the targets announced in MIS do not align with the target of the HRD OP and the SAR. The Interim Evaluation has no further comment to that observation.

With respect to the relevance of the indicators the Interim Evaluation has found no serious objections insofar the indicators make sense for the stakeholders.

1.4 Measure 1.4To improve public employment services

Specific Objective:• To improve the quality and efficiency of public employment services especially by

strengthening administrative capacity.




12Number of İŞKUR staff who participated in PES trainings

301 1000 1200MoLSS Programme Reports, Final Beneficiary Reports

13Number of Monitoring Mechanisms

0 1 1

The mechanism itself, MoLSS Programme Reports, Final Beneficiary Reports




157

Count Indicators Baseline Target Expected contribution

12

Number of İŞKUR staff who participated in PES trainings

301 1000 Data on the indicators has not been provided in the progress report; it was noted as an information that shall be submitted to the OS.

13Number of Monitoring Mechanisms

Number of monitoring mechanisms established for ensuring the sustainability of monitoring of ALMP through cooperation with Provincial Employment and Vocational Training Boards.

0 1

14

Number of İŞKUR provincial offices transformed into model offices by being modernised and becoming user-friendly to provide customer-focused public employment services

23

15

Number of İŞKUR staff in newly established model offices that increased their knowledge/abilities on job search methods, customer relations and managerial skills

450

16

Number of İŞKUR staff that increased their knowledge/abilities on conducting labour market analyses

160

17

Number of İŞKUR and MoLSS staff at central level responsible for IT related issues that increased their knowledge/ability on Software development training, Security Training, Database Training, and Network Training

50

18

Divided into two: 1) Increase in the percentage of qualified software relating to labour market information system of İŞKUR

19

2) Increase in the percentage of the usage of software quality standard

0%

158


20

Revised: Increase in the percentage of qualified Internet based services of İŞKUR

200

21

Number of İŞKUR staff at provincial levels that increased their knowledge/abilities on job and vocational guidance services

12

22

Result in the OP, Output in the OIS:Number of the unemployed people benefited from job and vocational guidance services and increased their knowledge on job and career perspectives

18654 22000(Male: 11000;

Female: 11000)

23

Number of İŞKUR staff responsible for designing the Turkish Occupational Dictionary that increased their knowledge/abilities on ISCO 2008 to adopt Turkish Occupational Dictionary

10

24

Number of İŞKUR and MoLSS staffs at provincial levels that increased their knowledge/abilities on monitoring ALMP

500

25

Revised: Increase in the percentage of effective of ALMP measures determined in the 15 growth centres.

26

Number of the members of Provincial Employment and Vocational Training Boards and the experts working in cooperation with these Boards who increased their knowledge/ability regarding employment issues

430 members and experts

159


27

Number of staff of MoLSS, İŞKUR and representativeness of social actors who increased their knowledge on European Employment Strategy, employment policies of the Member States (best practices), urgent employment policy measures in case of global crisis and the Member States practises in designing employment policies

Trainings: 100 people in total;Internships: 30 people in total;

20 from İŞKUR, 5 from MoLSS and 5

from social partners

28

Number of research centres established in the library at the disposal of MoLSS to assist the staff designing employment policies.

0 1

29

Number of models designed cooperation and coordination between MoLSS and Ministry of Science, Industry and Technology is increased regarding designing employment policies.

0 1 model through a

policy paper

30Number of call centres (7/24 functioning) established

0 1

31

Number of İŞKUR staff who gained the ability and knowledge on working in a call centre and informing people in an efficient way

NA (Data will be collected when

the measure implemented)

125


1.4.1 Interim Evaluation’ comments to Output Indicators underneath Measure 1.4

The TAT has found no reliable data on the indicators in the MIS, and again no raw data has been submitted to the TAT.

The OIS, which is defined as a so-called Operating Identification Sheet, has transformed the indicators of the programme, and not to the easier route; the 2 output indicators of the HRD OP suddenly appear to be 17 where one of the additional 15 was a former result indicator.

The result indicator which now is an output indicator is: “Number of the unemployed people benefited from job and vocational guidance services and increased their knowledge on job and career perspectives”. The transformation of the single result indicator to an output indicator is correct; basically it is an output indicator. The reason which is methodological is given within the context in the section on Result Indicators below as well as in the methodological annex; the target group only concerns those who receive the benefit from the programme.

160

Otherwise, the general point on behalf of the Interim Evaluation is that the OIS describes the content of the indicators. For instance, “Number of Monitoring Mechanisms” (Indicator no. 13 above) is not sharp enough, since we simply do not know what the content is. In this case, the OIS redefined the indicator to: “Number of monitoring mechanisms established for ensuring the sustainability of monitoring of ALMP through cooperation with Provincial Employment and Vocational Training Boards”. Obviously, far more precise.

For some reason the OIS does not specify the meaning of Output Indicator 12 “Number of İŞKUR staff who participated in PES trainings”. However, exactly that appear divided into several new indicators within the OIS, which clearly tells us the content of the trainings. And that appears to be the main point in the additional output indicators; they all appear to specify the content and the exact measure of the indicator.

Though it is not possible to evaluate the quality of the data in this section, there appear no obvious practical hindrances to obtaining data from a theoretical point of view; all indicators are well-defined and sharp, and it should be fairly easy to create routes for the feedback data needed.

The Interim Evaluation has no further comments to the indicators from a qualitative, theoretical point of view insofar the additional indicators are purely serve internal purposes. If not, the Interim Evaluation has to issue a warning, which will be further described below: There are already far too many indicators within the HRD OP40. The section below presents a solution to the obvious difficulties in decreasing the number of indicators, which solution is fairly similar to the model mentioned here.


To enhance investment in human capital by increasing the quality of education, improving the linkage between education and labour market, and raising enrolment rates at all levels of education, especially for girls

2.1 Measure 2.1To increase enrolment rates particularly for girls at all levels of education and vocational training

Specific Objectives: • To increase enrolment rates particularly for girls in secondary education and VET • To decrease drop-outs particularly for girls• To increase vocational skills and competences of labour force, particularly of girls and

women• To promote awareness-raising for parents on importance of education, especially for girls.

Table 2.1.1 Output indicators on Measure 2.140 This note was already mentioned by the ex-ante evaluation.

161



32 Number of educational staff having participated in vocational guidance services trainings

0 (There was no such a collective

training except some individual

ones)

1720 2920 Participant List MoLSS Programme Reports, Final Beneficiary Reports

33 Number of parents who are given trainings

8248 parents have been

reached between 2001-2006 (in

Adıyaman, Bingol,

Diyarbakır, Erzurum,

Şanlıurfa, Hatay)

+ 10750 18750 Attendance Sheets,, MoLSS Programme Reports, Final Beneficiary Reports

34 Number of NGO members having participated in trainings

NA 2150 3550 Participants List, MoLSS Programme Reports, Final Beneficiary Reports

35 Number of protocols between local administrations enterprises schools NGOs and social partners in order to increase girls’ enrolment rates

NA (There was no such holistic

protocols)

43 53 Protocols signed between local administrations enterprises schools NGOs and social partners

36 Number of families individually visited to persuade to send their daughters to school

NA (It will be determined after

the measure is implemented)

2000 7000 Attendance sheets, MoLSS Programme Reports, Final Beneficiary Reports

37 Action plans prepared 0 43 43 MoLSS Programme Reports, Final Beneficiary Reports

38 Secondary vocational schools equipped

77 % 80-85% 85% School enrolment ratesMoNE statistics, MoLSS Programme Reports, Final Beneficiary Reports


Table 2.1.2 Output indicators on Measure 2.1 as found in SARCount Indicators Baseline Target Expected

contributionHRD OP OIS32 Number of

educational staff participated in vocational guidance services training

Revised: Number of teachers participated in trainings about vocational guidance services at schools

0 (There was no such a collective

training except some individual

ones)

1720 (40 teachers from 43

provinces)

23


Revised: Number of parents trained by programmes aiming at raising the awareness of parents on the importance of education

8248 parents have been

reached between 2001-2006 (in

Adıyaman, Bingol,

Diyarbakır, Erzurum,

Şanlıurfa, Hatay)

plus 10750 4001

34 Number of NGO members participated in trainings

Revised:Number of NGO members participated in trainings aiming at raising the awareness of

NA 2150 320

162

Count Indicators Baseline Target Expected contributionparents on the

importance of education

35 Number of protocols between local administrations, enterprises, schools, NGOs and social partners in order to increase girls’ enrolment rates

Revised: Number of protocols signed between public institutions, social partners, municipalities, universities and NGOs aiming at increasing cooperated activities for supporting education of girls

NA (There was no such holistic

protocols)

43 24


Revised: Number of parents families reached through campaigns aiming at raising their awareness on the importance of education for persuading them to send their daughters to school

NA (It will be determined after

the measure is implemented)

20000 5639App.(-) 15000

39

Number of pilot projects implemented to increase the quality of VET education and girls’ schooling and quality education

0 Approximately 50- 80 projects

X

40

Revised: Number of VET programmes prepared according to the regional needs with the cooperation of all partners

NA (It will be determined after the measure is implemented)

Not accessible data yet

41

Revised: Number of schools (secondary education institutions especially those for girls) equipped in the pilot provinces to contribute to girls’ schooling and quality education

0 (not exceeds 15% of total budget)

Not accessible data yet

42

Number of detection and monitoring services for students particularly for girls under the risk of dropping out or who have already dropped out in primary and secondary schools

0 1 Not accessible data yet


Table 2.1.3 Output indicators on Measure 2.1 as found in MIS

163

Ref. Indicator Target Achieved32 Number of educational staff having participated in

vocational guidance services trainings 2089 274

33 Number of parents who are given trainings 18164 468734 Number of NGO members having participated in trainings 545 5635 Number of protocols between local administrations

enterprises schools NGOs and social partners in order to increase girls’ enrolment rates

481 451


12698 2955

37 Action plans prepared38 Secondary vocational schools equipped



Again, the Interim Evaluation had no access to raw data.

In general and as seen above, the OIS helps in determining the content and accordingly the precise measure. For some reason there appear to be some errors connected to indicator 18, since the HRD OP states a target of 2,000 while the SAR ambitiously raise the target to 20,000. The targets in the MIS are in general misleading.

Furthermore, the SAR (as well as the MIS as it seems) has removed indicator 37 and 38. It has not been possible to track the indicators even through the MIS. On the other hand, the OIS has identified 4 additional output indicators, of which one of them appears to be a revised indicator 38: “Number of schools (secondary education institutions especially those for girls) equipped in the pilot provinces to contribute to girls’ schooling and quality education”.

Unfortunately, the new indicator does not explicitly define the equipment needed to fulfil the objective; are we talking about gender-specific toilet facilities, specific learning materials addressed to girls (whatever that might be) or something else? What is quality education for girls? And how is quality gender specific? The indicator is not precise enough which actually is quite surprising due to the solid update of the other HRD OP named indicators (which appeared much undone in the HRD OP).

Otherwise, the Interim Evaluation have no further comments to the either the updated HRD OP-named output indicators or the new indicators though we still wonder what happened to output indicator 19 on action plans.

2.2 Measure 2.2To improve the content and quality of Vocational and Technical Education

Specific Objectives: • To increase the quality of vocational and technical education and training addressing

curriculum, teachers, learning environment and school managers• To increase flexibility of the education system by facilitating vertical and horizontal

transitions between different types of programmes in the VET system• To make VET an attractive option for students• To promote cooperation between social partners, schools and VET centres and private

sector

Table 2.2.1 Output indicators on Measure 2.2

164


Target (2011)

Source of Verification

43 Number of developed / revised and implemented modular programs in secondary and post secondary vocational schools by region, by training institution / school and by domain

42 ISCED-97 vocational/educational fields

and 192 vocational branches developed under

SVET

250 vocational branches

260 Programmes approved by Board of Education and YOK, MoLSS Programme Reports, Final Beneficiary Reports

44 Number of comprehensive and scheduled awareness raising events organised with the cooperation of private sector to increase awareness raising among the sector by region and by economic sector

0 (except voluntary campaigns conducted by

NGOs or private companies nationwide).

Min. 30 42 Conference reports

45 Number of partnership protocols between vocational institutions, social partners and private sector in 12 NUTS II regions

N.A (There is no such collective protocols except

individual attempts which might be determined when

the measure is implemented).


46 Number of secondary and post-secondary VET schools with improved capacity and programs by region and by training institution / school

N.A. 45 63 MoNERecords of higher VET schools of the related regional universities , MoLSS Programme Reports, Final Beneficiary Reports



contributionHRD OP OIS43 Number of

developed / revised and implemented modular programs in secondary and post secondary vocational schools by region, by training institution / school and by domain

Number of modular VET curricula developed and/or revised on Judicial Practices, Agriculture, Maritime and Health areas

42 ISCED-97 vocational /

educational fields and 192

vocational branches

developed under SVET

In the OP: 250 vocational

branches In the OIS: 4 sets of modular VET

curricula

44 Number of comprehensive and scheduled awareness raising events organised with the cooperation of private sector to increase awareness raising among the

Number of communication strategies developed through awareness raising activities

0 (except voluntary campaigns conducted by NGOs or private companies nationwide).

Min. 30

165

Count Indicators Baseline Target Expected contributionsector by region

and by economic sector


Number of partnership protocols between vocational institutions, social partners and the private sector in 12 NUTS II Regions

N.A (There is no such collective protocols except individual attempts which might be determined when the measure is implemented).

In the OP:30 In the OIS: Minimum

20


Revised: Number of modular VET curricula developed and/or revised on Judicial Practices, Agriculture, Maritime and Health areas

N.A. In the OP: 45 In the OIS: 4 sets of modular curricula

47

Revised: Number of teachers participated in technical training in the sector for increased rate of pedagogical and professional qualifications of administrators and teachers

N.A. In the OIS: 1800 teachers (350 teachers from Post Secondary Higher VET Schools and 1450 teachers from the MoNE) participated in the technical training in the sector

48

Revised: Number of teachers participated in trainings for increasing the quality of vocational and career guidance services in secondary vocational education institutions and post-secondary higher schools improved

N.A. In the OIS: 700 secondary school and 200 post-secondary higher school teachers

49

Revised: Number of teachers participated in Teacher Training in line with the newly updated curriculum

N.A. 1500 Teachers

50

Revised: Number of QASs in line with the European Quality Assurance Reference Framework for VET (EQARF) developed and piloted In the selected 20 post-secondary higher schools and 30 vocational educational institutions and revised according to

N.A. In the OIS: 30 vocational high schools) (20 post-secondary higher vocational schools)

166

Count Indicators Baseline Target Expected contributionthe results of the

piloting.

51

Revised: Number of departments provided with equipment support that are providing training in the fields of Judicial Practices, Agriculture, Maritime and Health in the selected 20 pilot post-secondary higher vocational schools and METARGEM

N.A. In the OIS: 30



The Interim Evaluation recognizes again the revision done by the OIS, which all, more or less, appear to be highly justified and needed. One single comment should be that the revised indicator 22, Number of communication strategies developed through awareness raising activities, still appear slightly weak and could use some more sharp and precise definitions: Perhaps there is a link somewhere between awareness raising activities and development of communication strategies (if not directly being the same); in that case it would certainly help to know.

Otherwise, there appear no practical objections to the revised indicators except for the number of indicators which again appear to be way too many. The TAT had no access to data, but based upon experience with indicators there should not be any practical hindrances for obtaining the data in case the information lines can be established.

It might be worth to note that some of the newly revised indicators do not apply to the HRD OP Regions alone indicating that in case regional data should be provided and relevant for the indicators, the stakeholder should find ways to ensure data from all NUTS 2 regions of Turkey.

II.3 Priority axis 3 To increase adaptability of workers, enterprises and entrepreneurs, in particular by promoting lifelong learning and encouraging investment in human resources by enterprises and workers

3.1 Measure 3.1Promoting the development and implementation of coherent and comprehensive strategies for lifelong learning

Specific Objectives: • To provide basic skills and competences especially for the disadvantaged and women• To improve quality of non-formal training in line with the LLL perspective• To develop a comprehensive new approach to valuing learning to build bridges between

different learning context and learning forms and to facilitate access to individual pathways of learning.

• To strengthen the existing VET institutions for organizing VET courses in line with LLL strategies and according to the ECVET principles.

167

• To support establishment of Vocational Standards Development, Testing and Certification Centres (VOC-TEST) as accredited bodies of vocational standard development, testing and certification under the monitoring and supervision of Vocational Qualifications Authority.

• To support the establishment of National Qualification System• To promote the linkage between LLL, education system and labour market


Count Indicator Baseline (2006) Target (2009) Target (2011)


52 Participants certified and tested in terms of vocational knowledge and skills in 10 VOC-TEST by region and by VOC-TEST Centre

0 5000 Participants (50 % Female, 50 % male)

8000 Enrolment statistics Attendance rates, MoLSS Programme Reports, Final Beneficiary Reports

53 Number of Vocational Standards developed by VOC-TEST Centres and endorsed by VQA by VOC-TEST Centre

0 Min. 11 80 MoNE records, VQA records, vocational standards issued, MoLSS programme reports, final beneficiary reports

54 Number of participants certified on basic skills courses by region and by VOC-TEST Centre

84662 15000 18000 Certificates issued, MoLSS Programme Reports, Final Beneficiary Reports

55 Number of people who participated in employability courses provided by networks

NA (There was no such a

network before)

M: 900F: 900

M: 1500F: 1500

Participant List, MoLSS Programme Reports, Final Beneficiary Reports

56 Number of teachers who participated in the technical training in the sector by region

NA (Data will be collected when

the measure implemented)

900 1500 Participant List

57 Number of people who participated to internships provided by these networks (60-65% of participants are female, 40-35% are male.20-25 % of participants are graduates of primary education, 80-75% are secondary education)

NA (There was no such a

network before)

M: 600F: 600

M: 1000F: 1000

MoNE Records, Work Place Records, MoLSS Programme Reports, Final Beneficiary Reports

58 Number of teachers trained on new methods of monitoring, evaluation and measurement for LLL

NA (Innovative approach for

Turkey, there is no meaningful

data)

900 1500 Attendance sheets Surveys, MoNE Reports, MoLSS Programme Reports, Final Beneficiary Reports

59 Number of additional training activities on basic skills and further development of basic skills for the adults, particularly for women

5127 860 + 1435 Attendance sheets Surveys, MoLSS Programme Reports, Final Beneficiary Reports

168



60 Number of networks among formal and non-formal educationproviders, business life, NGOs and social partners providing courses for increasing employability of people

0 60 90 Protocols signed Courses opened Statistics on employed trainees (comparisons between before and after the courses), MoLSS Programme Reports, Final Beneficiary Reports



contributionHRD OP OIS52 Participants certified

and tested in terms of vocational knowledge and skills in 10 VOC-TEST by region and by VOC-TEST Centre

3000 participants certified and tested in terms of vocational knowledge and skills in 10 VOC Test Centres.

0 5000 (3000 in OIS) participants

(50% female, 50% male)

285


Minimum 11 Occupational Standards endorsed by VQA

0 Min. 11 15


Trainings on basic skills and further development of basic skills for the adults particularly for women in formal and non-formal vocational institutions in 12 NUTS II Regions provided.

84662 15000 962+120: 1082 App. (-) 13000


1800 participants in employability courses by these networks

NA (There was no such a network before)

M: 900, F: 900 306 App. (-) 1500


900 teachers participated in the technical training in the sector

NA (data will be collected when the measure implemented)

900 118


1200 participants to internships provided by these networks

NA (There was no such a network before)

M: 600 F: 600 131

58 Number of teachers trained on new methods of

Revised: Number of teachers trained on new methods for

NA (Innovative approach for Turkey, there is

900 34

169

Count Indicators Baseline Target Expected contributionmonitoring, evaluation

and measurement for LLL

monitoring, evaluation and measuring LLL activities that are developed, tested and implemented.

no meaningful data)


Revised: Number of “trainings on basic skills and further development of basic skills for the adults particularly for women in formal and non-formal vocational institutions in 12 NUTS II Regions provided.”

5127 860 17


Number of focus groups networks established

0 60 26



The OIS does not add new indicators. However, it redefines and clarifies the existing indicators. Not all of the revisions have been equally clever; Indicators 55 to 57 in OIS are basically not indicators. They are targets.

Otherwise, there appear to be no formal objections to the indicators on behalf of the Interim Evaluation.

3.2 Measure 3.2To increase adaptability of employees and employers by investing more in human capital

Specific Objectives: • To improve adaptability of employees in SMEs by facilitating their access to training, in

particular for low-skilled workers, • To improve adaptability of employers in SMEs by designing and disseminating innovative

and sustainable forms of work organization, which support labour productivity and quality at work,

• To increase training capacities of social partners, NGOs, VET Institutions and public bodies in order to establish capacity to facilitate adaptability of the employees and employers


170



61 Number of employees who participated in trainings on work organisation following training needs analysis

N.A. 1200 3400

Records of Final BeneficiariesMoLSS Programme Reports

62 Number of employers who participated in trainings organized following a training needs analysis

N.A. 1200 2400


63 Number of social partner staff participated in training for trainers

N.A. 600 1800


64 Number of institutions (NGOs, trade unions, public institutions etc.) benefited from consultancy services on the improvement of the adaptability of employers and employees for social partners, universities, public and private VET Institutions, NGOs, public bodies and enterprises

N.A. 50 130




contributionHRD OP OIS61 Number of employees

who participated in trainings on work organisation following training needs analysis.

Revised: Number of employees participated in trainings

NA OP: 1200 OIS: 3000 (total number of

employees - preferably of

SMEs whose employers also

participate in trainings)

65 Revised: Number of study visits by SME employees who have who have participated and successfully completed the trainings to SMEs in an EU country

OIS: max. 120 man/day in total

66 Revised: Number of SMEs visited in developed and industrialized provinces with the participation of employers who have participated and successfully completed the trainings

at least 25 SMEs

67 Revised: Number of staff of

OIS: 600

171

Count Indicators Baseline Target Expected contributionchambers, social

partners, Provincial Employment and Vocational Training Board staff and ÇASGEM staff increased their training capacities to be trainers

68 Revised: Number of gap analysis and trainings needs analysis at company level in each 15 growth centres

at company level: OIS: 15

also, in at least 15 most

developed provinces of

Turkey 69 Revised: Number

of vocational school teachers participated in on- the- job trainings

NA 200


Revised: Number of employers participated in the trainings

NA OP: 1200OIS: 2000

70 Revised: Number of study visits to SMEs in EU with the participation of employers

NA 2 study visits, max. 80 man/

day in total

71 Revised: Number of workshops by the participation of employers from developed regions at sectoral level.

NA 5

72 Revised: Number of visits to selected SMEs in developed and industrialized provinces with the participation of employers and employees

NA at least 25 SMEs

73 Revised: Number of models of innovative and sustainable forms of work organisation for SMEs which have participated in and successfully completed the trainings

NA at least 5 models for 20 SMEs

74 Revised: Number of SME provided by training and for the establishment of support and coordination centres within the chambers and 1 within TOBB in each of which non-

NA OIS: 600

172

Count Indicators Baseline Target Expected contributionkey experts and

chamber staff will work

75 Revised: Number of awareness raising meetings for raising the awareness of employers and employees on the need for more training and consultancy for increasing adaptability and competitiveness

NA 15 (approximately

1500 employers and 1500

employees will participate in

these meetings)


Revised: Number of chamber staff accredited as trainers

NA OP: 600 OIS: at least 6 chamber staff accredited

as trainers76 Revised: Number

of study visits by chamber staff and centre coordinators to the training and consultancy centres established by a chamber in the EU

NA 2 study visits (max. 60

man/day in total)

77 Revised: Number of online database including information on stakeholders, resources, the training needs, training providers, announcements regarding career opportunities, decisions of Provincial Employment and Vocational Education Boards that will serve for chambers, SMEs, civil society organisations and vocational schools is established

NA 1


NA 50

78 Revised: Number of web- sites

NA one

173

Count Indicators Baseline Target Expected contributionestablished for

enhancing the cooperation and coordination among relevant stakeholders

79 Revised: Number of conferences established for increasing industry- university cooperation

NA one



The OIS add or revise the indicators of the HRD OP adding some 15 new indicators into the framework41.

The Interim Evaluation has found no reason to make explicit comments to any of the revised indicators in the OIS.


To promote an inclusive labour market with opportunities for disadvantaged people, with a view to their sustainable integration into the labour force and combat all forms of discrimination in the labour market

4.1 Measure 4.1To increase employability of disadvantaged persons, facilitate their access to labour market, and eliminate barriers for their entrance into labour market

Specific Objectives:• To increase employment and labour force participation rate of disadvantaged persons • To facilitate disadvantaged persons’ access to labour market • To eliminate barriers for disadvantaged persons’ entrance into labour market


Count Indicator Baseline (2006) Target (2009) Target (2011) Source of Verification

80 Number of people having participated in guidance and counselling services

Ex-prisoner and ex-convicts = 188 for 3 years(There is no helpful data on the people with disabilities)

M: 2000F: 2000

M: 3500F: 3500

MoLSS Programme Reports, Final

Beneficiary Reports

81 Number of people having benefited from rehabilitation programmes

Ex-prisoner and ex-convicts 114 people for 3 years 1712 people with disabilities regarding SHÇEK

M: 1500F: 1500

M: 3000F: 3000


Beneficiary Reports

41 This time we have indicated the new added indicators by italics

174


rehabilitation centres for 3 years in NUTS II regions.

82 Number of people having participated in courses in scope of ALMPs

395 women with disabilities, 1015 men with disabilities, 12 women ex-prisoners, 93 men ex-prisoners for 3 years under ALM Project.

1185 women with

disabilities,3045 men

with disabilities,

36 women ex-prisoners, 279

men ex-prisoners

1985 women with

disabilities,

5045 men with

disabilities,76 women

ex-prisoners,479 men ex-

prisoners


Beneficiary Reports

83 Number of people who are reached through the awareness raising activities

651 people are reached in Adıyaman,

Çankırı, Gaziantep,

Kahramanmaraş, Kastamonu,

Kayseri, Malatya, Sinop, Şanlıurfa in

the scope of Awareness

Raising activities of Administration

for the Disabled in 2007

100000 170000 Surveys, MoLSS Programme

Reports, Final Beneficiary Reports



As noted in the theoretic introduction above output indicators are in general not that difficult to define since they appear almost natural from the definition of the measures, actions and target groups. Furthermore, output indicators are by nature almost always of quantitative nature, since the main purpose of output indicators is to measure the direct effect of the effort; that is numbers.

Output indicators on Social Inclusion are no exception. The difficulties in determining Output Indicators on Social Inclusion are connected with two things: 1) difficulties in defining the target groups and 2) obtaining data.

In other countries, notably Member States of the EU, the discussion of the target groups continues and basically never stops; it is a political topic, obviously. As examples Social Inclusion could easily be fully-functional persons lacking basic reading abilities, it could be immigrants, even second generation immigrants, and it could be unemployed in age groups of 55 and above. Some pessimistic researchers (semi-opinion-makers) might argue that smokers tend to be Social Excluded these years, since some enterprises follows the new-economics, puritan American business philosophy, has banned smoking from the grounds, and where smokers cannot be deployed42.

Accordingly, definitions differ across countries and perhaps quite rightly so. When it comes to the definition of Social Excluded there is no golden rule, though a lot of international NGOs might think differently and though UN have defined some minimum criteria. From a methodological, theoretic and scientific point of view that discussion is often not an issue; the definition has become a matter of taste.

42 And that could soon generate Social Exclusion by definition

175

The difficulties in Social Inclusion arise when it comes to obtain reliable data and statistics. We might have a good idea of the target groups, the marginalised and social excluded, but often we do not keep track of them – we might count them when the address public institution but otherwise we cannot see where they live.

Turkey is obviously no exemption. Having defined the target groups to be mainly ex-prisoner, ex-convicts, and people with disabilities, the next problem appears to find reliable data to constitute the baseline where we would expect SSI to have some basic data43.

The Interim Evaluation Report has no objections to the definition of target groups insofar a note within the system explicitly defines what constitute disability44. We shall expect that there is a law framework which clarifies the target group sharply. We shall assume that the prime Operation Beneficiaries (MoLSS, İŞKUR, MoNE and SSI) can redefine the target groups in later programmes if needed.

Neither has the Interim Evaluation found reason to raise fingers of the data (targets and baselines). The comment in the HRD OP on the new approach which is signalled in the baselines justifies a baseline of “close to zero”.

The only comment which was found relevant is the strange gender difference in the target of Number of people having participated in courses in scope of ALMPs45; apparently, the measure favours men to women.

4.2 Measure 4.2Better functioning and coordination among the institutions and mechanisms in the field of labour market and social protection particularly in order to facilitate the integration of disadvantaged persons into the labour market

Specific Objectives:• To improve the institutional capacities of public service providers to disadvantaged

persons in their employment related services, and the functioning of existing mechanisms, • To ensure better coordination among public service providers to disadvantaged people,

including between central and regional/local authorities and in cooperation with NGOs, in employment related services



Target (2011)


84 Number of staff of social protection institutions and NGOs who participated in trainings on services related to the disadvantaged persons

N.A. (There are no meaningful data. Data will be collected when the measure is implemented)

1500 N.A. MoLSS Programme Reports, Final Beneficiary Reports

85 Number of common monitoring and analysis system established

0 1 N.A. MoLSS Programme Reports, Final Beneficiary Reports

86 Number of staff of related institutions and NGOs in that area who participated in trainings on the formation, usage and implementation of

2069 staff in SYDGM in central and local level.222 staff in SSI in central level.1394 staff in İŞKUR.

5500 in total.

N.A. MoLSS Programme Reports, Final Beneficiary Reports

43 Unfortunately, the TAT received no data from SSI and background data from TUIK/TURKSTAT are simply not detailed enough.44 The definition of “disability” differs significantly across countries.45 The target mentions 1185 women and 3045 men.

176


Target (2011)


central database. 20 staff in Administration of Disabled

87 Number of disadvantaged persons registered to central database, monitoring and analysis systems.

683,269 disadvantaged persons in SSI

database in NUTS II level

161,097 disadvantaged persons in SYDGM

project database.20,626 people with

disabilities in the database of

Administration for Disabled in NUTS II

regions.

1,250,000 disadvantaged persons

N.A. MoLSS Programme Reports, Final Beneficiary Reports



In general, the Interim Evaluation has no additional comments to the comments mentioned in sub-section 9.4.1.1. The only very light comment could be that the target of 1,250,000 disadvantaged persons under Output Indicator 87 raises an inferior question: What if you cannot find 1,250,000 disadvantaged persons in the named NUTS 2 Regions?

The question illustrates that somehow the indicator is strange since it pursue something which basically is not the issue; the issue is to have the disabled persons registered (and not the number). It might indicate that the target should be expressed as a fraction of total disabled persons.

The problem might be that we do not know the exact number of total disabled persons, but that fact would just justify the irony of the question raised.

We will allow this comment to stay as a light recommendation; the recommendation is not vital.

II.5 Priority axis 5: Technical Assistance

5.1 Measure 5.1Support for Programming, Management, Implementation, Monitoring, Control, Evaluation and Dissemination Activities

Specific Objectives:• The main aim of this measure is to provide support to the OS in different steps of the IPA

process and to increase the capacity of the relevant central and local partners in the related issues.


Count Indicator Baseline (2006)

Target (2009)

Target (2011)


84 Minimum number of trainings for the staff of the Operating Structure related to improvement of knowledge and practice of IPA.

N.A. 24 48 MoLSS Programme reports

85 Minimum number of trainings on the usage of MIS

N.A. 9 19 MoLSS Programme Reports

86 Minimum number of trainings for OS staff to improve their knowledge on the IPA-ESF and


177


Target (2009)

Target (2011)


differences between them87 Minimum number of site visits for

the monitoring of the projectsN.A. 80 160 MoLSS Monitoring

and Information System, project and programme reports

88 Minimum Number of evaluation activities

N.A. 2 3 MoLSS programme reports

89 Minimum number of prepared analyses, studies, strategy papers, manuals etc.




The Interim Evaluation has no comments to the output indicators and their targets.

5.2 Measure 5.2Support for development of absorption capacity of final beneficiaries

Specific Objectives:• The main aim of this measure is to improve absorption capacity of the final beneficiaries in

order to develop a project pipeline in the framework of HRD OP.



90 Minimum number of capacity building activities (trainings, workshops etc.)

Available 90 140 MoLSS Programme reports



The Interim Evaluation does not understand the indicator and even lesser the baseline.

Minimum number of capacity building activities does not make sense; it is simply not specific and sharp and can subsequently hardly be measured. The indicators should specify the areas of expertise which is sought improved for obvious reasons46.

The baseline “available” is beyond understanding.

5.3 Measure 5.3Information and publicity activities

Specific Objectives:• The objective of this measure is to assist the implementation of obligations arising from

Article 62 of IPA Implementing Regulation concerning the promotion of the HRDOP and its operations and informing entities interested in receiving support from the HRDOP, as well as the general public, about the opportunities provided by the assistance and its outcomes.


46 From experience with previous output indicators it appears plausible to assume that the OIS (Operation Identification Sheets) has specified and clarified which capacity building activities we are talking about. It has not been possible to obtain access to the OIS within the time limit of this report.

178


91 Minimum number of information events (seminars, workshops etc.)

Available 50 100 MoLSS Programme reports



The Interim Evaluation refers to the comments in sub-section 5.2.1.

179

PART III: Result Indicators

Part III presents the introduction to the study of Result Indicators. However, some further notes to the study appear necessary.

First, the induction to the study of indicators describes the important difference between Outcome and Impact Indicators, which in the current HRD OP is merged to Result Indicators. Perhaps the merging of the two can be justified since it appears easier to explain, the Interim Evaluation finds reason to believe that the merging has generated some methodological shortcomings to future evaluations by the merge. Accordingly, the Interim Evaluation will try to categorize the named indicators into Outcome and Impact indicators.

Second, while the study of the relevance of output indicators can be done fairly easy, simply because output indicators are so closely linked to final beneficiaries, the study of the relevance of Outcome and not least Impact Indicators becomes far more exquisite: With respect to Outcome Indicators we have to incorporate the complete potential participants of the target groups, and with respect to Impact we potentially have to look at the overall welfare for the regions in question.

With respect to the study of the relevance of the impact indicators, the main point is to find the link to the priorities and its measures, of course, but basically even that might not be enough. In most cases it would be far more obvious to understand the more global objective of Human Resources Development in general and notably for the country and the regions in question. We shall return to these last aspects in the final sections of this Part which summarizes the findings of the study of the Indicators of the HRD OP.

In the evaluation of Result Indicators we basically do not need data to study the relevance. Referring to the general introduction to the Interim Evaluation Report and the methodology notes in the introduction to the study on Indicators in this Annex, we regret that we have no data to study the quality of the data, and in order to create time series and cross-section analysis to study the efficiency of the measure, the indicators, and their targets, which not least would have been important for Impact assessment. However, when it comes to Outcome and more so Impact, the delayed progress of the implementation of the HRD OP would in any case not have provided us means to make impact assessments.

The study of the single result indicators follows a set of “tests” based upon the following questions:1. Is the indicator understandable and measurable from common practice?2. Is the indicator an Outcome or an Impact Indicator?3. Does the Indicator mirror or reflect a relevant and sustainable process?4. Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and the

social and economic conditions facing the NUTS 2 Regions of HRD OP?5. Are data available and are the data consolidated ensuring means to do solid evaluation?

The study of each indicator is rounded by: 6. Overall conclusion and preliminary recommendation

Not all indicators stand the test; in that case the study of the indicator will be terminated by a conclusion / recommendation.

180

III.1 Priority axis 1

To attract and retain more people in employment, particularly by increasing labour force participation of women, and decrease unemployment rates especially for young people

III.1.1 Measure 1.1To promote women’s participation into the labour market, and increase female employment, including those formerly employed in agriculture

Specific Objectives:• To improve employability of women• To support women entrepreneurship• To diminish cultural and other obstacles that hinder women’s participation in labour market

Table 1.1.1 Result indicators on Measure 1.1, Priority axis 1



1

Increased rate of women having entered into labour market (having the qualification and motivation to seek a job) after participating in courses or counselling

88 % (According to İŞKURAIP Project Data for 3 years)

95 % 95% İŞKUR Job Seeking Records, MoLSS Programme Reports, Final Beneficiary Reports

2

Increased rate of women employed through the courses and counselling

27 % (According to İŞKUR data for 3 years in NUTS II regions)

35 % 23% İŞKUR Job Placement Records, SSI Records, MoLSS Programme Reports, Final Beneficiary Reports

3

Increased rate of women entrepreneurs

25.6 % (According to KOSGEB and İŞKUR data)

30 % 20% KOSGEB Records, SSI Records, MoLSS Programme Reports, Final Beneficiary Reports


Table 1.1.2 Result indicators on Measure 1.1, Priority axis 1 as found in SARCount Indicators Baseline Target Expected


1


88 % (According to İŞKURAIP Project Data for 3 years)

95 % 2484

2


27 % (According to İŞKUR data for 3 years in NUTS II regions)

35 % 1094

3


25.6 % (According to KOSGEB and İŞKUR data)

30 % 928

PYE: 464


Table 1.1.3 Output indicators on Measure 1.1, Priority axis 1 as found in MIS

181

Ref. Indicator Target Achieved1 Increased rate of women having entered into labour market

(having the qualification and motivation to seek a job) after participating in courses or counselling

5,313 3,342

2 Increased rate of women employed through the courses and counselling

3,194 787

3 Increased rate of women entrepreneurs 2,182 789Source: Monitoring Information System, Operating Structure, MoLSS

1.1.1 Result Indicator no. 1

1. Is the indicator understandable and measurable from common practice?

The first step in the study is obviously to understand the indicator. The indicator reads: “Increased rate of women having entered into labour market (having the qualification and motivation to seek a job) after participating in courses or counselling”. In order to understand the complexity of the indicator it appears necessary to decompose it.

First, we have to understand the term “entering the labour market” correctly. It must be a mistake, a mistake which is repeated in the Measure; obviously, the authors must have meant “entering the labour force”47. In other words the indicator reads, that we would be happy if women steps out of inactivity and into employment or unemployment48.

But the indicator does not stop there. It further requests a rate, or more sharply: “the rate of women getting a job or registers as unemployed after participating in courses or counselling”. Again, we have to make presumptions: What rate? What is the denominator? Is the rate the reciprocal of drop-out rates (or failure rates) from courses or counselling49? Or does the rate measure the difference between those women who get a job after the named support divided by women who obtain a job without participating and without support? Or is it number of women getting a job or get unemployed divided by number of inactive women? Presumably not the last if we should believe the targets, but generally no answer is given; and the MIS only provides numbers50.

Next, and this point will be raised a couple of times in this chapter, the indicator mention a need for an “increase”. Obviously, that must be a mistake. Indicators never speak of increase or decrease and never give any signs of directions; that is the “job” of the baselines and the targets of course.

Finally, we have an even more gigantic problem in the formulation to the requirements to the women who actually find a job (if that is what it means): Exactly how do we measure women who have the qualification and motivation to seek a job? And yes, both are equally impossible, qualification and motivation, though, admittedly, qualification perhaps can be measured to some extent, but appears to be quite symbolic in this context where the women actually obtains a job.

Measuring “right” qualifications for the job is a dangerous task which only few labour market analysts would embark upon. Just imagine asking your manager the following: “Tell me, Sir, do you think you

47 Of course, we could alternatively assume that the authors mean “getting a job” but that would bring the indicator to become the same as indicator 2.48 It does not indicate that it has to be registered employment, which is a matter of Measure 1.3, but we basically do not know, and perhaps that is intentionally.49 E.g. if 6 women obtain a job after participating in a training course with 10 participants in total we have a fraction of 0.6, indicating that we have a drop-out rate (or perhaps a failure rate) of 0.4. Does that mean that the indicator, the rate, measure successes? Or should we go even further as presume that the word “rate” determines “speed” or “growth”?50 We have, as mentioned in Chapter 8 above, data on “number of persons having benefited from guidance and counselling services by region (NUTS 2) and gender” from İŞKUR, but we still do not know what rate means.

182

are qualified for your job, you are doing?” Of course, we could make screenings of all job places, which could include collecting of CV’s and educational diplomas allowing us to do some assessment of the qualification levels for certain job positions, but if this woman obtained a job from an employer who willingly gave her the contract, would we say that she is not qualified?

In other words, using criteria like “qualification” and “motivation” in indicators should only be done with reason and not least care. We have to know how to measure it, and in this case it is plain impossible. Unless, of course, the participants end up being unemployed51.

Accordingly, the Interim Evaluation does not understand the indicator. Furthermore, it cannot be measured since we do not know the dominator and since motivation cannot be tested. Even if ignored these facts and tried to measure it, we would not be able to proof that the women actually have the right qualifications for the jobs (still in case the participation is crowned with some success, that is). In other words: The indicator is incomplete, unfinished, undone, and not thought trough.

2. Is the indicator an Outcome or an Impact Indicator?

Assuming that the right words should have been something in the neighbourhood of “number of women getting a job or register as unemployed after participating in courses or counselling” the indicator can qualify to be an Outcome Indicator in two specific cases:

1. Insofar the purpose of the measure is to increase the number of registered women, or2. Insofar we talk of courses or counselling in a very broad sense. The broad sense indicates a

necessity that the measures relates to courses or counselling outside the scope of HRD OP, though still within the NUTS 2 Regions.

The first cause is basically the purpose of Measure 1.3 as mentioned, and it does not include women who find a job in the informal sector. But then again, support of the informal sector has never been a real target for any public measures. So, perhaps the indicator tries to seduce women to become registered basically and perhaps as a sort of step 1 leaving the goodies to Result Indicator 2 for those who obtain a job. If so, it would be far more feasible to name the indicator by its right name: Number of formerly inactive women registered as unemployed”. But that is basically not what the indicator say, and still it simply does not make sense to talk of courses and counselling then.

In case the indicator aims at courses or counselling in a very broad sense the measure more clings to training of staff of İŞKUR as well as to ensure better trainers (or so), hoping to increase the quality of the services provided and thereby ensuring a higher number of women who actually obtains a job.

Accordingly, we have to look at the measure, which reads: “To improve employability of women”. But that is not enough, since we further have to understand the basic actions (the projects and their contents) in order to see if they are designated to improve the quality of the courses or counselling. Exactly that has not been possible within the short time of this evaluation project, but having a quick glimpse at the output indicators under Measure 1.1 it appears quite obvious that we are counting participants – and not counting staff and teachers trained.

That leaves only one conclusion: The so-called Result Indicator, even if we assume that we understand it, is neither an Outcome nor an Impact Indicator. Changing the indicator into a fraction or a rate does change nothing mainly since the indicator so clearly tries to ensure that the training courses or the counselling service is improved for the selected few who obtain the benefit thereof.

51 Which hardly can be a priority, or what?

183

It is obviously just another Output Indicator if anything52, though presumably an Output Indicator in a slightly wider scope since it might incorporate other actions taken by İŞKUR. It simply measures the number of women who circles through the public funded Active Labour Market Measures and get through it with success53.

In order to qualify as an Outcome indicator we would have required the indicator to incorporate the measure which reads: “To promote women’s participation into the labour market, and increase female employment, including those formerly employed in agriculture” as well as the Priority “To attract and retain more people in employment, particularly by increasing labour force participation of women, and decrease unemployment rates especially for young people”54. In that case, we might have found it more indicative to look for labour force participation rates or perhaps even employment rates by gender, where the Outcome Indicator could specify “women”.

Preliminary recommendation55: Redefine the indicator or perhaps delete it.

An obvious doable alternative would be a workable and highly practical well-defined Outcome Indicator of “Labour Force Participation Rate for Women” or perhaps “Employment rate for women”

In other words, assuming that the Interim Evaluation are right in its search for Outcome and Impact Indicators amongst the Result Indicators of the HRD OP, the study of the first indicators was fairly disappointing. On the other hand, the Interim Evaluation believes that the above comments expressed through a light tone of almost “oral writings” ensure better understanding of how to define indicators in the future. The Interim Evaluation will, as will be seen from the following sections below, bring the issue up front in the final recommendations in the end of this very chapter.



“Increased rate of women employed through the courses and counselling” is definitely more sharply formulated that indicator 1 above. However, the points about the words “rate” and “Increased” still remains; we simply do not know what the denominator is, but we would presume it to be the total number of participants in “courses and counselling”56.

2. Is the indicator an Outcome or an Impact Indicator?Having redefined the indicator to measure “Number of women employed after participating in courses or after having received counsel divided by total number of women participating in courses and counselling” we stand in the same dilemma as under indicator 1 above: Are we into measuring the quality of the training courses / the counselling service, or do we have a specific target group to reach?

52 The “if anything” is not just another comment to the difficulties to see what the HRD OP stipulates to measure, but also the fact that measuring “number of women getting a job after participating in courses or counselling” quite easily could be valued to be slightly out of the scope of the actions taken within the projects. 53 Though still with a focus on improving the “systems” / institutions rather than the situation for the target group as mentioned below.54 The priority clearly states “Labour Force participation” which brings the discussion of “entering the labour market” into perspective. And, right, now, after the study of the first indicator, we are in doubt of the meaning of Measure 1.1; it would have been reasonable to suggest that at least part of the objective / priority / goal of the HRD OP was to help decreasing the number of inactive, particularly aimed at women (noting the social structure within the HRD OP Regions).55 Throughout this section we use preliminary recommendations only since additional studies of the indicators from a more overall viewpoint follows below. The Chapter will conclude by stating final recommendations56 Well, we did not dare to mention it under indicator 1, but basically in that case it would be wrong to use the term “rate”; “Share” would be a more proper term.

184

Well, women in general cannot be the target group; that would be far too global, of course, but what about the dominator of total women having benefited from the public service? That would be logic if we keep our eyes focused on the indicator alone. But recall that the measure talks of “promoting women's participation into the labour market (labour force?), and increase female employment, including those formerly employed in agriculture”. The measure speaks for a target group of women OUTSIDE the labour force, obviously. So, the question would be: Would women outside the labour force benefit from women obtaining a job after being trained? Obviously not; there is simply not a single element of sustainability in-built in the indicator.

And that leads us back to the same story and the same conclusion as under indicator 1: Either the indicator speaks of the quality of the service or it is not an Outcome indicator.

The recommendation under indicator 1 remains.



“Increased rate of women entrepreneurs” is plain talk in case we ignore the word “increase” and assume that we know what an entrepreneur is. In common language for daily use we believe that entrepreneurs own and run their own businesses. We shall accordingly presume that women entrepreneurs are women who start their own business as owners57. Self-employed, that is58.

The rate is not specified and the denominator could (again) be a number of things: Women in general / female working age population (15-64) / women in labour force / total employed women / male “entrepreneurs” etc.59. We shall presume that we speak of total employed women60.


Basically, the entrepreneurs are not specifically mentioned as prime target in the priority and the measure, so it tends to be difficult to assess the logic and the relevance of the indicator. Though it appears to be a long shot to recruit new entrepreneurs among a group of women outside the labour force it appears to make sense to encourage entrepreneurship among women. In that case it would correct to place it as an Outcome indicator.

3. Does the Indicator mirror or reflect a relevant and sustainable process?

Ensuring encouragement for starting own business, including ensuring that the basic platform for doing that is in place (in terms of financial support, legislative advisory, insurance systems, etc.), is always relevant, perhaps even sustainable though that might take some time to show. But of course, that depends on content which requires a far more qualitative indicator.

57 Does that exclude women elected to be chairman of the board of shareholders and women who are deployed as managing or executive director? Presumably the latter but not the former if we should understand the term “entrepreneurs” correctly. 58 Which normally includes freelancers (e.g. journalist, photographers, consultants, models, actors, etc.)59 Likewise, the indicator do not clarify “full-time” versus “part-time” which easily could be essential in start-up of enterprises.60 Since that would be the most logic in accordance to international practice. However, that does not seem plausible from the baseline and the target; can it really be true that 25 per cent of total women in employment are self-employed?

185

If the main new enterprise start-ups emerge within a production or a market with no or limited demand, the initial benefits might soon turn into a disaster, not just on a private account but also for the local economy. Examples could be small-time shops and small-time craftsmanship (e.g. home-based knitting ware and small-time tourist merchandise). If the effort, however, are directed into sub-sectors with larger potentials and excess demand, the effort is alternate prosperous.

To answer that question we need far more qualitative information than given by the indicator, which seen from an evaluation point of view it is basically not a requirement. The only answer to the question of sustainability is “time must show”, while the answer to the question of relevance appears to be positive, since stimulating enterprise start-ups is generally reckoned to beneficial for the society (international experience)61.

4. Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and the social and economic conditions facing the NUTS 2 Regions of HRD OP?

The relevance of the indicator subject to the Priority, the measure, the specific objectives, and the social and economic conditions facing the NUTS 2 Regions of HRD OP, is however far more questionable. The following table summarizes the framework of the indicator (Priority, Measures, and Objectives) and tries to provide some brief comments to each of the elements within.

Framework Relevance of the Indicator Check

1) Attract and retain more people in employment

Assuming that the recruitment of women comes from inactivity we can speak of “attracting”. It is not evident that entrepreneurship provide means of sustainability for those who succeed, though

?

2) Increase labour force participation of women

Assuming that the recruitment of women comes from inactivity: YesIf not: No.

()

3) Decrease unemployment rates especially for young people

Assuming that the recruitment of women comes from unemployment: Yes. No long term effect is guaranteed, thoughIf not: No effect and not relevant

()

4) Promote women’s participation into the labour market (which we understand as Labour Force)

Assuming that the recruitment of women comes from inactivity: YesIf not: No

()

5) Increase female employmentAssuming that the recruitment of women comes from inactivity: Yes ()

6) Include those formerly employed in agriculture

It depends. The indicator do not exclusively focus on that segment ?

7) Improve employability of women

Assuming that the recruitment of women comes from inactivity: YesIf not: No. The indicator is not directed towards measure of employability

()

8) Support women entrepreneurship Yes, that is the idea 9) Diminish cultural and other

obstacles that hinder women’s participation in labour market

It sounds plausible on labour markets mainly dominated by men

61 Again, and as mentioned, that could be questioned (and perhaps it should) in case the new enterprises do not stand a fair chance on a private market. Somehow and somehow strangely, the question of demand is hardly ever addressed. However, approximately 10 years ago EC Commission launched a new indicator on “Survival rates of newly born enterprises” which is defined as “Newly born enterprises of year n that are still active in year n+3”. The indicator was former part of European Employment Strategy guideline. We will not recommend using this indicator here; basically enterprise-start-ups are not the prime aim of the HRD OP and its Priorities.

186

Accordingly, the main question is the extent the new entrepreneurs are recruited from inactivity or at least either the pool of unemployed women. But even when the recruitment comes from the pool of employed the indicator appears to be accepted from a very general point of view.

The reservation from the Interim Evaluation is solely of more qualitative, and perhaps more subjective nature; 1) in case the new entrepreneurs are former employed a large part of the overall intention of the Priority appear to be lost, and 2) in case some kind of sustainability is not directly ensured from other measures and actions (do the new enterprises survive?), the net impact might soon turn to become negative.

5. Are data available and are the data consolidated ensuring means to do solid evaluation?

The fifth and final test is data availability. Unfortunately data is not available according to İŞKUR62. The same conclusion has been given by the Operating Structure in MoLSS.

However, it should be possible to develop some basic raw indicators from the Household Labour Force Survey conducted by TÜİK/TURKSTAT. A set of data is available from the internet and called “Employment by status in employment [15 years old and over-thousand people]: Employer and own account worker” which series provides data for all NUTS 2 regions of Turkey since 200463.

However, the Interim Evaluation is not sure that the data is solid enough which concern includes a denominator, and furthermore it was not possible to provide data on gender from TUIKs home pages. Accordingly, we recommend the Operating Structure to contact TÜİK/TURKSTAT in order to find means to obtain meaningful data series on the specific indicator.

Wherever data from KOSGEB and İŞKUR is available, these data sets, which will be un-official, can be used for quantitative validation purposes64.

By means of LFS the evaluators can further obtain data from other NUTS 2 regions providing a solid fundament for measuring the effect.

6. Overall conclusion and preliminary recommendation

The Interim Evaluation has no objection towards the Outcome Indicator, except from some minor reservations. The reservations are: 1) in case the new entrepreneurs are formerly employed a large part of the overall intention of the Priority appears to be lost, and 2) in case some kind of sustainability is not directly ensured from other measures and actions (do the new enterprises survive?), the net impact might soon turn to become negative.

The Interim Evaluation recommends the Operating Structure to contact TÜİK/TURKSTAT to find ways to obtain meaningful and consolidated data based on LFS.

62 See table 1 in this Annex: The response from İŞKUR.63 LFS data series64 The note here concerns a methodological note on the substance of data generated from administrative records; as a general rule of thumb the more we move towards Outcome and not least Impact assessment the requirement to the statistics becomes of paramount importance: Never use anything else than official statistics for measures of Impact. With respect to Outcome we can use specially designated surveys and perhaps administrative records, where the administrative records are able to feed in on official statistics (qualitative) or otherwise be used as control. Recall, that administrative records are NOT validated (in terms of terminology we can hardly call administrative record “statistics”, though that word is being used widely in some central administrations).

187

1.1.4 Feedback from survey on indicators among the stakeholders on Result Indicator 1-3What is the problem of the indicators under this measure, if any? Share NNet impact cannot be measured since Impact can arise from other factors in the environment

12.7% 8

Data is not available 6.3% 4Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of reports, etc.)

7.9% 5

Data differs from region to region (not standardized) 14.3% 9Data are not systematically collected (e.g. timing differs between projects and/or regions)

9.5% 6

Data is only generated from on-going projects (we cannot see what status are in locations without projects, before the project is launched, and after the project has been concluded)

9.5% 6

The indicator does not mirror the real target / is not relevant 7.9% 5The target group is not clearly defined 4.8% 3There is no baseline data (we cannot see and measure improvements) 11.1% 7The indicator is too complicated to calculate (if based upon aggregated data) 9.5% 6Other (please specify) 6.3% 4

The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders view on where the main problems lie65.

III.1.2 Measure 1.2To increase employment of young people

Specific Objective:• To promote integration of young people into the labour market




4

Increased rate of young people employed following the courses, internship and counselling activities

In average 22.3 % according to the İŞKUR data in NUTS II regions.

30 % 23% İŞKUR Job Placement Records, SSI Records, MoLSS Programme Reports, Final Beneficiary Reports

5

Increased rate of young entrepreneurs

7.5 % According to KOSGEB data in NUTS II regions

15 % 15% MoIT Records, SSI Records, MoLSS Programme Reports, Final Beneficiary Reports




4


In average 22.3 % according to the İŞKUR data in NUTS II regions.

30 %

PWE: 35%

4152

PWE: 1094

5

Increased rate of young entrepreneurs

7.5 % According to KOSGEB data in NUTS II regions

15%

PWE: 30%

464


65 We agree that the questions should be raises to each and every indicator, but that would have created a far too long questionnaire.

188


The Result indicator is basically the same as Result Indicator no. 2 above. Accordingly, we refer to the comments noted in subsection 10.1.1.2. The preliminary recommendation from subsection 10.1.1.2 remains.


The Result indicator is basically the same as Result Indicator no. 3 above. Accordingly, we refer to the comments noted in subsection 10.1.1.3. The preliminary recommendation from subsection 10.1.1.3 remains.

With respect to data we recommend to contact TÜİK/TURKSTAT in order to obtain data by both gender and age.

1.2.3 Feedback from survey on indicators among the stakeholders on Result Indicator 4-5

What is the problem of the indicators under this measure, if any? Share NNet impact cannot be measured since Impact can arise from other factors in the environment 12.1% 4

Data is not available 12.1% 4Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of reports, etc.) 15.2% 5

Data differs from region to region (not standardized) 9.1% 3Data are not systematically collected (e.g. timing differs between projects and/or regions) 12.1% 4


12.1% 4

The indicator does not mirror the real target / is not relevant 9.1% 3The target group is not clearly defined 0.0% 0There is no baseline data (we cannot see and measure improvements) 6.1% 2The indicator is too complicated to calculate (if based upon aggregated data) 6.1% 2The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders view on where the main problems lie.

III.1.3 Measure 1.3To promote registered employment

Specific Objective:• This measure aims to promote registered employment through strengthening the capacity

of social security institution, provision of awareness raising and ensuring coordination among relevant public institutions and social partners.




6

Decrease in the rate of complaints (advice slip) on the unregistered employment to the Insurance Inspection Board

2432 individual advice slips

- 15 % = 2067

-%5 = 2310

SSI Insurance Inspection Board Records, MoLSS Programme Reports, Final Beneficiary Reports

7Increase in the number of bilateral projects on promoting registered employment

N.A. (Data will be collected when the measure is


189

between the relevant partners implemented)

8

Increase in the rate of registered people in 43 city centres through the bilateral projects and local partnerships

66.3 % 66.6 % 66.6% MoLSS Programme Reports, Final Beneficiary Reports




6


Revised: Percentage of decrease in the rate of complaints (advice slip) on the unregistered employment to the Social Security Inspection Board

2432 individual advice slips

- 15 % = 2067

3

7

Increase in the number of bilateral projects on promoting registered employment between the relevant partners

N.A. (Data will be collected when the measure is implemented)

12 158

8


The rate of registered employees in the 12 NUTS II regions.

66.3 % 66.6 % 1033




“Decrease in the rate of complaints (advice slip) on the unregistered employment to the Insurance Inspection Board” is not common practice as an indicator. Nevertheless, and as in the first 5 indicators, we should neglect the word “decrease” which has nothing to do in an indicator.

The advice slip, or rather complaints, is reporting on irregular (informal) work. A decrease in number of “complaints” should, according to the stakeholders, indicate that fewer undertake irregular work.

And then we have to understand the point in “rate”, which again is based upon pure guess work. Well, it simply does not make sense, not even after looking at the baseline. The target measures a decrease, which is a very basic analytic process, and nothing valuable. So, no, rate does not make sense at all.

That leaves us with a raw indicator saying “number of complaints (advice slip) on the unregistered employment to the Insurance Inspection Board”. The Insurance Inspection Board does not exist anymore; the complaints are now received by telephone through the hotline, through inspection, etc.

As mentioned the indicator is not common international practice. Data should be available, though the TAT has received no reply to our requests as mentioned several times before.


190

The reports of informal activities appear to be an Outcome Indicator, since it mirrors activities for the complete target group.


As explained to the TAT the number of complaints should be an indicator of irregular activities. However, the TAT is not entirely convinced that the indicator precisely measures the magnitude of irregular activities, and it tends to be plausible to suggest that number of complaints easily could vary according to other factors than the activities related to the HRD OP, including perhaps economic development and general wage increases in the formal labour market.

Of course, economic development and general wage increases in the formal labour market could be seen as being a benefit for the society, but does a decrease in “number of complaints on irregularities” due to these factors actually measure the magnitude of irregularities, and could we hope that economic development and general wage increases in the formal labour market would stop irregularities? Presumably not, of we should trust international experience and theory of informal sectors.

And that leads us to a reservation about the doubt on the ability to reflect a relevant and sustainable process and progress through the indicator. It is not perfectly clear to the TAT that this measure by itself is sustainable; will it work when times get tough and the economic crisis hits the NUTS 2 Regions?


The relevance of the indicator subject to the Priority, the measure, the specific objectives, and the social and economic conditions facing the NUTS 2 Regions of HRD OP, is studied in the following table which, as above, summarizes the framework of the indicator (Priority, Measures, and Objectives) and tries to provide some brief comments to each of the elements within.

Framework Relevance of the Indicator Check

1) Attract and retain more people in employment

No. The indicator do not address issues of attracting and retaining employment; it simply measure irregularities

2) Increase labour force participation (of women)

No. Complaints changes nothing; it is doubtful that the employer will hire the same if it was formal and registered

3) Decrease unemployment rates

(especially for young people)No

4) Promote registered employment No. The indicator tells nothing about registered employment; it just measure irregularities

The indicator might tell us about the magnitude of the problem, and accordingly the cause of action, but it will not be able to measure progress towards our priorities, measures, and objectives.

Using a negative or reciprocal indicator of the development we wish to achieve will always lead to this conclusion. It does not disqualify the indicator per se.


191

According to SSI data is very available. As mentioned the TAT have had no means to test the quality of the data, and we are not able to draw upon international practice.


It is not easy to draw up conclusions on the “number of complaints on irregularities related to work” as a viable Outcome Indicator for a programme which serves to promote registered employment. The indicator is presumably established as a measure of the magnitude of the problem, to which measurement the TAT still lacks proof of creditability; at least it is not a standard way to measure the size of the informal sector.

Of course, it is always nice when citizens stop making complaints, but the TAT is not convinced that a stop in the number of complaints should be received as a proof of a problem which no longer exists. Similarly, the TAT is not at all convinced that a decrease in the number of complaints received should be taken as a proof of a decreasing scale of the informal sector. At least, that TAT remains to be convinced thereof.

Accordingly, the Interim Evaluation is in no position to make recommendations. If the stakeholders full-heartedly believe in the measure the Interim Evaluation has no objection.



Leaving out the term “Increase”, the “number of bilateral projects on promoting registered employment between the relevant partners” appears fairly straight insofar we know who the relevant partners are.


The indicator is unfortunately neither an Outcome not an Impact Indicator. The indicator measure output of certain initiatives.



Again we have to omit the word “Increase”. The term “rate” in “rate of registered people” is not explained. We would presume that we are talking about employment rate, which makes sense.

Accordingly, the indicator reads “the employment rate among registered people in 43 city centres through the bilateral projects and local partnerships”. The words has been changed in the OIS into “the employment rate in the 12 NUTS II regions”, which is slightly more sharp, since “employees” appear to be more in line with the overall measure than “people”. Furthermore, the revision has omitted the “… through the bilateral projects and local partnerships” which has made the indicator far stronger.

2. Is the indicator an Outcome or an Impact Indicator?The employment rate in the 12 NUTS II regions is without any reasonable doubt an Outcome Indicator.


192

Employment rates are a very commonly used Outcome Indicator and have proven to be solid and transparent. When it comes to documentation of a sustainable process under the heading of promoting registered employment it is the most obvious.


We do not need a table here: Yes, without any doubt the indicator is highly relevant, perhaps the most relevant given the measure.


Data is very available and solidly supported by TÜİK / TURKSTAT which makes it even far more viable. Given that the indicator is highly transparent and available for all NUTS 2 regions, not just in Turkey but in all EU Member States, the indicator can be used in international benchmark as well ensuring a solid evaluation in due time.

The data is used in the Socio-Economic background analysis of Chapter 3.


The Interim Evaluation gives credit to the use of this indicator. Further comments will follow in later sections of this chapter.




Data differs from region to region (not standardized) 4.9% 3Data are not systematically collected (e.g. timing differs between projects and/or regions) 9.8% 6Data is only generated from on-going projects (we cannot see what status are in locations without projects, before the project is launched, and after the project has been concluded) 13.1% 8

The indicator does not mirror the real target / is not relevant 8.2% 5The target group is not clearly defined 0.0% 0There is no baseline data (we cannot see and measure improvements) 11.5% 7The indicator is too complicated to calculate (if based upon aggregated data) 9.8% 6Other (please specify) 4.9% 3The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders view on where the main problems lie.

III.1.4 Measure 1.4To improve public employment services

Specific Objective:• To improve the quality and efficiency of public employment services especially by

strengthening administrative capacity.Table 1.4.1 Result indicators on Measure 1.4, Priority axis 1

193



9

Increase in the number of unemployed people counselled

18654 M: 11000F: 11000

M:12500F: 12500

İŞKUR Records, MoLSS Programme Reports, Final Beneficiary Reports

10

Increase in the placement rates of İŞKUR

22.3 % 30% 30% Surveys, Polls, MoLSS Programme Reports, Final Beneficiary Reports




9Increase in the number of unemployed people counselled

This indicator has been turned into an Output Indicator Revised: Increase in the number of qualified reports, policy documents and actions produced by the Provincial Employment and Vocational Training

11

Number of communication campaigns organized for increasing the visibility of İŞKUR

NA (Data will be collected when the measure implemented)

43

12

Revised: Increase in the percentage of the registration rate to İŞKUR

10Increase in the placement rates of İŞKUR

22.3% 30%


The original Indicator of “number of unemployed people counselled” has been deleted from the pool of Result Indicators and entered pool of Output indicators in the OIS; quite justified. Three new indicators have been added, of which the first will be referred to as “Result Indicator Number 9” in the following.



Again omitting the use of the term “Increase” the indicator reads: “the number of qualified reports, policy documents and actions produced by the Provincial Employment and Vocational Training”. The indicator is understood with ease and presumably easy to measure.


194

The indicator is so obviously an Output Indicator. It might not be related to the HRD OP exclusively, and the institution might not be related to the public employment service, but that does not make the indicator less an Output indicator.



Omitting the term “Increase” the indicator is “placement rates of İŞKUR”. The indicator should be measurable insofar data from İŞKUR is available66.


The indicator is an Outcome indicator in this case since the target group is the public employment service (İŞKUR) and since the measure is to improve the efficiency of İŞKUR.


Well, a relevant and sustainable process can sometimes be tricky to value, especially when we speak of public service institutions; in present years and in many countries public service has a negative value. However, from an objective point of view there is no doubt that the indicator reflects a relevant and sustainable process, mainly justified by the measure itself which supports the capacity building of İŞKUR.


The Outcome indicator appears to be relevant mainly since it so closely reflects the measure. There is no need for a table to prove that.


Data are available as noted. The data is consolidated by İŞKUR, but the data are based upon administrative records. As noted above on data generated from administrative records, the general rule of thumb is that the more we move towards Outcome and not least Impact assessment the requirement to the statistics becomes of paramount importance. In general we only accept official statistics for measures of Impact. With respect to Outcome we can use specially designated surveys and perhaps administrative records, where the administrative records are able to feed in on official statistics (qualitative) or otherwise be used as control since administrative records are NOT validated (in terms of terminology we can hardly call administrative record “statistics”, though that word is being used widely in some central administrations).

In this case we use this comment as a reservation to the use of the indicator; the Interim Evaluation strongly recommends making sure that the data on placement rates are used with care.


66 Which they are indeed

195

The last comment serves as a general conclusion and recommendation. The Interim Evaluation has no further comments to the use of the indicator.



“Number of communication campaigns organized for increasing the visibility of İŞKUR” is indeed understandable.


The indicator is by all means an Output Indicator since the indicator measure output from public production.



The 12th Result Indicator reads “Increase in the percentage of the registration rate to İŞKUR”. Again the use of “Increase” has to be deleted, but how should we understand an “increase in the percentage of the registration rate”? It is simply impossible to understand.

We will for the remaining part of the study presume that the author means “number of registered unemployed”.


“Number of registered unemployed” can easily go for an Outcome Indicator. In the case of Turkey and in the case of the HRD OP the Interim Evaluation would even say that it is a solid Outcome Indicator. The reason will follow shortly.


Assuming that the indicator is understood correctly as “number of registered unemployed” the first point would be a question: Is that a new ‘negative’ or ‘reciprocal’ indicator67?

Given the circumstances which Turkey faces and acknowledging the challenges facing the stakeholders to bring the NUTS 2 Regions up to pace institutionally and not least in welfare terms, the Interim Evaluation agrees with the indicator. But more so, and especially noting the volume of the informal sector and the challenges facing Turkey in the pathway to make better and more sustainable jobs, perhaps even jobs which obey the gradually more intensified progress towards OSH regulation, the Interim Evaluation Report can see the indicator as a token of progress.

Accordingly, the Interim Evaluation agrees that the indicator reflects a sustainable process.


67 Not that it disqualifies an indicator; it just has to be taken into account in the evaluation of the strength of the indicator not least in terms of relevance for the objectives, priorities, etc.

196

Since the measure is to improve the quality of public employment services it appears quite obvious that the indicator is relevant, not least noting the comments above about the challenges facing Turkey.


Indeed data are available and used in the Socio-Economic Study of chapter 3. However, the data is based upon administrative records which reinforce the reservation noted above under Indicator 10. The data from İŞKUR cannot be used for international benchmarking68.


The Interim Evaluation agrees with indicator conditional that the Interim Evaluation has understood the indicator. The Interim Evaluation issues a light reservation to the use of administrative records, but sees no doable alternative in this case.







To enhance investment in human capital by increasing the quality of education, improving the linkage between education and labour market, and raising enrolment rates at all levels of education, especially for girls

III.2.1 Measure 2.1To increase enrolment rates particularly for girls at all levels of education and vocational training

Specific Objectives: • To increase enrolment rates particularly for girls in secondary education and VET • To decrease drop-outs particularly for girls

68 Reservation should further be made to cross-regional benchmarking due to the uncertainty in administrative records. The Interim Evaluation is sure that İŞKUR does every possible effort to ensure valid data through QA measure within its organisation, but so does every country and still errors are very visible. The point is that errors occur and become highly visible notably through cross-section analyses, perhaps due to light divergences in local practices in registration.

197

• To increase vocational skills and competences of labour force, particularly of girls and women

• To promote awareness-raising for parents on importance of education, especially for girls.




13

Increased percentage of girls accessing to education particularly secondary education as a result of programme intervention.

77 % 80-85 % 85% School enrolment ratesMoNE statistics, MoLSS Programme Reports, Final Beneficiary Reports

14

Reduction in the number of girls dropping out of secondary education and VET

NA(It will be determined after the measure is implemented)

15 % reduction

17% School enrolment rates, MoNE statistics, MoLSS Programme Reports, Final Beneficiary Reports

15

Increase in the number of girls graduating from secondary education and VET


15 % increase

17% School enrolment rates, MoNE statistics, MoLSS Programme Reports, Final Beneficiary Reports




13


Increase in the access rate of education, particularly secondary education for girls

77% Increased percentage: 80- 85% 3- 8% increaseOutput indicator: 50 000 girls

491

14


Decrease in girls' dropout rates at secondary education and VET schools


15% reduction No accessible data yet

15



15% increase No accessible data yet


Table 2.1.3 Output indicators on Measure 1.1, Priority axis 1 as found in MISRef. Indicator Target Achieved

13Increased percentage of girls accessing to education particularly secondary education as a result of programme intervention.

12,808 1,880

14 Reduction in the number of girls dropping out of secondary education and VET

4,878 -240

15 Increase in the number of girls graduating from secondary education and VET

9,133 -14Source: Monitoring Information System, Operating Structure, MoLSS


198


Indicator 13 reads “Increased percentage of girls accessing to education particularly secondary education as a result of programme intervention”. The percentage of girls accessing to education is not understood, since we (again) lack a denominator. We shall assume that the author might have meant “number of girls accessing to education particularly secondary education as a result of programme intervention relative to total girls in the same age group”.

Nevertheless, the indicator has been revised in the OIS into “Increase in the access rate of education, particularly secondary education for girls”. We shall presume that access means enrolment, which simply transform the indicator into something which in common use is titled “enrolment rates”.

Accordingly: “enrolment rates of girls with special focus on secondary education levels”.


It is generally not normal to have a measure, objective or a priority which is named after an indicator, but we will leave that out of here. Yes, the indicator is by all means a well-known Outcome Indicator.


Certainly, the indicator reflects a relevant and sustainable process which is of high priority of the HRD OP.


Yes, the indicator is very relevant for the HRD OP and the challenges facing the NUTS 2 Regions.


Data is available from TÜİK / TURKSTAT ensuring a solid platform for evaluation.


The Interim Evaluation has no further comments.



Indicator 13 reads “Reduction in the number of girls dropping out of secondary education and VET” later in the OIS changed to “Decrease in girls' dropout rates at secondary education and VET schools”. Omitting the “decrease-thing” the indicator is: “Dropout rates for girls at secondary education and VET schools”. That is a well-known indicator.


The indicator is an Outcome Indicator by nature.3. Does the Indicator mirror or reflect a relevant and sustainable process?

199

The indicator reflects a relevant and sustainable process and addresses a general problem throughout the world.


Indeed, the indicator is relevant for the overall purpose of HRD OP as well as the challenges facing the NUTS 2 Regions.


The TAT has not obtained any data on the subject. The data is commonly based upon administrative records which generate a general warning. Otherwise, data should be within reach with some effort from Ministry of National Education.


Making the reservation of the quality of the data based upon administrative records, mainly administrative records originating from single and more or less autonomous institutions with presumably divergent approaches to QA measures, the Interim Evaluation has no additional comments to the use of the Outcome Indicator.



Indicator 13 reads “Increase in the number of girls graduating from secondary education and VET” which transformed into something readable and doable becomes: “number of girls graduated from secondary education and VET”. Again, the indicator is well-known and commonly used.


Also this indicator is an Outcome Indicator by nature.


The indicator is relevant for the overall process of the HRD OP.


Indeed, the indicator is relevant for the overall purpose of HRD OP as well as the challenges facing the NUTS 2 Regions.


Data are available from TÜİK/TURKSTAT.


200

The Interim Evaluation has no comments to the use of the Outcome Indicator.




Data differs from region to region (not standardized) 8.7% 2Data are not systematically collected (e.g. timing differs between projects and/or regions) 4.3% 1


13.0% 3


III.2.2 Measure 2.2To improve the content and quality of Vocational and Technical Education

Specific Objectives: • To increase the quality of vocational and technical education and training addressing

curriculum, teachers, learning environment and school managers• To increase flexibility of the education system by facilitating vertical and horizontal

transitions between different types of programmes in the VET system• To make VET an attractive option for students• To promote cooperation between social partners, schools and VET centres and private

sector


Count Indicator Baseline (2006) Target (2009)Target (2011) Source of

Verification

16

The level of satisfaction of teachers, students and entrepreneurs who participated in the activities regarding the content and quality of Vocational and Technical Education.

N.A.(Baseline data will be determined when the measure implemented)

50 % 70% Surveys, MoLSS Programme Reports, Final Beneficiary Reports

17

Increasing demands for 45 pilot vocational schools.

N.A. (It is not possible to measure it at the moment)

Min. 20-25 % increase in enrolment for each individual supported vocational school

Min. 20 % increase in enrolment for each individual supported vocational school

Student pre-registration records, MoLSS Programme Reports, Final Beneficiary

201

Count Indicator Baseline (2006) Target (2009)Target (2011) Source of

Verification(demands for 45 pilot vocational schools will be increased to 20-25 % after the implementation of the project in 45 pilot institutions)

(demands for 45 pilot vocational schools will be increased to 18 % after the implementation of the project in 45 pilot institutions)

Reports

18

Teacher training in line with the newly updated curriculum.

N.A. 1500 Teachers 4000 Teachers MoLSS Programme Reports, Final Beneficiary Reports


Table 2.2.2 Result indicators on Measure 1.1, Priority axis 1 as found in SARCount

Indicators Baseline Target Expected contributio

nHRD OP OIS

16


Increased percentage of the teachers, students and entrepreneurs satisfied with the content and quality of Vocational and Technical Education.

N.A.(Baseline data will be determined when the measure implemented)

50 % (satisfaction of teachers, students and entrepreneurs will be increased to 50 % after the implementation of measure in 45 pilot institutions)

17


N.A. (It is not possible to measure it at the moment)

Min. 20-25 % increase in enrolment for each individual supported vocational school (demands for 45 pilot vocational schools will be increased to 20-25 % after the implementation of the project in 45 pilot institutions)

18Teacher training in line with the newly updated curriculum.

The indicator is not mentioned in the SAR




Indicator 16 reads “the level of satisfaction of teachers, students and entrepreneurs who participated in the activities regarding the content and quality of Vocational and Technical Education” later in the OIS revised to read: “Increased percentage of the teachers, students and entrepreneurs satisfied with the content and quality of Vocational and Technical Education”.

Omitting “increase” and changing the word “percentage” into the more correct term “share”, the indicator is “the share of teachers, students and entrepreneurs satisfied with the content and quality of Vocational and Technical Education”.

202

The indicator calls for designated surveys. “Satisfaction” is a difficult measure to handle but the method is widely used. From a more theoretical standpoint it tends to become non-sense to use the indicator since there will be no baseline, and since satisfaction is extremely subjective and not explicitly specified from a qualitative point of view. But that discussion is not designed for an Interim Evaluation Report.


“Satisfaction” as an indicator is still undergoing theoretical discussions on where to place it; Output or Outcome. For the sake of the purpose of this very report the Interim Evaluation will accept the indicator as an Outcome Indicator.


Yes, the indicator, if done properly, reflects a relevant and hopefully sustainable process toward gradually better, relevant and market-oriented trainings.


The indicator is highly relevant subject to the Priority and its measures.


Apparently, no data are available yet, but presumable the proper means has been established to ensure data.


The Interim Evaluation has some general theoretical reservation towards the indicator, but accepts is relevance. The Interim Evaluation has no further comments.



Indicator 17, “Increasing demands for 45 pilot vocational schools” is not at all understood. The word “Demand” has to be specified and transformed into something measurable.

From the description of the target it appears that the stakeholders want to measure demand as “increase enrolment” which is pure non-sense; enrolment is not a unified measure of pure demand, since various factors, including institutional, structural / economic, and infra-structural factors, have heavily impact on students’ choice of education.

The point is, of course, that demand measure voluntary behaviour. In order to use enrolment rates as a measure of demand, the measure has to be cleaned from all factors which are not of behavioural origin. That task might be impractical perhaps even impossible.

Another way to measure demand is to establish surveys directed towards those pupils / students who consider their choice of education.

203

In general, the Interim Evaluation is uncertain of the meaning of the indicator and cannot find the way it is supposed to be performed. For the sake of the report the Interim Evaluation presume that the indicator reads “enrolment rates in vocational schools”.

In that case the indicator is the same as Indicator 13.



Indicator 18 reads: “Teacher training in line with the newly updated curriculum”. The indicator is not mentioned in the SAR.

The Interim Evaluation does simply not understand the indicator at all. Presumably it is established to measure the “number of teachers trained in newly developed curriculum”. In that case the indicator is an Output Indicator by nature.






III.3 Priority axis 3 To increase adaptability of workers, enterprises and entrepreneurs, in particular by promoting lifelong learning and encouraging investment in human resources by enterprises and workers

III.3.1 Measure 3.1Promoting the development and implementation of coherent and comprehensive strategies for lifelong learning

Specific Objectives: • To provide basic skills and competences especially for the disadvantaged and women• To improve quality of non-formal training in line with the LLL perspective• To develop a comprehensive new approach to valuing learning to build bridges between

different learning context and learning forms and to facilitate access to individual pathways of learning.

204

• To strengthen the existing VET institutions for organizing VET courses in line with LLL strategies and according to the ECVET principles.

• To support establishment of Vocational Standards Development, Testing and Certification Centres (VOC-TEST) as accredited bodies of vocational standard development, testing and certification under the monitoring and supervision of Vocational Qualifications Authority.

• To support the establishment of National Qualification System• To promote the linkage between LLL, education system and labour market



Target (2009)


19

Share of primary education and secondary education graduates among participants certified and tested in terms of vocational knowledge and skills in 10 VOC-TEST Centres

30 % to be graduates of primary education; 70 % to be graduates of secondary education


Enrolment statistics, Attendance rates, MoLSS Programme Reports, Final Beneficiary Reports

20

Increased percentage of participants who obtained a certificate in vocational knowledge and skills

313.584 (2005 data from Public Education Centres)

10-15 % increase

16-20% Enrolment statistics, Attendance rates, MoLSS Programme Reports, Final Beneficiary Reports

21

Increased percentage of participants who obtained a certificate in basic skills


15-20 % increase

16-20% Enrolment statistics, Attendance rates, MoLSS Programme Reports, Final Beneficiary Reports

22

Increased percentage of those who found work after completing an employability course

NA 10-15 % 16-18% Enrolment statistics, Attendance rates, MoLSS Programme Reports, Final Beneficiary Reports




19



190

20

Increased percentage of participants who obtained a certificate in vocational


10-15 % increase 384 (LLL) + 2977 (PWE) + 3068 (PYE) App. (-) 20000

205

Count Indicators Baseline Target Expected contributionknowledge and

skills

21



15-20 % increase 962+120:1082 App. (-) 15000

22


NA 10- 15% %63 (194/306) (LLL)


Table 3.1.3 Output indicators on Measure 3.1, Priority axis 3 as found in MISRef. Indicator Target Achieved

19


418 158

20 Increased percentage of participants who obtained a certificate in vocational knowledge and skills

2,178 1,243

21 Increased percentage of participants who obtained a certificate in basic skills

2,100 1,718

22 Increased percentage of those who found work after completing an employability course

575 209




Indicator 19 reads “Share of primary education and secondary education graduates among participants certified and tested in terms of vocational knowledge and skills in 10 VOC-TEST Centres”. The indicator is understood though the phrasing could need a hand.


It is not crystal clear to the TAT what the specific purpose of the related specific objective is: “To support establishment of Vocational Standards Development, Testing and Certification Centres (VOC-TEST) as accredited bodies of vocational standard development, testing and certification under the monitoring and supervision of Vocational Qualifications Authority”. And that fact makes is slightly difficult to understand the indicator.

If the purpose is to “support” the VOC-TEST centres forcing primary education and secondary education graduates to enter the school might soon appear to be for the benefit of establishing the VOC-TEST Centres and not the in the pursuit of promoting Lifelong Learning as such. And in that case the indicator appears to be an Output Indicator.

In other words, the Interim Evaluation is in doubt of the relevance of the specific object as well as the indicator. However, for the sake of the report the Interim Evaluation ignores this light reservation.

206


Since the Interim Report ignores its general reservation towards the objective and the indicator, and instead believes the indicator serves a purpose, the Interim Evaluation can agree that ensuring certificates for primary education and secondary education graduates in terms of vocational knowledge and skills could potentially reflects a relevant process.


Basically, the TAT does not know if the indicator is relevant to the overall purpose of the HRD OP. The TAT is not convinced at all.


The TAT has received no data on the indicator, which cannot be given by TÜİK/TURKSTAT. The TAT is in doubt of the purpose of the indicator’s relevance, and cannot answer the question of the data quality from a theoretical perspective.


Since the Interim Evaluation is uncertain about the benefit of this indicator other far more vital stakeholders might sense the same. It is highly recommended to ensure a solid knowledge of the purpose of the indicator and from there take a discussion of the relevance of the indicator. Basically, the Interim Evaluation sees no point in having the indicator as an Outcome indicator at all.



Indicator 20 reads “Increased percentage of participants who obtained a certificate in vocational knowledge and skills”. The denominator in “percentage of…” is not understood, and a baseline of 313,584 does simply not give a clue; it appears to be a number and not a percentage.

The Interim Evaluation assumes the correct indicator is “number of participants who obtained a certificate in vocational knowledge and skills” which has a target which stipulates an increase69.


Determination of the difference of Output and Outcome will depend on the definition of the word “participants”. In case “participants” refer to specific programmes initiated as actions or measure, the indicator will be an Output indicator. In case the author basically thought of participants who by own means and effort are engaged in training the indicator will obviously be an Outcome indictor. The degree of voluntary behaviour is accordingly important.

The Interim Evaluation acknowledges the difficulties in finding the right phrasing and shall subsequently assume that the intention refers to the latter of the two. Accordingly, the indicator is an Outcome Indicator.

69 A main source of the problem is the overall measure, obviously. The measure seeks to stimulate Lifelong Learning which makes it fairly difficult to spot a natural denominator. Percentage of day-students (initial VET/TVET) would not be reasonable, and the same goes with total students as well as total employees.

207


Upgrading the labour force is always relevant.


The indicator is relevant to the measure and the challenges facing the NUTS 2 Regions for sure.


The TAT has seen no data. The TAT shall assume that data on number of participants who obtained a certificate in vocational knowledge and skills is very available or at least the data can easily be produced.


The Interim Evaluation has no comment to the choice of Outcome Indicator but to sharpen the name of the indicator.



Indicator 21 reads “Increased percentage of participants who obtained a certificate in basic skills” which generates the same mystery as in indicator 20 above. The denominator is simply not clear.

Reformulating the indicator to “number of participants who obtained a certificate in basic skills” can be understood70.

The line of arguments follows the points noted under indicator 20 above.



Indicator 22 reads: “Increased percentage of those who found work after completing an employability course”. Presuming that the denominator is the total participants in employability courses the indicator is “number of participants from employability course who obtains a job after completion relative to total participants on the employability courses initiated”.


The indicator is an Outcome indicator.


The indicator reflects a relevant process of the HRD OP.


70 Please note the comments to the phrasing “participants” under Indicator 20 above. The problem is not severe in this indicator since the indicator basically refers to the effect “employment”.

208

The Interim Evaluation would have expected to see some sort of qualitative approach to the issue, which could have been added with a requirement of e.g. “number of participants from employability course who obtains a job which mirrors the upgraded qualifications gained after completion …”

Otherwise, the indicator appear to be appropriate and in line with its purpose.


The indicator will have to be based upon combining administrative records which generates a general warning (a reservation rather) as noted several times above. In addition, the indicator might imply a need for follow-up procedures which can be slightly man-power consuming; at least the administrators / training schools have to do tracer studies.


The Interim Evaluation has no additional comments to the indicator.






III.3.2 Measure 3.2To increase adaptability of employees and employers by investing more in human capital

Specific Objectives: • To improve adaptability of employees in SMEs by facilitating their access to training, in

particular for low-skilled workers, • To improve adaptability of employers in SMEs by designing and disseminating innovative

and sustainable forms of work organization, which support labour productivity and quality at work,

• To increase training capacities of social partners, NGOs, VET Institutions and public bodies in order to establish capacity to facilitate adaptability of the employees and employers.


209


Target (2009)


23

Increased adaptability of workers in terms of having extra skills or abilities related to their position (percentage of participants increasing their salary or being promoted or having extra responsibilities)

N.A. 15 % 15% Surveys, Records of Final BeneficiariesMoLSS Programme Reports

24

Increased adaptability of employers in terms of having extra skills or abilities in order to be competitive enough (percentage of participants increasing the number of workers, change of the branch of activity )

N.A. 15 % 15% Surveys, Records of Final BeneficiariesMoLSS Programme Reports




23


NA 15%

24


NA 15%

25

Increased training activities on the requirements of adaptability given by social partners to their members

NA 20%



210


Indicator 23 reads: “Increased adaptability of workers in terms of having extra skills or abilities related to their position (percentage of participants increasing their salary or being promoted or having extra responsibilities)”.

The Interim Evaluation finds it very difficult to understand how the indicator should be measured. Basically, there are two distinct indicators which are combined, of which second is mentioned in parenthesis. Both of the two indicators appear to create severe difficulties in measurement. In fact, the TAT has never found similar practice elsewhere.

The problem of the first part (the first indicator) is to find the workers who have achieved extra skills and abilities related to their position; it will presumably necessitate solid surveys within a very large sample of establishments. Secondly, all data has to be coded by ISCO-level as well as by training courses achieved. As far as the TAT knows Turkey has no database on education and training programmes by domain, including LLL training programmes. And finally, time is essential; participation in trainings five or ten years ago does not really count anymore, and yet we would not expect progression to happen as soon as the training has been concluded.

The problem with the second indicator is related to the last part; it appears quite impossible to obtain detail data on individual level. In addition, it appears far too narrow to believe that salaries and promotion is given by upgrade skills alone.

Accordingly, the indicator, however defined, is almost criminal problematic and not least extremely difficult to evaluate upon if data should be available in a million years.


The indicator is an Outcome indicator insofar the target group of the indicator covers all workers and not just is directed to a selected number of participants in selected training programmes. The use of the word “participants” in the second part of the indicator’s name is accordingly problematic. If the indicator only measures those who has participated (perhaps relative to those who has not participated) the indicator is an Output indicator.

Since we would believe that the idea of the indicator is to measure the wage level and job functions within a restricted pool of workers, who has recently participated in a training course, a relate the study to an average or at least a selected control group, the TAT tend to believe that the indicator is designed as an output indicator.

3. Feedback from survey on indicators among the stakeholders on Result Indicator 23




The indicator does not mirror the real target / is not relevant 5.1% 2The target group is not clearly defined 5.1% 2There is no baseline data (we cannot see and measure improvements) 5.1% 2

211

The indicator is too complicated to calculate (if based upon aggregated data) 20.5% 8Other (please specify) 2.6% 1The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders view on where the main problems lie.


Relevant or not, data available or not, the Interim Evaluation does not believe in the indicator for a number of reasons. The Interim Evaluation recommends deleting the indicator.



Indicator 24 reads: “Increased adaptability of employers in terms of having extra skills or abilities in order to be competitive enough (percentage of participants increasing the number of workers, change of the branch of activity)”.

The Interim Evaluation admires the effort of trying to measure “adaptability” both in indicator 23 and now yet again in indicator 24.

The phrasing in the parenthesis, which presumably was meant to explain the indicator, is not understood: What is meant by “change of branch of activity” and how can that ever be relevant for Human Capital Investments? The TAT assumes that “participants increasing the number of workers” indicate the number of enterprises which experience employment growth after the employer has participated in a certain training course under the heading of Human Capital Investments.

The indicator is not measurable and even lesser practical. Perhaps a survey can be designed, but it is highly doubtful that the outcome of it would be valid, significant and interesting.


The indicator, even if it worked, would be an Outcome Indicator, though somehow only inferior related to the trainings and the programme. The reason for accepting the indicator as Outcome Indicator is solely that it measures a potential side-effect arising from certain, named and specific training programmes for a selected few final beneficiaries. It becomes an Outcome Indicator simply because it is not a direct Output Indicator; a more correct term would be “induced output”, but that would be too technical in this context.

3. Feedback from survey on indicators among the stakeholders on Result Indicator 24

What is the problem of the indicators under this measure, if any? Share NNet impact cannot be measured since Impact can arise from other factors in the environment

18.6% 8

Data is not available 11.6% 5Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of reports, etc.)

9.3% 4

Data differs from region to region (not standardized) 4.7% 2Data are not systematically collected (e.g. timing differs between projects and/or regions) 11.6% 5Data is only generated from on-going projects (we cannot see what status are in locations without projects, before the project is launched, and after the project has been concluded)

7.0% 3

The indicator does not mirror the real target / is not relevant 2.3% 1

212

The target group is not clearly defined 7.0% 3There is no baseline data (we cannot see and measure improvements) 7.0% 3The indicator is too complicated to calculate (if based upon aggregated data) 16.3% 7Other (please specify) 4.7% 2The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders view on where the main problems lie.


The indicator will not work for various reasons. Even if data exist employment growth is hardly ever correlated with the skills of the employer. And changing branch or economic activity is way out of bounds in this context.

The Interim Evaluation highly recommends deleting this indicator.



Indicator 25 reads: “Increased training activities on the requirements of adaptability given by social partners to their members”. The indicator is not mentioned in the HRD OP; it appears only in the OIS.

The TAT understands the indicator and agrees that it might be measurable through surveys, though far from common practice.


Again, and as in indicator 24, the indicator, even if it worked, would be an Outcome Indicator simply because we cannot term it correctly as “induced output”. It measures a side-effect of a given programme.


The Interim Evaluation recommends deleting the indicator.

3.2.4 Recommendations to Result Indicator no. 23-25

Apparently, the Interim Evaluation proposes to delete all the chosen Result Indicators underneath Measure 3.2 partly since the selected indicators will be highly difficult to measure on objective grounds and partly since the indicators are designed to measure side-effects which might arise from a billion other (more relevant) reasons that the participation in a certain training programme.

Of course, the Interim Evaluation cannot leave it with that. The problem with this measure is the measure itself: “To increase adaptability of employees and employers by investing more in human capital”. The measure has presumably inspired the stakeholders (the authors) to design impossible indicators; impossible since the authors have become blind by focusing too narrow on some single words in a far broader context; “adaptability” and “Human Capital Investment”. Perhaps the authors could not see that the terms are hardly measurable in quantitative terms.

Sometimes we simply have to believe in positive outcome. Sometimes we do not need proof to know that our effort pays off. But more so, when it comes to measuring the un-measurable other methods and techniques are called for; techniques which cannot be applied and conducted within a fortnight.

213

The effect of investment in Human Capital is proven and well-documented in scientific reports. If we want to proof the effort we do, we have to apply a scientific approach, simply because the topic cannot be measured from smart techniques alone. It would most presumably take a long range of indicators to measure a presumed effect (perhaps through advance statistical techniques like e.g. Factor Analysis), and most presumably the conclusions will be weak (not significant) given the short time the new paradigms have had an effect.

That fact might well look as an easy solution for a TAT doing an Interim Evaluation, but there is more to it. The main point is to understand that some topics / measures / ideas / initiatives are simply not designed to be measured in smart numbers. But that does not necessarily indicate that we lack means to evaluate our effort; in this case it would be far cleverer to look at long term effects, which basically calls for Impact indicators.

Impact may arise from various reasons, and we would never link impact indicators directly to single measures. However, we would expect to see a change, gradually, as time passes on. Sometimes the changes or the adjustments happen relatively fast allowing us to measure the impact, say, within a year. Other times, impact takes longer time to show.

In case we want to measure “adaptability” and “Human Capital Investment” we are simply without means. These terms do not come in terms of numbers and sizes. Not directly at least. And furthermore, the complete context of the measure is basically a matter of change in behaviour, which never comes easy and never is done by training courses alone.

In order to keep this section short, the Interim Evaluation would give priority to defining Impact indicators even though the Impact indicators may work on much broader and general level than the single measure.

An alternative would be to keep the above mentioned induced output indicators, broaden them up to mirror a wider group (all employers with a certain economic sector), and then ask the Trade Unions, the Chambers, and the Employers’ Association to conduct surveys once a year on one single issue: Contribution to Human Capital Investment (or rather: “How much did your company spend on training of your employees?”). Obviously, that measure would be the correct Outcome Indicator in this context.The final recommendation here from the Interim Evaluation would be: Never do the impossible and do not build monsters, where it is not needed! Monitoring and Evaluation is of paramount importance, agreed, but no decision maker or stakeholder expects progress reports and evaluation reports to be a scientific and long-lasting discipline.

Far too often Monitoring and Evaluation consumes far too many resources. Basically, that is a very poor investment. The stakeholders want answers now. The main point is to design few smart indicators – it is not to create a monster which no one can comprehend.


To promote an inclusive labour market with opportunities for disadvantaged people, with a view to their sustainable integration into the labour force and combat all forms of discrimination in the labour market

III.4.1 Measure 4.1To increase employability of disadvantaged persons, facilitate their access to labour market, and eliminate barriers for their entrance into labour market

214

Specific Objectives:• To increase employment and labour force participation rate of disadvantaged persons • To facilitate disadvantaged persons’ access to labour market • To eliminate barriers for disadvantaged persons’ entrance into labour market



Target (2009)

Target (2011)


26


90 % 95 % 95% İŞKUR Job Seeking Records, MoLSS Programme Reports, Final Beneficiary Reports

27


20 % people with disabilities13 % ex-prisoners

30 % people with disabilities19.5 % ex-prisoners

%35 people with disabilities

%25 ex-prisoners

İŞKUR Job Placement Records, SSI Records, MoLSS Programme Reports, Final Beneficiary Reports




Indicator 26 speaks again of a rate which is not clearly defined. Rephrasing the indicator it becomes: “the number of disadvantaged persons entered into labour market (having the qualification and motivation to seek a job) following the rehabilitation programmes, courses and counselling”.

As in indicator 1 above the indicator speaks of “labour market” which we translate to “labour force”, and as in indicator 1 above the indicator speaks of qualification and motivation which cannot be measured and is if not further specified.


Accordingly, the indicator should read: “number of disadvantaged persons entered into labour force (employed or unemployed) following the rehabilitation programmes, courses and counselling” which is an Output indicator. The argument is again the same as in Indicator 1.


Let us then turn to indicator no. 27, which obviously will receive the same comments as Indicator no. 2 above. Accordingly, “number of disadvantaged persons obtaining a job after participating in rehabilitation programmes, courses and counselling”.

The indicator is again an Output indicator.

215

4.1.3 Feedback from survey on indicators among the stakeholders on Result Indicator 26 and 27What is the problem of the indicators under this measure, if any? Share NNet impact cannot be measured since Impact can arise from other factors in the environment 20.0% 4




4.1.4 Recommendation for Indicator 26 and 27

The Interim Evaluation recommends doing the same with the two indicators (indicator 26 and 27) as above under indicator 1 and 2; to establish an employment rate / labour force participation rate, or at least obtain an indicator of number of disabled persons entering the labour force.

Of course, we expect that to increase after the commencement of the HRD OP and establishing rehabilitation programmes, courses and counselling, but we do not specify that (since that turns the indicators into Output indicators).

III.4.2 Measure 4.2Better functioning and coordination among the institutions and mechanisms in the field of labour market and social protection particularly in order to facilitate the integration of disadvantaged persons into the labour market.

Specific Objectives:• To improve the institutional capacities of public service providers to disadvantaged

persons in their employment related services, and the functioning of existing mechanisms, • To ensure better coordination among public service providers to disadvantaged people,

including between central and regional/local authorities and in cooperation with NGOs, in employment related services



Target (2009)


28

Increased satisfaction on the services both in institutional and target group level (by staff of institutions and by disadvantaged persons)

N.A. ( Data will be collected when the measure implemented)

25 % N.A. Surveys, MoLSS Programme Reports, Final Beneficiary Reports



216

The evaluation of indicator 28 follows the same line of arguments as noted under indicator 16 above. The indicator is an Outcome indicator and relevant. However, the indicator necessitates surveys which makes it difficult to specify baselines and makes it impractical on a long term scale71.

The Interim Evaluation has no further comments.

4.2.2 Feedback from survey on indicators among the stakeholders on Result Indicator 28What is the problem of the indicators under this measure, if any? Share NNet impact cannot be measured since Impact can arise from other factors in the environment 13.9% 5




III.5 Priority axis 5: Technical Assistance To improve the coordination, control and implementation of HRD OP and to assist the Operating Structure and relevant institutions in the further programming, monitoring, evaluation and implementing tasks.

Specific Objectives• Strengthening the necessary capacity at central level to efficiently develop, implement,

evaluate, monitor and control the IPA funds in the period 2007-2011 within the framework of HRDOP and improving the information about the ESF structures and best examples, providing assistance in the transition to Decentralised Management without Ex-ante controls of the EU Delegation.

• Increasing the absorption capacity of the final beneficiaries and administrative capacity of stakeholders which may get more responsibilities in the upcoming period.

• Increasing the information and public awareness with respect to the preparation for and effective use of the IPA funds in Turkey in line with the HRDOP.

III.5.1 Measure 5.1Support for Programming, Management, Implementation, Monitoring, Control, Evaluation and Dissemination Activities

Specific Objectives• The main aim of this measure is to provide support to the OS in different steps of the IPA

process and to increase the capacity of the relevant central and local partners in the related issues.

There is no Result Indicator aligned with the measure.

71 It is doubtful that the surveys will be conducted after the completion of the HRD OP.

217

III.5.2 Measure 5.2Support for development of absorption capacity of final beneficiaries

Specific Measures• The main aim of this measure is to improve absorption capacity of the final beneficiaries in

order to develop a project pipeline in the framework of HRD OP.


III.5.3 Measure 5.3Information and publicity activities

Specific Objectives• The objective of this measure is to assist the implementation of obligations arising from

Article 62 of IPA Implementing Regulation concerning the promotion of the HRDOP and its operations and informing entities interested in receiving support from the HRDOP, as well as the general public, about the opportunities provided by the assistance and its outcomes.


III.5.4 Recommendations to Priority axis 5

The lack of indicators, notably Outcome Indicators, under Priority axis 5 is understandable to some extent; the complete axis and the underlying measures are devoted to technical assistance which only works for a temporary period.

Nevertheless, Outcome indicators could be established since the purpose of the technical assistance is to make capacity building activities for the operation beneficiaries, the Ministries and their sub-ordinates, as well as the operating structure. Obviously, the increase capacity does not only apply to the management of external aid programmes; hopefully, the main aim of the technical assistance programmes is to increase the overall capacity within the involved organisations and hereby ensure better means to carry out daily tasks.

In that case perhaps the specific objectives clings too much to the IPA, though that is understandable; perhaps the specific objectives makes the stakeholders blind for the long term effect of the technical assistance and thus blind when it comes to define and establish Outcome Indicators.

The Interim Evaluation Report will propose recommendation to both Outcome and Impact indicators in a more generic and general section below, partly since both Outcome and Impact indicators should be based upon a broad perspective where the overall purpose the HRD OP in general should be taken into account, and partly since the list of indicator within the HRD OP for Turkey is far too long.

However, in case the Operating Structure and the other main beneficiaries find it useful to establish indicators for each of the priority axes within the HRD OP, Interim Evaluation could propose the following Outcome Indicators72:

• The alignment in the institutional structure with recommendations in JAP, where the target could be fewer comments in the chapters under negotiation with the EU. The measure could be ratified by EC documents on the progress of adjustment73;

72 The following Outcome Indicators are merely means as inspiration and are not tested73 It could also be the number of law packages in alignment with the EU passing through the parliament

218

• Satisfaction with the proposed institutional changes recommended, which could be based upon surveys within the benefitting organisations74. The indicator could further be supported by the following:o The institutional changes followed by recommendations on behalf of the technical

assistance75

PART IV. Reflexions upon the studies in Part I, II and IIIIV.1 Findings, main conclusions and recommendations

The Interim Evaluation has been able to recognize some 95 Output indicators from the HRD. In addition we further investigated 28 Result indicators, which can be divided into the following:

• 9 indicators are Output indicators• The remaining 19 indicators are all Outcome Indicators of which:

o 10 Outcome Indicators are workable in practice o 4 Outcome Indicators could work with some light adjustmentso 5 Outcome Indicators cannot work in its present form and needs to be adjustedo We found not even one single Impact Indicator

In total we have identified 104 Output Indicators, 19 Outcome Indicators and 0 (zero) Impact Indictors. Of the 19 Outcome Indicators, ten of them passed the Evaluation without remarks, 9 of the indicators can be workable with some adjustments. 5 Result Indicators cannot work in practice.

The two tables in the end of this Annex presents the conclusions and recommendations from the study on both the Output Indicators in Part II as well as the conclusions and recommendations from the study on Result Indicators in Part III.

Obviously, we are talking about a very high number of indicators, which makes it close to impossible to undertake evaluation reports; each indicator will have to examined in time series analysis throughout the implementation and the following years, and each of them has to be examined in various cross-sections studies, not just on regional levels but toward target groups and control groups. Using a conservative estimate based upon experience we could easily suggest that each indicator would require close to 5 tables, with presumably another 5 tables in annex, and close to 6 pages description. Due to the number of indicators it would generate evaluation reports of 700 pages, which obviously would take months to produce. And yet the HRD OP does not even have any Impact Indicators making it impossible to evaluate the effect of the HRD OP.

Keeping track of a record of 104 output indicators is very appreciable, but no one bother read the lengthy report the evaluation will create in that case, basically indicating that the system creates a sequence of reports where only a fraction of the conclusions will be read, understood and acted upon.

In that respect we have to recall that the burden of monitoring and evaluation does not fall on single M&E experts on central level; the burden is transferred to the complete system, perhaps in some cases even to final beneficiaries; I.e. finding and creating data, controlling data, ensuring timely input, ensuring quality procedures in order to produce data of sufficient quality, follow-up, and making comments. Forcing discipline through does not make better data and input to the evaluation reports if the burden obviously is caused by some few central stakeholders’ unreasonable need to know every single detail. And that comment speaks for some few words on the Monitoring Information System, MIS, which appears to have turned into a nightmare, at least if we should believe the main stream of comments

74 Doing surveys within the public sector is generally much easier than market surveys within the private sector75 Agreed, it would be very difficult to set targets here, since we cannot set a value number of changes and it is even not certain that it is clever to expect the number to decrease. The measure should be a qualitative measure.

219

received from both central and regional stakeholders. We have added some few comments to the Monitoring Information System in Chapter 5 above.

IV.1.1 Impact Indicators

The programme lacks Impact Indicators. Without impact indicators it becomes impossible to see the real effects of the programme. By real effects we mean the impact on the local / regional economy, the labour market and the welfare.

The Impact indicators will become vital for the final evaluation of the programme. The final evaluation will require means and tools to do a fair and solid evaluation of the impact of implementing so far 431 projects in 12 NUTS 2 Regions. In order to make the impact assessment data should be secured for a long time period ensuring solid means to do impact studies across time. And the indicators should be agreed upon long before the evaluators arrive; it would be impossible for the evaluators to establish the evaluation criteria and the indicators on top of the impact assessment.

The complete lack of impact indicators tells us that the difference between output, outcome and impact has never been understood by the stakeholders. But even more it tells us that the stakeholders do not even know what impact indicators are76.

Let us repeat: Impact Indicators measures the consequences of the outcomes in terms of wider objectives (for example literacy rates, health improvement). The definition covers the wider effects of the outcomes but there might also be higher level impacts, related to broader objectives – growth and income poverty, for example. In order to turn it into a net impact indicator the effect of other economic, educational, and employment programmes have to be assessed and accordingly subtracted.

Let us provide some examples from this HRD OP context.

Example 1. Education. What is Impact here?

Priority axis 2: To enhance investment in human capital by increasing the quality of education, improving the linkage between education and labour market, and raising enrolment rates at all levels of education, especially for girls

Measure 2.2: To improve the content and quality of Vocational and Technical Education

Specific Objectives: o To increase the quality of vocational and technical education and training addressing

curriculum, teachers, learning environment and school managerso To increase flexibility of the education system by facilitating vertical and horizontal

transitions between different types of programmes in the VET systemo To make VET an attractive option for studentso To promote cooperation between social partners, schools and VET centres and private

sector

The following indicators were selected:1) The level of satisfaction of teachers, students and entrepreneurs who

participated in the activities regarding the content and quality of Vocational and Technical Education.

2) Increasing demands for 45 pilot vocational schools.

76 Otherwise we would have seen at least one impact indicator

220

3) Teacher training in line with the newly updated curriculum.

But what is impact of improving education? Is that smiling students? Larger enrolment? Training of teachers?

The reason for investment in education (for students) is obviously to obtain better job opportunities. The reason for investing public funds in education is obviously because we believe it creates larger opportunities to establish a well-developed economy. Accordingly, we do not invest in education for the sake of teachers, school directors or just in order to park some teenagers for a period of time: It serves a purpose, that is.

Impact measure the purpose. Here it is obvious to establish an Impact indicator which mirrors and reflects advancements in education. The most prominent Impact indicator of them all is subsequently: Productivity.

Other indicators could be:• Literacy• Health indicators (we shall return to them)• GDP / inhabitant and economic growth measures• Increased production of advanced goods (some regions experience more or less

a mono-cultural production)

In order to establish Impact indicators we need to look up from the computer and the books and think: Why do we do this? What do we want to achieve?

The next step in Impact indicators is to find indicators with solid and consistent data. In other words we need official statistics, hopefully well-documented international harmonized indicators allowing us to see how much we, Turkey, has improved relatively to the others.

We, the TAT, cannot propose indicators to the HRD OP. That would be madness since proposing indicators necessitate that you are a real part of the system. The TAT had meetings with some of the key stakeholders where they were allowed to reflect over the chosen indicators. The meetings were inconclusive. However, the meetings made clear that the chosen indicators were insufficient and in some case directly wrong. Most of the comments have been integrated in the study of Result indicators above.

The meetings would be an obvious place to start a reform of the indicators. Indicators have to make sense; they should reflect the environment and they should be understood by the real stakeholders.

The table below presents some ideas to pathways for designing the type of indicators which are needed for larger programmes, including impact indicators. Further well-proven and ready-made indicators are briefly presented in Appendix to the Study on Indicators in the end of this study.

Finally, the meeting with the DG OSH in the Ministry of Labour made very clear to all, including the TAT, that the HRD OP seriously lack indicators on OSH and health. It appears obvious to include DG OSH in future programmes, but also in the revision of the indicator system of the present HRD OP.

Table 9.6.1 Example of pathways within education to design of indicators using examples of combined markers with direction of targets77

77 The mentioned effect is scientifically proven. Please note that “marker” represent themes, and not necessarily indicators. Please note that the impact markers could fall into every category of inputs

221

INPUT OUTPUT OUTCOME IMPACT

More spending on education

More teachers hired Better learning

materials Better schools

(physical)

More books at libraries borrowed

More local citizens educated

Faster economic growth Multi-cultural production Knowledge Society More visits to movie

theatres Better living conditions

(income)

222

INPUT OUTPUT OUTCOME IMPACT

General increase in the standard of education

Better and more advanced learning materials

National Qualification Frameworks developed

increase in salaries and wages

more local citizens qualified for attractive well-paid jobs

higher national living standard in general

increased gender equality poverty reduction longer life expectancy

Increased educational opportunities/variety of education

Increased number of citizens with a wider variety of skills and qualification

Less drop-outs

number of unskilled get semi-skilled or skilled

increased number in higher education

number of women and men training and education increased

quality of the workforce improves

higher productivity faster and sustainable

growth fewer and healthier

children improved family economy

(better housing) less drugs and alcohol

misuse less criminality improved childcare in

families and through institutions

Improved quality in educational material and curricula

Increased number of citizens with a wider variety of skills and qualification

Better standard of the workforce

A wider/broader understanding of the society and on-going changes

More citizens with a globalised and cosmopolitan orientation

Less conflicts on the labour market

“Green” economicsTailor made training courses for special target groups

Increased number educated among those who previously did not finish education or were unable to follow the education requirement

Less drop-outs

Better standard of the workforce

A wider/broader understanding of the society and on-going changes

Social consciousness less criminality higher national living

standard in general savings on national

finances increased gender equality poverty reduction

Labour market and training statistical data increased and standards improved

Increased number of and variety in data available

improved information about the labour market development

improved information on labour market needs

improved knowledge about relevant training needs nationally and in the regions

improved conditions for providing tailor made training courses

Improved conditions for political decision making

Better allocation of financial resources

Loss of money on irrelevant or less relevant training courses reduced

IV.1.2 The Multi-level System

The multi-level system is introduced in Chapter 8. The main point is to establish a system of indicators which feed into each other across the levels. Since the administration of the HRD OP in Turkey appears to like to know everything it would be an obvious solution.

A note warning is, however, that creating a multi-level system does not make the evaluation easier and the real danger is to limit time-consuming report-writing in multiple directions. The need to

223

understand what is necessary to know and what is not essential is accordingly almost reinforced by creating a multi-level system of indicators, since the multi-level system, where different indicators feed into different carefully pre-selected monitoring reports, soon can create a new need for still more reports (now that they are doable).

The objective of the multi-level system is basically quite simple. Different stakeholders within the system need different information. And above all there are the main stakeholders, perhaps the donors, who just want to know that the overall programme works in accordance to expectations.

Differentiation, or “who needs what”, is to direct reports to the key stakeholders in accordance to their interest areas, e.g. the operations they are assigned to administrate and manage.

When the selection has been made, the template for the reports can be designed ensuring an easy and smooth study and writing process. In most cases, most of the stakeholders in fact agree on a very slim template based upon the most indicative indicators. And often there is basically no need to create multi-level reports as well.

The next point is of course to understand and prepare the multi-level system for adjustments. Adjustments pops up when initiatives (or perhaps projects) exceed the limits and targets, and / or becomes more or less de-railed subject to their intensions78. It is in that case the multi-level system shows its real strengths; the point is that in case we want to know where the problem lies we need data, and in case the system has data ready at hand, the adjustment can be made promptly. That is obviously far better than being forced to call in investigators to dig through layers of reason. However, an early warning system like this comes with a fairly high cost.

In the context of the HRD OP it would be plausible to suggest the following levels:1. Overall general level with indicators on the overall impact of the complete programmes (all

four types of indicators)2. Operations level; separate report systems for each Priority axis (input, output and outcome

indicators)3. Project or micro-level (mainly input and output indicators)

Each level should have a very limited number of indicators. Furthermore, it is of vital importance that most of the indicators are inter-linked if not directly the same, reducing the overall number of indicators. And then finally, it is important to agree on the reporting standards, both in terms of data input and reporting. In most cases on subordinate levels simple progress reports (extracts of tables with few comments) should be sufficient. But it is of vital importance that data is available at the time for reporting.

IV.1.3 Main Conclusions and recommendations

In the Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010, the following comments to the functionality of the indicators were found:

“…some difficulties were encountered in determining the actual situation in terms of achievement of indicators. Although there were some problems stemming mainly from the fact that practice of monitoring the indicators was at its initial stage, these kinds of preliminary studies were beneficial from many aspects; for example, importance of entering the “performance indicators” were emphasized by the MEU and the RGMTT Experts, and the grant beneficiaries were requested to enter the targets and status of achievement in terms of

78 Recall that a too positive and strong effect can be just as problematic as weak and no effect at all.

224

indicators determined within their project. In addition, some revisions were made in the indicators in terms of expression and wording, also some indicators were divided in different parts for converting the indicator into a measurable format.”[SAR, p. 38]

Somehow, the Interim Evaluation finds the adjustments insufficient. At least the first and perhaps prime recommendation is…:

… to revise the current set of indicators, which revision should bring the number of indicators significantly down. The need for a reform is urgent since the current system of indicators are insufficient, inappropriate and yet far too ambitious. The Interim Evaluation proposes to bring the total number of indicators down to 18 indicators: six Output, six Outcome, and six Impact Indicators79.

I.e. the current system is:• Insufficient since the current set of indicators lack Impact measures• Inappropriate since many indicators have no doable baselines, lack data, lack

infrastructure, or are just not measuring the right (defined wrongly)• Too ambitious due to the number of indicators

The Interim Evaluation has of course suggestions to how ease the administrative burden of a far too ambitious monitoring and evaluation programme. The suggestions fall in three main categories:

4. Revise the indictors toughly until a decent and doable target is reached80; Understand the basics in evaluation and design a list of indicators which summarizes the main and basic idea and purpose of the overall programme;

5. Establish a multi-level system of indicators, ensuring that the main evaluation reports only contains a very small number of smart indicators. The multi-level system does not decrease the overall workload of the complete system, but at least the main work falls on the central level;


The first suggestion is surely the hardest one. On the other hand the TAT has been informed that many central level key stakeholders have never seen the present set of indicators81. And besides, it is not just a matter of a far too long list of indicators; the complete programme has no means to make evaluations due to the complete lack of impact indicators.

The second solution, which draws upon the introduction in Chapter 7, is further explained in a section below.

The third suggestion leads to open up for a far more flexible administration where it perhaps would be fair to delegate monitoring and even parts of the evaluation to other stakeholders than the central level Operation Structure. Agreed, it is exactly what the stakeholders try to achieve, but many of the stakeholders cannot find the logic yet notably on regional level. And exactly that fact makes the “burden” a real burden and difficult to create flexible solutions.

The three suggestions can easily be implemented simultaneously.

79 The mix may vary slightly, of course80 The study of indicators calls for a complete reform of the indicators. Perhaps it would be better to start from scratch.81 That goes, strangely, notably for the CGMTs by the way, which might be another point to add to our study of the management structure. It is a complete puzzle why the central and key stakeholders were caught by surprise when presented to the Result Indicators of the HRD OP. The Interim Evaluation did not follow up on the point except from this very light note.

225

Finally, the Interim Evaluation has noted that the following Outcome Indicators can be merged into single indicators without any further difficulties:

Indicator 15, 20, 21 could be merged (graduates) Indicator 3 and 5 could be merged (female and young entrepreneurs) Indicator 13 and 17 could be merged (enrolment rates)

That easy trick would reduce the number of Outcome Indicators by four. The Interim Evaluation has further considered merging some of the indicators into one single indicator on “employment rates” (see summary table on Result Indicators below). However, that step necessitates a redefinition of the selected “Result Indicators”.

226

IV.2 Summary of the examination of Output Indicators in Part II

IV.2.1 The score system usedSCORE INTERPRETATION Count Pct.

A The indicator is understood and acknowledged. The indicator is workable, indicating that smaller changes in definition might be needed in order to make it measurable and practically

17 18%

B The indicator is understood. The indicator is workable, but changes in definition or ways to generate a measurable indicator is needed in order to make it work for the evaluator / monitoring expert

31 33%

C The indicator does not work in its present form and needs to be sharpened or revised in order to work from an evaluator’s point of view

43 45%

0 The indicator does not work in its present form either because it is not measurable or because the measures need to be defined

4 4%

IV.2.2 Summary table on Output IndicatorsCount /

Ref.HRD OP OIS SCORE Conclusions / comments Recommendation

1 Number of women participated in entrepreneurship courses for women

Number of women participated in entrepreneurship courses

B It is extremely difficult to operate with indicators where the baselines can be doubted.

Furthermore, we cannot see what is HRD OP generated and what is basic ALMM

Clean the indicator through measures of QA in data collection. Define its focus (what it appear to measure) and create a doable baseline somewhere which appears convincing (the present baseline is no way convincing)

2 Number of women participated in child / elderly care trainings

Number of women participated in child/elderly care training

C It appears slightly difficult to understand the indicator and from where it originates as the priority, the measures, and the specific objective never address a certain political focus on driving women into childcare and elderly care.

It sounds strange that the labour markets of the 12 NUTS Regions should have address a specific or rather exclusive demand for women in childcare and elderly care; why not men?

We cannot see how the indicator aligns with the overall horizontal issue of equal opportunities

In general, remove indicators which appear to contradict the horizontal principles and tend to distort or create biases onto the labour market without any reason. It is highly recommended to assure objective Indicators.

227

Count / Ref.

HRD OP OIS SCORE Conclusions / comments Recommendation

3 Number of women who participated in employment oriented courses for women

Number of women participated in employment guaranteed courses for women

B The indicator requires solid data similar to the data provided by İŞKUR

The data has to be cleaned and cannot be used in its raw form (data from 2006 and onwards)

Keep the indicator on female participation on employment oriented training courses provided that the data are validated, consistent, and based upon solid QA measures of control. An additional comment would be to measure both female and male participants in all employment oriented training courses in the monitoring reports, and not just Number of women having participated in employment oriented training courses for women, since every study needs a control group for indicating the effect of the specific programme.

4 Number of women benefited from guidance and counselling services

Number of women benefited from guidance and counselling service

B Same as no. 3 Same as no. 3; i.e.:Keep the indicator on Number of women benefited from guidance and counselling services provided that the data are validated, consistent, and based upon solid QA measures of control. Again the additional would be to ensure data on both genders as well as obtain data from other Turkish regions in order to ensure control groups.

5 Revised: Number of analyses activities of the grant scheme results including the main lessons learned

C Not particular sharp and precise (what defines “analyses” and what define “lessons learned”)

Can hardly be measured consistently

Redefine the indicator and make it measurable

6 Revised: Number of compendiums published for successful granted projects and results

C Is number of compendiums really an essential measure? The indicator completely lacks measures of quality

Consider the significance and the relevance of including the indicator which has no measure of quality connected to it. Include measures of quality

7



C Strange divergences in targets No raw data available which makes the

indicators highly doubtful In principle the same comments as

indicator 1

There is basically no theoretical or logical objection towards the indicator insofar consistent data is available. It is highly recommended to find ways to agree on the target (as well as the achievements). An additional note could be that the indicator clearly matches indicator 1 under specific objective 1 studied above; it makes sense to have indicators which are fruitful for multiple purposes. Further, it would again be recommendable to ensure valuable control groups to see the net effect of the measure

228

Count / Ref.


8

Number of young people who participated in employment guaranteed courses

Number of young people who participated in internship, employment guaranteed courses and apprenticeship programs

B No raw data The baseline aggregates the three named

indicators We cannot be sure that the same

participants are not counted twice But more so, it is generally not particular

fair and just to separate the counting into three brackets since they are mutually related to an obvious extent

Aggregate the indicators into one indicator but keep the three indicators on an internal operational level, providing strict guidelines to coding and quality assurance (QA)Eliminate the third indicator on guidance when related to the other two indicators but keep the indicator as measuring separate actions only related to guidance and counselling (that would presumably necessitate a far lower target)Ensure solid guidelines to separate the two former actions from each other, and make explicit notes to what to be reported and measured in case of drop-outs

9

Number of young people who participated in internship and apprenticeship programs

B

10



C

11 Number of established local partnerships

Number of permanent local partnerships to promote registered work established and operating

A No raw data received The OIS sharpen the indicator, which is

good No baseline (except from “0”) which is very

doable and okay Some mess in targets and data, including

between MIS and OIS

Clean the data. Otherwise, okay indicator insofar it makes sense for the stakeholders

12 Number of SSI staff at central and local level who participated in trainings

Revised: Number of SSI, relevant institutions' staff and social partners who participated in trainings to promote registered employment improved.

A No raw data received The OIS sharpen the indicator, which is

good No baseline (except from “0”) which is very

doable and okay Some mess in targets and data, including

between MIS and OIS

Clean the data. Otherwise, okay indicator insofar it makes sense for the stakeholders

13 Number of staff from social partners who participated in trainings on National Strategy

14Number of İŞKUR staff who participated in PES trainings

C System-related indicator, which necessitates relation to impact

Not particular qualitative / sharp General comment (indicator 12-31): Threat

of too many indicators

The indicator needs to be sharpened and focused towards HRD OP (there is a need to separate it from ordinary İŞKUR practice)

229

Count / Ref.


15 Number of Monitoring Mechanisms

Number of monitoring mechanisms established for ensuring the sustainability of monitoring of ALMP through cooperation with Provincial Employment and Vocational Training Boards.

B System-related indicator, which necessitates relation to impact

“Monitoring mechanisms” have to be defined clearly which necessitate a solid and consistent guideline allowing the evaluator to use the indicator

General comment (indicator 12-31): Threat of too many indicators


16 Number of İŞKUR provincial offices transformed into model offices by being modernised and becoming user-friendly to provide customer-focused public employment services


“Model offices” and “user-friendly” have to be defined clearly which necessitate a solid and consistent guideline allowing the evaluator to use the indicator



17 Number of İŞKUR staff in newly established model offices that increased their knowledge/abilities on job search methods, customer relations and managerial skills


It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed



18 Number of İŞKUR staff that increased their knowledge/abilities on conducting labour market analyses





230

Count / Ref.


19 Number of İŞKUR and MoLSS staff at central level responsible for IT related issues that increased their knowledge/ability on Software development training, Security Training, Database Training, and Network Training





20 Divided into two: 1) Increase in the percentage of qualified software relating to labour market information system of İŞKUR


It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed: Is it really relevant for the HRD OP??



21 2) Increase in the percentage of the usage of software quality standard


It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed: Is it really relevant for the HRD OP??



22 Revised: Increase in the percentage of qualified Internet based services of İŞKUR





231

Count / Ref.


23 Number of İŞKUR staff at provincial levels that increased their knowledge/abilities on job and vocational guidance services





24 Result in the OP, Output in the OIS:Number of the unemployed people benefited from job and vocational guidance services and increased their knowledge on job and career perspectives

C The indicator is not clear and can hardly be measured from a qualitative viewpoint

The indicator needs to be sharpened and focused towards HRD OP (there is a need to separate it from ordinary İŞKUR practice)The indicator has to be redefined

25 Number of İŞKUR staff responsible for designing the Turkish Occupational Dictionary that increased their knowledge/abilities on ISCO 2008 to adopt Turkish Occupational Dictionary





26 Number of İŞKUR and MoLSS staffs at provincial levels that increased their knowledge/abilities on monitoring ALMP





232

Count / Ref.


27 Revised: Increase in the percentage of effective of ALMP measures determined in the 15 growth centres.





28 Number of the members of Provincial Employment and Vocational Training Boards and the experts working in cooperation with these Boards who increased their knowledge/ability regarding employment issues





29 Number of staff of MoLSS, İŞKUR and representativeness of social actors who increased their knowledge on European Employment Strategy, employment policies of the Member States (best practices), urgent employment policy measures in case of global crisis and the Member States practises in designing employment policies


Not particular qualitative and sharp General comment (indicator 12-31): Threat

of too many indicators


233

Count / Ref.


30 Number of research centres established in the library at the disposal of MoLSS to assist the staff designing employment policies.




The indicator needs to be sharpened and focused towards HRD OP (there is a need to separate it from ordinary MoLSS practice)

31 Number of models designed cooperation and coordination between MoLSS and Ministry of Science, Industry and Technology is increased regarding designing employment policies.


What does “models” mean? It is by no means clear how the

measurement should be done consistently; clear definitions and guidelines are severely needed



32 Number of call centres (7/24 functioning) established




33 Number of İŞKUR staff who gained the ability and knowledge on working in a call centre and informing people in an efficient way


It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed: What is “efficient”?



34 Number of educational staff participated in vocational guidance services training

Revised: Number of teachers participated in trainings about vocational guidance services at schools

C No raw data received System-related indicator, which

necessitates relation to impact It is by no means clear how the


The indicator needs to be sharpened and focused towards HRD OP (there is a need to separate it from ordinary practice)

234

Count / Ref.



Revised: Number of parents trained by programmes aiming at raising the awareness of parents on the importance of education

B No raw data received It is by no means clear how the


Otherwise: OK, since it just call for a quantitative number irrespectively of the quality


36 Number of NGO members participated in trainings

Revised: Number of NGO members participated in trainings aiming at raising the awareness of parents on the importance of education

C No raw data received It is by no means clear how the


Furthermore: It is not clear what role NGO Members have in relation to the priority


37 Number of protocols between local administrations, enterprises, schools, NGOs and social partners in order to increase girls’ enrolment rates

Revised: Number of protocols signed between public institutions, social partners, municipalities, universities and NGOs aiming at increasing cooperated activities for supporting education of girls



measurement should be done consistently; clear definitions and guidelines are severely needed: What is “protocols” and is there no requirement to “content” and quality?



Make sure that the indicator is relevant for HRD OP!


Revised: Number of parents families reached through campaigns aiming at raising their awareness on the importance of education for persuading them to send their daughters to school

A No raw data received Otherwise: OK, since it just call for a

quantitative number irrespectively of the quality

None, except for perhaps adds some requirements for quality

235

Count / Ref.


39 Number of pilot projects implemented to increase the quality of VET education and girls’ schooling and quality education

B No raw data received System-related indicator, which


measurement should be done consistently; clear definitions and guidelines are severely needed: What is “quality”?


The indicator needs to be sharpened and focused towards HRD OP (there is a need to separate it from ordinary practice)Quality needs to be defined

40 Revised: Number of VET programmes prepared according to the regional needs with the cooperation of all partners


necessitates relation to impact “All partners” is either too ambitious or

simply non-sense Otherwise: OK, since it just call for a


The indicator needs to be sharpened and focused towards HRD OP (there is a need to separate it from ordinary practice)Define a minimum target of “partners consulted”

41 Revised: Number of schools (secondary education institutions especially those for girls) equipped in the pilot provinces to contribute to girls’ schooling and quality education



measurement should be done consistently; clear definitions and guidelines are severely needed: What does “equipped” mean? / What does “quality” mean?



Define “equipped” and make a minimum of requirements to be targeted (sharply) in order to help the evaluators

Define “quality”

42 Number of detection and monitoring services for students particularly for girls under the risk of dropping out or who have already dropped out in primary and secondary schools



measurement should be done consistently; clear definitions and guidelines are severely needed: There is no indication of what defines a detector!


Define “detection and monitoring services” and make a minimum of requirements to be targeted (sharply) in order to help the evaluators

236

Count / Ref.


43 Number of developed / revised and implemented modular programs in secondary and post-secondary vocational schools by region, by training institution / school and by domain

Number of modular VET curricula developed and/or revised on Judicial Practices, Agriculture, Maritime and Health areas

A No raw data received System-related indicator, which

necessitates relation to impact Otherwise: OK, since it just call for a


44 Number of comprehensive and scheduled awareness raising events organised with the cooperation of private sector to increase awareness raising among the sector by region and by economic sector

Number of communication strategies developed through awareness raising activities


necessitates relation to impact The indicator appear slightly weak and

could use some more sharp and precise definitions: Perhaps there is a link somewhere between awareness raising activities and development of communication strategies (if not directly being the same); in that case it would certainly help to know




Number of partnership protocols between vocational institutions, social partners and the private sector in 12 NUTS II Regions


necessitates relation to impact It would perhaps be nice to indicate some

requirements to “partnership protocols”; it must be more than just a piece of paper with some signatures




Revised: Number of modular VET curricula developed and/or revised on Judicial Practices, Agriculture, Maritime and Health areas

A No raw data received System-related indicator Same as no. 43

237

Count / Ref.


47 Revised: Number of teachers participated in technical training in the sector for increased rate of pedagogical and professional qualifications of administrators and teachers





48 Revised: Number of teachers participated in trainings for increasing the quality of vocational and career guidance services in secondary vocational education institutions and post-secondary higher schools improved





49 Revised: Number of teachers participated in Teacher Training in line with the newly updated curriculum





238

Count / Ref.


50 Revised: Number of QASs in line with the European Quality Assurance Reference Framework for VET (EQARF) developed and piloted In the selected 20 post-secondary higher schools and 30 vocational educational institutions and revised according to the results of the piloting.

A No raw data received System-related indicator, which

necessitates relation to impact

The indicator needs to be slightly sharpened and focused towards the priorities of the HRD OP (there is a need to separate it from ordinary practice)

51 Revised: Number of departments provided with equipment support that are providing training in the fields of Judicial Practices, Agriculture, Maritime and Health in the selected 20 pilot post-secondary higher vocational schools and METARGEM


necessitates relation to impact It is not entirely clear what “equipment

support” means


52 Participants certified and tested in terms of vocational knowledge and skills in 10 VOC-TEST by region and by VOC-TEST Centre

3000 participants certified and tested in terms of vocational knowledge and skills in 10 VOC Test Centres.

B No raw data received The OIS transformation is not an indicator:

It is a target! Otherwise OK; it is just a simple number

The indicator needs to be slightly sharpened and focused towards the priorities of the HRD OP (there is a need to separate it from ordinary practice)Redefine OIS

239

Count / Ref.



Minimum 11 Occupational Standards endorsed by VQA

C No raw data received The OIS transformation is not an indicator:

It is a target! “Developed” is not defined; there must be

some minimum requirements Is it occupational standards or vocational

standards we are measuring? Shall we agree on “occupational standards” (since “vocational standards” tend to be non-sense)

The indicator needs to be sharpened and focused towards the priorities of the HRD OP (there is a need to separate it from ordinary practice)

Redefine OIS Indicate ways to measure “developed”


Trainings on basic skills and further development of basic skills for the adults particularly for women in formal and non-formal vocational institutions in 12 NUTS II Regions provided.



“Number of…” has to be added to the context



1800 participants in employability courses by these networks


It is a target! It is by no means clear how the

measurement should be done consistently; clear definitions and guidelines are severely needed:

What separate an “employability course” from any other course

Network is not defined


Redefine OIS Define “employability course”; there must be

some basic requirements, otherwise forget it Define “network”


900 teachers participated in the technical training in the sector


necessitates relation to impact The OIS transformation is not an indicator:




Redefine OIS

240

Count / Ref.



1200 participants to internships provided by these networks



measurement should be done consistently; clear definitions and guidelines are severely needed:

Network is not defined


Redefine OIS Define “network”

58 Number of teachers trained on new methods of monitoring, evaluation and measurement for LLL

Revised: Number of teachers trained on new methods for monitoring, evaluation and measuring LLL activities that are developed, tested and implemented.

0 No raw data received System-related indicator, which


measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “new”; “LLL activities”; “developed”; “tested”; and “implemented”?


Redefine indicator


Revised: Number of “trainings on basic skills and further development of basic skills for the adults particularly for women in formal and non-formal vocational institutions in 12 NUTS II Regions provided.”



measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “trainings” and how to measure “further development”???


Redefine indicator

241

Count / Ref.



Number of focus groups networks established



measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “focus groups network” and how to measure them???


The indicator needs to be sharpened in general in order to avoid an arbitrary and non-consistent monitoring and evaluation

61 Number of employees who participated in trainings on work organisation following training needs analysis.

Revised: Number of empoyees partiticipated in trainings


measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “trainings”?


The indicator needs to be sharpened in general in order to avoid an arbitrary and non-consistent monitoring and evaluation

65 Revised: Number of study visits by SME empoyees who have who have participated and successfully completed the trainings to SMEs in an EU country

0 No raw data received Why invite employees on study visits??? It is by no means clear how the

measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “trainings” and “trainings to SMEs” not least; how to measure “successfully”?


The indicator needs to be redone completely

66 Revised: Number of SMEs visited in developed and industrialized provinces with the participation of employers who have participated and successfully completed the trainings


measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “trainings”; how to measure “successfully”?


The indicator needs to be redone completely

242

Count / Ref.


67 Revised: Number of staff of chambers, social partners, Provincial Employment and Vocational Training Board staff and ÇASGEM staff increased their training capacities to be trainers


measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “increased” and how to measure “their training capacities”?


68 Revised: Number of gap analysis and trainings needs analysis at company level in each 15 growth centres

B No raw data received Criteria needs to be added in order to avoid

excess studies where there is limited need “gap” needs to be clearly defined


Criteria needs to be added in order to avoid excess studies where there is limited need

“gap” needs to be clearly defined69 Revised: Number of

vocational school teachers participated in on- the- job trainings



measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “participation” and “on- the- job trainings”?



Revised: Number of employers participated in the trainings

B No raw data received It is by no means clear why employers need

to be trained It is by no means clear how the

measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “trainings”?


70 Revised: Number of study visits to SMEs in EU with the participation of employers

A No raw data received It would have been nice if there were some

assessment involved in “participation”; otherwise OK

243

Count / Ref.


71 Revised: Number of workshops by the participation of employers from developed regions at sectoral level.

A No raw data received It would have been nice if there were some

assessment involved in “participation”; otherwise OK

72 Revised: Number of visits to selected SMEs in developed and industrialized provinces with the participation of employers and employees

A No raw data received It appear to be captured by the above

indicators indicating no need for additional measurement

Delete the indicator or make a combined indicator with nos. 70-72

73 Revised: Number of models of innovative and sustainable forms of work organisation for SMEs which have participated in and successfully completed the trainings

0 No raw data received It is by no means clear how the

measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “models”; “innovative”; “sustainable”; “forms of work organisation for SMEs”; “participation”; and “successfully”?

The indicator needs to be defined; it cannot be measured in its current form

74 Revised: Number of SME provided by training and for the establishment of support and coordination centres within the chambers and 1 within TOBB in each of which non-key experts and chamber staff will work

0 No raw data received The indicator is not understood; what does

“Number of SME provided by training and for the establishment of support and coordination centres” mean???

What does “1 within TOBB in each of which non-key experts and chamber staff will work” mean???

It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “training”; “establishment”; “support”; “coordination”?

The indicator needs to be defined; it cannot be measured in its current form

244

Count / Ref.


75 Revised: Number of awareness raising meetings for raising the awareness of employers and employeeson the need for more training and consultancy for increasing adaptability and competitiveness

B No raw data received The indicator needs a measurable

dimension of outcome of meetings

The indicator needs to be sharpened


Revised: Number of chamber staff accredited as trainers

A No raw data received The indicator needs to be sharpened and focused towards the priorities of the HRD OP (there is a need to separate it from ordinary practice)

76 Revised: Number of study visits by chamber staff and centre coordinators to the training and consultancy centres esteblished by a chamber in the EU

A No raw data received It would be nice to know why the indicator

only counts “chamber staff and centre coordinators”

Perhaps there needs to be a focus on defining the content of the visits

The indicator needs to be slightly sharpened

245

Count / Ref.


77 Revised: Number of online database including information on stakeholders, resources, the training needs, training providers, announcements regarding career opportunities, decisions of Provincial Employment and Vocational Education Boards that will serve for chambers, SMEs, civil society organisations and vocational schools is established

A No raw data received Baseline is presumed to be “0” and target

“1”; it is a process indicator! Accordingly, the word “Number of…” appears silly: Just write “Establishment of…”

System-related indicator, which necessitates relation to impact




measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “benefited”; “consultancy services”; “improvement”; and “adaptability”?

The indicator needs to be defined; it cannot be measured in its current form without severe difficulties in ways to make consistent valued measurements

246

Count / Ref.


78 Revised: Number of web- sites established for enhancing the cooperation and coordination among relevant stakeholders

A No raw data received Baseline is presumed to be “0” and target

“1”; it is a process indicator! Accordingly, the word “Number of…” appears silly: Just write “Establishment of…”

System-related indicator, which necessitates relation to impact

It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed to measure “successfully achieved”

The indicator needs to be slightly sharpened, otherwise very OK

79 Revised: Number of conferences established for increasing industry- university cooperation

A No raw data received The indicator is not really sharp, but we

shall assume that everyone knows what we are measuring (“number of conferences” on what?; it is easy to make a conference…)

OK, insofar everyone knows what we measure

80 Number of people having participated in guidance and counselling services

B Same as no. 3 The indicator needs to be sharpened and focused towards the priorities of the HRD OP (there is a need to separate it from ordinary practice)

Same as no. 3; i.e.: Keep the indicator on Number of people

benefited from guidance and counselling services provided that the data are validated, consistent, and based upon solid QA measures of control.

Find and define control groups.81 Number of people

having benefited from rehabilitation programmes

B We would perhaps like to see some indication of content of the rehabilitation programmes


Keep the indicator on Number of people benefited from guidance and counselling services provided that the data are validated, consistent, and based upon solid QA measures of control.

Find and define control groups.

247

Count / Ref.


82 Number of people having participated in courses in scope of ALMPs

C We would perhaps like to see some indication of content of the rehabilitation programmes

ALMP is not just a HRD OP oriented measure; there has to be a way to see net impact of the HRD OP


Keep the indicator on Number of people benefited from guidance and counselling services provided that the data are validated, consistent, and based upon solid QA measures of control.

Find and define control groups.83 Number of people

who are reached through the awareness raising activities

C “Reached” is not at all precise! How to define “awareness raising

activities”? We would perhaps like to see some

indication of content of the awareness raising activities


84 Number of staff of social protection institutions and NGOs who participated in trainings on services related to the disadvantaged persons



measurement should be done consistently; clear definitions and guidelines are severely needed, mainly in terms of “participation”


85 Number of common monitoring and analysis system established


It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed: “common”; “system”


86 Number of staff of related institutions and NGOs in that area who participated in trainings on the formation, usage and implementation of central database.


It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed: “participated”; “trainings”; “formation”; “usage”; “implementation”; “central database”


Definitions are required in order to perform monitoring and evaluation

248

Count / Ref.


87 Number of disadvantaged persons registered to central database, monitoring and analysis systems.


It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed: Is one person registered just a count or is there some degree of “disadvantaged” or other parameters which has to be taken into account? How do we separate between “almost normal / perhaps otherwise strong job-seeker” and “disadvantaged” (are smokers included to make an extreme example relevant to some enterprises)?


88 Minimum number of trainings for the staff of the Operating Structure related to improvement of knowledge and practice of IPA.

A System-related indicator, which necessitates relation to impact

It is by no means clear how the measurement should be done consistently; clear definitions and guidelines are severely needed: What defines “trainings”? Is that just a brief and friendly advice?

No real comments, except for the point that it might be difficult to measure relevance for the HRD OP on a simple measure of counts

89 Minimum number of trainings on the usage of MIS

B It is a target!!! Delete the word “minimum”! System-related indicator, which

necessitates relation to impact “Number of trainings” is not measurable:

Number of participants or number of training programmes?

What is meant by “usage”?

No real comments. However: Define “Number of trainings” to make it

measurable Define “usage” to make it measurable

90 Minimum number of trainings for OS staff to improve their knowledge on the IPA-ESF and differences between them


necessitates relation to impact “Number of trainings” is not measurable:

Number of participants or number of training programmes?

What is meant by “knowledge”?

No real comments. However: Define “Number of trainings” to make it

measurable Define “knowledge” to make it measurable

91 Minimum number of site visits for the monitoring of the projects

A It is a target!!! Delete the word “minimum”! System-related indicator, which

necessitates relation to impact Otherwise OK

No real comments

249

Count / Ref.


92 Minimum Number of evaluation activities


necessitates relation to impact “Number of evaluation activities” is not

measurable

No real comments. However: Define “Number of evaluation activities” to

make it measurable

93 Minimum number of prepared analyses, studies, strategy papers, manuals etc.


necessitates relation to impact “Number of …” is not measurable

No real comments. However: Define “Number of …” and the ways to

measure it to make it measurable

94 Minimum number of capacity building activities (trainings, workshops etc.)





95 Minimum number of information events (seminars, workshops etc.)





250

IV.3 Summary of the examination of Result Indicators in Part III

IV.3.1 The score system usedSCORE INTERPRETATION Count Pct.

A The indicator is understood and acknowledged. The indicator is workable, indicating that smaller changes in definition might be needed in order to make it measurable and practically 10 36%

B The indicator is understood. The indicator is workable, but changes in definition or ways to generate a measurable indicator is needed in order to make it work for the evaluator / monitoring expert 4 14%

C The indicator does not work in its present form and needs to be sharpened or revised in order to work from an evaluator’s point of view 5 18%

0 The indicator does not work in its present form either because it is an Output Indicator, it is not measurable, or because the measures need to be defined 9 32%

IV.3.2 Summary table of comments, conclusions, and the stakeholders’ view on the Result indicators studiedCount HRD OP OIS “Real name” Type Survey Comment Score

OK Not good

Don’t know

1


Number of women in labour force

after participating in courses or counselling

Output Indicator

82 % 7 % 11 % “Qualification” and “motivation” can hardly be measured.

It is recommended to redefine the indicator to “labour force participation rate” or “employment rate”

0

2


Number of women employed after participating in courses or after having received

counsel divided by total number of

women participating in courses and counselling

Output Indicator

73 % 10 % 17 % It is recommended to redefine the indicator to “employment rate”

0

251

Count HRD OP OIS “Real name” Type Survey Comment ScoreOK Not

goodDon’t know

3


Number of female entrepreneurs

Outcome indicator

73 % 10 % 17 % It is recommended to contact TÜİK / TURKSTAT in order to provide solid data

A

4


young people employed

following the courses, internship

and counselling activities

Output Indicator

78 % 5 % 17 % 0

5Increased rate of young entrepreneurs

Number of young entrepreneurs

Outcome indicator

58 % 9 % 33 % It is recommended to contact TÜİK / TURKSTAT in order to provide solid data

A

6


Revised: Percentage of decrease in the rate of complaints (advice slip) on the unregistered employment to the Social Security Inspection Board

number of complaints (advice

slip) on the unregistered

employment to the Insurance

Inspection Board

Outcome Indicator

29 % 18 % 53 % The Interim Evaluation is not sure of the purpose of the indicator

doubt on the ability to reflect a relevant and sustainable process

The indicator do not address issues of attracting and retaining employment

The indicator tells nothing about registered employment

Data based upon administrative records

C

7

Increase in the number of bilateral projects on promoting registered employment between the relevant partners

number of bilateral projects on promoting registered

employment between the

relevant partners

Output Indicator

53 % 10 % 37 % Redefine or delete 0

8


The rate of registered employees in the 12 NUTS II regions.

Employment rates Outcome Indicator

57 % 7 % 36 % No comments A

252


goodDon’t know

9

Revised: Increase in the number of qualified reports, policy documents and actions produced by the Provincial Employment and Vocational Training

Number of qualified reports, policy documents

and actions produced by the

Provincial Employment and

Vocational Training

Output Indicator

0

10Increase in the placement rates of İŞKUR

placement rates Outcome indicator

63 % 9 % 28 % Data based upon administrative records issues a warning

B

11

Number of communication campaigns organized for increasing the visibility of İŞKUR

Number of communication

campaigns organized for increasing the

visibility of İŞKUR

Output Indicator

0

12

Revised: Increase in the percentage of the registration rate to İŞKUR

Number of registered

unemployed

Outcome Indicator

No comments A

13


Increase in the access rate of education, particularly secondary education for girls

enrolment rate of females with

special focus on secondary

education levels

Outcome Indicator


14


Decrease in girls' dropout rates at secondary education and VET schools

Dropout rates for girls at secondary

education and VET schools

Outcome Indicator


253


goodDon’t know

15


number of girls graduated from

secondary education and

VET

Outcome Indicator


16


Increased percentage of the teachers, students and entrepreneurs satisfied with the content and quality of Vocational and Technical Education.

the share of teachers, students and entrepreneurs satisfied with the

content and quality of Vocational and

Technical Education

Outcome Indicator

57 % 6 % 37 % The indicator is based upon surveys which is doable, but leaves no baseline

Satisfaction is relative and difficult to measure; it needs to be defined

B

17


enrolment rates in vocational schools

Outcome Indicator

52 % 4 % 44 % “Demand” is non-sense in the context and not understood

Enrolment rates are okay

B

18

Teacher training in line with the newly updated curriculum.

number of teachers trained in newly developed

curriculum

Output Indicator

50 % 6 % 44 % 0

19


Share of primary education and

secondary education

graduates among participants

certified and tested in terms of vocational

knowledge and skills in 10 VOC-TEST Centres

Outcome Indicator with reservations

54 % 5 % 41 % The Interim Evaluation cannot see and understand the purpose of the indicator

Data are not available from TÜİK / TURKSTAT

The indicator should be redefined

C

254


goodDon’t know

20

Increased percentage of participants who obtained a certificate in vocational knowledge and skills

number of participants who

obtained a certificate in vocational

knowledge and skills

Outcome Indicator

70 % 4 % 26 % “participants” has to be clearly defined

A

21


number of participants who

obtained a certificate in basic

skills

Outcome Indicator

67 % 4 % 29 % “participants” has to be clearly defined

A

22


number of participants from

employability courses who

obtains a job after completion relative to total participants

on the employability

courses initiated

Outcome indicator

62 % 8 % 30 % A

23


? “induced” output

indicator

53 % 6 % 41 % Many reservations (see context) C

255


goodDon’t know

24


? “induced” output

indicator


25

Increased training activities on the requirements of adaptability given by social partners to their members

Number of training activities on the requirements of

adaptability given by social partners to their members

“induced” output

indicator


26


number of disadvantaged

persons entered into labour force

(employed or unemployed) following the rehabilitation programmes, courses and counselling

Output Indicator

63 % 5 % 32 % 0

27


number of disadvantaged

persons obtaining a job after

participating in rehabilitation programmes, courses and counselling

Output indicator

65 % 5 % 30 % 0

256


goodDon’t know

28

Increased satisfaction on the services both in institutional and target group level (by staff of institutions and by disadvantaged persons)

satisfaction on the services both in institutional and

target group level

Outcome indicator

58 % 11 % 31 % The indicator is based upon surveys which is doable, but leaves no baseline

Satisfaction is relative and difficult to measure

B

257

Appendix to the study on Indicators: Well-developed indicators

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)Unemployed, total No.- by gender No.- By age groups No.- By duration of unemployment, gender and age

No.

- By education and gender No.- By last occupation (ISCO) and gender (or work experience)

No.

- By criteria on special needs, gender and age

No.

- By Other criteria, e.g. organised/non-organized (members of trade unions)

No.

Unemployment rate, total Ratio- by gender RatioLong-term unemployment rate Ratio- by gender RatioUnemployment gender gap Ratio The difference in unemployment rates between

women and men in percentage pointsUnemployment rate gaps for people at a disadvantage

Ratio Difference in unemployment rates for disadvantaged groups (according to national definitions) and the overall unemployment rate, in percentage points

Unemployment rate gap between non EU and EU nationals

Ratio Unemployment rate gap between non EU and EU nationals, in percentage points

Inflow into long-term unemployment Ratio Share of young/adult unemployed becoming unemployed in month X, still unemployed in month X+6/12 without any break of more than one month (28-31 calendar days).

Youth unemployment ratio Ratio Total unemployed young people (15-24 years) as a share of total population in the same age bracket (by gender)

Employed, total No.- by gender No.- By age groups No.- By occupation (ISCO) and gender No.- By economic sector (NACE) and gender No.

- By other criteria, including self employed, managers, etc. (organised/non-organized)

No.

Employment gender gap The difference in employment rates between men and women in percentage points

Employment rate Ratio- by gender Ratio- By age groups RatioEmployment gender gap in full-time equivalent

Ratio The difference in employment rates measured in full-time equivalent between men and women in percentage points

Employment gender gap by age group and educational attainment

Ratio The difference in employment rates between men and women in percentage points, by age group (15-24, 25-54, 55-64) and by education level (less than upper secondary, upper secondary and tertiary education, according to the ISCED classification)

Gender segregation 1 Ratio 1. Gender segregation in occupations, calculated as the average national share of employment for women and men applied to each occupation; differences are added up to produce a total amount of gender imbalance presented as a proportion of total employment (ISCO classification).

Gender segregation 2 Ratio 2. Gender segregation in sectors, calculated as the average national share of employment for women and men applied to each sector; differences are added up to produce a total amount of gender imbalance presented as a proportion of total employment (NACE classification).

Employment rate full-time equivalent Ratio Total hours worked divided by the average annual

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)number of hours worked in full-time jobs, calculated as a proportion of total population in the 15-64 age bracket.

Employment rate in services Ratio Number of employed persons working in the services sector (in main job) aged 15-64 as percentage of the population of the same age group. (by gender)

Labour Force, total No.- by gender No.- By age groups No.Activity rate Ratio Share of employed and unemployed persons in total

population of working age 15-64Labour supply growth Ratio Annual change in labour supply (including employed

and unemployed in working age 15-64). (by gender)Inactive population, total No.- by gender No.- By age groups No.- By main reason for inactivity No. E.g. retired, students, children, disabled or early

retired, ‘housewives’Social burden I Ratio Number of Inactive relative to no. of Labour ForceSocial burden II Ratio Number of Inactive relative to no. of EmployedDependent Elderly Ratio Dependent elderly men and women over 75 as a

proportion of all men and women over 75. Breakdown by: living in specialised institutions, help at home (other than by the family), and looked after by the family

Average exit age from the labour force Ratio The average age of withdrawal from the labour market, based on a probability model considering the relative changes of activity rates from one year to another at a specific age

Labour market gaps for disadvantaged groups

No. Gaps on the labour market (such as employment, unemployment and inactivity gaps), for disadvantaged groups (such as disabled people, ethnic minorities, immigrants, low skilled people, lone parents, etc.) according to national definitions

Transitions into employment and training Ratio Transitions of unemployed people into employment and training from year n to year n+1

Transitions by pay level Ratio Transitions between non-employment and employment and within employment by pay level (gross monthly earnings) from year n to year n+1

Transitions by employment status Ratio Transitions between employment, unemployment and inactivity from year n to year n+1

Diversity of contractual and working arrangements

Ratio Total employees in part-time and/or fixed-term contracts plus total self-employed as % of persons in employment. Employees in non-standard employment (parttime and/or fixed-term) as % of total employees. (Breakdown by part-time, fixed-term, part-time and fixed-term, by reason, by gender.)Total self-employed as % of total persons in employment. (Breakdown by part-time, by gender.)

Transitions by type of contract Ratio Transitions between non-employment and employment and within employment by type of contract from year n to year n+1

Working poor Ratio Number of working poor as % of working population, calculated separately for wage and salary employees and self-employed. (by gender)

Poverty (low-wage) trap Ratio The marginal effective tax rate on labour income taking account the combined effect of increased taxes on labour and in-work benefits withdrawal as one increases the work effort (increased working hours or moving to a better job). Calculated as the ratio of change in personal income tax and employee contributions plus change (reductions) in benefits, divided by increases in gross earnings,

259

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)using the "discrete" income changes from 34-66% of APW.Breakdown by family types: one earner couple with two children and single parent with two children.

Unemployment trap Ratio The marginal effective tax rate on labour income taking account the combined effect of increased taxes and benefits withdrawal as one takes up a job. Calculated as the ratio of change in gross income minus (net in work income minus net out of work income) divided by change in gross income for a single person moving from unemployment to a job with a wage level of 67% of the APW.

Taxation on low-wage earners Ratio Tax wedge on labour cost: ratio of income tax plus employee and employer social contributions including payroll taxes less cash benefits divided by the labour costs for a single earner earning 67% of the APW.

Undeclared work Ratio Size of undeclared work in national economy (e.g. as share of GDP or persons employed)

Gender pay gap 1 Ratio Difference between men’s and women’s average gross hourly earnings as percentage of men's average gross hourly earnings (for paid employees at work 15+ hours). Breakdown by public and private sectors.

Gender pay gap 2 Ratio 1. Gender pay gap by age group and educational attainment (difference between men’s and women’s average gross hourly earnings as percentage of men's average gross hourly earnings for paid employees at work 15+ hours).

Gender pay gap 3 Ratio Gender pay gap, adjusted for sector, occupation and age

Employment impact of parenthood Ratio The absolute difference in employment rates without the presence of any children and with the presence of a child aged 0-6 (age group 20-50)

Childcare Ratio Children cared for (by formal arrangements* other than by the family) up to 30 hours a usual week / 30 hours or more a usual week as a proportion of all children of the same age group. Breakdown by: Children aged under 3 (0-2 years), Children aged between 3 years and the mandatory school age (during the day and outside pre-school hours), Children aged between mandatory school age and 12 years in compulsory primary (or secondary) education (during the day and outside school hours).

Lack of care for children and other dependants

Ratio Share of persons (age group 15-64) who would like to work but are not searching for a job / who are working part-time due to lack of suitable care facilities, in relation to the total population of the age group. Breakdown by: lack of care services for children; lack of care services for dependant adults (i.e. ill, disabled, elderly relatives or friends); lack of care services for both children and dependant adults.

Vacancies, total No.- By economic sector (NACE) and occupation (ISCO)

No.

- By types of vacancies No. E.g. Permanent and temporary jobs, New job creation

Transparency of job vacancies Ratio Share of job vacancies advertised through Member States' public employment services that are accessible on a common platform by jobseekers throughout the EU

Vacancies per unemployed Ratio Trend of the ratio between the total number of the stock of vacancies compared to the total number of unemployed (v/u ratio)

260

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)LMP expenditure No.- by activity No.- relative to GDP Ratio Active/passive LMP expenditure as % of GDPActivation of unemployed, total No.- by gender No.- By age groups No.- By duration of unemployment, gender and age

No.

- By education and gender No.- By last occupation (ISCO) and gender (or work experience)

No.

- By criteria on special needs, gender and age

No.

- By Other criteria, e.g. organised/non-organized (members of trade unions)

No.

Preventative services Ratio Share of young/adult unemployed becoming unemployed in month X, still unemployed in month X+6/12, and not having benefited from intensive counselling and job-search assistance. (LMP category 1) [target value 0%=full compliance] (by gender)

New start (a) Ratio Share of young/adult unemployed becoming unemployed in month X, still unemployed in month X+6/12, and not having been offered a new start in the form of training, retraining, work experience, a job or other employability measure. (New start = a job or a measure from LMP categories 2-7) [target value 0%=full compliance] (by gender)

New start (b) Ratio Share of young/adult unemployed becoming unemployed in month X, still unemployed in month X+6/12, and not having been offered a new start in the form of training, retraining, work experience, a job or other employability measure. (New start = job or measure LMP categories 1-7) [target value 0%=full compliance] (by gender)

Activation of long-term unemployed Ratio Number of long-term registered unemployed participants in an active measure (training, retraining, work experience or other employability measure) in relation to the sum of the long-term unemployed participants plus registered long-term unemployed (yearly averages). Broken down by types of measures and gender. (LMP categories 2-7)

Follow-up of participants in active measures 1

Ratio 1. Rate of inflow of LMP participants into employment (3 or 6 months after participation in a measure)

Follow-up of participants in active measures 2

Ratio 2. Rate of return of LMP participants into unemployment (3 or 6 months after participation in a measure)

Enterprises, total No.- By Economic Sector No.Enterprise births Ratio Gross birth rate of new enterprises as a percentage

of total stock of active enterprisesSurvival rates of newly born enterprises Ratio Newly born enterprises of year n that are still active

in year n+3Employment in newly established enterprises Ratio Number of persons employed in newly born

enterprises (in year n) and in surviving enterprises (set ups in years n-3, n-2 and n-1) in relation to the number of persons employed in all active enterprises (in year n)

Work places No. Can be indicative when cross-regional commuting appears

- by economic sector No.Trends in accidents at work Ratio The evolution of the incidence rate, defined as the

261

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)number of accidents at work per 100 000 persons in employment

Trends in occupational diseases RatioRegional disparities - coefficients of variation: Employment

Ratio Standard deviation of employment divided by the weighted national average (age group 15-64 years). (NUTS II)

Regional disparities - coefficients of variation: Unemployment

Ratio Standard deviation of unemployment divided by the weighted national average (age group 15-64 years). (NUTS II)

Regional disparities – Underperforming regions

Ratio 1. Share of underperforming regions in terms of employment and unemployment (in relation to all regions and to the working age population/labour force) (NUTS II) (by gender).2. Differential between average employment/unemployment of the underperforming regions and the national average in relation to the national average of employment/unemployment (NUTS II) (by gender).

Job Satisfaction Ratio Satisfaction with type of work in present jobReal unit labour costs Ratio Growth in total compensation per employee

adjusted for labour productivity and GDP deflatorLabour productivity Ratio Total annual output divided by number of occupied

population and hours worked (GDP in PPS per person employed/per hour worked relative to EU average)

Working time 1 Ratio 1. Average weekly number of hours usually worked per week defined as the sum of hours worked by full-time employees divided by the number of full-time employees

Working time 2 Ratio 2. Trends in average effective annual working time per employed person.

Overtime work Ratio Number of employees for whom the number of hours actually worked exceeds the number of hours usually worked due to overtime as a % of all employees. (by gender)

Implicit tax rate on employed labour Ratio Ratio of total taxes on employed labour (personal income taxes plus employees' and employers' social security contributions plus payroll taxes) divided by the total compensation of employees plus payroll taxes

Population No.- By gender No.- By age groups No.- By economic activity (unemployed, employed, students, children, retired, etc.)

No.

- By education and gender No.- By ethnicity, citizenship or nationality No.Day population No.Night population No.Interregional migration measures No.Mortality rates RatioBirth rates RatioFertility rates RatioGDP No. GDP growth rates and GDP/c can easily be

calculated if needed. Official statistics is preferred if available. Regional GDP in absolute numbers can be used for reference purposes (for instance, in order to calculate rough productivity measures per sector).

GDP growth Ratio Annual average of GDP growthGrowth in labour productivity Ratio Growth in GDP per capita of employed population

and per hour workedGross Value Added No. Relative to employed persons it is a rough indicator

262

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)of labour productivity

- by economic sector and region No. Relative to employed persons it is a rough indicator of labour productivity

Monthly labour costs by region No. If wage level by economic sector is available it might be indicative, but monthly labour costs is international comparable.

- by economic sector No. If wage level by economic sector is available it might be indicative, but monthly labour costs is international comparable.

Price indexes by region Ratio Production prices (when available); consumer prices; regional measures (when available)

FDI Foreign Direct Investment by region Ratio Relative to GDP. FDI is a highly complex measure as it is a net measure. In some areas it is yet another growth indicator.

- by economic sector Ratio Relative to GDP. FDI is a highly complex measure as it is a net measure. In some areas it is yet another growth indicator.

(Domestic) Gross Investment Ratio Relative to GDP. This indicator can be very hard to obtain on regional level. Net investment might be available, but this indicator is not as good as gross investment.

- by economic sector Ratio Relative to GDP. This indicator can be very hard to obtain on regional level. Net investment might be available, but this indicator is not as good as gross investment.

Export of Goods and Services Ratio Relative to GDP- by economic sector Ratio Relative to total ExportImport of Goods and Services Ratio Relative to GDP- by economic sector Ratio Relative to total ImportTrade balance Ratio Relative to GDPCurrent Account Ratio Relative to GDPCapital Account Ratio Relative to GDPPublic Expenditure Ratio Relative to GDP- by major brackets 1 Ratio Relative to total public expenditure. By

governmental and local (non-governmental)- by major brackets 2 Ratio Relative to total public expenditure. By sectorPublic debt Ratio Relative to GDPPupil/Teacher Ratio (PTR) Ratio Average number of pupils per teacher at a specific

level of education, based on headcounts for both pupils and teachers

- by ISCED levels RatioNet enrolment ratio in primary schools (MDG indicator No. 6)

Ratio Net primary enrolment ratio is the ratio of the number of children of official school age (as defined by the national education system) who are enrolled in primary school to the total population of children of official school age

Gender parity index in primary, secondary and tertiary education

Ratio Ratio of female to male gross enrolment ratio. A GPI of 1 indicates parity between sexes; a GPI between 0 and 1 means a disparity in favour of boys/men; a GPI greater than 1 indicates a disparity in favour girls/women.

Gross intake rate to the last grade of primary school (UIS proxy for Primary completion rate (MDG goal 2))

% Total number of new entrants in the last grade of primary education, regardless of age, expressed as a percentage of the population of the theoretical entrance age to the last grade." The number of new entrants is the total number of children in the last grade of primary school minus the number of repeaters

Students first choice of Education %Repetition rates in primary and secondary schools

% Number of repeaters in a given grade in a given school year, expressed as a percentage of enrolment in that grade the previous school year.

263

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)Drop-out rates in primary and secondary school

% Percentage of pupils or students who drop out from a given grade in a given school year. It is the difference between 100% and the sum of the promotion and repetition rates.

Early School Leavers No. Or in fraction of total school population (drop out rate)

- by types of shools No. Or in fraction of total school population (drop out rate)

- by domain No. Or in fraction of total school population (drop out rate)

Youth literacy rate (MDG indicator no. 8) % Literacy rate of 15–24 year-olds, or the youth literacy rate, is the percentage of the population 15–24 years old who can both read and write with understanding a short simple statement on everyday life.

Educational attainment of 22 year olds % In percentage of all 22 year olds

Students No.- by domain and gender No.Graduates No.- by domain and gender No.Participation in CVT No.Number of agreements between Gvt. (VT supply) and economic sectors/branches (VT demand) on the basis of studies of market perspectives of economic sector/branches

No. Strategic Partnership between government and professional federations/chambers etc. to improve market relevance of VT provision, following surveys undertaken by different sources (internal/external) on emerging/declining economic sectors.

Percentage of training courses updated further to skills/competencies identified at enterprise level

% Percentage of training courses updated further to skills/competencies identified at enterprise level to total number of training courses

Number of VT centres built/ rehabilitated/ equipped

No. Number of VT centres built/ rehabilitated/ equipped

Percentage of trainers trained % Percentage of trainers having completed training out of available trainers in the system

Ratio VT trainers/VT trainees Ratio Average number of trainees per trainer, based on headcounts for both trainees and trainers.

Enrolment rate in VT as compared to number of VT applications

% Absorption capacity of training centres as compared to training applicants

Completion rate % Percentage of trainees successfully completing (or graduating from) VT programs in a given year as compared to the total number of trainees enrolled that year

Percentage of VET graduates hired after training

% Number of VET graduates hired in a given year, expressed as a percentage of total VET graduates.

Existence of a demand driven strategy/reform of VT for employment

No. A qualitative indicator for measuring of policy progress, to which the answer is ‘yes’ or ‘no’ within a timeframe.

Public expenditure on VT as a percentage of Education and Social affairs budget

% Government spending on Vocational Training as a percentage of total public expenditure on education and social affairs.

% of EC contribution within sector reform financial gap

% EC spending (funding) for VT as a percentage of sector reform financial gap (ie the difference between the financial needs for the implementation of the reform and the funds available to the sector/country)

Ratio public sector versus private sector VT centres

Ratio

Average cost by trainee Ratio

264

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)Number of studies of market perspectives of economic sector/branches

No. Surveys undertaken by different sources (internal/external) on emerging/declining economic sectors.

Number of conventions between Gvt. (VT supply) and economic sectors/branches (VT demand)

No. Strategic Partnership between government and professional federations/chambers, etc. To improve VT demand/supply.

Ratio of training courses updated further to skills/competencies identified at enterprise level

Ratio Ratio of training courses updated further to skills/competencies identified at enterprise level to total number of training courses

Ratio trainers/trainees Ratio Average number of trainees per trainer, based on headcounts for both trainees and trainers.

Ratio residential/alternate/ apprenticeship training (initial training)

Ratio Efficiency of apprenticeship or alternate training

Number of VT centres built/rehabilitated No. Number of VT centres built/rehabilitated

Number of courses/CV reoriented towards 'competency based approach' (CBA)

No. CBA is supposed to reproduce business environment at VT school /centre level

Number/ratio of trainers trained No. Proportion of trainers having completed training

Ratio available training places (supply)/would be trainees/employees (demand)

Ratio Number of potential trainees per available training course.

N./ratio of apprentices hired after training per skill, per sector and per region

Ratio Number of VET graduates hired in a given year, expressed as a percentage of total VET graduates.

Index of stakeholders satisfaction (trainees, employers, community government) through surveys

Index Index associated with the perception of stakeholders regarding the quality and effectiveness of the VET program

Increased efficiency / effectiveness of orientation / information services (rate of 'clients' referred to VT)

% Percentage of trainees having used the VET orientation information services

Enrolment rate in VT % The number of trainees enrolled in VT programs, expressed as a percentage of the number of workers in the corresponding sector.

Literacy rate of managers and employees % Number of literate managers and employees, expressed as a percentage of the total number of managers and employees

Reduction of % of working children under 15 % Percentage of children under 15 employed.

Rate of mobility within enterprises % Measures the ability of workers/managers to change easily employment, disaggregated by gender, age group, profile and sector of economic activity.

Productivity increase within enterprise/economic branch

Ratio Output per unit of labor within enterprise/economic branch

Insertion rate of active population by gender, age and level of study

% Labour Insertion rate of active population by gender, age and level of study

Distribution of population by level of education

% Evaluate changes in the distribution of population by education level

(Un)employment trends in target sector/region/group

% Trends in unemployment rate, ie the percentage of the total labor force that is unemployed but actively seeking employment and willing to work

Educational attainment for population (15 years+) by age, gender, ISCED, and Domain

%

Participation rates in TVET as a percentage of all participants of education/training

%

265

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)Percentage of apprentices to total TVET participants

%

Distribution of apprentices/applied secondary education by sex and specialization (number or percentage) to total participants

%

Percentage of vocational education students to total TVET participants

%

Distribution (number or percentage) of vocational education students by sex and type of education.

%

Percentage of students in technical/ technician education to total TVET participants

%

Distribution (number or percentage) of participants in technical / technician education by sex, age and specialisation

%

Gross enrolment rate in TVET by region, gender, and Domain

Ratio

Net enrolment rate in TVET by region, gender, and Domain

Ratio

Completion rates in TVET by programme, and gender

Ratio

Dropout rates in TVET by gender and programme

Ratio

Dropout rates in CVT by gender and programme

Ratio

Percentage of graduates in TVET by gender and programme.

%

Public educational expenditure in TVET to the total public expenditure

%

Public educational expenditure in CVT to the total public expenditure

%

Public expenditure per student by type of education and training

%

Relative proportions of public and private investment in educational institutions.

%

Funds of TVET by sources and type of education and training.

%

Funds of CVT by sources and type of education and training.

%

Cost per contact hour, by programme (all educations and trainings)

EUR

Graduate cost per programme (all educations and programmes)

EUR

Teaching load per teacher (all educations and programmes)

Hours per week

Teachers’ qualifications by gender, programme and years of experience

?

Expenditure on training and up-qualification of teaching staff by institution

EUR

Workplace training: Participation in job-related continuing education and training by

266

INDICATOR UNIT COMMENTS (DEFINITIONS, ETC.)labour force statusWorkplace training: Participation rates in continuing education and training by economic activity and size of enterprise

Ratio

Workplace training: Average duration of training undertaken by employed adults aged 25–64 in continuing education and training

Days

Participation in adult education / training by age and sex

%

267

ANNEX B1. List of Meetings held

Regional visits

NoCOUN

T INSTITUTION DATE TIME1 KASTAMONU

1.1 1 RGMTT 13/07/2011 15:001.2 2 İŞKUR Provincial Directorate 14/07/2011 10:001.3 3 Kastamonu Chamber of Comm. And Industry 14/07/2011 11:301.4 4 Kastamonu Public Training Centre and Evening Art School 14/07/2011 13:302 VAN

2.1 5 Association for Manufacturer & Enterprise 18/07/2011 16:002.2 6 RGMTT 19/07/2011 10:002.3 7 İŞKUR Provincial Directorate 19/07/2011 14:002.4 8 Van Chamber of Commerce and Industry 19/07/2011 16:002.5 9 Yüzüncüyıl University 20/07/2011 10:003 SAMSUN

3.1 10 RGMTT 19/07/2011 10:003.2 11 Karadeniz Sanayici ve İşadamları Dernekleri Federasyonu, KASİF 19/07/2011 16:004 ŞANLIURFA

4.1 12 RGMTT 21/07/2011 10:004.2 13 İŞKUR Provincial Directorate 21/07/2011 14:004.3 14 Şanlıurfa Chamber of Commerce & Industry 22/07/2011 10:004.4 15 MoNE Local Education Authority 22/07/2011 13:004.5 16 Harran University 25/07/2011 14:004.6 13 İŞKUR Deputy Provincial Directorate (repeat) 26/07/2011 09:304.7 17 Karacadağ Development Agency 26/07/2011 11:005 GAZİANTEP

5.1 18 RGMTT 27/07/2011 10:005.2 19 İŞKUR Provincial Directorate 27/07/2011 14:005.3 20 Gaziantep Commercial Chamber 27/07/2011 15:306 TRABZON

6.1 21 İŞKUR Provincial Directorate 09/08/2011 10:006.2 22 RDA 09/08/2011 14:006.3 23 Chamber of Commerce 10/08/2011 10:006.4 24 TOKADER, TRABZON 10/08/2011 14:00

Meetings with central institutions

List of meetings held on central levelCount No INSTITUTION DATE TIME

1 1 Ministry of Development 21/07/2011 16:002 2 TURKSTAT 29/07/2011 10:003 3 TESK 02/08/2011 13:304 4 Min. of National Education / CGMT 03/08/2011 10:005 5 TISK 04/08/2011 13:306 6 SSI / CGMT SSI 04/08/2011 16:007 7 TURK-IS 09/08/2011 16:008 8.1 MoLSS: DG Labour and DG OSH 09/08/2011 10:009 9 National Qualification Authorities 10/08/2011 10:00

10 10 YOK / CoHE 10/08/2011 14:00Operating Structure of HRD OP, MoLSS:

11 11.1 Programme Management Unit 11/08/2011 10:0012 11.2 Procurement Unit 11/08/2011 16:0013 11.3 Project Management Unit 12/08/2011 15:3014 11.4 Financial Management Unit 11/08/2011 13:3015 11.5 Quality Assurance and Control Unit 12/08/2011 11:3016 11.6 Monitoring and Evaluation Unit 17/08/2011 11:0017 11.7 Information, Publicity and TA Unit 12/08/2011 14:0018 12 Ministry of Science, Industry and Technology 12/08/2011 09:3019 13 EUD 15/08/2011 14:0020 14 NIPAC 15/08/2011 16:00

268

Count No INSTITUTION DATE TIME21 15 TOBB 16/08/2011 14:0022 16 HAK-IS 16/08/2011 10:3023 17 National Authorizing Officer 17/08/2011 13:3024 18 National Fund 17/08/2011 13:3025 19 CFCU 17/08/2011 15:3026 20.1 İŞKUR: Indicators 18/08/2011 10:0027 20.2 İŞKUR: Overall Management and CGMT 18/08/2011 14:0028 8.2 MoLSS: DG Internal Audit 19/08/2011 11:00

Other meetings and similar appearances

List of other meetings held, including management meetings, and other appearances until the date of submission the draft Interim Evaluation Report

No Purpose VENUE DATE TIME1 Kick-Off Meeting ÇSGB 16/05/2011 10:002 HRD Awareness conference Hilton, Ankara 18/05/2011 10:003 Weekly Meeting ÇSGB 23/05/2011 10:004 Weekly Meeting ÇSGB 30/05/2011 10:305 Weekly Meeting ÇSGB 13/06/2011 10:306 Monthly Meeting ÇSGB 23/06/2011 10:007 Information Meeting City Hotel, Ankara 24/06/2011 13:308 SMC Meeting Trabzon 07/07/2011 09:309 Weekly Meeting ÇSGB 11/07/2011 10:3010 Weekly Meeting ÇSGB 01/08/2011 10:3011 Weekly Meeting ÇSGB 15/08/2011 10:3012 Monthly Meeting ÇSGB 23/08/2011 13:30

List of people met during meetings

The list is reconstructed from lists of participants; not all names and titles have been readable.

CENTRAL LEVEL

Operating Structure, MoLSSEsat AKTAŞOĞLU, EU Expert Coordinator, MOLSSUğurtan TAŞKINER, EU Expert, Programme Management Unit, MoLSSAhmet KILINÇ, Prog. Expert, Programme Management Unit, MoLSSBurcu SAĞLAM, Monitoring & Evaluation Expert, MOLSSMelahat GÜRAY, EU Expert, Project Management UnitEnder TAN, Asst. EU Expert, Quality Assurance & Control Unitİrem YAZAR, Asst. EU Expert, Financial Management UnitSelin AYTAŞ, Asst. EU Expert, Project Management UnitFeride Bahar ÖZDEMİR, Financial Manager, MOLSSHüseyin Ali Ali TANGÜREK, Financial Management Unit, MoLSSMelih AKIN, Coordinator, Procurement UnitEsra DÜZENLİ, Quality Assurance and Control Unit, MoLSSSalih ENİŞ, Coordinator, Information and Publicity Unit, MoLSSŞükrü Zafer DOKUZER, Assistant EU Expert Information and Publicity Unit, MoLSSHülya TEKİN, Senior Grant Manager, Project Management Unit, MoLSS

EUDPetek KOVANCU SHEHRIN, Sector ManagerNuman ÖZCAN, Sector ManagerCaner DEMİR, Sector Manager

269

National IPA CoordinatorNazlı Hezar TANRISEVER, Ministry for EU Affairs, CoordinatorA. Deren Doğan YAVUZ, Ministry for EU Affairs, DirectorÜzeyir BAŞER, Ministry for EU Affairs, J. ExpertM. Süreyya SÜNER, Ministry for EU Affairs, Director

National Authorising Officer and National Fund/TreasuryAykin Berk PAKEL, NAOSD, Expert Fatih TATLILIOĞLU, NF, Expert Bayram ÜSTÜNAY, NF, Expert Gülşen DEMİR AY, NAOSD, Expert Özkan ÖZKARDEŞ, NF, Expert Şehnaz ÖZER, NAOSD, DN Chief

CFCUAygül ÖZDEMİR, Contract Manager, CFCUM. Fatih ERKOÇ, Finance Manager, CFCUBarbaros Murat KÖSE, CoordinatorM.Fatih YILDIZ, Grant Scheme Coordinator

İŞKURTuğba ÇİMŞİR, MHIE, VHKİJülide KUŞCUOĞLU, MHIE, VHKİŞerafettin POLAT, MHIE, VHKİHalil YILMAZ, MHIE, VHKİNazan KAHRAMAN, İŞKUR, Sef.Serkan Hasan KOZE, MHIE, VHKİMuhittin BOZÖYÜK, MHIE, Mat.İrfan GÜMÜŞ, Sb MdDerya Duyar COŞKUN, MHIE, MemurSecil KABAR, OCU Member, SefKivilcim SARA, OCU Member, Tr. ExpertErcan AKTEPE, İŞKUR MHIE, SefHakan ÖZ, OCU Member, Tr. Expert Burhan ARYAN, İŞKUR, PWE OCUElif ŞAHİN, İŞKUR, Project Branch MngSerkan YÜCEL, İŞKUR, Şb.Md.Yrd.

Participants in Information MeetingSibel TUĞ, TISK, Research, Training and External Relations ExpertBurcu Miraç DIRAOR, SPO, Pl. ExpertGökhan GÜDER, SPO, Head of DepartmentIrem YAZAR, MOLSS, Assist. EU ExpertFeride Bahar OZDEMIR, MOLSS, Assist. EU ExpertEsat AKTAŞOĞLU, MOLSS, Coordinator of Mon. and Evaluation UnitAygül OZDEMİR, CFCU, Contract ManagerAslı Meryem ŞIVGIN, KOSGEB, ExpertSelin DÜNDAR, Şahin SERİM, Project Coordinator, HAKİŞYusuf ŞAHİN, Karadeniz Teknik University

270

Ministry of DevelopmentIşıl BOZKURT, Ass. Planning ExpertSerdar POLAT, Planning ExpertFulya YATMAT, Planning ExpertSinem ÇAPAR DİRİÖZ, Planning Expert

Ministry of Science, Industry and TechnologyE.Meriç Meriçli TAŞAN, RCP-CIC, Head of Prog.DepartPınar İRDEM, RCP-CIC, Chief of Peop.DivAyşe AYDIN, RCP-CIC, Monitoring Assist.ExpSefa MAVİŞ, RCP-CIC, IPA Assist. Exp /MEDOya ERYİĞİT, RCP-CIC, IPA Assist. Exp /PDSayime ÇELİK, RCP-CIC, IPA Assist.Exp /PDOsman Cihan SELVİ, RCP-CIC, IPA Assist.Exp /PDTolga ŞEFLEK, RCP-CIC, IPA Assist.Exp /PD

TÜİK/TURKSTATEnver TAŞTI, Head of Department, Social Statistic DepartmentGüzin ERDOĞAN, Head of Social Sector Group, Social Sector StatisticsSüheyla TULUMBACI, Education Stat. Ass. Expert, Social Sector StatisticsCengiz ERDOĞAN, Head of Group, Pri. Stat. GroupMurat KARAKAŞ, Head of Group, IYKGNevin UYSAL, Team Leader of Vital Stat., Population Demogra.Ebru GÖK, Expert, Foreign Relation RepVedat METİN, Mathematician, Publication and DataDilek ÖZSOY, Head of Group, National Account. GroupEmel URAL, Team Leader, Population and Migration TeamDidem SEZER, Team Leader, Labour Force

Ministry of National Educationİbrahim Nail BURAL, Director of Monitoring & Evaluation BranchEmrullah ÜNAL, Monitoring & Evaluation Dept.Elvan KAHYAOĞLU, Director-Increasing Girls’ Educ. ProjectEda ÖZBEK, Expert-Strengthening Secondary Education ProjectMurat MIDAS, Expert – Strengthening Special EducationDr. İsmail DEMİR, Expert - Promoting LLL ProjectNamık Kemal YILDIZ, Coordinator – (Girls’ Education ProjectGöksel KÖROĞLU, Coordinator of LLL Central Monitoring TeamKürşat Kutalmış YALÇIN, PCC, Grand Monitoring Coordinator

Social Security Institution, SSIVarol DUR, Project Coordinator

Ministry of Labour and Social SecurityCanan KAMİ, General Directorate of Labour, Assistant Labour ExpertMehmet Said APUPULAR, DGOHS, Assistant OSH ExpertSelçuk YAŞAR, DGOHS, Assistant OSH ExpertCeren Seda ERDEM, DG Labour, Labour ExpertMurat ANDAÇ, Auditor, Internal AuditMürsel ÇUKUR, Auditor, Internal AuditŞerif OLGUN ÖZEN, Auditor, Internal AuditCengiz ULUTAŞ, Head of the Unit, Internal Audit

271

YÖK /CoHEÖzgül TORTOPDr. Ömer AÇIKGÖZ, AdvisorH.Serdar YÜCESU

TESKZehra KAYA, Education and Planning DirectorZeynep GAZİOĞLU, EU and Foreign Relations Expert

TİSKBülent PİRLER, Secretary GeneralSibel TUĞ, Research, Training and External Relation ExpertTuba Burcu ŞENEL, Research, Training and External Relation Expert

TÜRK-İŞGüldane KARSLIOĞLU (Researcher, Assistant Expert)Namık TAN, Researcher

MYK (National Qualification Authority)İsmail ÖZDOĞANAhmet GÖZÜKÜÇÜK, Firuzan SİLAHŞÖR, Head of Occupational Standards Department

HAK-İŞ Confederation, AnkaraDr. Osman YILDIZ (Assistant to President & Responsible for International Relations)Jülide SARIEROĞLU

TOBBWerner GRUBER, Acting Manager, EU Project Development & Monitoring DivisionEsin ÖZDEMİR, EU Department ExpertBelgin YILMAZ, Assistant Expert

REGIONAL LEVEL

KastamonuSibel ÖZSAVAŞ, RGMTT Kastamonu Alpaslan KULUŞ (Data Collection & Control Operator), RGMTT KastamonuMuhammed Hayati TABAN (Youth Employment-Project coordinator)Adnan ARSLAN, Provincial Director, İŞKUR, KastamonuSermin GOLOP, Kastamonu Public Education Centre and Evening Art School Aydin Faris GÖLOĞLU, Kastamonu Public Education Centre and Evening Art SchoolZeliho KUBATOGLU, Kastamonu Public Education Centre and Evening Art School Zeki GENCOGLU, Kastamonu Public Education Centre and Evening Art SchoolCihan CILBIRCIOĞLU, Genel Sekreter Yardımcısı, Deputy Secretary General, Kastamonu Chamber of Commerce and IndustryAdil Levent BAŞ, KOSGEB, Sinerji Odağı Dış Uzmanı, Kastamonu Chamber of Commerce and Industry

VanMahmut GEDİK, Association for Manufacturer and Enterprise, VanElvan ÜREY, RGMT, VanNurcan ALPASLAN, RGMT, VanMurat AKKAYUN, RGMT, VanAbdülkerim ARVAS, İŞKUR, Van

272

Elvan UREY, İŞKUR, VanArzu ÇİPLAK, Project Coordinator, Chamber Commerce and Industry, VanDoç. Dr. Kenan GÜLLÜ, Coordinator, AB-PA office, Yüzüncüyıl University, Van

SamsunYaşar BIYIKLI, General Secret., Kasif Office, SamsunGülşen GÜLTEKİN, Project Coordinator, Kasif Office, SamsunMehmet HUNTURK, RGMTT, SamsunGülay ŞENER, RGMTT, SamsunHasan KILIÇASLAN, RGMTT, SamsunNurcan GÜRSES, RGMTT, SamsunA.Kemal OZBİŞİRİCİ, RGMTT, SamsunSoner DOĞANYILDIZ, RGMTT, SamsunHicran KARADOĞAN KINIK, RGMTT, SamsunBurcu F.YAZICI, RGMTT, SamsunRukiye DOĞAN, RGMTT, Samsun

ŞanlıurfaMehmet CENGİZ, RGMT, ŞanlıurfaHalil ŞİLAN, RGMT, ŞanlıurfaMahmut KAYA, İŞKUR Regional Directorate, ŞanlıurfaNimet İNCE, Project Coordinator, Chamber of Commerce and Industry, ŞanlıurfaHanefi ŞAHİN, Educ. Director, Provincial Directory of National Education, ŞanlıurfaProf. Dr. Abuzer PINAR, Dean, Harran University, ŞanlıurfaAyşegül ÖZBEK, Specialist, Local office of RDA, Şanlıurfa/Karacadağ Development Agency

GaziantepEmirhan Hikmet ASLAN, Regional Monitoring Expert, RGMTT, Gaziantepİhsan CANPOLAT, RGMTT, GaziantepMurat BAĞLIBEL, RGMTT, Gaziantepİlyas BULDUK, İŞKUR, GaziantepAtilla AKARSLAN, İŞKUR, GaziantepŞenay YEŞİLTAŞ, Project Assistant, Chamber of Commerce, GaziantepOkan ASLAN, Project Coordinator, Chamber of Commerce, GaziantepEmel ERÇETİN, Project Coordinator, Chamber of Commerce, GaziantepSenay Copur, Project Coordinator, Chamber of Commerce, GaziantepFiliz Hosukoglu, Project Consultant, Chamber of Commerce, Gaziantep

TrabzonNurettin KARAL, BHIT Coordinator, İŞKUR, TrabzonUmit ORHAN, General Coordinator, Chamber of Commerce, TRABZONZeki Mert BARUT, Chamber of Commerce, TRABZONÇağlar EMİRZEOĞLU, TOKADER, TRABZONOnur ADIYAMAN, RDA, TRABZON

273

Annex B2. Terms of Reference

SPECIFIC TERMS OF REFERENCE Providing Technical Assistance for the First Interim Evaluation of Human Resources

Development Operational ProgrammeFWC BENEFICIARIES 2009 – LOT 9: Culture, Education, Employment and Social

EuropeAid/127054/C/SER/multi

1. BACKGROUND

2. DESCRIPTION OF THE ASSIGNMENT Global objective Specific objective(s) Requested services Required outputs

3. EXPERTS PROFILE Number of requested experts per category and number of man-days per expert Profile required

4. LOCATION AND DURATION Starting period Foreseen finishing period or duration Planning including the period for notification for placement of the staff as per art 16.4 a) Location(s) of assignment

5. REPORTING Content Language Submission/comments timing Number of report(s) copies

6.ADMINISTRATIVE INFORMATION Interviews if necessary indicating for which experts/position When in the interest of the project, possible limits to subcontracting Language of the specific contract Request for a succinct methodology when needed Management team member presence required or not for briefing and/or debriefing Other authorized items to foresee under ‘Reimbursable’ For riders only : operational conditionality for intermediary payment if foreseen as per article 7.2 b) of

the Special Conditions Others

ANNEX I: Tax and Customs ArrangementsANNEX II: Evaluation Grid

274

1. BACKGROUND

With the aim of benefiting from financial assistance provided under the IPA, Turkish Ministry of Labour and Social Security (MoLSS), designated as the OS (Operating Structure) of Component IV which is Human Resources Development, have prepared the Operational Programme (HRD OP) covering the period of 2007-2009 with the involvement of relevant stakeholders. HRD OP was designed to “address the main challenges Turkey faces in the fields of employment, education and social inclusion” with a geographical concentration approach, especially on the 12 NUTS II Regions82 of Turkey.

There are five priority axes defined under the HRD OP: The first priority axis aims to attract and retain more people in employment through the

measures of enhancing the employment of women and young people, promoting registered employment and improving the quality of the public employment services.

The second priority axis seeks to enhance investment in human capital through education, especially with the measures of increasing enrolment of girls and the quality of Vocational and Technical Education (VET).

The third priority axis was designed for increasing adaptability of both employees and employers through promoting life-long learning and investing more in human capital.

The fourth axis aims to the integration of the disadvantaged persons to the labour market through increasing their employability and contributing to the improvement of the institutions functioning in the area of labour market and social protection.

The fifth priority axis of the HRD OP is the technical assistance. It tries to achieve this objective through the measures to support the HRD OS for preparatory, management, monitoring, administrative support, information, evaluation and control activities; development of absorption capacity of final beneficiaries through information and publicity activities.

This assignment will be carried out under the fifth priority axis of the HRD OP which aims to support the MoLSS as the OS of the HRD OP. Through this assignment, MoLSS will be supported in its duties related to the interim evaluation of the HRD OP.

2. DESCRIPTION OF THE ASSIGNMENT2.1. Global ObjectiveTo make an independent evaluation of the overall relevance, effectiveness and efficiency of the implementation of the HRD OP with a view to provide all the stakeholders with reliable data, analysis and feedback that will help the upcoming OP revision exercise.

2.2. Specific ObjectivesThe specific objectives of the assignment are:

- to assess the relevance (in particular with the strategic documents such as the Strategic Coherence Framework83, Multi-Annual Indicative Planning Document84, employment guidelines, strategic community guidelines and the Lisbon Strategy), efficiency, effectiveness of the programme.

- to assess the complementarity between the HRD OP and Regional Competitiveness Operational Programme (RC OP),

- to provide data on the output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring system,

- to assess the achievement of the horizontal principles,

- to identify best practices, factors for success and failure and provide recommendations to improve the remaining programming and implementation of the HRD OP.

2.3. Requested Services After the signing of the framework contract, following procedures will be pursued:

1-Inception phase

82 The regions whose GDP per capita is less than 75% of the average GDP per capita of Turkey.83 Please seehttp://www.ikg.gov.tr/document.html84Pleaseseehttp://www.ikg.gov.tr/document.html

275

http://www.ikg.gov.tr/document.html

http://www.ikg.gov.tr/document.html

The inception phase starts with a kick off meeting to be held at the premises of the Beneficiary. The aim of the “kick-off” meeting should be, amongst other:

- introduction of the parties and any information or other assistance to be provided by the MoLSS/ Contracting Authority/ the participants to the Consultant,

- discuss any specific issues that require further attention.

After the kick of meeting, within 10 calendar days, the Consultant shall prepare an inception report mainly by revising the “methodology” that was presented within the proposal on the basis of the feedbacks received from the MoLSS, Evaluation Sub-Committeemembers of Sectoral Monitoring Committee85, European Union Delegation to Turkey (EUD), European Union Secretariat General (EUSG) and Central Finance and Contracts Unit (CFCU).

The inception report shall also include an executive summary and detailed work plan of management of the assignment including the plan about the visits to the determined Regional Grant Monitoring and Technical Assistance Teams86 (RGMTTs), time schedule with clear milestones for the implementation, specific activities to be carried out, identification of risks that might occur and recommendations to overcome them.

At the end of the 1st month of the assignment, the inception report is expected to be approved.

2- Information MeetingThe assignment will contribute to gaining experience and improving capacity on programme evaluation. To that aim, a half a day meeting on evaluation of HRD OP for the Internal Coordination Team within the Beneficiary with the members of the SMC Evaluation Sub-Committee will be organized by the Consultant after the completion of inception phase.

3-Analysis phaseHRD OP will be evaluated through the evaluation questions listed below and the evaluation method adopted in the inception phase:

Evaluation Question 1:

What is the overall relevance, efficiency and effectiveness of the HRD OP at the level of priority and measures in particular as regards;

The appropriateness of the management structures formed at the levels of Operation Structure and Operation Beneficiaries?

The use and financial allocation of financial assistance? The quality of projects designed to achieve the objectives, targets and indicators determined

in the HRD OP?

Evaluation Question 2: What is the complementarity between the HRD OP and RC OP in particular as regards achieving the objectives, targets and indicators determined in the HRD OP and RC OP?


85 The Sectoral Monitoring Committee, established in accordance with Article 59 of the IPA IR, includes the relevant institutions functioning within the IPA structure such as Operating Structure for HRD OP, National IPA Coordinator (NIPAC), National Authorizing Officer (NAO), Strategic Coordinator for Components III and IV (State Planning Organisation) and EUD as well as the representatives from the social partners and civil society. The Committee meets at least twice a year and reviews at each meeting the progress made towards achieving the specific targets of the OP. Evaluation Sub-Committee has been established under the SMC. It shall support the independence of the evaluation function, review and check the quality of works of external team (using an agreed quality check list) including inception and draft evaluation reports and issue comments. Evaluation Sub-Committee is composed of representatives selected among the SMC members.86With the aim of providing technical assistance to potential and final Grant Beneficiaries and carrying out monitoring of grant projects at local level, Regional Grant Monitoring and Technical Assistance Teams (RGMTTs) were established under the responsibility of the MoLSS in 12 provinces within the eligible NUTS-II regions (namely Trabzon, Kars, Samsun, Erzurum, Gaziantep, Elazığ, Batman, Kastamonu, Şanlıurfa, Sivas, Kahramanmaraş, Van). The RGMTTs are consisting of at least 9 experts from the provincial directorates of Turkish Employment Agency (ISKUR), Ministry of National Education (MoNE) and Social Security Institution (SSI).

276

What are the output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring system?


What are the results of the programme in achieving the horizontal principles during the design and implementation of Operations?

Evaluation Question 5: What are the best practices, factors of success and failure regarding the planning and implementation process of the HRD OP? What recommendations can be proposed to the actors involved in the programming, determining financial allocations, monitoring and evaluation?

These questions will be directed to the target institutions mentioned below:

Operating Structure Operation Beneficiaries (i.e; Ministry of National Education [MoNE], Turkish Employment

Agency [ЭЮKUR], Social Security Institution [SSI] and The Union of Chambers and Commodity Exchanges of Turkey [TOBB])

Evaluation Sub-Committee members of Sectoral Monitoring Committee Ministry of Industry and Trade Central Grant Monitoring Teams (CGMTs87) Regional Grant Monitoring and Technical Assistance Teams (RGMTTs) CFCU National Fund (Undersecretary of Treasury) National Authorizing Officer (Undersecretary of Treasury) Strategic Coordinator for Components III and IV (State Planning Organisation) National IPA Coordinator (EU Secretariat General) European Union Delegation to Turkey

The responses given to these evaluation questions by the aforementioned institutions will be the main source of data for the “Interim Evaluation Report”.

Tasks to be accomplished with respect to evaluation question #1

a. Analysis of Previous Evaluation’s Results

Review the results of the ex-ante evaluation carried out by external independent team under the responsibility of the Operating Structure within the framework of the EU-supported Technical Assistance Project “Support to the SPO to Build Capacity at Central, Regional and Local Level to Implement Economic & Social Cohesion Measures" in September 2007.

b. Analysis of the Relevance of the HRD OP

Analyze and review the major developments since 2007 at national and EU level that influence the HRD OP,

Analyze the HRD OP’s coherence with the objectives of pre-accession assistance, strategic documents such as the SCF, Lisbon Strategy, MIPD, employment guidelines, strategic community guidelines,

Make a SWOT analysis of the beneficiaries and stakeholders,

Analyze the accuracy, clarity, quality, usefulness and internal consistency of the overall objectives, purposes, targets, indicators and eligible activities at the priority and measure level of the HRD OP.

c. Analysis of the HRD OP’s Effectiveness and Efficiency

87 Each Operation Beneficiary having a Grant Scheme establishes a Central Grant Monitoring Team (CGMT) composed of minimum 5 experts under the coordination of the OS. The CGMT is co-chaired by an expert from the Operation Beneficiary and an Operation Coordinator from the OS. The CGMT is responsible for monitoring and controlling the technical implementation of the projects under the grant schemes.

277

Review programme outputs and results in relation with the expenditures incurred at priority and measure levels; in the light of financial and physical progress,

Collect data on indicators not provided through the regular monitoring system,

Make an overall examination of the HRD OP’s programming, monitoring and evaluation structures and procedures (in particular of the content, timeliness and compliance) to provide an assessment of the public administration systems’ efficiency and effectiveness in terms of constructing the necessary mechanisms for attaining the objectives under HRD OP,

Assess the concrete progress of the HRD OP on the basis of the indicators,

Examine the efficiency of Operation Selection Criteria,

Analyze the role of national bodies, the relations/co-ordination with local authorities, institutions, beneficiaries and other stakeholders, the commitment of all the stakeholders,

Assessment of local absorption and implementation capacities,

Analyze the financial management of the budget,

Analyze the accuracy of the budget allocations under measures.


The Consultant should analyze the complementarity between HRD OP and the RC OP and prepare a thematic report annexed to the “Interim Evaluation Report” by analyzing the extent to which,

RC OP’s objectives have been achieved through programming and implementation of the HRD OP,

RC OP has influenced the management structures of the OS and Operation Beneficiaries of HRD OP.


The Consultant will analyze and provide data on output and result indicators agreed upon in the HRD OP that cannot be obtained through monitoring system.


A thematic report annexed to the “Interim Evaluation Report” in order to evaluate the progress made towards meeting the horizontal principles during the design and implementation of the operations under HRD OP will be prepared. These horizontal issues are:

equal opportunities for men and women; sustainable development & environmental protection; participation of civil society; geographic, sectoral and thematic concentration; concerns of disadvantaged persons; good governance


The Consultant will analyze and determine the best practices and the factors of success and failure regarding the planning and implementation process of the HRD OP. Moreover, the Consultant will propose recommendations to the actors involved in the programming, determining financial allocations, monitoring and evaluation.

With the completion of the analysis, conclusions should be drawn by the Consultant based on robust findings of the interim evaluation. It is expected from Consultant to provide recommendations through follow-up tables for the next programming period of the HRD OP; identify and formulate later phases on the basis of lessons learned during the evaluation. Recommendations should indicate deadlines, be realistic and pragmatic. If options for future action are available, these should be identified and ranked taking into consideration their costs. These conclusions and recommendations will be an integral part of the Interim Evaluation Report.

4- Preparation of Draft Version of the “Interim Evaluation Report” and its Annexes (Thematic Reports, Conclusions and Recommendations)

278

The Consultant shall produce the draft version of the “Interim Evaluation Report” and its annexes, on the basis of the findings of the analysis, three months after the commencement of the assignment. After Quality Assurance and Control Unit (QACU) within the OS checks the draft report and its annexes, they will be issued in soft copy for comments of the target institutions listed above and DG Employment, Social Affairs and Equal Opportunities. The comments on the report and its annexes will be issued in 10 calendar days.

5- Briefing MeetingA half-day briefing meeting will be organized by the Consultant within 3 days after the completion of draft version at the premises of the Beneficiary. The aim of the briefing meeting is to provide information and receive feedbacks/comments about the draft version of the “Interim Evaluation Report” from the senior representatives of the target institutions. Findings and recommendations about the implementation and planning of HRD OP will be particularly stressed in the briefing meeting.

According to the comments received from the target institutions, DG Employment, Social Affairs and Equal Opportunities and the senior representatives attended to briefing meeting, the Consultant will make necessary revisions within 5 working days from the receipt of comments.

6- Submission of the Final Version of the “Interim Evaluation Report” and its Annexes (Thematic Reports, Conclusions and Recommendations)At the middle of the 4th month, the Consultant will send the revised report to the OS together with a table indicating how the comments have been handled. QACU of the OS will review the revised version of the “Interim Evaluation Report” and the soft copy of the report will be submitted to the target institutions listed above and DG Employment, Social Affairs and Equal Opportunities for feedbacks. They will provide feedbacks within 10 calendar days. According to the comments received, the Consultant will make necessary revisions within 5 working days and finalize the preparation of the report.

Finally, the final version of the report will be discussed in the Sectoral Monitoring Committee meeting, which is foreseen to be held in June 2011, for approval. Then, the approved “Interim Evaluation Report” would be sent to the EC.

Totally 10 copies of the final version of the Interim Evaluation Report, with all its annexes will be published and distributed to CFCU (1 copy), EUD (1 copy), EUSG (1 copy) andMoLSS (7 copies) in hard copy. However, the rest of the target institutions will be provided with the final version of the Interim Evaluation Report via e-mail.

After the approval of the “Interim Evaluation Report”, the booklets which include an executive summary of the “Interim Evaluation Report” of HRD OP will be submitted to the related parties. After approval of the draft copies, the 50 copies of the booklets will be published and submitted to the MoLSS to be disseminated to the relevant institutions.

Month Task

February 2011 Commencement of the assignment

Preparation and approval of the Inception Report

Organising information meeting

March – April 2011 Analysis phase

May 2011 Submission of the draft version of the “Interim Evaluation Report”

Receipt of comments/feedbacks

Organising briefing meeting

Submission of the revised version of the “Interim Evaluation Report”

Receipt of comments and finalizing the preparation of the report

Table 1: Indicative time-table for the assignment

279

With respect to the monitoring structure, proposed methodology will be as follows:The OS of the HRD OP is responsible from the tasks related to the monitoring and implementation of the assignment.

Coordination Meetings: Weekly coordination meetings will be held with the participation of Consultant and Internal Coordination Teamto assess the progress of the assignment and propose corrective measures. The weekly coordination meetings, which will be held at the OS’s premises in Ankara, will be chaired by the Beneficiary.

Monthly Coordination Meetings: Monthly meetings will be held with the participation of Consultant, Internal Coordination Team, Internal Audit Department, Evaluation Sub-Committee Members of SMC, the CFCU and the EUD in order to advice, guide and control the Consultant. Monthly meetings will be held at the premises of the Beneficiary.

Consultant will prepare the agenda prior to the meeting. The minutes of weekly and monthly meetings and submit to the parties involved in 3 working days for feedbacks and comments in order to ensure up-to date monitoring of the tasks.

2.4. Required Outputs Types of

indicatorsIndicator Target (2011) Source of verification

ResultIncreased capacity of MoLSS and EU Coordination Department in tasks related to programming, implementing and evaluation

90% of recommendations adopted andimplemented according to deadlines set in thefollow-up tables

Sectoral Monitoring Report, Sectoral Annual Report, MoLSS Programme Report, revised HRD OP, monitoring sheets, final beneficiary reports

Result Increased awareness of the internal coordination team within the Beneficiary and members of the SMC evaluation sub-committee with respect to evaluation of the HRD OP.

Approximately 20 people attended to the information meeting give positiveanswers to the questionnaires on the quality andusefulness of meeting delivered by the Consultant.

Sectoral Monitoring Report, Sectoral Annual Report, MoLSS Programme Report, revised HRD OP, monitoring sheets, questionnaires

Output An inception report prepared within 10 calendar days following the kick of meeting

1 Inception Report

Output An information meeting with the Internal Coordination Team within the Beneficiary and members of the SMC Evaluation Sub- Committee

1 Minutes of the meeting, participant list

Output A briefing meeting with the senior representatives of target institutions

1 meeting with the participation of at least 1 senior representative from each of the structures within the target institutions

Minutes of the briefing meeting, participants list

Output “Interim Evaluation Report” 1 Final “Interim Evaluation Report”Output Number of hard copies of the “Interim

Evaluation Report” with its annexes10 Copies

Output Number of booklets of executive 50 Booklets

280

summary of the “Interim Evaluation Report”

Output Number of visits to target institutions for interviews and data collection

Min. 17(min. 5 visits to RGMTTs,min. 12 visits to the structures within the target institutions)

Sectoral Annual Report, Sectoral Monitoring Committee Report,Reports received from the Consultant

Output Thematic report annexed to the “Interim Evaluation Report” assessing the complementarity between the implementation of HRD OP and RC OP

1 Thematic Report, “Interim Evaluation Report”

Output Thematic report annexed to the “Interim Evaluation Report” on meeting the horizontal issues during the preparation and implementation of HRD OP

1 Thematic Report, “Interim Evaluation Report”

3. EXPERTS PROFILE 3.1. Number of requested experts per category and number of man-days per expertThe Consultant shall make a team of one senior expert and two junior experts available for the successful completion of this assignment. 195 working days including the international travel days will be totally allocated to the experts for the implementation of the assignment. The Consultant shall ensure that these experts work in cooperation with each other.

The experts and the indicative number of working days of the experts are given in the table below:

No Experts Category Number of Working Days1 Team Leader - Expert on Employment Senior 652 Expert on Education Junior 653 Expert on Tendering, Financial Management and

Contract/Project ManagementJunior 65

TOTAL 195

3.2. Profile required (education, experience, references and category as appropriate)The experts to be proposed shall have the necessary professional qualifications and experience required to successfully deal with the issues and to cover all the activities indicated in this Terms of Reference. If the team proves unable to meet the level of quality required for drafting the reports, the consulting team shall provide, at no additional cost to the Beneficiary, an immediate technical support to the team to meet the required standards.

The qualifications, skills and experience listed below are the minimum required skills except the ones specified as assets.

Expert 1: Team Leader - Expert on Employment

Qualifications and skills:

Bachelor’s degree in economics or social sciences or administrative sciences, or any relevant field of study linked with human resources development. In the absence of a university degree the expert will have an equivalent professional experience of minimum 14 years in the relevant fields.

Graduate/post graduate degree in employment and/or social policy field will be an asset. Good command of both written and spoken English Full computer literacy

General professional experience:

At least 10 years of professional working experience in the sector(s) related to the lot.

281

Specific professional experience:

At least 5 years of experience in employment and social policy areas, Experience in setting out indicators and making statistical analysis, preferably through an

experience in the preparation of an HRD OP, Experience in an EU-funded project as a team leader/project manager, Experience in evaluation and/or designation and/or implementation of EC Structural and

Cohesion funds and/or pre-accession assistance programmes in the field of HRD conducted in EU candidate countries or member states.

Experience in the following area(s) will be an asset:o Drafting/updating Operational Programmes, o Preparation of sectoral annual and final reports, o Checking the progress/monitoring reports of the projects (major projects),o Programme evaluation.

Specialization on the employment of women, youth and/or disadvantaged people, public employment services and/or unregistered employment will be an asset.

Expert 2: Education Expert Qualifications and skills:

University degree in education or social sciences or any other related field. In the absence of a university degree the expert will have an equivalent professional experience of minimum 7 years in the relevant fields.

Good command of both written and spoken English Full computer and internet literacy

General professional experience:

At least 3 years of professional experience in the sector(s) related to the lot.

Specific professional experience: At least 1 year of experience in evaluation and/or designation and/or implementation of

education programmes related with employment and human resources development, Experience in EC Structural and Cohesion funds and/or pre-accession assistance

programmes in EU candidate countries or member states, Experience in making statistical analysis, Experience in EU projects will be an asset. Specialization on vocational training, lifelong learning and/or adaptability of workers/employers

will be an asset.

Expert 3: Tendering, Financial Management and Contract/Project Management ExpertQualifications and skills:

University degree in finance, economics or any other related field. In the absence of a university degree the expert will have an equivalent professional experience of minimum 7 years in the relevant fields.

Good command of both written and spoken English Full computer literacy

General professional experience: At least 3 years of professional experience in the sector(s) related to the lot.

Specific professional experience: At least 1 years of experience in the field of evaluating and/or implementing financial planning,

financial management and/or budget planning of international programmes, Experience in making statistical analysis, Experience in tendering procedures executed by the managing authorities / implementing or

intermediate bodies under Structural and Cohesion Funds, especially the European Social Fund; or executed by the CFCUs in EU candidate countries or member states,

Experience in the field of human resources development programmes will be an asset.

282

4. LOCATION AND DURATION 4.1. Starting periodThe indicative starting period for this assignment is February 2011.

4.2. Foreseen finishing period or durationThe foreseen duration of the assignment is 150 calendar days.

4.3. Planningincluding the period for notification for placement of the staff as per art 16.4 a)The Consultant, at the beginning of the assignment, will have an initial briefing meeting (kick-off meeting) at the premises of MoLSS in Ankara with the representatives of the MoLSS, CFCU, EUD, EUSG and other related institutions. Necessary documents such as HRD OP, RC OP, results of the ex-ante evaluation will be submitted to the Consultant in this meeting. The kick-off date will be determined by an administrative order after the signature of the contract.

Work-plan, time-management and organization of the team members (including designation of a Team Leader and allocation of the duties among experts) for the efficiency and success of the assignment are the responsibility and duty of the Consultant.

4.4. Location(s) of Assignment

The country of the assignment is Turkey and working place is in Ankara at the premises of the Ministry of Labour and Social Security. However, min. 5 visits to RGMTTs for interviews are also foreseen. The RGMTTs to be visited will be listed in the inception report. Except the visits to target institutions for interviews and data collection, the Consultant will perform the tasks at the premises of the Ministry of Labour and Social Security/EU Coordination and IPA Management Department.

Address:

ЭnцnьBulvarэ No: 42 C Blok, Kat: 4 PK 06100 Emek/ANKARA

5. REPORTING

5.1. ContentThe Consultant should submit the inception report in accordance with the provisions explained in detail under the “Requested Services”.

The Consultant should also submit a final report that reflects the outcome of all tasks performed. The final report should include the synopsis of the assignment, observations related to implementation. All relevant technical papers covering the “Interim Evaluation Report” and the findings of the interviews to be made with the target institutions including the RGMTTs, reports, and papers prepared from the start to the end of the assignment will be attached to the final report.

The Consultant should submit these reports to the related parties: the MoLSS, Evaluation Sub-Committee Members of SMC, EUD, EUSG and CFCU.

5.2. LanguageAll reports shall be prepared and submitted in English.

5.3. Submission/comments timing

After the kick-off meeting the Consultant will prepare an inception report within 10 calendar days and submit it to the related parties (MoLSS, Evaluation Sub-Committee Members of SMC, EUD, EUSG and CFCU).

The Consultant shall submit the final report not later than 7 calendar days after completion of the assignment.

283

The MoLSS, Evaluation Sub-Committee Members of SMC, EUD, EUSG and CFCU have 10 calendar days to provide comments on the submitted reports. Necessary revisions will be made by the Consultant within 5 working days from the receipt of the comments. In the absence of any comments from the related parties within 10 calendar days after the submission of the reports, they will be considered to be tacitly approved.

5.4. Number of report(s) copies

The Consultant shall submit the reports to the related parties for comments in electronic version readable by a Microsoft Office application. The final versions of the reports will be submitted to the related parties both in hard copy (one copy to each party,) and in electronic version readable by a Microsoft Office application.

6. ADMINISTRATIVE INFORMATION

6.1. Interviews if necessary indicating for which experts/position

No interview is foreseen.

6.2. When in the interest of the project, possible limits to subcontracting

No subcontracting is foreseen.

6.3.Language of the specific contract

Language of the specific contract is English.

6.4. Request for a succinct methodology when needed

The tenderers shall propose a methodological approach for this evaluation taking into consideration the objectives, evaluation questions and quality control requirements. Hence, the proposal should include an implementation plan and allocation of the duties among experts as well as an “evaluation methodology” for the programme evaluation such as interviews, focus groups, case study, counterfactuals to provide robust findings. This list is not exhaustive, any other methods and the appropriate number of analysis can be proposed.

6.5. Management team member presence required or not for briefing and/or debriefing

Management team member presence is not required for briefing and/or debriefing.

6.6. Other authorized items to foresee under ‘Reimbursable’The budget for this assignment should include (1) fees, and (2) reimbursable expenses. The reimbursable costs shall include i) per diems, ii) international travel costs of the experts (6 travels), iii) inter-city travel costs (min. 10 travels), (iv) translation costs (350 pages), (v) consecutive interpretation costs for interviews and visits to regions (min. 10 half-days), and iv) reproduction costs of 50 booklets.

MoLSS and target institutions will provide appropriate furnished office space for the experts for the period of their stay in Ankara and related running costs (telephone and fax, excluding international telephone). MoLSS and relevant institutions will provide adequate staff time and timely and easy access to all pertinent information requested by the Framework Contract Team, possibly in English language. Target institutions will be notified through official letter about the visit of the Consultant and office space will be requested.

The Consultant shall ensure that experts are adequately supported and equipped. In particular it shall ensure that there is sufficient administrative, secretarial and interpreting provision to enable experts to concentrate on their primary responsibilities. It must also transfer funds as necessary to support its activities under the contract. The Framework Contract experts will be required to bring their own computers/lap tops. Eventual communication (international phone calls), photocopy costs of programme evaluation materials shall be covered by the Consultant.

284

6.7. For riders only: operational conditionality for intermediary payment if foreseen as per article 7.2 b) of the Special Conditions

No interim payment is foreseen. An interim payment may be foreseen in the rider to a Specific Contract if at least one of the conditions identified in the article 7.2 (b) of the Special Conditions is fulfilled.

6.8. OthersThe assignment will be implemented and supervised by Turkish Ministry of Labour and Social Security - EU Coordination Department as the Operating Structure and the Beneficiary. Also, it is responsible for the overall evaluation of the HRD OP.

An Internal Coordination Team, composed of 4 experts/assistant experts, will be formed within the OS and it will be functioning separately from the management and control system. The Internal Coordination Team is planned to be the direct counterpart of the Consultant and it cooperates and ensures the preparation and delivery of the relevant documentation to be used during the implementation of the assignment. The Consultant and Internal Coordination Team will meet once a week in the Coordination Meetings to discuss issues related to the implementation of the assignment. The Consultant and Internal Coordination Team may come together when needed for smooth ongoing of the assignment.

Evaluation Sub-Committee (ESC) established under the SMC of the HRD OP, shall transmit its comments on any guidance document about evaluation and monitoring systems; support the independence of the evaluation function; and review and check the quality of works of the Consultant (using an agreed quality check list) including inception and draft evaluation reports and issue comments. The OS should compile all relevant comments provided by the ESC about the reports/outputs of the Consultant and submit them in a clearly defined structure and timetable.

Senior Representative of Operation Beneficiary (SROB) will act as the official representative of the MoLSS for this assignment and will bear overall responsibility for successful implementation of the assignment through cooperating with the Consultant and ensuring that the team performs its tasks in accordance with the pre-defined deadlines and the standard of quality required. He will formally approve the reports and the technical documents. He will ensure the coordination for the development and proper implementation of the assignment.

The Central Finance and Contracts Unit (CFCU) is the Contracting Authority and will be responsible for all procedural aspects of the tendering process, contracting matters and financial management, including payment of contract activities.

Contact Information

Contact person at the CFCU

Mr. Muhsin AltunProgrammeAuthorising Officer – CFCU DirectorTel: +90 312 295 49 00Fax: +90 312 286 70 72E-Mail: [email protected]: EskiєehirYolu 4. km. 2.Cad. (Halkbank Kampьsь) No: 63 C-Blok 06580 Sцрьtцzь/Ankara Tьrkiye

Publicity and VisibilityThe Consultant shall take all necessary measures to publicize the fact that the European Union has financed the Program.

In addition, the Consultant shall take the necessary measures to ensure the visibility of the European Union financing or co financing. These measures must comply with the rules laid down and published by the Commission on the visibility of external operations: http://ec.europa.eu/europeaid/work/visibility/index_en.htm

285

All projects /contract implemented under this programme shall comply with the Visibility Guidelines for European Commission Projects in Turkey published by the EU Delegation to Turkey, at http://www.avrupa.info.tr/AB_Mali_Destegi/Gorunurluk,Visi.html

All communication and visibility activities should be carried out in close co-operation with the CFCU. The CFCU is the main authority in charge of reviewing and approving visibility-related materials and activities. Before initiating any information, communication or visibility material and activity, consultants and implementing partners should seek the approval of the CFCU in writing.

The EU-Turkey cooperation logo should be accompanied by the following text:

“This project is co-financed by the European Union and the Republic of Turkey.”

Whether used in the form of the EU-Turkey cooperation logo for information materials or separately at events, the EU and Turkish flag have to enjoy at least double prominence each, both in terms of size and placement in relation to other displayed logos and should appear on all materials and at all events as per the Communication and Visibility Manual for European Union External Actions. At visibility events, the Turkish and the EU flag have to be displayed prominently and separately from any logos.

Logos of the beneficiary institution and the CFCU should be clearly separated from the EU-Turkey partnership logo and be maximum half the size of each flag. The logos will not be accompanied by any text. The CFCU and beneficiary logo will be on the lower left-hand corner and lower right-hand corner respectively. The consultant logo with the same size will be in the middle of the CFCU and beneficiary logo. If the consultant is a consortium, only the logo of the consortium leader will be displayed.

Any publication by the Consultant, in whatever form and by whatever medium, including the Internet, shall carry the following or a similar warning: “This document has been produced with the financial assistance of the European Union”. In addition, the back cover of any such publications by the Consultant should also contain the following disclaimer: “The contents of this publication is the sole responsibility of name of the author/Consultant/implementing partner – and can in no way be taken to reflect the views of the European Union”.

Taxand VAT ArrangementsThe contract is, as a rule, exempt from all taxes and duties, including value added tax (VAT) and Special Consumption Tax (SCT), Motor Vehicle Tax, Special Communication Tax, and/or taxes of equivalent effect, stamp or registration duties or any other charge having equivalent effect, pursuant to the provisions of Articles set out in the Framework Agreement88 signed between Turkey and the EC in 2008, extract of Article 25 & 26 of which are provided in Annex I.

The Contractor shall accordingly complete the necessary formalities with the relevant authorities to ensure that the goods and services required for performance of the Contract are exempt from taxes, customs, import duties, levies and/or taxes of equivalent effect, and stamp or registration duties or any other charge having equivalent effect.

Annexes to the Terms of Reference:

ANNEX I: Tax and Customs ArrangementsANNEX II: Evaluation Grid

88 This is an extract of Article 25 and 26 of the IPA Framework Agreement signed between Turkey and the EC on 11.07.2008, and adopted as law by Turkish Parliament (no: 5824) on 03 December 2008 and which was published in the Turkish Official Gazette on 07 December 2008, no: 27077. It has been put into force by the Government Decree, no 2008/14450 that was published in the Official Gazette no: 27090 (Complementary Issue), dated 24 December 2008. Please, also refer to the Framework Agreement and the Communiqués issued by the Ministry of Finance (the MoF) thereto for further information, especially for exemption scope and implementation procedure, which are available at the MoF’s website at: http://www.gib.gov.tr/index.php?id=1055.

286

http://www.gib.gov.tr/index.php?id=1055

Annex B3. Generic Question Frame for meetings

1. How does the HRD OP impact on work planning within the organisationDo you make annual work plans for your organisation?Have you assigned certain tasks to certain specific employees?Is HRD OP management part of the overall work plan?

2. How many projects are related to your organisation / institution?Can you tell me, briefly, about them?Can you, briefly, tell me the process from tender through implementation and monitoring to final end evaluation / what is your duties during the different stages?Note: these questions can be checked through documents; the point is to obtain an understanding of the stakeholders’ perception of their own roles

3. How does the HRD OP impact on daily routines and the organisation of work

4. How do you organise the administration of HRD OP?Is the management of HRD OP an administrative burden? Have you any ideas to how it could be done less bureaucratic and more smoothly?

5. How appropriate are the management structures formed at the levels of Operation Structure and Operation Beneficiaries? What could be done differently in the whole system? What could be done differently in your institution?

6. What are your direct partner institutions (to whom you report, who report to you, with whom you communicate, to whom you send or from whom you receive documents according to formal document flows and procedures)? How would you assess cooperation with each of the identified partner institutions? Please describe any experiences/practices with other institutions that you would like to change and what would you like to change?

7. How is the priory axes impact on your daily work1) attract and retain more people in employment (women, young) 2) investment in human capital through education and VET (especially for women/girls)3) promoting life-long learning4) integration of the disadvantaged persons to the labour market5) technical assistance

8. How do the HRD OP and priority axes meet?

9. Do the projects within the HRD OP mirror the local needs? How do you assess local needs?

10. What sort of routines is used for follow-up procedures, for monitoring and evaluation, and for registration (see and observe)

11. How is the communication between your organisation and the Ministry / central level?

12. To which extent is registration and monitoring procedures done in paper / PC / national MIS (see

287

QF for Information Systems)

13. To which extent is the data and information stream digitalised?

14. What sort of quality assurance procedures for data and information collected through the system?

15. Can we trust data and information collected? From your organisation? From local level?

16. In general, how do you see the formal procedures in HRD OP?Is your main contact to the projects formal or non-formal – and which of the two serves your purpose best?(try to provide examples of informal communication)

17. Have you experienced any problems in finding and in visitation of participants in projects (for instance training programmes etc.)?- women- young- disabled

18. How are the facilities for disabled / women?Do you perform on-site visits and controls?How do you perform that (on-site visits and controls)?

19. How is the regional distribution of participants?Are they just from the major cities and growth centres?

20. How is the dialogue with employers?How is the outcome of the HRD OP (e.g. trainings) perceived by the local employers?

a. Employers participating in the programmeb. Employers NOT participating in the programme

Is there a risk of distortion of the local labour market due to the programme?

21. How well does the HRD OP meet the horizontal issues: equal opportunities for men and women; sustainable development & environmental protection; participation of civil society; geographic, sectoral and thematic concentration; concerns of disadvantaged persons; good governance

E.g.: How does civil society impact on the HRD OP (and vice versa)? Does the civil society participate? Who is the civil society according to your opinion?What is good governance?

COOPERATION, SOCIAL DIALOGUE AND PARTNERSHIP APPROACH22. How do you assess the cooperation between your institution and:EmployersTraining schools and educational centresChambersLocal authorities

288

MoLSSİŞKURMoNE

23. Do your organisation take part in defining priorities, indicators, targets, and roles – or does the initiative mainly originates from other (e.g. central organisations, ministries, etc.)

24. How does the partnership approach work on local and regional level?

SWOT25. What do you consider to be your organisation’s / institution’s main strengths in programming and implementing the HRD OP?Do you use your strength or are there things you could have done more efficiently?

26. What do you consider to be your organisation’s / institution’s main Weaknesses in programming and implementing the HRD OP?How do your weaknesses impact on the HRD OP implementation?

27. What do you consider to be your organisation’s / institution’s main Opportunities in programming and implementing the HRD OP?How can the opportunities come into effective use for the forthcoming programme?

28. What do you consider to be your organisation’s / institution’s main Threats in programming and implementing the HRD OP?How do the threats impact on the HRD OP implementation?

289

Annex B4 Basic Survey on the functionality of the HRD OP: Questionnaire

Page 1Evaluation of the HRD OP: BasicImportant note: The Interim Evaluation Team guarantees anonymity. All information entered in this section is only meant to ensure that the survey covers all sectors and participants in the HRD OP. The information only serves internal purposes on the Interim Evaluation Team and will be removed from our report.

1. Please enter the name of your organisation European Commission or similar including

• National IPA coordinator• EUD• EUSG• Other

Ministry of Labour and Social Security (MoLSS) Turkish Employment Agency (İŞKUR)

• İŞKUR Central level (Ankara)• İŞKUR Regional level• İŞKUR Local level

Ministry of National Education, MoNE, or related, including:• The Council of Higher Education (YÖK)• Vocational Qualification Authority (MYK)• University• Public Training Institutions involved in initial vocation or technical training (VET/TVET)• Public Training Institutions involved in Formal Education• Public Training Institutions involved in Adult Education (LifeLong

Learning) Private Training Institutions; please specify your involvement in HRD OP:

• Initial vocation or technical training (VET/TVET)• Formal Education• Adult Education (LifeLong Learning)

Ministry of Development / State Planning Organization (DPT)• Central level (Ministry of Development)• Regional Development Agency

State Organization for the Development of Small and Medium Sized Businesses (KOSGEB) Small and Medium Sized Enterprises (KOBİ) Turkish Statistical Institute (TÜİK)

Page 2Evaluation of the HRD OP: Basic2. Please specify your type of organisation

Governorship NGO/ Occupational Organization and Chambers; please specify:

• Chamber of Commerce and Industry• Exporters’ Associations• Trade union• Employer association

Other (please specify)

290

Page 3Evaluation of the HRD OP: Basic3. Please specify your title / seniority

4. Please specify your main task connected to HRD OP (please tick your prime task) SMC Member, including subcommittee Monitoring Selection of projects Inspection of projects Other (please specify)

5. Please specify your regionTR 00 Central level (e.g. Ministry or central organisation in e.g. Ankara or Istanbul)TR 83 Samsun, Tokat, Çorum, AmasyaTR 72 Kayseri, Sivas, YozgatTR 90 Trabzon, Ordu, Rize, Giresun, Artvin, GümüşhaneTR B2 Van, Muş, Bitlis, HakkariTR B1 Malatya, Elazığ, Bingöl, TunceliTR 63 Hatay, Kahramanmaraş, OsmaniyeTR C2 Şanlıurfa, DiyarbakırTR A1 Erzurum, Erzincan, BayburtTR 82 Kastamonu, Çankırı, SinopTR C3 Mardin, Batman, Şırnak, SiirtTR C1 Gaziantep, Adıyaman, KilisTR A2 Ağrı, Kars, Iğdır, Ardahan

6. Please indicate your Municipality

Page 4Evaluation of the HRD OP: Basic7. To what extent do the measures mirror the local and regional needs in your region?

Criteria:o Agree o Moderately Agreeo Neutral o Disagree o I don’t know

Question:o Promoting women’s employment o Promoting youth employment o Promoting registered employment o Increasing quality of public employment services o Supporting girls’ enrolment and education o Strengthening Vocational and Technical Education o Strengthening links between Lifelong Learning, education system and labour marketo Increasing adaptability of employees and employers no Increasing integration of disadvantaged persons to the labour market

291

Page 5Evaluation of the HRD OP: Basic8. Do the selected HRD OP projects in your region reflect and address the local and regional needs?

Criteria:o Yes, to large extento Yes, to some extento Not really, we would like to add moreo Not at all o I don’t know

Questions:o Promoting women’s employment o Promoting youth employment o Promoting registered employment o Increasing quality of public employment services o Supporting girls’ enrolment and education o Strengthening Vocational and Technical Education o Strengthening links between Lifelong Learning, education system and labour marketo Increasing adaptability of employees and employers o Increasing integration of disadvantaged persons to the labour market

Page 6Evaluation of the HRD OP: Basic9. Do you find reason to revise any of the measures mention in the aforementioned questions perhaps subject to the situation within your region?

Criteria:o YES o NO o I don’t know

Questions:o Promoting women’s employment o Promoting youth employment o Promoting registered employment o Increasing quality of public employment services o Supporting girls’ enrolment and education o Strengthening Vocational and Technical Education o Strengthening links between Lifelong Learning, education system and labour market o Increasing adaptability of employees and employers o Increasing integration of disadvantaged persons to the labour market

10. In case you typed “YES” in question 9 could you please specify why? Criteria:

o Not applicable to our regiono Any measure distorts the labour marketo Other (specify below)

Questions:o Promoting women’s employment o Promoting youth employment

292

o Promoting registered employment o Increasing quality of public employment services o Supporting girls’ enrolment and education o Strengthening Vocational and Technical Education o Strengthening links between Lifelong Learning, education system and labour market o Increasing adaptability of employees and employers o Increasing integration of disadvantaged persons to the labour market o Other (please specify)

Page 7Evaluation of the HRD OP: BasicTo what extent will the Programme meet its targets?11. How effective are the evaluation and selection criteria?Please state your opinion on the statements below.

Criteriao Agree o No opinion o Disagree o I do not know

Questions:o Evaluation and selection criteria are objective o The evaluation and selection criteria are fair or at least the only doable o The evaluation and selection criteria ensure that the most successful projects are

selectedo The evaluation and selection criteria ensure that only projects which makes an impact

on the local development are selectedo The evaluation and selection criteria ensure a balance in number of projects across

priorities axis within the regionso The evaluation and selection criteria ensure a fair balance in number of projects

across regions in accordance to regional capacity and needso The evaluation and selection criteria ensure that only projects which sufficiently

address the local and regional needs are selected

12. Please indicate how the evaluation and selection criteria can be improved in case you disagree with any of the above statements:

Page 8Evaluation of the HRD OP: Basic13. How effective and balanced are the evaluation and selection of projects?Please state your opinion on the statements below.


Questions:o The selection of projects are done on an objective basiso The selection of projects are balanced across priorities

293

o The selection of projects mirror the regional and local needs o The selection of projects are balanced across regions o The selection of projects are balanced within the 12 NUTS 2 regions o The selection of projects mirrors the need of less developed local areas within the

NUTS 2 regionso Regional stakeholders have sufficient influence on the selection of projects

Page 9Evaluation of the HRD OP: Basic14. Please consider the following statements


Questions:o Potential beneficiaries of the Programme have enough information on projecto evaluation and selection criteria.o Potential beneficiaries of the Programme have enough information on requirements

related to improvement of PTDs for final approval.o High number of project proposals received after the first call for tender (2nd Package)

has made it difficult to evaluate proposals properly.o Project evaluation and selection process is effective enough (for instance, time spent

compared with the results).o You have enough information on Programme monitoring indicatorso Programme indicators reflect Programme outputs/resultso Programme has effects that are not put forward by indicatorso Potential beneficiaries of the Program have been informed sufficiently about the

opportunities and benefits of the Programme.o Potential beneficiaries of the Program have been informed sufficiently about project

selection.o Potential beneficiaries of the Program have been informed sufficiently about

evaluation process.o Potential beneficiaries of the Program have been informed sufficiently about

developing projects.o Potential beneficiaries of the Program have been informed sufficiently about tendering

and implementation.

15. How could the selection of projects be improved?

Page 10Evaluation of the HRD OP: Basic16. Please indicate your view of the Operational Programme and its projects:

Criteria:o To a large extento To some extento Medium

294

o To a less degreeo Not at all o I do not know

Questions:o To what extent do you think the projects implemented in your region have increased

women employment?o To what extent do you think the projects implemented in your region have increased

youth employment?o To what extent do you think the projects implemented in your region have increased

registered employment?o To what extent do you think the projects implemented in your region have

strengthened public employment offices?o To what extent do you think the projects implemented in your region have increased

girls’ enrolment rate?o To what extent do you think the projects implemented in your region have

strengthened vocational and technical education?o To what extent do you think the projects implemented in your region have

strengthened links between education system and labour market?o To what extent do you think the projects implemented in your region have increased

adaptability of employees and employers?o To what extent do you think the projects implemented in your region have contributed

to integration of disadvantaged groups to labour market?

Page 11Evaluation of the HRD OP: Basic17. Please indicate your view on the functioning of the Regional Grant MonitoringTeams


Questions:o Responsibilities of Regional Grant Monitoring Teams, RGMTT, regarding the

management and implementation of HRD OP have been defined clearly.o Regional Grant Monitoring Teams, RGMTT, are effective and sufficient o Rotation principle ensures Regional Grant Monitoring Teams, RGMTT, to be effective

and efficient.o The time allocated to meet the task of the Regional Grant Monitoring Teams, RGMTT,

are sufficient and reasonablyo The Regional Grant Monitoring Teams, RGMTT, ensure effective support and backup

for the projectso The Regional Grant Monitoring Teams, RGMTT, ensure that the projects runs

effectivelyo The Regional Grant Monitoring Teams, RGMTT, ensure proper and timely inspection

of the projects in accordance with their monitoring role

18. How could the functioning of the Regional Grant Monitoring Teams be improved?

295

Page 12Evaluation of the HRD OP: Basic19. Please indicate your view on the functioning of the Sectoral Monitoring Committee


Questions:o Responsibilities of Sectoral Monitoring Committee Members regarding the

management and implementation of HRD OP have been defined clearly.o SMC meetings are effective and sufficient o Rotation principle ensures SMC meetings to be effective and efficiento Time allocated to discuss agenda during SMC meetings is enough.

20. How could the functioning of the Sectoral Monitoring Committee be improved?

Page 13Evaluation of the HRD OP: Basic21. To what extent do the achieved results correspond to the resources spent (financialresources, human resources, time)? Please consider the following statements:


Questions:o Human resources of MoLSS EU Coordination Department, EU Turkey Delegation,

Central Finance and Contracts Unit and beneficiary organizations is sufficient in terms of quality to fulfil tasks related to program management and implementation.

o Human resources of MoLSS EU Coordination Department, EU Turkey Delegation, Central Finance and Contracts Unit and beneficiary organizations is sufficient in terms of quantity to fulfil tasks related to program management and implementation.

22. According to you, what are the fundamental success factors (internal and external)which determine the efficiency of HRD OP up until now? Please state your opinion tothe box below by using key words or short sentences.

23. According to you what are the fundamental problematic areas (internal and external)which determines the efficiency of the HRD OP up until now? Please state your opinionto the box below by using key words or short sentences.

Page 14Evaluation of the HRD OP: Basic

296

This constitutes the end of the online evaluation questionnaire.THANK YOU FOR YOUR KIND PARTICIPATION!

297

Annex B5 Survey on Indicators: Questionnaire

Page 1Evaluation of the HRD OP: IndicatorsThis Questionnaire serves to obtain some brief opinions on the relevance and the effectiveness of the indicators selected for monitoring and evaluation of the HRD OP.

Background information:The HRD OP operates with two types of indicators: Result and Output Indicators.Results indicators serve to measure both Outcome and Impact (and not least net impact), while output indicators measures output, which is the basic result from undertaken a specific action (e.g. 20 trainees trained). The HRD OP does not explicitly define input indicators.

In here we have copied the Result indicators as they are mentioned in the programme.We do not ask you to comment on the theory of evaluation and monitoring purposes. However, we kindly ask you to provide some few comments on the relevance of the indicator subject to the Measure and the Objective.

Page 2Evaluation of the HRD OP: BasicImportant note: The Interim Evaluation Team guarantees anonymity. All information entered in this section is only meant to ensure that the survey covers all sectors and participants in the HRD OP. The information only serves internal purposes on the Interim Evaluation Team and will be removed from our report.

1. Please enter the name of your organisation European Commission or similar including

• National IPA coordinator• EUD• EUSG• Other

Ministry of Labour and Social Security (MoLSS) Turkish Employment Agency (İŞKUR)

• İŞKUR Central level (Ankara)• İŞKUR Regional level• İŞKUR Local level

Ministry of National Education, MoNE, or related, including:• The Council of Higher Education (YÖK)• Vocational Qualification Authority (MYK)• University• Public Training Institutions involved in initial vocation or technical training (VET/TVET)• Public Training Institutions involved in Formal Education• Public Training Institutions involved in Adult Education (LifeLong

Learning) Private Training Institutions; please specify your involvement in HRD OP:

• Initial vocation or technical training (VET/TVET)• Formal Education• Adult Education (LifeLong Learning)

298

Ministry of Development / State Planning Organization (DPT)• Central level (Ministry of Development)• Regional Development Agency

State Organization for the Development of Small and Medium Sized Businesses (KOSGEB) Small and Medium Sized Enterprises (KOBİ) Turkish Statistical Institute (TÜİK)

Page 3Evaluation of the HRD OP: Basic2. Please specify your type of organisation

Governorship NGO/ Occupational Organization and Chambers; please specify:

• Chamber of Commerce and Industry• Exporters’ Associations• Trade union• Employer association

Other (please specify)

Page 4Evaluation of the HRD OP: Basic3. Please specify your title / seniority

4. Please specify your main task connected to HRD OP (please tick your prime task) SMC Member, including subcommittee Monitoring Selection of projects Inspection of projects Other (please specify)

5. Please specify your regionTR 00 Central level (e.g. Ministry or central organisation in e.g. Ankara or Istanbul)TR 83 Samsun, Tokat, Çorum, AmasyaTR 72 Kayseri, Sivas, YozgatTR 90 Trabzon, Ordu, Rize, Giresun, Artvin, GümüşhaneTR B2 Van, Muş, Bitlis, HakkariTR B1 Malatya, Elazığ, Bingöl, TunceliTR 63 Hatay, Kahramanmaraş, OsmaniyeTR C2 Şanlıurfa, DiyarbakırTR A1 Erzurum, Erzincan, BayburtTR 82 Kastamonu, Çankırı, SinopTR C3 Mardin, Batman, Şırnak, SiirtTR C1 Gaziantep, Adıyaman, KilisTR A2 Ağrı, Kars, Iğdır, Ardahan

6. Please indicate your Municipality

299

Page 5Evaluation of the HRD OP: IndicatorsPlease have a look at the measures and the specific objectives defined in the HRD OP and the following indicators which are meant to measure the outcome of the measures and steps taken. The sections are divided into the five priority axes. The questionnaire mentions briefly their measures and their so-called “specific objectives”. From here, and kindly asking you to consider the measure and objectives, we kindly ask you to have a view on the indicators.Our question is: Are these indicators relevant, are data / information achievable, and can the indicators be used in the future after the HRD OP has been concluded?

Page 6Evaluation of the HRD OP: IndicatorsPriority axis 1:To attract and retain more people in employment, particularly by increasing labour force participation of women, and decrease unemployment rates especially for young people

7. Assuming the Specific objective 1: Promote labour force participation and employment of women, including those formerly employed in agriculture. Are the following indicators relevant for the objective?

Criteria: o YES o NO o Do not know

Questions:o Increased rate of women having entered into labour market (having the qualification

and motivation to seek a job) after participating in the courses or counsellingo Increased rate of women employed through the courses and counsellingo Increased rate of women entrepreneurs

8. If no: What is the problem? Net impact cannot be measured since Impact can arise from other factors in the environment Data is not available Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of

reports, etc.) Data differs from region to region (not standardized) Data are not systematically collected (e.g. timing differs between projects and/or regions) Data is only generated from on-going projects (we cannot see what status are in locations

without projects, before the project is launched, and after the project has been concluded) The indicator does not mirror the real target / is not relevant The target group is not clearly defined There is no baseline data (we cannot see and measure improvements) The indicator is too complicated to calculate (if based upon aggregated data) Other (please specify)

300

Page 7Evaluation of the HRD OP: Indicators9. Assuming Specific objective 2: To increase youth employment.Are the following indicators relevant for the objective?


Questions:o Increased rate of young people employed following the courses, internship and

counselling activitieso Increased rate of young entrepreneurs

10. If no: What is the problem? Net impact cannot be measured since Impact can arise from other factors in the environment Data is not available Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of



Page 8Evaluation of the HRD OP: Indicators11. Assume Specific objective 3: To promote registered employmentAre the following indicators relevant for the objective?


Questions:o Decrease in the rate of complaints (advice slip) on the unregistered employment to

the Insurance Inspection Boardo Increase in the number of bilateral projects on promoting registered employment

between the relevant partnerso Increase in the rate of registered people in 43 city centres through the bilateral

projects and local partnerships

12. If no, what is the problem? Net impact cannot be measured since Impact can arise from other factors in the environment Data is not available Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of

reports, etc.)

301

Data differs from region to region (not standardized) Data are not systematically collected (e.g. timing differs between projects and/or regions) Data is only generated from on-going projects (we cannot see what status are in locations


Page 9Evaluation of the HRD OP: Indicators13. Assume Specific objective 4: To improve public employment services.Are the following indicators relevant for the objective?


Questions:o Increase in the number of unemployed people counselledo Increase in the placement rates of İŞKUR




Page 10Evaluation of the HRD OP: IndicatorsNext consider Priority axis 2: To enhance investment in human capital by increasing the quality of education, improving the linkage between education and labour market, and raising enrolment rates at all levels of education, especially for girls

15. Assume Specific objective 1: To increase enrolment rates particularly for girls at alllevels of education and vocational training.Are the following indicators relevant for the objective?

Criteria:

302

o YES o NO o Do not know

Questions:o Increased percentage of girls accessing to education, particularly secondary

education, as a result of programme interventiono Reduction in the number of girls dropping out of secondary education and VET o Increase in the number of girls graduating from secondary education and VET




Page 11

17. Assume Specific objective 2: To increase the quality of education especially invocational education and training.Are the following indicators relevant for the objective?


Questions:o The level of satisfaction of teachers, students and entrepreneurs who participated in

the activities regarding the content and quality of Vocational and Technical Education.o Increasing demands for 45 pilot vocational schools.o Teacher training in line with the newly updated curriculum.



without projects, before the project is launched, and after the project has been concluded) The indicator does not mirror the real target / is not relevant The target group is not clearly defined

303

There is no baseline data (we cannot see and measure improvements) The indicator is too complicated to calculate (if based upon aggregated data) Other (please specify)

Page 12Evaluation of the HRD OP: Indicators19. Assume Specific objective 3: To develop innovative approaches to improve linkage between education and labour marketAre the following indicators relevant for the objective?


Questions:o No Indicators are available




Page 13

21. Assume Specific objective 4: To promote a “partnership approach” to modernize VET systemAre the following indicators relevant for the objective?




reports, etc.) Data differs from region to region (not standardized)

304

Data are not systematically collected (e.g. timing differs between projects and/or regions) Data is only generated from on-going projects (we cannot see what status are in locations


Page 14

Priority axis 3:To promote a “partnership approach” to modernize VET system.23. Assume Specific objective 1: To promote Life Long Learning (LLL) Opportunities under a LLL Strategy.Are the following indicators relevant for the objective?


Questions:o Share of primary education and secondary education graduates among participants

certified and tested in terms of vocational knowledge and skills in 10 VOCTEST Centres

o Increased percentage of participants who obtained a certificate in vocational knowledge and skills

o Increased percentage of participants of participants who obtained a certificate in basic skills

o Increased percentage of those who found work after completing an employability course

Page 15




Page 16

25. Assume Specific objective 2: To improve quality of non-formal trainings

305

Are the following indicators relevant for the objective? Criteria:

o YES o NO o Do not know





Page 17

27. Assume Specific objective 3: To increase adaptability of employeesAre the following indicators relevant for the objective?


Questions:o Increased adaptability of workers in terms of having extra skills or abilities related to

their position (percentage of participants increasing their salary or being promoted or having extra responsibilities)

o Increased training activities on the requirements of adaptability given by social partners to their members



without projects, before the project is launched, and after the project has been concluded) The indicator does not mirror the real target / is not relevant

306

The target group is not clearly defined There is no baseline data (we cannot see and measure improvements) The indicator is too complicated to calculate (if based upon aggregated data) Other (please specify)

Page 18

29. Assume Specific objective 4: To increase adaptability of employers in SMEsAre the following indicators relevant for the objective?


Questions:o Increased adaptability of employers in terms of having extra skills or abilities in order

to be competitive enough (percentage of participants increasing number of workers, change of the branch of activity)




Page 19

31. Assume Specific objective 5: To promote well-functioning of the National Qualifications SystemAre the following indicators relevant for the objective?


Questions:o Number of certificates issued by VOCTEST Centres


reports, etc.)

307



Page 20Evaluation of the HRD OP: IndicatorsPriority axis 4:To promote an inclusive labour market with opportunities for disadvantaged people, with a view to their sustainable integration into the labour force and combat all forms of discrimination in the labour market.

33. Assume Specific objective 1: To facilitate sustainable integration of the disadvantaged into the labour market.Are the following indicators relevant for the objective?


Questions:o Increased rate of disadvantaged persons entered into labour market (having the

qualification and motivation to seek a job) following the rehabilitation programmes, courses and counselling.

o Increased rate of disadvantaged persons employed following the courses and counselling services




Page 21

308

35. Assume Specific objective 2: To improve the functioning and coordination of institutions and mechanisms in the field of labour market and social protection, particularly in order to facilitate the integration of disadvantaged persons into the labour market.Are the following indicators relevant for the objective?


Questions:o Increased satisfaction on the services both in institutional and target group level (by

staff of institutions and by disadvantaged persons)




Page 22Evaluation of the HRD OP: IndicatorsPriority axis 5:Technical Assistance37. Assume Specific objective 1: Strengthening the necessary capacity at central level to efficiently develop, implement, evaluate, monitor and control the IPA funds in the period 2007-2009 within the framework of HRD OP and improving the information about the ESF structures and best examples, providing assistance in the transition to Decentralised Management without ex-ante control of the EC DelegationAre the following indicators relevant for the objective?



38. If no, what is the problem? Net impact cannot be measured since Impact can arise from other factors in the environment Data is not available

309

Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of reports, etc.)



Page 23Evaluation of the HRD OP: Indicators39. Assume Specific objective 2: Increasing the absorption capacity of the final beneficiaries and administrative capacity of stakeholders which may get more responsibilities in the upcoming periodAre the following indicators relevant for the objective?






Page 2441. Assume Specific objective 3: Increasing the information and public awareness with respect to the preparation for and effective use of IPA funds in Turkey in line with the HRD OPAre the following indicators relevant for the objective?

310






This constitutes the end of the online evaluation questionnaire on indicators.THANK YOU FOR YOUR KIND PARTICIPATION!

311

The content of this publication is the sole responsibility of the Consultant and can in no way be taken to reflect the views of the European Union.

The content of this publication is the sole responsibility of the Consultant and can in no way be taken to reflect the views of the European Union.

312

Date post:	30-May-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Abbreviations · Web viewThe Interim Evaluation TAT, however, accentuates that the SWOT in this...

Documents