Annual Activity Report Annexes - European Commission · eac_aar_2013_annexes_final Page 1 of 75...

eac_aar_2013_annexes_final Page 1 of 75

2013

Annual Activity Report

Annexes

DG Education and Culture

27 March 2014

Ref. Ares(2014)978408 - 28/03/2014


TABLE OF CONTENTS

ANNEX 1: STATEMENT OF THE RESOURCES DIRECTOR.................................................................................... 3

ANNEX 2: HUMAN AND FINANCIAL RESOURCES ............................................................................................ 4

ANNEX 3: ANNUAL ACCOUNTS AND FINANCIAL REPORTS ................................................................................ 8

ANNEX 4: MATERIALITY CRITERIA ............................................................................................................ 27

ANNEX 5: INTERNAL CONTROL TEMPLATE(S) FOR BUDGET IMPLEMENTATION (ICTS) ......................................... 29

ANNEX 6: IMPLEMENTATION THROUGH NATIONAL OR INTERNATIONAL PUBLIC-SECTOR BODIES AND

BODIES GOVERNED BY PRIVATE LAW WITH A PUBLIC SECTOR MISSION ................................................ 59

ANNEX 7: AARS OF EXECUTIVE AGENCIES................................................................................................. 65

ANNEX 8: DECENTRALISED AGENCIES........................................................................................................ 66

ANNEX 9: PERFORMANCE INFORMATION INCLUDED IN EVALUATIONS.............................................................. 67


ANNEX 1: Statement of the Resources Director

I declare that in accordance with the Commission’s communication on clarification of the

responsibilities of the key actors in the domain of internal audit and internal control in

the Commission1, I have reported my advice and recommendations to the Director-

General on the overall state of internal control in the DG.

I hereby certify that the information provided in Parts 2 and 3 of the present AAR and in

its annexes is, to the best of my knowledge, accurate and exhaustive.

Brussels, 28 March 2014

[signed]

Arturo CABALLERO BASSEDAS

1 SEC(2003)59 of 21.01.2003.


ANNEX 2: Human and Financial resources

Table 1: Human Resources by ABB activity

Establishment Plan posts External Personnel Total Code ABB

Activity ABB Activity AMP AAR AMP AAR AMP AAR

15 02 Lifelong learning, including multilingualism 233 230 36 45 269 275

15 04 Developing cultural and audiovisual cooperation in Europe 55 51 8 9 63 60

15 05 Encouraging and promoting cooperation in the field of youth and sports 39 40 6 9 45 49

15 07 People — Programme for the mobility of researchers 22 22 13 7 35 29

15 AWBL-01 81 (*) 75 9 16 90 91

15 AWBL-01 Central Library 53 54 12 12 65 66

15 AWBL-01 Traineeship Unit

Administrative support for the Directorate-General for Education and Culture

15 13 2 2 17 15

15 AWBL-02 Policy strategy and coordination for the Directorate-General for Education and Culture 48 (*) 57 4 5 52 62

Total 546 542 90 105 636 647


Establishment Plan posts *15 AWBL-01 The decrease in the number of posts under this budget line reflects the DG's concerted effort in recent years to increase the proportion of front-line operational staff and decrease the proportion of administrative support staff.

*15 AWBL-02 The increase in the number of posts under this budget line is determined by the need to return posts to DG HR representing taxation and contributions to the central redeployment pool in 2014. As these posts are kept in reserve at Director-General level, they are counted under the same budget line as the Director-General.

External Personnel The methodology for calculating the external staff figures is different for AMP and AAR: - in the AMP, the external staff has to match the final HR allocation as decided by the Commission; - in the AAR, the figure is calculated based on a Sysper2 snapshot on 31 December 2013. Therefore, no relevant comparison of the two figures can be carried out. As reference, on 01/01/2013, there were 104 external staff compared to 105 on 31/12/2013 (AAR), while the AMP figure is 90.


Table 2: Financial Resources by ABB activity (EUR) implementation of Commitment Appropriations (CA)

Administrative expenditure Code

ABB Activity

ABB Activity Operational expenditure

(1) (2) Total

15 02 Lifelong learning incl. multilingualism

1 678 159 235 1 292 040 34 629 070 1 714 080 345

15 04 Cultural cooperation and Media

193 188 101 533 283 15 320 173 209 041 557

15 05 Cooperation in youth and sport 218 363 011 408 951 4 424 880 223 196 842

15 07 People 1 072 136 300 3 074 092

1 075 210 392

15 AWBL- 15 AWBL-01 2 489 627 10 534 034

13 023 661

15 AWBL- 15 AWBL-02 139 547 139 547

Total 3 161 846 647 72 845 697 3 234 692 344

(1) Heading 5 appropriations managed by the DG (global envelope) XX 01 02 (2) BA lines (XX 01 04) and, when relevant XX 01 05 and XX 01 06.


Table 3: IMPLEMENTATION OF THE GLOBAL ENVELOPE - BUDGET LINES CONCERNED: 15 01 02 11 00 01 TO 15 01 02 11 00 06 (based on information received from BUDG services following the 2015 Budget circular)

(IN EUROS) APPROPRIATIONS 2013 (C1) APPROPRIATIONS carried over (C8)

BUDGET LINE* BUDGET LINE DESCRIPTION

AVAILABLE APPROPRIATIONS

2013

COMMITMENTS

2013

PAYMENTS

2013

AMOUNTS OF APPROPRIATIONS

CARRIED OVER FROM 2012

% IMPLEMENTATION ON APPROPRIATIONS CARRIED OVER FROM

2012

15.010211.00 2,701,309.00 (*)52,673.03

15.010211.00.01.10 Mission expenses 820,000.00 755,258.00 78,112.50 94.96

15.010211.00.01.30 Representation expenses 5,000.00 1,889.00 0.00 0

15.010211.00.02.20 Meeting costs 1,170,000.00 1,169,787.00 379.48 100

15.010211.00.02.40 Conference costs 281,816.00 144,537.00 97,704.79 100

15.010211.00.03 Meetings of committees 110,000.00 108,101.00 8,253.95 100

15.010211.00.04 Studies and consultations 219,912.00 34,547.00 430,391.97 100

15.010211.00.05 Development of management and information systems

94,581.00 43,143.00 140,705.62 100

15.010211.00.06 Further training and management training

149,146.00 149,146.00 46,547.00 125,236.14 78.97

TOTAL 2,850,455.00 2,850,455.00 2,304,078.00 933,457.48 91.11

(*) € 52,673.03 have been decommitted


ANNEX 3: Annual accounts and financial reports

Annex 3 Financial Reports - DG EAC - Financial Year 2013

Table 1 : Commitments

Table 2 : Payments

Table 3 : Commitments to be settled

Table 4 : Balance Sheet

Table 5 : Economic Outturn Account

Table 6 : Average Payment Times

Table 7 : Income

Table 8 : Recovery of undue Payments

Table 9 : Ageing Balance of Recovery Orders

Table 10 : Waivers of Recovery Orders

Table 11 : Negotiated Procedures (excluding Building Contracts)

Table 12 : Summary of Contracts (excluding Building Contracts)

Table 13 : Building Contracts

Table 14 : Contracts declared Secret


Additional comments

1) Article 128 FR Transparency The average time to grant according to Article 128 FR is respected by DG EAC. The average period of 6 months in Article 128 (2a) FR is at 2,6 months at DG EAC and the average of 3 months in Article 128 (2b)FR is at 1,7 months at DG EAC. 2) The current receivables presented in Table 4 and Table 9 have a difference in balance of € 100.479,29. This difference between SAP and the data warehouse is related to transactions recorded on different profit centers for technical accounting reasons. DG BUDG is informed and consulted, but not able to solve this issue. The receivable amount presented in Table 9 is the correct one and at Commission level the difference is balanced.


TABLE 1: OUTTURN ON COMMITMENT APPROPRIATIONS IN 2013(in Mio €)

Commitment appropriations

authorised

Commitments made %

1 2 3=2/1

Title 15 Education and culture

15 15 01 Administrative expenditure of the Èducation and culture- policy area 76,40 72,85 95,35 %

15 02 Lifelong learning, including multilingualism 1508,00 1429,53 94,80 %

15 04 Developing cultural and audiovisual cooperation in Europe 11,22 10,64 94,79 %

15 05 Encouraging and promoting cooperation in the field of youth and sports 225,92 193,51 85,65 %

15 07 People - Programme for the mobility of researchers 1,48 0,41 27,94 %

Total Title 15 1823,02 1706,93 93,63%

Total DG EAC 1823,02 1706,93 93,63 %

* Commitment appropriations authorised include, in addition to the budget voted by the legislative authority, appropriations carried over from the previous exercise, budget amendments as well as miscellaneous commitment appropriations for the period (e.g. internal and external assigned revenue).


TABLE 2: OUTTURN ON PAYMENT APPROPRIATIONS IN 2013 (in Mio €)

Chapter Payment

appropriations authorised *

Payments made %

1 2 3=2/1

Title 15 Education and culture

15 15 01 Administrative expenditure of the Èducation and culture- policy area 87,51 69,95 79,93 %

15 02 Lifelong learning, including multilingualism 1463,29 1397,65 95,51 %

15 04 Developing cultural and audiovisual cooperation in Europe 7,80 7,75 99,42 %

15 05 Encouraging and promoting cooperation in the field of youth and sports 205,16 177,77 86,65 %

15 07 People - Programme for the mobility of researchers 2,88 1,38 47,82 %

Total Title 15 1766,65 1654,50 93,65%

Total DG EAC 1766,65 1654,50 93,65 %

* Payment appropriations authorised include, in addition to the budget voted by the legislative authority, appropriations carried over from the previous exercise, budget amendments as well as miscellaneous payment appropriations for the period (e.g. internal and external assigned revenue).


TABLE 3 : BREAKDOWN OF COMMITMENTS TO BE SETTLED AT 31/12/2013 (in Mio €)

2013 Commitments to be settled Commitments to be

settled from

Total of commitments to be settled at end

Total of commitments to be settled at end

Chapter Commitments 2013 Payments 2013 RAL 2013 % to be settled financial years previous

to 2013

of financial year 2013(incl corrections)

of financial year 2012(incl. corrections)

1 2 3=1-2 4=1-2/1 5 6=3+5 7

Title 15 : Education and culture

15 15 01

Administrative expenditure of the Èducation and culture- policy area

72,82 60,04 12,77 17,54 % 0,22 12,99 11,21

15 02 Lifelong learning, including multilingualism

1429,53 1222,24 207,28 14,50 % 39,06 246,35 221,71

15 04

Developing cultural and audiovisual cooperation in Europe

10,64 6,11 4,53 42,58 % 1,52 6,04 3,35

15 05

Encouraging and promoting cooperation in the field of youth and sports

193,51 158,85 34,66 17,91 % 12,03 46,69 40,29

15 07

People - Programme for the mobility of researchers

0,41 0,00 0,41 100,00 % 1,27 1,69 2,65

Total Title 15 1706,90 1447,24 259,66 15,21% 54,10 313,76 279,21

Total DG EAC 1706,90 1447,24 259,66 15,21 % 54,10 313,76 279,21



TABLE 4 : BALANCE SHEET

BALANCE SHEET 2013 2012

A.I. NON CURRENT ASSETS 132142667,8 160513166,9

A.I.1. Intangible Assets 2805998,57 1.570.312,46

A.I.5. LT Pre-Financing 129336669,3 158.942.854,46

A.II. CURRENT ASSETS 1001946838 956646166,8

A.II.2. Short-term Pre-Financing 984208551,5 929.885.120,04

A.II.3.2. Current Receivables and Recoveries 17738286,08 26.761.046,72

A.II.5. Cash and Cash Equivalents 0,00

ASSETS 1134089505 1117159334

P.III. CURRENT LIABILITIES -26097721,22 -9239166,74

P.III.4. Accounts Payable -26097721,22 -9.239.166,74

LIABILITIES -26097721,22 -9239166,74

NET ASSETS (ASSETS less LIABILITIES) 1107991784 1.107.920.166,94

P.I.2. Accumulated Surplus / Deficit 11291191 0

Non-allocated central (surplus)/deficit* -1119282975 -1107920167

TOTAL 0,00 0,00

It should be noted that the balance sheet and economic outturn account presented in Annex 3 to this Annual Activity Report, represent only the (contingent) assets, (contingent) liabilities, expenses and revenues that are under the control of this Directorate General. Significant amounts such as own resource revenues and cash held in Commission bank accounts are not included in this Directorate General's accounts since they are managed centrally by DG Budget, on whose balance sheet and economic outturn account they appear. Furthermore, since the accumulated result of the Commission is not split amongst the various Directorates General, it can be seen that the balance sheet presented here is not in equilibrium. Additionally, the figures included in tables 4 and 5 are provisional since they are, at this date, still subject to audit by the Court of Auditors. It is thus possible that amounts included in these tables may have to be adjusted following this audit.


TABLE 5 : ECONOMIC OUTTURN ACCOUNT

ECONOMIC OUTTURN ACCOUNT 2013 2012

II.1 SURPLUS/ DEF. FROM OPERATING ACTIVT 1409518440 1239045681

II.1.1. OPERATING REVENUES -164227691,5 -154265559

II.1.1.1. Own resource and contributions -23.908.897,00 -22.983.427,00

II.1.1.2. Other operating revenue -140.318.794,48 -131.282.132,00

II.1.2. OPERATING EXPENSES 1573746131 1393311240

II.1.2.1. Administrative Expenses 62.096.038,65 61.922.242,66

II.1.2.2. Operating Expenses 1.511.650.092,69 1.331.388.997,35

II.2. SURPLUS/DEF. NON OPERATING ACTIVIT -4738891,16 -5791541,74

II.2.1. FINANCIAL OPERATIONS -4738891,16 -5791541,74

II.2.1.1. Financial revenue -4.752.696,78 -5.794.621,51

II.2.1.2. Financial expenses 13.805,62 3.079,77

ECONOMIC OUTTURN ACCOUNT 1.404.779.548,70 1.233.254.139,27


TABLE 6: AVERAGE PAYMENT TIMES FOR 2013 - DG EAC

Legal Times

Maximum Payment

Time (Days)

Total Number of Payments

Nbr of Payments

within Time Limit

Percentage Average

Payment Times (Days)

Nbr of Late Payments Percentage

Average Payment Times

(Days)

20 75 68 90,67 % 9,68 7 9,33 % 21,14

21 65 62 95,38 % 13,74 3 4,62 % 22,67

29 74 74 100,00 % 8,43

30 1389 1324 95,32 % 16,13 65 4,68 % 38,89

44 1 1 100,00 % 50

45 8177 8152 99,69 % 10,38 25 0,31 % 80,08

50 9 9 100,00 % 16

60 1256 1255 99,92 % 17,23 1 0,08 % 68

75 3 3 100,00 % 18

90 16 16 100,00 % 32,88

105 1 1 100,00 % 16


11066 10964 99,08 % 102 0,92 %

Average Payment Time

12,23 11,90 47,69


Target Times

Target Payment

Time (Days)


Nbr of Payments

within Target Time

Percentage Average

Payment Times (Days)

Nbr of Late Payments Percentage

Average Payment Times

(Days)

20 392 313 79,85 % 13,59 79 20,15 % 27,96

21 65 62 95,38 % 13,74 3 4,62 % 22,67

29 74 74 100,00 % 8,43

30 10420 10282 98,68 % 11,54 138 1,32 % 45,17

45 1 1 100,00 % 56

50 6 6 100,00 % 15

60 5 5 100,00 % 16,6

75 13 12 92,31 % 27,42 1 7,69 % 85

90 1 1 100,00 % 16


10977 10755 97,98 % 222 2,02 %

Average Payment Time

12,17 11,62 38,97


Suspensions

Average Report

Approval Suspension

Days

Average Payment

Suspension Days

Number of Suspended Payments

% of Total Number


Amount of Suspended Payments

% of Total Amount

Total Paid Amount

6 96 472 4,27 % 11066 48.985.561,70 2,97 % 1.648.450.549,09

Late Interest paid in 2013

DG GL Account Description Amount (Eur)

EAC 65010000 Interest expense on late payment of charges 0,00

EAC 65010100 Interest on late payment of charges New FR 13 805,62

13 805,62


TABLE 7 : SITUATION ON REVENUE AND INCOME IN 2013 Revenue and income recognized Revenue and income cashed from Outstanding

Chapter Current year RO Carried over RO Total Current Year RO Carried over RO Total balance

1 2 3=1+2 4 5 6=4+5 7=3-6

52 REVENUE FROM INVESTMENTS OR LOANS GRANTED, BANK AND OTHER INTEREST 5964028,27 1112369,13 7076397,4 5742375,44 1102334,36 6844709,8 231687,6

57 OTHER CONTRIBUTIONS AND REFUNDS IN CONNECTION WITH THE ADMINISTRATIVE OPERATION OF THE INSTITUTION

981595,32 60053,48 1041648,8 981595,32 3293,48 984888,8 56760

59 OTHER REVENUE ARISING FROM ADMINISTRATIVE MANAGEMENT 653494,88 0 653494,88 653494,88 0 653494,88 0

60 CONTRIBUTIONS TO UNION PROGRAMMES 167696056,2 31680,24 167727736,4 167696056,2 31680,24 167727736,4 0

66 OTHER CONTRIBUTIONS AND REFUNDS 64712401,58 19283403,3 83995804,88 54175074,01 18073146,21 72248220,22 11747584,66

70 INTEREST ON LATE PAYMENTS 2416,71 347,72 2764,43 2273,12 347,72 2620,84 143,59

Total DG EAC 240009992,9 20487853,87 260497846,8 229250869 19210802,01 248461671 12036175,85


TABLE 8 : RECOVERY OF UNDUE PAYMENTS (Number of Recovery Contexts and corresponding Transaction Amount)

INCOME BUDGET RECOVERY ORDERS ISSUED IN 2013 Error Irregularity TOTAL Qualified TOTAL RC(incl. non-qualified) % Qualified/Total RC

Year of Origin (commitment) Nbr RO Amount Nbr RO Amount Nbr RO Amount Nbr RO Amount Nbr RO Amount

2001 1 7.486,51 1 7.486,51 2 39.041,56 50,00% 19,18%

2005 2 2.443,61 2 2.443,61 24 434.844,76 8,33% 0,56%

2006 1 973,91 1 973,91 34 316.215,03 2,94% 0,31%

2007 1 5.872,13 11 1.845.141,96 12 1.851.014,09 56 5.166.900,88 21,43% 35,82%

2008 1 16.182,39 6 329.061,95 7 345.244,34 93 15.249.062,15 7,53% 2,26%

2009 1 1.510,04 1 1.510,04 120 39.965.987,11 0,83% 0,00%

2010 1 40.000,00 1 40.000,00 39 3.374.945,62 2,56% 1,19%

2011 2 51.767,28 2 51.767,28 92 1.219.181,78 2,17% 4,25%

2012 2 39.074,29 2 39.074,29 76 912.543,63 2,63% 4,28%

No Link 1 9.553,64 1 9.553,64 9 29.541,80 11,11% 32,34%

Sub-Total 2 22.054,52 28 2.327.013,19 30 2.349.067,71 620 68.259.577,63 4,84% 3,44%

EXPENSES BUDGET Error Irregularity OLAF Notified TOTAL Qualified TOTAL RC(incl. non-qualified) % Qualified/Total RC

Nbr Amount Nbr Amount Nbr Amount Nbr Amount Nbr Amount Nbr Amount

INCOME LINES IN INVOICES

NON ELIGIBLE IN COST CLAIMS 7 49809,31 28 4063459,3 35 4113268,61 40 4.222.810,56 87,50% 97,41%

CREDIT NOTES 15 365760,34 15 236658,05 30 602418,39 239 896.866,38 12,55% 67,17%

Sub-Total 22 415569,65 43 4300117,35 65 4715687 279 5119676,94 23,30% 92,11%

GRAND TOTAL 24 437624,17 71 6627130,54 95 7064754,71 899 73379254,57 10,57% 6,43%


TABLE 9: AGEING BALANCE OF RECOVERY ORDERS AT 31/12/2013 FOR EAC

Number at 01/01/2013

Number at 31/12/2013 Evolution

Open Amount (Eur) at

01/01/2013

Open Amount (Eur) at

31/12/2013 Evolution

1999 1 1 0,00 % 8.000,00 8.000,00 0,00 %

2002 3 2 -33,33 % 136.900,51 118.079,19 -13,75 %

2003 2 2 0,00 % 38.667,00 38.667,00 0,00 %

2004 2 2 0,00 % 19.500,00 19.500,00 0,00 %

2005 9 9 0,00 % 109.757,47 109.757,47 0,00 %

2006 1 -100,00 % 10.061,96 -100,00 %

2007 1 1 0,00 % 79.532,80 79.532,80 0,00 %

2008 5 4 -20,00 % 597.659,39 97.659,39 -83,66 %

2009 7 2 -71,43 % 180.164,56 20.706,16 -88,51 %

2010 3 2 -33,33 % 692.265,00 663.934,76 -4,09 %

2011 1 1 0,00 % 8.329,31 8.329,31 0,00 %

2012 43 2 -95,35 % 18.607.015,87 112.885,78 -99,39 %

2013 24 10.759.123,99

78 52 -33,33 % 20.487.853,87 12.036.175,85 -41,25 %


TABLE 10 : RECOVERY ORDER WAIVERS IN 2013 >= EUR 100.000

Waiver Central Key

Linked RO Central Key

RO Accepted Amount

(Eur)

LE Account Group

Commission Decision Comments

Total DG

Number of RO waivers

DG EAC has no recovery order waivers above € 100.000 in 2013.


TABLE 11 : CENSUS OF NEGOTIATED PROCEDURES - DG EAC - 2013

Procurement > EUR 60,000

Negotiated Procedure

Legal base Number of Procedures Amount (€)

Art. 134.1(b) 2 832.339,89

Total 2 832.339,89


TABLE 12 : SUMMARY OF PROCEDURES OF DG EAC EXCLUDING BUILDING CONTRACTS

Internal Procedures > € 60,000

Procedure Type Count Amount (€)

Internal Procedures > € 60,000

Exceptional Negotiated Procedure without publication of a contract notice (Art. 134 RAP)

2 832.339,89

Open Procedure (Art. 122.2 IR) 3 19.089.900,00

Open Procedure (Art. 127.2 RAP) 5 1.219.494,00

TOTAL 10 21.141.733,89


TABLE 13 : BUILDING CONTRACTS

Total number of contracts :

Total amount :

Legal base Contract Number Contractor Name Description Amount (€)

No data to be reported


TABLE 14 : CONTRACTS DECLARED SECRET

Total Number of Contracts :

Total amount :

Legal base

Contract Number Contractor Name Type of

contract Description Amount (€)

No data to be reported


ANNEX 4: Materiality criteria

In line with Commission guidelines, the deficiencies leading to reservations should fall within the scope of the declaration of assurance. The following types of possible deficiencies can be relevant in DG EAC’s context:

• Significant repetitive errors detected during ex post controls or supervision exercises. The frequency and duration of the errors will be the determining factor to judge their significance.

• Significant weakness in one of the control systems identified by auditors, in supervision exercises, or in the assessment of the implementation of the internal control standards. A particularly relevant case for DG EAC would be identified weaknesses in the control chain of National Agencies.

• Situation where a major critical issue that is of relevance to the declaration has been identified by the European Court of Auditors, the Internal Audit Service or the Internal Audit Capability.

• Situation where the DG knows that it does not have sufficient evidence from internal control systems or audit coverage.

• Situation where the DG has evidence that a significant risk remain unmitigated. • A significant risk for the reputation of the Commission.

When significant weaknesses are identified, a quantification of the amount at risk should be carried out when possible.

As the DG implements its budget through three very different implementation modes (see section 2.1-2.2), which have different risk profiles and which each have their own control and supervision arrangements, it is considered that observed quantified weaknesses should be assessed against the part of the budget spent in each specific implementation mode. As the Directors of the executive agencies are Authorising Officers by Delegation for the parts of the programmes delegated to them, they take responsibility for this spending in their declaration of assurance in the AARs of the executive agencies. Consequently, the operational spending through executive agencies is not part of the declaration of assurance of the Director General of the parent DGs, only the subsidy for the EA's operating budget. The following diagram gives an overview of the discussed basis for assessing materiality for the two other implementation modes.

Overview of basis for assessing materiality

LLP 15.02

Culture 15.04

MEDIA 15.04

Youth 15.05

People 15.07

Indirect management through NAs

Direct management by


DG EAC

In order to better capture the multi-annual nature of programmes and control activities, a multi-annual approach to the calculation of error rates has been introduced as from AAR 2012. The detected error rate calculated in the 2013 AAR is based on audits carried out over the prior 3 years. Cashed recoveries related to the audit findings (no extension of audit findings is possible in the context of DG EAC’s beneficiary population) are deducted to arrive at the residual error rate. If the amount at risk surpasses 2% of the budget for a specific implementation mode during the multiannual reporting period, a reservation should be considered.

In order to quantify the weaknesses, a detected error rate is obtained through random ex post audits for each implementation mode.

Where the deficiency consists of an observed serious weakness in the control system, it will not always be possible to quantify the amount at risk based on observed errors. This can be in particular relevant for the indirect management through National Agencies, where, in the framework of the single audit model, the DG's assurance is mainly based on a verification of the functioning of the control system. In such a case, the following steps are taken to calculate the percentage of the budget at risk:

• Analyse the effectiveness of the control system of each NA/NAU combination and conclude, based on all available information (audits, visits, Declarations of Assurance, reporting…), if the system gives acceptable, partial or no assurance.

• Identify the part of the budget that has been executed through systems with acceptable, partial and no assurance.

• Deduct any relevant suspensions of payments and any financial corrections to obtain the maximum open exposure.

• As an approximation for the potential exposure, consider that 20% of the calculated maximum open exposure is at risk for NA/NAUs with no assurance; 5% of the exposure for NA/NAUs with partial assurance; the observed error rate in random testing for NA/NAUs with acceptable assurance.


ANNEX 5: Internal Control Template(s) for budget implementation (ICTs)

Procurement management

Stage 1: Programming, evaluation and selection of tenders

A - Preparation, adoption and publication of the Annual Work Programme and Calls for tender and other procurement procedures

Main control objectives: Ensuring that the tenders submitted meet the objectives, priorities and needs set by DG EAC; Compliance. Main risks It may happen (again) that… Mitigating controls How to determine coverage,

frequency and depth How to estimate the costs and benefits of controls Control indicators

Risk that the annual work programme and the subsequent calls for tender or other procurement procedures do not adequately reflect the objectives and priorities or are not consistent with DG EAC legal bases. Risk that the needs are not well defined (operationally and economically) and that the procedure to procure was inappropriate.

Definition of needs and procedures to be launched in the Work program or other planning tools. Tenders discussed during directorial management meetings and sometimes at the level of the Directors Board. Planning of procedures and monitoring at the level of central financial unit, collaboration with Service Support to Directors (SuDs) and Operating Units (OU).

Coverage / Frequency: 100% of procurement procedures (nature of checks may differ for specific contracts under non-competitive Framework Contracts) Depth: In-depth analysis of procedures maybe differentiated depending on the type of procedure

Risk of high quality offers not being submitted due to the choice of the procedure and/or the specifications

Regular preparatory meetings between central financial unit and OU before launching the procedures

Coverage / Frequency: 100% of procurement procedures (nature of checks may differ for specific contracts under non-

Costs: Cost of staff involved in procurement planning Benefits: Qualitative benefits: Comprehensive Work Programme and planning give assurance that procedures meet DG EAC objectives, priorities and needs and that the procedures are appropriate to the needs.

Effectiveness: % of number of calls and other procurement procedures launched / number of calls and other procurement procedures planned in WP or other planning tools Efficiency: Cost-Effectiveness:


Main risks It may happen (again) that… Mitigating controls How to determine coverage,


Financial circuit : Central Financial Unit validation and AOS approval on specifications

competitive Framework Contracts) Depth: In-depth analysis of the procedures maybe differentiated depending on the type of procedure

B - Selecting and awarding: Evaluation, ranking and selection of tenders Main control objectives: Ensuring that the tenders submitted meet the objectives, priorities and needs set by DG EAC; Compliance; Prevention of fraud.



Risk of most advantageous offer not being selected, due to a deficiencies in the selection/evaluation process

Documented procedures available for OU together with templates on intranet (Based on Business Process Management analysis) Appointment of the evaluation committee members who might be assisted for important calls by independent experts Validation by the AO of the evaluation stage In addition, if applicable: opinion of advisory bodies

Coverage / Frequency: 100% of procurement procedures (nature of checks may differ for specific contracts under non-competitive Framework Contracts) Depth: In-depth analysis of procedures maybe differentiated depending on the type of procedure

Costs: Cost of staff involved in the selection/evaluation of calls. If applicable, cost of the external experts Benefits: A robustselection/evaluation process allows the DG to cover its needs in an efficient way and limits the possible litigations High quality services provided

Effectiveness: % of number of evaluated tenders / total number of tenders received Efficiency: Cost-Effectiveness:




Training sessions on new regulatory provisions for procurement given to staff involved in contracts management. Systematic checks on operational and legal aspects performed before signature of the contract.


Stage 2: Contracting and monitoring the execution

Main control objectives: : Ensuring that the tenders submitted meet the objectives, priorities and needs set by DG EAC. Ensuring that the operational results (deliverables) are of good value and meet the objectives and conditions; ensuring that the related financial operations comply with regulatory and contractual provisions; prevention of fraud; ensuring appropriate accounting of the operations; compliance.



Risk of payments which are not in accordance with the applicable contractual and regulatory provisions. Risk of non-implementation of the contract (the tasks foreseen are not, totally or partially carried out in accordance with the technical description and requirements foreseen in the contract) Risk of inadequate quality or not reaching policy objectives

Effective external communication and guidance to the tenderers. Preparatory meetings for sensitive and high risk contracts. Operational and financial checks in accordance with the financial circuits and the provisions of the contractual documents in order to ensure that each stage (including amendment) of the procedure/work programme is correctly and fully implemented. Monitoring measures are put in place also at the moment of interim reports. Supervision and authorisation by the AO during the procedure/work programme implementation

Coverage / Frequency: 100% of procurement procedures Depth: In case of doubts at the time of payment, more in-depth controls are put in place

Costs: Cost of staff involved in this phase Benefits: Qualitative and quantitative benefits : Detection of errors before payment, sound financial management and respect of contractual provisions Reduction of the committed amounts for contracts and more efficient use of funds due to risk detection.. Reduction of the error rate Operational results are of good value and meet the objectives and conditions

Effectiveness: Number of suspended or refused reports/total number of reports Efficiency: Cost-Effectiveness:


Stage 3: Ex-post controls

A - Reviews, audits and monitoring Main control objectives: Measuring the level of error in the population after ex-ante controls have been undertaken; detect and correct any error or fraud remaining undetected after the implementation ex-ante controls; identifying possible systemic weaknesses in the ex-ante controls, or weaknesses in the rules



Risk of not detecting by ex-ante controls errors, non-compliance with specifications or potential frauds The ex-post controls focus on the detection of external errors (e.g. made by beneficiaries and contractors), which is the main driver of the error rate.

At intervals carry out ex-post controls and external audits of a representative sample of operations to measure the level of error in the population after ex-ante controls have been performed Additional sample if necessary to address specific risks At intervals IAC audits carried out on DG EAC procedures Ex-post controls and audit planning and monitoring of the action plan implementation. In case of systemic error being detected, extrapolation to all similar non-audited procedures.

MUS and random Selection

Audit programme of approximately 10 MUS and 10 randomly selected items per year on procurement, with a coverage of 3%, and a 100% depth of testing.

Period checks of procurement transactions in the context of the regular accounting control process

Costs: Cost of staff involved in the coordination and execution of the ex-post controls and audits. Benefits: Reduction in the number and budget value of the errors detected by the auditors. Improvement of ex-ante controls or risk approach in ex-ante controls by feeding back findings from audits. Improvement in rules and guidance from feedback from audits.

Effectiveness:

Consistent error rate data for the AAR.

Efficiency:


Grants direct management

Stage 1: Programming, evaluation and selection of proposals

A - Preparation, adoption and publication of the Annual Work Programme and Calls for proposals and other grants procedures

Main control objectives: Ensuring that the proposals submitted meet the objectives and priorities set by DG EAC; Compliance; Prevention of fraud



The annual work programme and the subsequent calls for proposals or other grant procedures do not adequately reflect the objectives and priorities, are incoherent and/or the essential eligibility, selection and award criteria are not adequate to ensure the evaluation of the proposals. The annual work programmes are not consistent with DG EAC legal bases.

Hierarchical validation within the authorising department Inter-service consultation, including all relevant DGs; adoption by the Commission Explicit allocation of responsibility. Validation of calls or other grant procedures by the central financial unit before launching the procedure

Coverage / Frequency: 100% Depth: All work programmes and calls or other grant procedures are thoroughly reviewed at all levels, including for operational and legal aspects. In-depth controls are put in place for riskier grant procedures.

Costs: estimation of cost of staff involved in the preparation and validation of the annual work programmes and calls or other grant procedures. Benefits: Qualitative benefits. A good Work Programme and well prepared calls or other grant procedures give assurance that applications meet DG EAC objectives and priorities. Clear planning/programming ensure simplification of procedures Quantitative benefits: number of applications or number of work program objectives met.

Effectiveness: % of number of calls or other grant procedures launched / number of calls or other grant procedures planned in WP Efficiency: Cost-Effectiveness:




In the preparation of simplified form of grants, the categories of eligible costs are not clearly identified in advance

Methodologies on calculating simplified form of grants have been prepared. Commission decisions were taken on their use. The method for determining lump sums, unit costs or flat-rate financing is based on either adequate statistical data, certified historical data or usual accounting practices of the beneficiary. Regular meetings between central financial unit and Operational Unit (OU) were organised to discuss the methodologies and to harmonize the use of this form of grants at the lever of the DG Regular monitoring/follow-up of their implementation in DG EAC.

Coverage / Frequency: 100% of procedures concerned by simplified forms of grants Depth: in-depth analysis for categories of costs with high degree of complexity and risk

Costs: estimation of cost of staff involved in the preparation and implementation of simplified form of grant Benefits: Qualitative benefits refer to simplification of grants procedures and consequent ease of application.

Effectiveness: % number of grant procedures concerned by simplified forms of grants/total number of grant procedures Efficiency: Cost-Effectiveness:


B - Selecting and awarding: Evaluation, ranking and selection of proposals Main control objectives: Ensuring that the proposals submitted meet the objectives and priorities set by DG EAC; Compliance; Prevention of fraud



The evaluation, ranking and selection of proposals is not carried out in accordance with the established procedures, the objectives and priorities and the eligibility, selection and award criteria defined in the annual work programme and subsequent calls for proposals and/or invitations to submit proposals.

Documented procedures available to operational staff with relevant guidance documents such as templates e.g., via the intranet (based on Business Process Management analysis) Appointment of the evaluation committee members who might be assisted for important calls by independent experts Evaluation committee members are selected for their technical expertise and independence (e.g. sworn declaration for conflicts of interests) Validation by the AO of the evaluation stage In addition, if applicable: opinion of advisory bodies; comitology where appropriate Training sessions on new

100% of individual or spontaneous applications are evaluated. More in depth analysis can be carried out depending on the type of grants

Costs: estimation of cost of staff involved in the evaluation process. If applicable, cost of the external experts Benefits: a robust evaluation process contributes to assurance on the good implementation of the WP

Effectiveness: % of number of evaluated applications / total number of applications received Efficiency: Cost-Effectiveness:




regulatory provisions for grants given to staff involved in grant management to improve performance Systematic checks on operational and legal aspects performed before signature of the grant agreement or decision.

Stage 2: Contracting

Main control objectives: : Ensuring that the proposals accepted meet the objectives and priorities set by DG EAC; Compliance Main risks It may happen (again) that… Mitigating controls How to determine coverage,


Procedures do not comply with the regulatory framework.

Documented procedures available to operational staff with relevant guidance documents such as templates e.g., via the intranet(based on Business Process Management process analysis) Project Officers implement evaluators' recommendations in contracting with applicants

100 % of proposals are checked by financial initiators and verifiers before commitment Depth may be differentiated depending on the nature of the grant or beneficiary

Costs: estimation of cost of staff involved in the contracting process. Benefits: Qualitative benefits: The mitigating controls put in place to ensure the respect of rules and procedures contribute to avoiingd possible errors at the stage of contracting.

Effectiveness: % of errors or challenges to contractual procedures Efficiency: Cost-Effectiveness:




Hierarchical validation of proposed adjustments. Ex-ante verification of the proposed budget Signature of the grant agreement or decision by the AO.


Stage 3: Monitoring the execution

Main control objectives: ensuring that the operational results (deliverables) from the projects are of good value and meet the objectives and conditions; ensuring that the related financial operations comply with regulatory and contractual provisions; prevention of fraud; ensuring appropriate accounting of the operations



The actions foreseen are not, totally or partially carried out in accordance with the technical description and requirements foreseen in the grant agreements or decisions and/or the amounts paid exceed what is due in accordance with the applicable contractual and regulatory provisions.

Kick-off meetings (if applicable) and "launch events" involving the beneficiaries in order to avoid beneficiary reporting errors. Effective external communication about guidance to the beneficiaries. Operational and financial checks in accordance with the financial circuits and the provisions of the contractual documents in order to ensure that each stage (including amendment) of the action/work programme is correctly and fully implemented. Supervision and authorisation by the AO during the action/work programme implementation.

100% of the projects are controlled each time a payment is made or an amendment is issued. In case of doubts at the moment of the payment, more in-depth controls are put in place.

Costs: estimation of cost of staff involved in this phase. Benefits: any reduction of the committed amounts for grants due to risk materialization. (data not available so far) Reduction of the error rate. Operational results from the projects are of good value and meet the objectives and conditions

Effectiveness: % of errors detected at an operational or financial level Efficiency: Cost-Effectiveness:




Audit certificates for final payments required in compliance with FR.

Non-detection of grants with high risk of non-implementation when no bank guarantee is requested.

Methodology on evaluation of pre-financing risks and guidelines on pre-financing payments are in place in conformity with FR Training sessions on new regulatory provisions for grants given to staff involved in grants management Workshops on risk assessment for staff concerned Verification at the stage of payments

100% of grants for which a pre-financing is foreseen are subject to a risk analysis An in-depth evaluation of risks is carried out for each grant concerned by a pre-financing, based on a set of criteria (value, duration, type of grant, subject, financial capacity, type of payments).

Cost: estimation of cost of staff involved in risk analysis Benefits: assurance that riskier actions are identified and mitigating measures (e.g. guarantees) are taken in order to preserve EU financial interests.

Effectiveness: % number risk analysis/number of grants concerned by pre-financing payment Efficiency: Cost-Effectiveness:


Stage 4: Ex-post controls

A - Reviews, audits and monitoring Main control objectives: Measuring the level of error in the population after ex-ante controls have been undertaken; detect and correct any error or fraud remaining undetected after the implementation ex-ante controls; identifying possible systemic weaknesses in the ex-ante controls, or weaknesses in the rules



Risk of not detecting by ex-ante controls errors or non-compliance with specifications or frauds The ex-post controls focus on the detection of external errors (e.g. made by beneficiaries , which is the main driver of the error rate.

At intervals carry out ex-post controls and external audits of a representative sample of operations to measure the level of error in the population after ex-ante controls have been performed. Additional sample to address specific risks, where appropriate. At intervals IAC audits carried out on DG EAC procedures Ex-post controls and audit planning and monitoring of the action plan implementation. In case of systemic error detected extrapolation to all the non-audited similar procedures.

MUS Selection

Audit programme of approximately 10 externally contracted audits per year on projects, with a coverage of 10%, and a 100% depth of testing.

Period checks of grant transactions in the context of the AAR and the regular accounting control process

Costs: Cost of staff involved in the coordination and execution of the ex-post controls and audits. Benefits: Reduction in the number and budget value of the errors detected by the auditors. Improvement of ex-ante controls or risk approach in ex-ante controls by feeding back findings from audits. Improvement in rules and guidance from feedback from audits.

Effectiveness:

Consistent error rate data for the AAR.

Efficiency:

Cost-Effectiveness:


B - Implementing results from ex-post audits/controls Main control objectives: Ensuring that the (audit) results from the ex-post controls are implemented.



Risk of non-implementing of action plans for correcting errors, irregularities and cases of fraud detected

Ex-post supervision strategy, performed by external auditors, AOSD decision based on the recommendation of the auditors and proper follow up of recoveries.

100% of results are implemented

Costs: Cost of staff involved in the implementation of the audit results. Benefits: Budget value of the errors, detected by ex-post controls/audits, which have actually been corrected

Effectiveness:

Recovery of ineligible amounts

Efficiency Indicators:


Indirect management – National Agencies

Stage 1 – Establishment (or prolongation) of the mandate to the entrusted entity.

Main control objectives: Ensuring that the legal framework for the management of the relevant funds is fully compliant and regular (legality & regularity), delegated to an appropriate entity (best value for public money, economy, efficiency), without any conflicts of interests (anti-fraud strategy).

Main risks It may happen (again) that… Mitigating controls How to determine coverage

frequency and depth How to estimate the costs

and benefits of controls Possible control indicators

The establishment (or prolongation) of the mandate of the National Agency is affected by legal issues, which would undermine the legal basis for the management of the related decentralised EU funds (via that particular National Agency).

For the period 2007-2013, the Member States/participating countries provide the Commission with the preliminary assurances (ex-ante declaration of assurance) that the minimum conditions established by the legal bases are fulfilled by the National Agencies

Before the Commission signs a contract with the National Agency, DG EAC a) reviews the ex-ante declaration/ex-ante compliance assessment b) approves the work programme of the National Agency.

After each year of programme implementation, the National Authorities provide an ex-post

Coverage/Frequency: 100%

Depth: Checklist includes a list of the requirements of the regulatory provisions to be complied with.

If risk materialises, funds delegated during the year(s) to the entrusted entity may be subject to error and irregularity

Costs: estimation of cost of staff involved in the preparation, adoption and selection work.

Benefits: The potential error which could affect the budget amount entrusted to the National Agency, if significant (legal) errors would otherwise be detected.

Effectiveness: Smooth transition to next programme;

Efficiency: Timely conclusion of process.


Main risks It may happen (again) that… Mitigating controls How to determine coverage



declaration of assurance on the proper management of the funds by the NAs, on the functioning of their control system and on the probity of the accounts presented in the NAs yearly reports.

DG EAC has set minimum standards of control and procedures for the NAs (Guide for National Agencies, updated annually) and issued Guidelines for National Authorities.

For the period 2014-2020 the same preliminary assurance is provided through the handover and ex-ante Compliance Process.

Negotiation of legal base with the Legislative Authority, following which the designation of the National Authority, National Agency and Independent Audit Body.


Stage 2 – Ex-ante (re)assessment of the entrusted entity’s financial and control framework (towards “budget autonomy”; “financial rules”).

Main control objectives: Ensuring that the entrusted entity is fully prepared to start/continue implementing the delegated funds autonomously while respecting all 5 ICOs.

Main risks It may happen (again) that… Mitigating controls

How to determine coverage, frequency

and depth

How to estimate the costs and benefits of controls Possible control indicators

The financial and control framework deployed by the National Agency is not fully mature to guarantee achieving all 5 ICOs (legality and regularity, sound financial management, true and fair view reporting, safeguarding assets and information, anti-fraud strategy).

For the Erasmus+ programme (2014-2020):

- Ex-ante Compliance Assessments for the new National Agencies are of poor quality

LLP/YiA (programmes 2007-2013):

- DG EAC has set minimum standards of control and procedures for the NAs (Guide for National Agencies, updated annually) and issued Guidelines for National Authorities.

Erasmus+ Programme (2014-2020):

Guidance documents on the designation of National Agencies under E+, on the ex-ante Compliance Assessments and on the transition/handover between National Agencies

Ex-ante assessment, conditional to sign a Delegation Agreement with the new National Agency

Guidance to Independent Audit Bodies

Hierarchical validation by the AOSD of Directorate B

Coverage/frequency: 100% of National Agencies

Depth : all ex-ante assessment are analysed with the same depth, independently of the level of decentralised funds that will be entrusted to the National Agency

Costs: estimation of cost of staff involved in the ex-ante assessment process (which includes missions).

Benefits: The potential error in the (average annual) total budget avoided.

Effectiveness: Positive performance of stakeholders achieved

Efficiency Indicators: Timeliness of process (guidance is clear and properly applied by stakeholders)


Stage 3 – Operations: monitoring, supervision, reporting (“representation” / “control around the entity”).

Main control objectives: Ensuring that the Commission is fully and timely informed of any relevant management issues encountered by the entrusted entity, in order to possibly mitigate any potential financial and/or reputational impacts (legality & regularity, sound financial management, true and fair view reporting, anti-fraud strategy).




Due to weak "modalities of cooperation, supervision & reporting", the Commission is not (promptly) informed of relevant management issues encountered by the National Agency, and/or does not (promptly) react upon notified issues by mitigating them or by making a critical observation and recommendation for them – which may reflect negatively on the Commission’s governance reputation and quality of accountability reporting.

The Commission – National Agency Agreement (for LLP/YiA) or Delegation Agreement (Erasmus+) specifying the control, accounting, audit, publication, etc related requirements – incl. the modalities on reporting back relevant and reliable control results

Analysis of yearly Declarations of Assurance prepared by National Authorities on its secondary controls (controls on systems of NAs, in particular on the operation of primary controls)

Analysis of audit reports commissioned by National Authorities under its secondary controls

Analysis of yearly NA reports which are certified by the National Authorities (yearly

Coverage: 100% of the National Agencies and Authorities are monitored/supervised. Frequency: annually

In case of operational and/or financial issues, measures are being reinforced.

The depth is the same for all National Agencies.

Costs:

- estimation of cost of staff involved in the actual (regular or reinforced) monitoring and supervising the National Agencies (which includes missions)

- cost of externalised financial audits of COM-NA Agreements of National Agencies

Benefits: Assurance on the budget amount entrusted to the National Agency.

Effectiveness: Conclusions reached on the basis of annual reports are confirmed by monitoring visits, audits and other supervisory activities.

Efficiency Indicators: Timeliness of delivery of annual reports

Cost effectiveness : Long term resource monitoring





Declaration of Assurance)

Risk-based supervisory visits to National Agencies and National Authorities

Financial audits on yearly sampled National Agencies

Regular reporting to Senior Management and in AAR of the supervision results.

Regular NA meetings, training of NA staff

Information seminars for National Authorities and regular updates and improvements of guidelines for them

If appropriate/needed:

- monitoring visits to National Agencies to support and provide advice

- individual targeted follow-up of critical recommendations

- precautionary measures

- intervention, e.g. via own audits on-the-spot,

- referral to OLAF


Stage 4 – Commission contribution: payment or suspension/interruption.

Main control objectives: Ensuring that the Commission fully assesses the management situation at the entrusted entity, before either paying out the (next) contribution for the operational and/or operating budget of the entity, or deciding to suspend/interrupt the (next) contribution (legality & regularity, sound financial management, anti-fraud strategy).




The Commission pays out the (next) contribution to the National Agency, while not being aware of the management issues that may lead to financial and/or reputational damage.

The Commission – National Agency Agreement (for LLP/YiA) or Delegation Agreement (Erasmus+) specifying the control, accounting, audit, publication, etc related requirements

Management review of the supervision results.

Ex-ante Operational Verification and Financial Verification, ‘in-depth’ as required

Hierarchical validation of contribution payment of operating grant and funds for decentralised actions and recovery of non-used funds for decentralised actions

If appropriate/needed: suspension or interruption of payments

Coverage: 100% of the contribution payments. Frequency: usually annually

The depth is the same for all National Agencies.

Costs: estimation of cost of staff involved in the (in-depth) OV and FV of the contribution payments/recoveries to/from the National Agencies.


Benefits in case of recovery or suspension/interruption: the amount and % value of budget recovered or not paid out

Effectiveness: Appropriate information available to support payment or suspension decision.

Efficiency Indicators: Timeliness of information and action when issues are noted


Stage 5 – Audit and evaluation, Discharge for decentralised agencies

Main control objectives: Ensuring that assurance building information on the entrusted entity’s activities is being provided through independent sources as well, which may confirm or contradict the management reporting received from the entrusted entity itself (on the 5 ICOs).




The Commission has not sufficient information from independent sources on the National Agencies' management achievements, which prevents conclusions being correctly drawn on the assurance for the budget entrusted to the entity – which may reflect negatively on the Commission’s governance reputation and quality of accountability reporting.

The Commission – National Agency Agreement (for LLP/YiA) or Delegation Agreement (Erasmus+) specifying the control, accounting, audit, publication, etc related requirements – incl. independent audit function and cooperation with IAS and ECA

Assessment of ex-post yearly Declarations of Assurance provided by National Authorities on its secondary controls.

NAs include in their yearly reports the results of primary controls (controls carried out by or under the responsibility of the National Agency on the actions that it manages). Types and minimum requirements for controls are set out by DG EAC and are legally binding for the National Agencies.

Coverage: sample as needed ( random for financial audits, value-targeted, risk-based for supervisory visits). Frequency: multiannual planning for financial audits; no pre-determined frequency for supervisory visits

The depth for financial audits is the same for all National Agencies. For supervisory visits, the depth is adapted to identified risks..

Costs: estimation of cost of staff involved in the coordination and execution of supervisory visits. Cost of the appointment of audit firms for the outsourced financial audits.

Benefits: The potential error in the (average annual) total budget avoided

Benefits: budget value of the errors with the beneficiaries detected and subsequently corrected.

Effectiveness: Assurance being provided via yearly Declarations of Assurance and corroborated by other supervision data.

Efficiency: Timeliness and administrative cost of process.





Own supervisory visits of National Agencies and/or National Authorities

Yearly sample of selected National Agencies for financial audit

- referral to OLAF


Indirect management – EACEA, REA, Cedefop, ETF and EIT

Stage 1 – Establishment (or prolongation) of the mandate to the entrusted entity (“establishment act” and “delegation act”).

Main control objectives: Ensuring that the legal framework for the management of the relevant funds is fully compliant and regular (legality & regularity), delegated to an appropriate entity (best value for public money, economy, efficiency), without any conflicts of interests (anti-fraud strategy).

Main risks Mitigating controls How to determine coverage



The establishment (or prolongation) of the mandate of the entrusted entity is affected by legal issues, which would undermine the legal basis for the management of the related EU funds (via that particular entity).

The mandate is not finalised in time to allow programme implementation to proceed by the Agency/ entity.

Existing and long established general legal bases (since 2003 for EAs), multiple mandate extensions in the past

Cost-benefit analysis carried out by external consultant and adjustments thereafter ensure value for money

Standard template provided by central services and standard wording

Hierarchical validation within the authorising department

Inter-service consultation, including all relevant DGs

Opinion of Council (Committee on Executive Agencies) and Parliament

Adoption by the Commission

Possibility to make necessary

Coverage/Frequency: 100%/once

Depth: Standard templates include all requirements of the regulatory provisions to be complied with.

If extreme challenge to the legality of the establishment and delegation materialises, regularity of funds delegated during the year(s) to the entrusted entity could be questioned.

Impact of late adoption would affect effectiveness and efficiency.

Costs: cost of staff involved in the preparation, adoption and selection work: this is a largely unavoidable cost and arises infrequently.

Benefits: Reputational risk of parent DGs intact.

Avoidance of possible additional costs which might result from delays.

Effectiveness: Quality of the legal work (basic act, LFS and delegation act, etc): number of control failures; number of initially negative CIS opinions and timeliness.

No litigation cases. No OLAF inquiries. No ECA criticism.

Efficiency: Ease of application – timeliness.


Main risks Mitigating controls How to determine coverage



clarifications to the text in the context of revision of the Delegation Act

Specific Financial Regulation

Common approach endorsed by EP, Council and Commission (July 2012) and Road map on decentralised Agencies


Stage 2 – Ex-ante (re)assessment of the entrusted entity’s financial and control framework (towards “budget autonomy”; “financial rules”).

Main control objectives: Ensuring that the entrusted entity is fully prepared to start/continue implementing the delegated funds autonomously with respect of all 5 ICOs.

Main risks Mitigating controls How to determine coverage,



The financial and control framework deployed by the entrusted entity is not fully mature to guarantee achieving all 5 ICOs (legality and regularity, sound financial management, true and fair view reporting, safeguarding assets and information, anti-fraud strategy).

Verification of the equivalence and guaranty of the system of internal control

As EACEA and REA are well established, the mitigating controls regarding a serious breakdown in the control framework can be regarded as at “cruising speed”.

The supervision arrangements constitute the principle control mechanism to ensure the high quality of EACEA’s and REA’s control structures are maintained.

The above situation applies by analogy to the Decentralised Agencies, which are even longer established and have a greater financial autonomy (including separate budgetary and discharge

Coverage/frequency: DG EAC does not carry out direct controls on the underlying transactions (except for EIT’s reinforced supervision by DG EAC), but uses the supervision mechanisms in place to ensure ICOs are being achieved.

Depth : if there was a (hypothetical) particularly serious issue, the parent DGs of the Agency can perform in-depth checks under Article 22 of the Delegation Act of EAs, or action via the steering committees for EAS or governing boards for Traditional Agencies.

Costs: not applicable, no ex-ante assessment process required.

Benefits: Reputational risk of parent DGs intact.

Effectiveness: Potential requirement for parent DGs to increase supervision due to deterioration in “Stage 3” indicators

Efficiency Indicators: Deterioration in “Stage 3” indicators of time-to-pay, etc.





process)

For EIT, full financial autonomy will be granted only after a positive ex-ante assessment of the management environment

Requiring prior approval for the riskier transactions (contracts above EUR 60 000 and grants)


Stage 3 – Operations: monitoring, supervision, reporting (“representation” / “control with the entity”).

Main control objectives: Ensuring that the Commission is fully and timely informed of any relevant management issues encountered by the entrusted entity, in order to possibly mitigate any potential financial and/or reputational impacts (legality & regularity, sound financial management, true and fair view reporting, anti-fraud strategy).




Due to weak "modalities of cooperation, supervision & reporting", the Commission is not aware of relevant management issues encountered by the entrusted entity, and/or does not react on a timely basis to issues notified by mitigating them or by making a reservation for them – which may reflect negatively on the Commission’s governance effectiveness and quality of accountability reporting.

For EAs, Delegation Act specifying the control, accounting, audit, publication, etc. related requirements including reporting of error rates and the reservations in the context of the Annual Activity Report.

For Traditional Agencies, monitoring via the governing board and the discharge process.

Monitoring or supervision of the entrusted entity (e.g. ‘regular’ monitoring meetings at operational level; review of reported control results and any underlying mngt/audit reports; representation and intervention in the Steering Committee, scrutiny of the Annual Activity Report, etc).

Management review of the supervision results.

Coverage: 100% of the entities are monitored/supervised. Frequency: monthly budget reports, quarterly coordination and Steering Committee meetings, etc.

For EAs, in case of operational and/or financial issues, measures can be reinforced under Article 22 of the Delegation Act to any depth that is deemed appropriate.

Costs: estimation of cost of staff involved in the actual (regular or reinforced) monitoring of the entrusted entities.


Effectiveness: Relevance and reliability of the overall performance of the supervision arrangements, and their coherence with the conclusions provided by external sources (evaiations, IAS, ECA, OLAF, complaints, etc.)

Efficiency Indicators: Timeliness of provision of information resource monitoring of administrative cost.






- reinforced monitoring of operational and/or financial aspects of the entity

- intervention, e.g. via own audits on-the-spot, by IAC or IAS

- potential escalation of any major governance-related issues with entrusted entities

- referral to OLAF


Stage 4 – Commission contribution: payment or suspension/interruption.

Main control objectives: Ensuring that the Commission fully assesses the management situation at the entrusted entity, before either paying out the (next) contribution for the operational and/or operating budget of the entity, or deciding to suspend/interrupt the (next) contribution (legality & regularity, sound financial management, anti-fraud strategy).




The Commission pays out the (next) contribution to the entrusted entity, while not being aware of the management issues that may lead to financial and/or reputational damage.

Reliance on “Stage 3” controls above, plus compliance with budget adopted by the Steering Committee/ governing board.

If appropriate/needed: suspension or interruption of payments

Coverage: 100% of the contribution payments. Frequency: twice yearly as specified in the Memorandum of Understanding

Depth covers the compliance and verification stages of the financial circuit.

Costs: estimation of cost of staff involved in the financial circuit of the contribution payments/recoveries to/from the entrusted entities.


Benefits in case of recovery or suspension/interruption: the amount and % value of budget recovered or not paid out due to identified irregularities.

Effectiveness: Number of queries regarding budgets or payments.

Efficiency Indicators: Time-to-pay/recover.


Stage 5 – Audit and evaluation, Discharge for decentralised agencies

Main control objectives: Ensuring that assurance building information on the entrusted entity’s activities is being provided through independent sources as well, which may confirm or contradict the management reporting received from the entrusted entity itself (on the 5 ICOs).




The Commission does not have sufficient information from independent sources on the entrusted entity’s management performance, which prevents conclusions being drawn on the assurance for the budget entrusted to the entity – which may reflect negatively on the Commission’s governance effectiveness and quality of accountability reporting.

Close review within the discharge and internal audit process of IAC, IAS and ECA findings and the entity’s response.


- own ex-post audit(s) on-the-spot, by the Parent DG, of the entity and/or its beneficiaries, normally only in the context of a serious breakdown in the control framework.

- potential escalation of any major governance-related issues with entrusted entities

- referral to OLAF

Coverage: Entity’s yearly audit sample of beneficiaries and multi-annual strategy. Sample as per the audit strategies of the IAC, IAS and ECA

Frequency: Entity’s IAC will perform multiple audits per year, ECA once a year, IAS less frequently.

Normally, the sample will be statistically representative to enable drawing valid management conclusions about the entire population during the programme’s lifecycle.

For EAs, the depth can be reinforced under Article 22 of the Delegation Act to any depth that is deemed appropriate.

Costs: estimation of cost of staff involved in the coordination and execution of the own audits and outsourced audits (if any).


Benefits: budget value of the errors with the entity’s beneficiaries detected by the own auditors, and subsequently corrected. Reputational benefits of clean audit opinion.

Effectiveness: Assurance being provided); consistent and credible error rate, residual error rate below tolerable threshold.

ECA opinion on the account and legality and regularity of the underlying transactions

Discharge granted

Efficiency: Resource monitoring, estimation of benefits.


ANNEX 6: Implementation through national or international public-sector bodies and bodies governed by private law with a public sector mission

1. Programmes concerned: Lifelong Learning and Youth in action (2007-2013); Socrates, Leonardo da Vinci and Youth (2000-2006)

2. Annual budgetary amount entrusted to these bodies in 2013: € 1,401.8 million (see table for breakdown by programme/agency)

3. Duration of the delegation : 2007-2013

4. Justification of recourse to indirect centralised management and justification of the selection of the bodies: the recourse to national agencies and their appointment by the Member States/participant countries are set in the legal bases of the programmes at

i. Article 8.(6)(b) of the Decision No 1719/2006/EC of the European Parliament and of the Council of 15 November 2006 establishing the Youth in Action programme

ii. Article 6(2)(b) of the Decision No 1720/2006/EC of the European Parliament and of the Council of 15 November 2006 establishing an action programme in the field of lifelong learning

5. Synthetic description of the implementing tasks entrusted to these bodies

i. Management of the project life cycle of the decentralised actions of the Programme: Information and publicity at national level on the Programme and, where applicable, the calls for proposals; Information and counselling of potential applicants; Receipt and evaluation of grant applications; Establishment and supervision of evaluation committees; Decision on the award of grants; Publication of information on supported projects; Commitment of funds for projects and signing of contracts with project beneficiaries; Pre-financing payments to beneficiaries; Monitoring projects, including visits to projects and thematic monitoring meetings; Analysis and control of final activity and final financial reports; Balance payments and recovery of funds; On-the-spot checks on projects; Reporting to the Commission and to the National authority.

ii. Monitoring and evaluating the decentralised actions of the Programme: Organisation of national thematic monitoring meetings between projects and of national participation in European thematic monitoring meetings; Organisation of national valorisation meetings bringing together project coordinators and potential users and of national participation in European valorisation events; Reporting on the impact of the Programme actions at national level; Studies, analyses and surveys on the Programme actions at national level; Contribution to national reports on implementation of the Programme and to evaluation of the Programme; Contribution to achieving synergies at national level with other Community programmes.


iii. Continuation and winding up of the actions from the Socrates (2000-06), Leonardo da Vinci (2000-06) and Youth (2000-06) programmes

LIST OF NATIONAL AGENCIES AND 2013 BUDGET BREAKDOWN

Country National Agency LLP 2013 Annual budgetary amount2 (€)

National Agency Youth in action

2013 Annual budgetary amount3 (€)

België EPOS vzw - Europese Programma's voor Onderwijs, Opleiding en Samenwerking - Agentschap

15.275.010 JINT v.z.w 2.634.836

Belgien Agentur für Europäische Bildungsprogramme VoG 744.190

Jugendbüro der Deutschsprachigen Gemeinschaft

282.180

Belgique Agence francophone pour l’éducation et la formation tout au long de la vie - AEF-Europe

10.835.925 Bureau international Jeunesse 1.793.460

Balgaria Human Resource Development Centre - HRDC 18.698.205

National Center "European youth programmmes and initiatives"

4.146.877

Ceska Republika DŮM ZAHRANIČNI SPOLUPRÁCE 26.837.137

Narodni institut deti a mladeze

4.566.761

Danemark The Danish Agency for Higher Education 15.695.547

The Danish Agency for Higher Education

3.519.646

Deutschland Nationale Agentur Bildung für Europa (BIBB) 53.160.880 JUGEND für Europa (JfE) 17.511.533

Deutschland

Nationale Agentur für EU-Hochschulzusammenarbeit, Deutscher Akademischer Austauschdienst e.V. (DAAD)

59.464.945

Deutschland

Nationale Agentur für EU-Programme im Schulbereich, Pädagogischer Austauschdienst (PAD) der Kultusministerkonferenz

19.421.175

Eire/Ireland Léargas The Exchange Bureau 8.202.484 Léargas The Exchange Bureau 3.430.012

Eire/Ireland Higher Education Authority 6.530.600

2 Decentralised funds and operating costs 3 Decentralised funds (including funds for Eastern Partnership Youth Window) and operating costs


Eesti Centre for Educational Programmes, Archimedes Foundation

8.425.353 Archimedes Foundation – Euroopa Noored Eesti Büro

2.764.386

Ellas Greek State Scholarship's Foundation (IKY) 30.089.027

Hellenic National Agency for the E.U. Programme YiA and Eurodesk

Youth and Lifelong Learning Foundation

4.682.128

España Organismo Autonoma Programas Educativos Europeos 105.310.414

Agencia Nacional Española del programa Juventud en Acción

11.562.136

France Agence Europe Education Formation France 114.487.000

Agence Française du Programme Européen Jeunesse en Action (AFPEJA)

Institut National de la Jeunesse et de l'Education Populaire (INJEP)

15.388.761

Italia Agenzia Nazionale LLP - Programma Settoriale Leonardo da Vinci - ISFOL

32.863.868 Agenzia nazionale per i giovani 12.665.936

Italia

Agenzia Nazionale LLP - Programmi settoriali Comenius, Erasmus, Grundtvig e Visite di Studio - AS Agenzia Scuola

70.665.059

Kypros Foundation for the Management of European Lifelong Learning Programmes

5.257.991 Youth Board of Cyprus 2.659.036

Latvija VIAA - State Education Development Agency) 11.310.888 Agency for International

Programs for Youth 2.969.156

Lietuva Education Exchanges support Foundation 16.400.569 Agency of International

youth co-operation 3.286437

Luxembourg

Anefore asbl - Agence Nationale pour le programme européen d'éducation et de formation tout au long de la vie

3.031.143 Service National de la Jeunesse 1.964.357

Magyarorszag Tempus Public Foundation 24.932.297

NCSSZI - Fiatalok Lendületben Programiroda NIFSP - Youth in Action Programme Office

4.563.866


Malta European Union Programmes Agency - EUPA 3.068.570 EUPA – European Union

Programmes Agency 1.948.195

Nederland Nederlands Nationaal Agentschap voor het Leven Lang Leren programma

40.023.094 Nederlands Jeugdinstituut (NJi) 5.839.101

Österreich

Österreichischer Austauschdienst - Agentur für Internationale Bildungs- und Wissenschaftskooperation (OeAD-GmbH)

22.087.947

Interkulturelles Zentrum Österreichische Nationalagentur "Jugend in Aktion"

4.102.086

Polska Foundation for the Development of Education System - FRSE 93.129.070

Polish National Agency of the Youth in Action

Programme - Foundation for the Development

of the Education System

11.632.141

Portugal Agência Nacional para a Gestão do Programa Aprendizagem ao Longo da Vida

28.641.597 Agência National para a gestao do programa Juventude em Acçao

4.901.038

Romania National Agency for Community Programmes in the Field of Education and Vocational Training

41.352.876

National Agency for Community Programmes in the Field of Education and Vocational Training

7.924.389

Slovenija

Centre of the Republic of Slovenia for mobility and European Educational and Training Programmes - CMEPIUS

9.948.926

Zavod MOVIT NA MLADINA

2.822.231

Slovensko SAAIC - National Agency of the Lifelong Learning Programme 17.018.234 IUVENTA – Slovak Youth

Institute 3.728.830

Suomi-Finland Centre for International Mobility CIMO (Kansainvälisen liikkuvuuden ja yhteistyön keskus )

20.735.236 Centre for International Mobility CIMO 3.591.275

Sverige National Agency by Swedish Council for Higher Education 23.398.030 Swedish Board for Youth

Affairs 4.483.798

United Kingdom British Council 64.644.480 British Council 15.572.294

United Kingdom

UK National Agency - Leonardo, Grundtvig & Transversal Programmes - Ecorys

36.635.660


Island

Landskrifstofa Menntaáætlunar Evrópusambandsins

National Agency for the Lifelong Learning Programme

4.054.379

Icelandic Youth Association (Ungmennafélag Íslands, UMFÍ)

1.951.413

Norge

Norwegian Centre for International Cooperation in Education - Senter for Internasjonalisering av utdanning - SIU

14.767.271 Barne-, ungdoms- og familiedirektoratet (BUFDIR)

3.458.949

Liechtenstein Agentur für Internationale Bildungsangelegenheiten 959.262 "Aha" Tipps und Infos für

Junge Leute 503.004

Turkye The Centre for EU Education and Youth Programmes 107.631.571

Turkish National Agency for LLP and YIA Programmes

13.950.432

Suisse ch Foundation for Confederal Co-operation 13.872.642

Ch Stiftung für eidgenössische Zusammenarbeit

1.558.239

Croatie Agencija za mobilnost i programme Euopske unije (AMPEU) 9.174.751

Agencija za mobilnost i programe Europske unije / Agency for Mobility and EU Programmes

2.609.172

former Yugoslav Republic of Macedonia

National Agency for European Educational Programmes and Mobility

1.413.251

National Agency for European Educational Programmes and Mobility

586.476;5

TOTAL LLP 1.210.196.554

TOTAL YiA 191.554.567,5

OVERALL TOTAL 1.401.751.121,5


LIST OF OTHER DELEGATED BODIES AND 2013 BUDGET BREAKDOWN

1. Programme concerned: Media 2007 2. Annual budgetary amount entrusted to these bodies in 2013: € 1,000,000 (see table

for breakdown by body) was committed and paid. 3. Duration of the delegation : 2010-2013 4. Justification of recourse to indirect management and justification of the selection of

the bodies: the recourse to third party bodies is set in the legal basis of the programme: Article 4 and annexe of the Decision No 1718/2006/EC of the European Parliament and of the Council of 15 November 2006 concerning the implementation of a programme of support for the European audiovisual sector (MEDIA 2007) - OJ L 327 of 24/11/2006. Thanks to the leveraging effect provided by delegated bodies, it is envisaged that the Fund will be able to guarantee a total amount of credits up to € 100-150 million during its lifespan on the basis of a 4 year budget of € 8 million. This action is implemented via indirect management according to Article 54(2)(c) of the Financial Regulation. It is managed by third party bodies on behalf of the European Commission. The selection of the following delegated bodies was the result of call for expressions of interest (n° O.J. 2010 / S 1-000123 of 02.01.2010). Delegated bodies are financial institutions with appropriate registration according to the rules applicable in their home country. They were selected among candidate national or international public-sector bodies or bodies governed by private law with a public-service mission established in one of the territories covered by the MEDIA Programme and providing adequate financial guarantees and adequate technical capacity.

5. Synthetic description of the implementing tasks entrusted to these bodies

The MEDIA Production Guarantee Fund set up in the framework of the MEDIA 2007 programme is aimed at facilitating the access of European audiovisual companies to private sources of financing. Delegated bodies share the financial risk related to the loans granted by banks to European independent audiovisual producers, by guaranteeing to cover a certain percentage of the loss in case of failure by the producer to reimburse the loan, thus encouraging local banks to grant credits to film producers.

Programme Delegated Body 2013 Annual Budgetary amount (€)

MEDIA 2007/ Production Guarantee Fund

IFCIC (Institut pour le Financement du Cinéma et des Industries turelles) - France

1,000,000

MEDIA 2007/ Production Guarantee Fund

Audiovisual SGR (Sociedades de Garantía Recíproca) - Spain

0


ANNEX 7: AARs of Executive Agencies


ANNEX 8: Decentralised agencies

Annual budgetary amount entrusted in 2013 by DG EAC

Decentralised Agencies Policy area Operational expenditure (€)

Administrative expenditure (€)

Total (€)

European Centre for the Development of Vocational 5 303 143

CEDEFOP

Training

Education and Training

12 430 000 17 733 143

ETF European Training Foundation

Education and Training 5 062 000 15 081 500 20 143 500

EIT

European Institute for Innovation and Technology

Innovation 121 612 400 4 898 533 126 510 933


ANNEX 9: Performance information included in evaluations

This Annex provides information on evaluations4 which have been finalised5 in 2013.

Europass 2nd evaluation (2008-2012)

ABB activity: 15.02 Type of evaluation: Regulatory instrument (R)

Summary of performance related findings and recommendations:

Europass is the framework for the transparency of qualifications and competences established by Decision 2241/2004/EC with the overall objective to increase the mobility of people in Europe for education and employment purposes. The purpose of the evaluation was to analyse the current role and tasks of Europass and its different documents within the current European policy framework and to suggest future Europass developments in synergy with related European initiatives and policies. The Europass initiative is highly successful overall. All the related European transparency and mobility tools and initiatives are complementary and coherent with Europass, although full synergies with EQF, ECVET and ESCO are still to be achieved. Europass is also instrumental in fighting unemployment: it contributes to helping people change their job or location and to gain further learning opportunities such as admission to educational institutions. However low qualified unemployed people especially often lacked information about Europass tools or found them too complex or confusing. A strong progress was observed, evidenced by a sharp rise in the awareness, usage and appreciation of Europass documents (particularly of Europass CV). This strong increase in usage was possible due to stable levels of funding. Despite great advances in using all Europass documents, the Europass Diploma Supplement and Certificate Supplement were still unknown by a large proportion of their intended users. Monitoring of Europass is not properly resourced and more systematic use of quantified targets could be pursued. The ways of accessing, developing and using Europass documents for disadvantaged groups (predominantly those with low levels of computer literacy, low qualified unemployed and immigrants) should be strengthened. Given its significant European added value, i.e. the creation of international, recognised and uniform tools for transparency of qualifications and skills and for mobility at European level, Europass should be maintained as a European policy

4 Surveys, rolling reviews, data collection, public consultations, legal implementation reports or other types of studies do not qualify as evaluations and are not part of this Annex.

5 Final Evaluation Report approved


initiative.

Availability of the report on Europa:

http://ec.europa.eu/dgs/education_culture/more_info/evaluations/index_en.htm

Evaluation of the Open Method of Coordination and the Structured Dialogue, as the Agenda for Culture's implementing tools at European Union level

ABB activity: 15.04 Type of evaluation: Other (O)


The Open Method of Coordination (OMC) and the Structured Dialogue process (SD) are the implementing tools of the European Union’s European Agenda for Culture intended to help bring together the various actors at EU and Member State (MS) level to take action in pursuit of shared objectives and to address cultural challenges of a European dimension. The OMC process is relevant to the Agenda for Culture’s policy objectives and the Work Plan for Culture 2011-2014 provides an improved and more integrated guiding framework and better defined objectives, compared with the previous generation of the process (2008-2010). Changes made under the second generation of OMC working groups have provided clearer direction, better quality outputs and have helped to focus activity better, although there remains potential for further improvements in outputs, for example through an evidence-based approach. Benefits and impacts mainly concern mutual learning, best practice exchange and the building of knowledge networks, rather than any far-reaching effects on national policies. There is potential to achieve greater impacts, through stronger connectivity and dissemination channels between OMC participants and key decision-makers at national level.

The SD objectives set have been achieved in relation to bringing the sector closer together, increasing the capacity to undertake advocacy work at EU level, and opening up new opportunities for exchanges between civil society and the Commission. The benefits of the SD process started to reduce after the first phase, especially in relation to the dialogue with the Commission. The work of the Platforms would have benefited from better alignment with the Council Work Plan for Culture and the OMC groups. European Culture Forums have played a valuable role in increasing awareness of European culture policies and in improving cooperation among sector organisations.






Evaluation of implementation of the European Quality Assurance Reference Framework for Vocational Education and Training (EQAVET) ABB activity: 15.02 Type of evaluation: Regulatory instrument (R)


The European Quality Assurance Reference Framework for Vocational Education and Training (EQAVET) is a European instrument to support development of national systems for quality assurance in vocational education and training. It was adopted in 2009 in form of a Recommendation of the European Parliament and the Council. The focus of the evaluation was to understand whether and how EQAVET influences national quality assurance (QA) systems and measures. EQAVET objectives remain relevant in the context of European priorities and policies in the field of education and training and beyond. In the context of national developments, the theme of quality assurance in VET remains of relevance as well. In principle EQAVET is complementary with other European instruments in the area of qualifications (EQF, ECVET), but in practice the complementarities need to be articulated more clearly, e.g. while the school-based aspects of QA in VET are well covered by EQAVET, the framework does not provide specific guidance for QA of work-based learning. The EU-level cooperation between EQAVET and European initiatives in quality assurance in other sectors of education is not systematic. The fact that EQAVET was adopted in form of a Recommendation strengthened the participation of countries in EQAVET and helped to formalise the role of National Reference Points (NRP). Directly or indirectly, EQAVET supported changes in quality assurance systems and measures in two thirds of countries analysed. However, there is currently very little evidence that EQAVET is contributing to the objective of making VET systems more transparent. The EQAVET Recommendation is vague on what constitutes an ‘approach aimed at improving quality assurance systems at national level’. This evaluation shows that a majority of EU countries have such approaches but these were not necessarily triggered by the existence of EQAVET. The content of the EQAVET Recommendation on quality cycle, criteria and descriptors does not lend itself easily for making national quality assurance systems transparent. The EQAVET governance structure is considered to be efficient. The expertise of the EQAVET secretariat is welcomed. The governance would benefit in greater involvement of representatives of stakeholders and representatives of organisations which have an interest in strengthening transparency of national quality assurance measures. The sustainability of changes induced by EQAVET is not yet clear. On one hand some countries are changing QA systems mainstreaming certain features of EQAVET. On the other hand EQAVET does not currently have a clear lever to trigger sustainable change of provider-level approaches to QA.





Evaluation of the European Commission and Council of Europe Youth Partnership Agreements 2007-2011/12



The EC-CoE Youth Partnership provides a framework for the joint development of cooperation and coherent strategy in the field of youth in Europe. It aims at creating better opportunities and improving access for young people, promoting active citizenship and fostering solidarity. The evaluation assessed the Partnership’s activities between 2007 and 2011/12. Overall, the Partnership’s objectives and activities are relevant to the needs of the target groups, but the approach needs to be articulated more clearly in terms of multiplier effects and the extent to which individual young people and/or organisations and policy-makers should be targeted. There is scope to rationalise the current framework to develop a clearer causal link between the specific and operational objectives. The robustness of the Youth Partnership’s framework of objectives and activities would be complemented and enhanced by the development of a complete logic for intervention that specifies types and numbers of intended outputs and results. This would help to develop clear goals for the Partnership. The general objective to establish a framework for cooperation has been achieved; the framework has fostered synergies and achieved complementarity with related programmes and policies across a range of institutions and organisations. The specific objectives are very broad and it is therefore difficult to assess the tangible contribution that Youth Partnership activity has made to each one. This suggests that the scope of the Youth Partnership’s activities may be too wide, risking over-stretch, with resources spreading too thinly. An effective area of joint cooperation has been the promotion of knowledge-based youth policy research. However, some of the methods used are not equipped to ensure that information stays up-to-date. The Partnership has also been effective in supporting youth work and building youth workers' capacity. The geographical focus of the Partnership is seen by sector stakeholders as effective, with strong linkages developed with partners in third countries. There is scope to improve the annual reporting in order to offer better information about the effectiveness of the Partnership's activities. There is evidence that the Youth Partnership model is not as efficient as it could be, and the operational effectiveness of the management and monitoring systems is not optimised. Operational efficiency may sometimes be constrained by the Partnership having to address two sets of organisational administrative cultures. The lack of a robust monitoring and evaluation system and the absence of agreed targets or expectations linked to the activities and outputs is a significant weakness; since this makes the impact of the activity very difficult to evaluate. Activities such as events were provided at a reasonable cost and unit costs did not appear excessive. There is potential to simplify and streamline procedures to improve operational efficiency, thereby making more resources available for key strategic tasks including systematic monitoring and evaluation.





Evaluation of the European Commission and European Youth Forum Operating Grant Agreements 2007-2011/12



The European Youth Forum (YFJ) is a platform at EU level which represents youth organisations vis-à-vis the EU and generates policy inputs from the youth field. The evaluation assessed the relevance, effectiveness, efficiency and sustainability of the YFJ and covered the period between 2007 and 2011/12. The European Youth Forum (YFJ) is a unique organisation at EU level, because of the lobbying work it does. It is a useful instrument for the EC as it provides structured opinions and input on youth-related topics. Organisations in the youth field find the YFJ’s activities and objectives relevant to them, mainly those related to representing youth organisations vis-à-vis the EU and contributing to the development of youth policies. The YFJ achieves, to varying degrees, all of its objectives and works according to the principles that are set in the legal base establishing the Youth in Action Programme. The YFJ has gained an important position in the youth field and has access to key decision makers, stakeholders and influential organisations. Therefore, the evaluators find that the YFJ is the most suitable mechanism at EU level to channel communication on youth-related issues with a broad range of young people in Europe. The YFJ's organisational structure is efficient, and the grant is proportional to the YFJ’s activities and status. However, the YFJ needs to improve the monitoring of its activities, outputs and achievements. The sustainability of the YFJ as such is very much dependent on the financial contribution of the EC.



Evaluation of the implementation of the European Qualification Framework Recommendation

ABB activity: 15.02 Type of evaluation: Regulatory instrument (R)


The European Qualifications Framework (EQF) was established by the EP and Council Recommendation to facilitate the comparison and recognition of qualifications throughout Europe, and thus to contribute to the wider objectives of promoting lifelong learning and increasing the employability, mobility and social integration of workers and learners in Europe. The evaluation covered the period of 2008-2012. Although only four countries achieved the first EQF milestone, i.e. to relate their national qualifications system to the EQF by the end of 2010, and six countries achieved the second EQF milestone, i.e. to ensure that all new qualification documents refer to the appropriate EQF level by the end of 2012, by the end of May 2013, 20 countries had related their national qualifications system to the EQF. Participating countries are more than ever aiming for qualifications systems



that are more transparent and clear and their commitment to the EQF objectives remains strong beyond the target dates set in the EQF Recommendation. The effort towards reaching the two EQF objectives should be intensified and remain a priority, even if the target dates have passed. Support to the EQF implementation – through the EQF advisory group, the EQF NCPs and their activities (including peer learning activities and projects) are more than ever needed. Cooperation with other mobility and transparency tools should be reinforced so as to ensure that the EQF is implemented in coherence with the objectives of these tools and that they all contribute to wider EU objectives of mobility and employability, as part of the EU2020 and ET2020. The EQF is already widely considered a reference tool in Europe and beyond. The main features of the EQF should remain stable so that this influence continues and expands, giving the EQF greater visibility. Data about implementation of the EQF should be continuously collected and analysed, and outcomes and impacts of the EQF Recommendations evaluated again, once the implementation has progressed further.



Evaluation of the European Centre for the Development of vocational Training (CEDEFOP).

ABB activity: 15.01 Type of evaluation: Internal Commission activity (I)


The evaluation provides information on whether CEDEFOP's activities are well chosen to provide support to the European VET policy agenda and takes into account implementing its objectives, as they had been set in its Founding Regulation and the current policy agenda During the evaluated period from mid-2006 to mid-2012 CEDEFOP was successful in implementing its objectives as they had been set in its Founding Regulation. CEDEFOP was able to embed in its activities the key EU policy developments and to meet the needs of its target groups. CEDEFOPs work did not duplicate the activities of any other actors at European, national or international level. The agency has been able to address promptly the ad hoc requests for support from the Commission. The evaluation recommends that in future the Centre should be focused on

• Fostering policy learning about EU initiatives at national level in the light of the European Semesters and advancing implementation of common EU principles and tools

• Putting focus on activities generating the highest policy impact: As the

balance between open source and policy support poles needs to be maintained, the open source should focus more on the activities under the 2012-2014 medium-term priorities.



• Fostering stakeholder relationships and learning among stakeholders:

CEDEFOP should continue and strengthen its networking activities. A systematic approach should be employed in following up the relations of the agency with the stakeholders.

• Collaboration with other agencies: the forthcoming review of the

founding regulations of the agencies should be taken as an opportunity to fine-tune the boundaries of their remits and scope for collaboration on the topics which have proven to create potential for overlap or synergies. This would ease the burden of search for synergies and potential collaboration activities.

• Improving governance and human resources management within

CEDEFOP: The recent CEDEFOP staff survey should be followed-up and its results should be used as a momentum for on-going organisational development and employee involvement.

• Improving performance management and evaluation within CEDEFOP:

The agency should work on strengthening its performance management systems (PMS), through further improvement of PMS and better implementation of activity based budgeting.



Evaluation of FP7- Marie Curie Actions - Interim

ABB activity: 15.07 Type of evaluation: Expenditure programme (E)


The People Specific Programme was established by Decisions 1982/2006/EC and 2006/973/EC with the aim to strengthen, quantitatively and qualitatively, the human potential in research and technology in Europe, by stimulating people to take up the profession of researcher, encouraging European researchers to stay in Europe, and to attract the best researchers from throughout the world to Europe. The interim evaluation assessed the Marie Curie Actions (MCA) and Researchers’ Night (RN) during the years 2007-2011. The programme remained highly relevant in terms of addressing the most pressing needs and challenges related to the implementation of research and innovation policy, to competitiveness and socio-economic needs of Europe as well as to researchers’ needs, particularly those relating to skills and networking, and especially for early career researchers. The Marie Curie Actions had a high European Added Value in terms of its unique design, scope effects, target groups addressed and the overall mix of instruments used. Researchers’ Night has developed into a prominent communication action. The European label added visibility and credibility to the events, but their belonging to the programme remains to be better communicated. The programme was attractive not only to the top-class universities but also to



the innovation leaders from the private sector. By funding almost 6,700 fellowships each year, the MCA were the leading research mobility scheme. Almost all fellows were very satisfied with the training received. Most researchers positively valued other aspects of mobility, such as networking. Most of the fellows who already finished their fellowships found employment (79%) and reported improvements in their career. The MCA enhanced the scope and excellence of academic research, allowed the valuable transfer of knowledge among the participating organisations and researchers, and contributed to commercialisation of research results. The MCA had a well-functioning and efficient overall management and implementation system. Almost all MC fellows were very satisfied with their fellowship experience. The programme had very high additionality: As many as 82% of the organisations whose projects were rejected for MC funding due to budgetary reasons had subsequently abandoned their projects. The majority of MC beneficiaries were likely to continue the research cooperation developed under the programme, but maintaining the same level and intensity of cooperation was difficult without further EU and national or regional funding. The adoption of the European Charter for Researchers and the Code of Conduct in the MCA produced a mixed structuring effect on the participating organisations, depending on the different national contexts and organisational factors.



Evaluation of the Education, Audiovisual and Culture Executive Agency (EACEA)

ABB activity: 15.01 Type of evaluation: Internal Commission activity (I)


The Education, Audiovisual and Culture Executive Agency (EACEA) was established in 2005 and assumed responsibility for the management of a number of EU programmes in the fields of education and vocational training, culture, audiovisual, citizenship and youth (plus external collaborations in some of these fields). This was the second evaluation of EACEA, covering the period from 1 May 2008 to 31 December 2011. The key analytical element was an assessment of the costs and benefits associated with delegating programme management to EACEA. EACEA has substantially achieved its objectives, with clearest progress against those related to efficient management of programmes and improving services to applicants and beneficiaries. There has been progress against all annual priorities set out in the Agency’s Annual Work Programmes: the strongest evidence of this concerns progress against the objective of streamlining application and selection processes, whereas there is less evidence of progress in terms of improving project implementation and reporting requirements. However, it should be borne in mind that this has been the main recent focus for improvements (with further improvements planned for the next generation of programmes).



Outsourcing the management of programmes continues to provide significant cost savings relative to the costs of in-house management by the Commission. This suggests that the potential cost-savings of out-sourcing have in fact been realised as a calculations based on current costs and workload suggest a saving of €93m for 2008-2011 and thus exceeding the forecast by over 100%.



Evaluation of European Capitals of Culture 2012 ABB activity: 15.01 Type of evaluation: Other (O)


The general objectives of the European Capital of Culture (ECoC) are to highlight the richness and diversity of European cultures and the features they share, as well as to promote greater mutual acquaintance between European citizens. ECoC remains of key importance complementary to other EU initiatives, e.g. the Culture and MEDIA Programmes, the European Agenda for Culture and other fields, such as youth, citizenship, education and training and regional development. The current evaluation covers two European Capitals of Culture in 2012: Maribor and Guimarães. The experience of 2012 shows that hosting a national competition does not (certainly not alone) guarantee a more successful ECoC, other factors such as strong local leadership, effective partnerships and continuity of funding in the development phase are likely to be of critical importance. Although the current monitoring arrangements are a significant improvement on those of previous years, they still do not ensure that cities fulfil all the commitments made at application, and during the monitoring stage. The evaluator suggests considering annual implementation plans or more regular reporting milestones than those set out under the strengthened monitoring procedures within the proposed new legal basis for ECoC. There is a demand for clearer commitments from a range of partners in financial terms: applicants should be encouraged to clearly state what funding is committed and what is ‘hoped for’ in their applications in order to avoid drastic reductions in budget. The experience of Guimarães in 2012 demonstrates the potential of ECoC to be reinforced by and add value to investments made by the ERDF, but also reinforces the importance of planning well in advance of the action year. It is suggested that future ECoC begin international marketing and commercial revenue generation strategies at an early point of the process, in order to maximise the potential wider economic benefits of the action. In both cities, cultural operators have gained valuable skills and experience and there are likely to be moderate effects in terms of enhancing the cities’ cultural offer. Continued impacts on cultural governance appear unlikely however, reinforcing the need for long-term strategy to be incorporated in selection and monitoring processes under the proposed new legal basis for ECoC.





Date post:	26-Jun-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Annual Activity Report Annexes - European Commission · eac_aar_2013_annexes_final Page 1 of 75...

Documents