Selection and monitoring for ERDF and ESF projects in the 2014-2020 period are still mainly outputs-oriented
About the report For the Cohesion policy funds during the 2014 – 2020 programme period, we examined how well Member States focus on results in selecting projects and how well the Commission and Member States can demonstrate, through their monitoring, that the EU budget is well spent. We found that although the Commission has taken various measures to increase results-orientation, selection procedures continue to emphasize outputs and absorption rather than results. Moreover, shortcomings in the monitoring arrangements made it difficult to assess the extent to which EU funding has contributed to the achievement of the EU’s and Member States’ objectives.
We recommend that, during the selection process, the Member States ensure a comparison of project applications, require beneficiaries to identify at least one real result indicator for each project to be included in the grant agreement and contributing to the OP indicators, and assess expected results and indicators in the evaluation report of the applications. The Commission should define common result indicators for the ERDF, improve reporting on performance, and ensure a meaningful performance review in 2019.
For the 2014-2020 programming period, the EU has allocated almost €349.4 billion towards achieving its objectives in terms of cohesion policy. This policy aims to support job creation, business competitiveness, economic growth, sustainable development and improvement in quality of life. To achieve these objectives, funding is allocated to projects delivered by beneficiaries in Member States. Authorities in Member States select projects to fund and monitor their implementation. It is critical that this funding is allocated effectively, namely through the delivery of expected results.II
In order to support this goal, the Commission implemented various measures aimed at increasing focus on results in the 2014-2020 period. In previous reports, we have welcomed the improvements that these have brought. But we have also highlighted serious weaknesses in the effectiveness of these measures, and the quality of the monitoring information relating to spending on Cohesion Policy. This report consolidates and develops these previous findings.III
We examined how well the focus on results is embedded within the way projects are selected, and how well the Commission and Member States can demonstrate the EU budget is well spent. As such, we have examined the selection and monitoring arrangements of projects using Cohesion funding.IV
We conclude that, despite longstanding intentions, the design of selection procedures and the processes themselves continue to emphasise outputs and absorption rather than results. Moreover, shortcomings in the monitoring arrangements made it difficult to assess the extent to which EU funding has contributed to the achievement of the EU’s and Member States’ objectives. Specifically, for the OPs we visited, we made the following findings:
- On project selection, potential beneficiaries were well informed and provided with appropriate support to access to EU funding. The procedures we examined were designed to support the selection of projects relevant to the OPs’ objectives, but selection criteria seldom required applicants to define quantified result indicators at project level. Furthermore, most project selection was done on a first-come first-served basis. Applications were only scored and ranked against each other in one selection procedure.
- On monitoring, we found that monitoring systems had only become functional at a late stage, mainly because of delays in adopting the legislative framework. There remain weaknesses in some of the IT systems used for collecting and aggregating monitoring data. Furthermore, slow progress in the audits of the respective monitoring systems introduced the risk that there now may not be enough time to take any necessary corrective action before the performance review in 2019.
- Monitoring information remains mostly outputs-oriented. The Commission’s main report to measure achievements presents progress for the main output indicators alongside the implementation of funding. However, there is only limited information on the achievement of results.
We therefore recommend the following:
- In order to ensure a consistent and genuine results-oriented approach to the selection of projects, Member States should ensure that a comparison between project applications takes place, require beneficiaries to define at least one genuine result indicator for each project and carry out an assessment of the expected results and indicators in the assessment report for the applications.
- In order to ensure a results-oriented monitoring, Member States should include one or more genuine and quantified result indicators in the grant agreement, which contribute to the result indicators set at OP level, and the Commission should define common result indicators for the ERDF based on a common definition of “results”.
- The Commission should improve the reporting on performance and ensure that a meaningful performance review takes place in 2019.
Cohesion policy is the EU’s main investment policy01
Cohesion policy is the EU’s main investment policy. It is aimed at supporting job creation, business competitiveness, economic growth, sustainable development and improvements in quality of life. Around one-third of the EU’s budget is allocated to cohesion policy. At current prices, this amounted to around €230 billion in the 2000-2006 programme period, €346.5 billion in the 2007-2013 period and €349.4 billion in the 2014-2020 period.02
Cohesion policy is delivered through the European Regional Development Fund (ERDF), the European Social Fund (ESF) and the Cohesion Fund (CF). These funds, together with the European Agricultural Fund for Rural Development (EAFRD) and the European Maritime and Fisheries Fund (EMFF), are part of the European Structural and Investment Funds (ESIF). They are implemented through operational programmes (OPs) which set out investment priorities and specific objectives1. These also describe how funding will be used during the programme period to finance projects. The OPs are implemented by the Member States and their regions. This means that the Member States and their regions select, monitor and evaluate projects.
Performance orientation is a key priority for the Commission and the Member States during the 2014-2020 period03
The Europe 2020 Strategy is the EU’s ten-year jobs and growth strategy. It was launched in 2010 to create conditions conducive to smart, sustainable and inclusive growth. With a view to meeting the objectives and targets set in the Europe 2020 Strategy, the Commission emphasised in 2010 the need to improve the effectiveness of the cohesion policy by focusing on results2.04
The legislative package for cohesion policy in the 2014-2020 programme period therefore introduced significant changes intended to increase focus on performance3. The main measures are the following:
- The introduction of ex-ante conditionalities (EACs)4 which require a Member State to fulfil certain conditions for the effective use of ESI funds (e.g. existence of policy/strategic frameworks) by the end of 2016. One of them, general ex-ante conditionality No 75, requires “the existence of a system of result indicators necessary to select actions which most effectively contribute to desired results, to monitor progress towards results”.
- The introduction of a performance reserve, setting aside part of the EU funding allocated to the OPs, to be released only if a subset of indicators (mainly spending/output indicators) reach pre-defined milestones. The release of this reserve in 2019 will be based on a performance review carried out in 20196.
- A reinforced intervention logic during the programming exercise7 requiring Member States to assess their needs and commit to a set of results before considering where and how to spend cohesion funds.
- A more consistent use of performance indicators to measure progress towards achieving results8.
The importance of project selection and monitoring to achieve performance05
In 2017, we concluded9 that the Commission and Member States had been successful in developing OPs with a more robust intervention logic. This means that the OPs set out in a consistent way the aims of the interventions (specific objectives/results) and how these will be achieved (required funding, actions to be undertaken and expected outputs).06
However, to achieve policy effectiveness, it is not only important to set clear goals and consider results during the programming exercise. It is essential that a results-oriented approach is followed through during the implementation phase of the OP, as it is mainly the quality of projects funded and their added value in terms of tangible results which will determine the effectiveness of a policy. Project selection, monitoring and reporting is thus essential for achieving performance. An overview of the implementation cycle of an OP and the bodies concerned is provided in Figure 1. The processes relevant to our audit are highlighted in yellow.
Main participants and their role in project selection and monitoring07
Projects are selected by the national and regional authorities responsible for managing the OPs. These managing authorities (MAs), or intermediate bodies (IBs) to which they delegate powers, lay down selection criteria, organise selection committees and decide which projects will receive European funding. The Commission is not involved in the selection of projects as opposed to its crucial role in the negotiation of OPs. However, the Commission is involved through its advisory role in the OPs monitoring committees10 where the methodology and criteria used for selecting projects are approved. It also provides guidance, based on best practices and shared experience with Member States.08
The responsibility for monitoring projects lies with the Member States. Managing authorities monitor the implementation of OPs and projects, aggregate monitoring information and submit Annual Implementation Reports (AIRs) to the Commission. The monitoring committee also reviews the implementation of the OPs and approves AIRs. Here also, the Commission’s role is limited to issuing guidance, examining the performance of OPs together with the MAs, and making observations on AIRs.09
In its reporting role nevertheless, the Commission submits its annual synthesis of Member States’ AIRs and strategic reports (in 2017 and 2019) to the discharge authority (European Parliament), and to the European Economic and Social Committee and the Committee of the Regions11.
Audit scope and approach10
This audit sought to answer the question of whether the selection of projects and monitoring systems for ERDF and ESF projects in the 2014-2020 programme period were results-oriented. Given the level of advancement of the OPs’ implementation, and building on the audits already carried out, this audit allowed us for the first time to follow the “results-oriented approach” through the whole implementation phase as the first results are becoming available. It is also of high relevance to the performance review scheduled for 2019.11
In the first part, we report on our analysis of selection procedures in place for the audited OPs. We assess the design of these selection procedures, especially the determination of the selection criteria and their result orientation, as well as the phase of evaluation of the applications. In the second part, we examine the management and control systems in place and the monitoring that has been carried out so far by the MAs/IBs of the visited OPs to measure operational performance at programme and national level. We review the reliability of data, its availability and capacity to produce relevant information for management and for reporting to the Commission.12
In total, we examined 34 projects (see list of projects in Annex I). We focused on the ERDF and ESF, and more specifically on three thematic objectives (TOs)12 (see a description of the 11 thematic objectives in Annex II):
- TO3, enhancing the competitiveness of SMEs (16 projects),
- TO8, promoting sustainable and quality employment and supporting labour mobility (12 projects), and
- TO9, promoting social inclusion and combating poverty and discrimination (6 projects).
We looked at projects selected directly by the MA (or intermediary body) through 20 selection procedures (see list in Annex III), from seven OPs located in four Member States: the Czech Republic, France, Italy and Finland (see list in Annex IV). These OPs and Member States were selected according to the materiality of funding allocated and the stage of the projects’ implementation. We selected the projects based on the amount of grant received and their level of advancement at the time of the audit. The priority axis, investment priorities and specific objectives to which the selected projects correspond are listed in Annex V. The related output and result indicators are listed in Annex VI for the ERDF and Annex VII for the ESF.14
We also reviewed the actions taken by the Commission in relation to project selection and monitoring of operational programmes for the ESF and the ERDF in general.
Project selection is insufficiently focused on results15
The selection procedures put in place by the managing authorities should prioritise the selection of those projects that are the most relevant to the objectives pursued and which have the greatest likelihood of delivering results, as well as ensuring that available funding is spent in line with the rules. We checked the extent to which results had been reflected in the design of selection procedures (e.g. the selection criteria used for assessing an application, the method used for assessing projects and the documentation available to potential bidders) as well as in the implementation of the selection procedures and thus in the projects selected.16
Previous ECA audit work13 showed that the Commission and Member States had been successful in developing OPs with a more robust intervention logic i.e. with a clear link between development needs, specific objectives and result indicators. This intervention logic should be reflected in project selection procedures. We consider that there are a number of key criteria that should be taken into account by MAs when designing procedures to ensure that they are results-oriented:
- the selection criteria applied should ensure that projects are aligned with OP objectives;
- the selection criteria should support the selection of projects that have not only quantified output indicators but also quantified result indicators to measure projects achievements;
- beneficiaries should be encouraged to submit applications to ensure that a high number of projects are available for selection;
- finally, the project selection should be based on a direct comparison of applications to identify and prioritise the projects best suited for funding.
The extent to which MAs applied these variables while processing the applications received and how they influenced the selection of results-oriented projects is described in the next paragraphs (see paragraphs 18 to 38).
The selection procedures support the selection of projects relevant to OPs’ objectives18
Two measures were introduced for the 2014-2020 programme period legislative framework which are designed to trigger the selection of projects relevant to OPs’ objectives:
- OPs must set out “guiding principles” for the selection of projects for each priority axis14;
- MAs must design selection procedures and criteria “that ensure the contribution of operations to the achievement of the specific objectives and results of the relevant priority”15.
In addition to these, the following requirements were also introduced for the 2014-2020 period to trigger the selection of more results-oriented projects:
- under general ex-ante conditionality 7 (see also paragraph 4), “a system of result indicators necessary to select actions which most effectively contribute to desired results” must be in place;
- MAs must also draw up selection procedures and criteria that “ensure the contribution of operations to the achievement of the […] results of the relevant priority”16.
The “guiding principles” act as guidance for project selection, detailing the main selection criteria to be used. Coupled with the description of the projects to be allocated funding, providing examples of such projects, and beneficiaries, they give a good overview of the type of projects to be funded under each specific objective. These principles are used as a basis for preparing the selection criteria, and may be referred to directly within selection criteria.21
We found that, in some cases, the principles were of a general nature; however, they all referenced the need for selected projects to be in line with the specific objectives of the OP, as described under each priority axis. The selection criteria, for the selection procedures we examined, were consistent with the guiding principles. Each procedure entailed at least one criterion about projects’ relevance and their contribution to the specific objective.22
Overall, the selected applications we examined were assessed against the criteria defined within the selection procedure. We found that the 34 projects we examined for this audit were relevant to the OP specific objectives.
The focus on results when assessing project applications was weak, and seldom included quantified results23
We examined how MAs carried out their selection of projects in practice, in particular the existence in the project proposal of clear expected results and objectives accompanied by clearly defined and quantified output and result indicators (linked to the specific objectives). We also assessed whether the MAs had assessed each applicant’s ability to achieve expected results.24
We found that 18 out of 20 of the selection procedures we examined included criteria requiring beneficiaries to describe their project’s expected results (see Annex VIII). However, only four out of 20 procedures included criteria requiring the quantification of result indicators at project level (selection procedures 17, 18, 19 and 20). And in three cases, these result indicators did not correspond directly to those defined at OP level (see example in Box 1).
Example of result indicators at project level different from result indicators at OP level: project No 32, selected through selection procedure No 17
|Result indicators at project level
(see Annex I, project No 32)
|Result indicators at OP level
(see also Annex VI, PA 1, SO 3d)
Our review of the 34 applications of selected projects corroborates these findings. All the applications provided quantified information for output indicators that corresponded to those at OP level, or such data could be obtained directly from the IT system (for instance for indicators such as number of enterprises receiving support). However, only 14 applications provided quantitative information on expected result indicators. In six of these, the result indicators did not correspond directly to the OP indicators. The other applications mainly provided a narrative description (see Figure 2 and see Annex VIII).
Regarding the assessment of applications, we found that the MAs had assessed the overall feasibility and likelihood of the projects to succeed. However, while there was an assessment of the achievability of the expected results for all 18 ERDF projects, this assessment was only performed for 2 of the 16 ESF projects we examined. We also found that the assessment of achievability remained vague in the assessment reports, except in seven cases17 which included a more detailed description about the relevance of the indicators and the likelihood of the targets being met.27
In terms of scoring the applications, we found that the provision of expected results was part of the scoring process in only 11 cases. This indicates that, as long as the envisaged projects were aligned with the type of actions set out in the selection procedure, the MAs assumed that it would contribute to the OP’s objectives. However, they did not focus on the magnitude of this contribution.
Beneficiaries were encouraged to apply for funding28
To ensure timely absorption throughout the programming period and at the same time ensure the focus on results, there is a need for a sufficient number of bidders able to provide good applications. The promotion of a call and related guidance to potential applicants by MAs is an important variable determining the overall effectiveness of selection procedures. A broader applicant population is likely to ensure that better projects are selected.29
The measures aiming at promoting calls to potential bidders were prepared by the MAs on the basis of the detailed requirements set in the EU regulation. They were submitted to the Monitoring Committees for approval as part of the OP’s communication strategy that is required by the Regulation18.30
We found that MAs had encouraged beneficiaries to participate in the selection procedures. First, MAs used several sources to promote a call to potential bidders. These included the involvement of Chambers of Commerce, local business associations, and online tools. MAs also relied on partners to circulate information on upcoming calls within their networks, based on the knowledge they acquired in the monitoring committees.31
Furthermore, the guidance on funding provided by MAs to applicants was publicly available and, except for one MA at the time of the audit, it set out clear conditions for receiving support, provided detailed information on the procedure for application and financing arrangements, referring explicitly to the selection criteria to be applied.
Selection procedures did not usually involve any comparison between project applications, resulting in a risk that the best projects will not be selected32
In our review of procedures, we checked that projects submitted for funding were assessed against each other. This is another important measure to ensure that the most results-oriented projects are selected.33
In the framework of shared management, selection procedures were defined by the MAs and approved by the monitoring committees, based on the requirements in the EU Regulations19 and the Commission’s guidance20. Neither the ESI Funds Regulations nor any Commission guidance specifies the type of selection procedure to be used; this is left to the MAs’ discretion.34
Out of the 20 procedures we examined, 10 were temporary calls for proposals, 6 were permanent calls, 3 were direct awarding procedures, and one a reiterated call (see also details in Annex III). We found that in all selection procedures except one (IT, OP Piemonte, No 13) no comparison between project applications had taken place. As a result, funding may not have been allocated to the best projects. This was mitigated to some extent in some of the OPs visited, where an informal (French and Finnish OPs) or a formal (Italy Puglia) pre-selection had taken place, allowing the MAs to limit the submission of applications to the most suitable projects. We found the following:
- Fifteen out of twenty procedures were based on the first-come first served principle, where an application was selected provided that it met the criteria and that sufficient funding was available. Of these, 13 had been subject to a formal or informal pre-selection. We noted that some had been required to reach a minimum score for selection (see selection procedures Nos 4 and 5 for the Czech OPEIC and 16, 17, 18, 19 and 20 for the Finnish OP).
- In the three direct awards, there was, by definition, no competition (see selection procedures No 1, 3 and 12).
- In one case (IT, OP Piemonte, selection procedure No 8)21, funding had been awarded to projects already selected during the previous 2007-2013 programming period simply by extending their duration. This allowed the IB to avoid interruptions to the provision of training to young people, and allowed it to start absorbing the funds. But it also prevented potential new applicants from participating.
The quality of monitoring data is at risk, and it is mostly outputs-oriented35
We checked that the monitoring systems allowed the systematic collection of data on specified indicators, thus enabling the measurement of the progress made at project, programme and EU levels. We also examined whether this had been reported on in a way that allowed the success of the financed projects, and thus the effective and efficient use of the EU budget, to be demonstrated.36
The following section contains our assessment of the necessary conditions to enable monitoring and reporting on the achievement of objectives. We first checked whether the MAs had set up functional monitoring systems before starting to implement the funds. We also looked at whether the performance information reported by the beneficiaries had been reliable, complete and timely. We also examined the role of the MAs and the Commission.37
Finally, we consider whether the systems in place produced results-oriented performance information; specifically, whether they produced meaningful aggregated information, as this is a key element to allow reporting on the effective use of funds at Member State and EU level via annual monitoring and reporting.
Assurance on the quality of monitoring data may not be provided in time for the 2019 performance review38
The monitoring of an OP’s progress towards its objectives is done on the basis of the data collected from various sources. This data is aggregated at OP and Member State level, and forms the basis for the reports sent to the OP’s monitoring committee and to the Commission. In order to present a correct view of the situation at OP or Member State level, it is essential that the data collected is reliable, complete and timely. To achieve this, the MAs have to set up a monitoring system including the collection of data and checks on the source of the data. In order to gain assurance on the quality of the data, the Commission relies on the audit work carried out by the audit authorities at Member State level as well as its own audit work.39
In the specific context of the performance review, it is also important that the data, and the systems used to produce them, be audited as early as possible, to allow Member State authorities to address any detected weaknesses regarding the data quality well before the end of June 2019. It is then that the 2018 Annual Implementation Reports (AIRs) – the basis for the performance review and the allocation of the performance reserve – will be sent to the Commission.
Extensive EU rules lay down how Member States should set up their monitoring systems for the 2014-2020 period40
With a view to achieving the systematic collection and monitoring of data produced at OP level, the regulatory framework for the 2014-2020 programme period introduced several changes concerning data collection and monitoring arrangements:
- MAs must put in place a management and control system (MCS) covering adequate systems for reporting and monitoring22.
- MAs are responsible for establishing a computerised system to record all the data related to the OPs indicators23 for monitoring and reporting purposes.
- Under EAC 7, MAs must put in place “a system of result indicators necessary to select actions which most effectively contribute to desired results, to monitor progress towards [those] results”24. OPs were required to demonstrate fulfilment of all the ex-ante conditionalities (EAC) by 31 December 201625 (see paragraph 4).
- By 31 December 201526, Member States were required to ensure that all exchanges of information between beneficiaries and a MA, Certifying Authorities (CA), AAs and IBs could be carried out electronically27.
The Commission has also established specific guidance28, for its staff and the Member States’ responsible authorities, on a common methodology for assessing management and control systems in the MS, including the existence of adequate systems for monitoring and reporting.
Performance information is not always reliable, complete and timely, and this has an impact on the implementation reports issued by the Member States
Data reported by the beneficiaries42
For the ERDF, in many cases, the data for result indicators is collected from national statistics bodies (see examples in Annex VI). The remainder comes from beneficiaries. For ESF, most of the data is reported via data collected from beneficiaries (see Annex VII). Managing authorities check that the data collected from beneficiaries is correct at project completion, when receiving their final report. During project implementation, checks can also be done on-the-spot, on a sample basis.43
A number of plausibility and consistency checks are carried out automatically by the systems in place on the data provided by the beneficiaries, especially for the ESF. MAs also have tools to monitor the completion of the data provided on participants which allow them to remind beneficiaries who have submitted incomplete data to provide further information.44
Our audit revealed issues regarding the quality of the data collected. We identified a general issue regarding the completeness and correctness of the data concerning participants to ESF co-financed actions. This data is classified as sensitive in national legislation29, so it is up to participants to decide whether to declare it. It is also difficult for the Member States and the Commission to check its accuracy. An additional weakness regarding the collection of data is presented in Box 2.
Issues regarding timeliness of the data
For the French national ESF OP, larger beneficiaries submitted their data for the 2014-2015 period late. This was corrected in 2016, and resulted in significant differences between the values reported in the 2015 AIR and those reported in the 2016 AIR for the year 2015.
In its audit of performance data reliability (2014-2020), the Commission reported on other issues regarding the reliability of data, for instance:
- data not collected which resulted in achievements not being recorded and thus not being taken into account in order to determine the progress of an OP, leading to an incorrect understanding of the situation;
- a misunderstanding on the MAs’ part regarding the definition of common indicators (for instance, Common Output indicator 26, 'Enterprises cooperating with research organisations') resulting in an overstatement of the number of companies reported under this indicator;
- double counting/failure to cap the value for indicator 'population served' at the number of area residents, resulting in inconsistent values.
Availability and reliability of performance information46
The existence of functional monitoring systems at the time MAs start implementing programmes is a key element ensuring that performance data is generated in time, which ensures that monitoring can take place.47
The late approval, in December 2013, of the legislative framework for the period 2014-2020 at EU level, delayed the development of the IT systems used for monitoring. For the ESF indicators, added complexity concerning the sensitivity of the data and the definition of the indicators, required additional discussions well into 2014. This further delayed the finalisation of the IT systems. In two cases, other factors led to the delays: reorganisation of the regions (France, OP Lorraine et Vosges), and the design of a new single IT system for all 2014-2020 ESI funds in the Czech Republic.48
The effects of the delay are reflected in the level of implementation of EAC 7 on the existence of a system of results indicators. At the end of 2016, the deadline to complete action plans, 50 plans remained incomplete, i.e. around 11 % of all OPs still did not fulfil this requirement. However, by the end of February 2018, all the action plans related to EAC 7 had been completed.49
In the first half of 2017, the fourth year after the start of the programme period, we found that some IT systems were still not fully functional; this poses risks to the completeness and accuracy of the data collected (see examples in Box 3). In its audit of performance data reliability (2014-2020)30, the Commission drew attention to the absence of electronic data exchange systems with beneficiaries, resulting in the manual encoding of the data on indicators by the MA into the system, a potential source of clerical errors.
Examples of IT systems still not fully functional and related risks
Czech Republic - OPE
- Problems with transferring data between the Labour Office and the OP. Data only started to be reported electronically to the OP from 31.3.2017. Until then, data was shared via Microsoft Excel files. This jeopardises the transfer of data between systems, especially given the large amount of data involved.
- The AIR 2016 was the first AIR containing relevant data.
The MA discovered inconsistencies in the data reported by the system for the AIR. This required to recalculate and fill them in manually in the AIR.
Italy - OP Puglia
The tools used for exchanging data with beneficiaries electronically were not working properly at the time of the audit31, and the system for collecting and storing the data was empty. Data was merely stored on staff computers. As a result, the aggregation of data with a view to producing the 2016 AIR (to be submitted to the Commission by 30 June 2017) was done manually, in separate files.
Implementation reports issued by the Member States50
The MAs must submit to the Commission an Annual Implementation Report (AIR) for each OP, which is approved by the Monitoring Committee. The AIR delivers key information on the implementation of the OP, mostly by providing quantitative and qualitative information on common and programme-specific indicators32. The 2016 AIRs, submitted in June 2017, were the first reports sent to the Commission on the “progress made towards achieving the objectives of the programme”33. The Commission carries out a quality review of the AIRs (consistency of number of participants with population of the areas covered, for instance).51
At the end of 2017, 26 AIRs had still not been accepted by the Commission due to missing data for the indicators used in the performance framework, inconsistencies in the data reported with the information provided on the implementation of the OP, or a lack of explanation for some of the data reported. Around 40 % of ERDF OPs had not reported any values for their performance framework indicators in 2017, on which there was no Commission analysis at the time of the audit.52
The shortcomings identified in the AIRs raise questions about the reliability of the information provided as well as the level of progress reached until now as presented in the documents issued by the Commission based on the AIRs34: the annual synthesis report to the Parliament and the Council, European Economic and Social Committee and the Committee of the Regions and, in 2017 and 2019 only, the strategic progress report, which summarises progress made.
Audits on management and control systems and the reliability of collected data started late, limiting the degree of assurance on the monitoring systems and reducing the time available to address detected weaknesses53
Appropriate arrangements for the verification and correction of the performance information are necessary to ensure its quality and hence, its use in the context of monitoring and reporting. Good practices requires audits on the systems producing the data - and associated corrective actions - to be carried out and finalised before the data is included in AIRs and strategic reports.54
Under the legislative framework, both the Member States’ audit authorities and the Commission35 audit the functioning of the monitoring systems. These audits should, in particular, ensure the existence of reliable systems for collecting, recording and storing performance data, and assess whether this performance data is reliable.
Audit authorities’ assessment of the functioning of the management and control systems55
An assessment of the systems in place at MA level was carried out during the early stages of the implementation of the OPs as part of the designation procedure36. In this procedure, an independent audit body (often the AA) was required to produce a report and an opinion on whether the MAs had fulfilled the criteria relating to the internal control environment, risk management, management and control activities and monitoring37. The designation was required to take place before the first submission of an interim payment for the OPs. Thus, without this designation, OPs cannot be reimbursed. At this stage, the audit covers the existence of systems and procedures, but it does not cover good implementation, which is checked in the context of the audit authorities’ system audits on management and control systems.56
Due to the late approval of the OPs and late finalisation of the IT systems (see paragraph 47), the designation process was late. This was also affected by national factors (reorganisation of the regions in France in 2015, complex management structures - for instance, the French PON FSE has 13 delegated managing authorities and 120 intermediary bodies). For the OPs examined in this audit, the MAs were designated between September 2016 and February 2018, more than two and four years into the programme period respectively (see Annex IV).57
Due to the late implementation of OPs and the resulting limited amount of data collected, systems audits were only beginning at the time of our audit visits i.e. during the first half of 2017. Any problems regarding the quality of data thus did not become apparent until the second half of 2017. This was almost four years after the start of the seven-year programme period and one-and-a-half years before the 2018 AIRs, which form the basis for the allocation of the performance reserve, are forwarded to the Commission.
The Commission’s audits58
In addition to relying on the audit work and systems in place in the Member States, the Commission also carries out its own audits. There are two types: “early preventive systems audits”, aimed at obtaining reasonable assurance at an early stage of the implementation that the management and control systems are functioning well, and “performance reliability data audits”. But given the aforementioned delays, and thus the late finalisation of the monitoring systems as well as the minimal reporting of relevant data, these audits were conducted later than planned. Especially for the ERDF, audits could only start after the 2016 AIRs were received. In total, by the end of 2017, the Commission managed to carry out 23 audits in 12 Member States: 12 performance reliability data audits and 11 early preventive systems audits.59
For most of these audits, reporting was ongoing at the beginning of 2018. According to the Commission, the preliminary results for the audited systems were rather positive, even though some shortcomings were highlighted. At the same time, however, around 40 % of ERDF OPs did not report any values for their performance framework indicators38 in the 2016 AIRs submitted in 2017. For these OPs, in February 2018, the Commission had not carried out audits to determine whether the absence of reporting was due to delays in implementation or a lack of a functioning data-reporting system.60
The late undertaking of the systems audits means that the overview of the monitoring systems’s status is not complete. Shortcomings are particularly susceptible to being identified late and there may not be sufficient time to apply the necessary adjustments. This jeopardises the implementation of the performance review in 2019, which will be based on data reported by Member States in the 2018 AIRs by 30 June 2019.
The performance review may not be based on realistic milestones61
In the OPs we visited, on the basis of the 2015 AIRs or in the monitoring information available at the time of the audit, we noted that there were already cases where OPs had achieved the targets set for 2023, as well as risks of underachievement. Requests for amendments in order to adjust the targets to a more realistic value, had not been submitted for the OPs visited. Only one (Czech Republic, OPEIC) had already asked for, and obtained, amendments, mainly in relation to the modification of allocations within priority axes. On the other hand, according to the Commission, in February 2018, out of the 17 480 indicators used in the existing 430 OPs, the corresponding target value for 2023 had been modified upwards in 1445 instances and downwards in 941 instances.62
This indicates that a number of milestones39 of the performance framework are no longer realistic. In case milestones need to be revised, this requires that amendments be made to the OP, which can only be done at the Member States initiative40. Given the duration of the amendment procedure (set in the Regulation at three months), there is a risk that the values reached at the end of 2018 by the performance-framework indicators are not compared with realistic milestones.
The information generated on the basis of the data collected is still mostly outputs-oriented63
Performance information is collected so that the Member States and the Commission can use it for reporting purposes, in order to be accountable for what has been achieved with funding.64
Result indicators and targets laid down in the grant agreements signed between beneficiaries and MAs are essential for measuring the achievement of results. However, several key issues in the set-up of the monitoring systems are affecting the measurement and monitoring of the contribution of projects to the achievement of the OP objectives.
A set-up which relies mostly on output indicators, especially in the early years of implementation for the ERDF, and a lack of quantified result indicators defined at project level65
The current set-up allows for a more systematic collection of data for a number of well-defined indicators, though the number of indicators is very high. The existence of common indicators also allows for the aggregation of this data at OP, Member State and EU levels. However, in the set-up of the logical framework for the ERDF, many result indicators at OP level are not directly related to the funded interventions. These indicators are often national indicators which take into account the influence of external factors, and their sources are mainly national statistics offices (see examples in Box 4 and see also Annex VI). These result indicators do not isolate the impact attributable to the ERDF interventions, and cannot therefore be considered immediate results of the OPs.
Examples of result indicators for which the data is collected from published statistics rather than funded projects
|France, OP Grand Est:
||Source: Banque de France.|
|Czech Republic, OPEIC:
||Source: Ministry of Industry and Trade.|
|Finland, OP Sustainable growth and work:
||Source: National statistics office of Finland.|
To mitigate the fact that results generated by ERDF projects often materialise (well) after project completion, at a time when there is no monitoring of the projects as such, MAs must carry out evaluations at priority axis level41. At least once during the programming period, these evaluations assess how support from the ESI Funds has contributed to the objectives for each priority. But given the late start of the implementation, the number of evaluations already carried out is still limited. As a consequence, for the time being, the progress of the OP is mainly measured through the output indicators.67
At project level, we found that result indicators were not systematically mentioned in the grant agreements (see paragraph 25). Only six42 out of the 18 ERDF projects examined in this audit contained results indicators in the applications and thus in the grant agreement. However, these result indicators were not the same as those used at OP level. This means that they cannot be used for reporting purposes or to measure the progress made in achieving OP objectives. In four other cases43, the results indicators were not quantified (i.e. no target was given) thus preventing an assessment of their contribution to the OP objective from being made and limiting the measurement of the project’s achievement to its outputs.68
For the ESF projects, result indicators were mentioned in 7 out of 16 projects. For the remaining projects, the results described were either not related to the OP indicators, or they were not quantified. In any given project, this makes it impossible to compare the value reached with an estimated target. However, in the case of the ESF, even if results and result indicators are not defined at project level or quantified, the measurement of the values for the common result indicators is still taking place, as project beneficiaries have to report those data in the MA’s IT systems.69
We have previously mentioned the lack of result indicators defined at project level44 in our annual reports. This makes it difficult to measure and to monitor the extent to which projects contribute to the achievement of the OP objectives.
The release of the performance reserve is based on indicators that are mostly input- and outputs-oriented70
Member States can demonstrate progress in programme implementation through milestones (to be reached by 2018) and target values (to be reached by 2023) for three types of indicators: financial, output and result. They can be complemented by key implementation steps (KIS) to measure projects that are on-going or scheduled to be started, but for which it is unlikely that any outputs can be achieved by December 2018. Overall, the indicators used in the performance framework must be representative of the actions under the priority axis. In order to trigger the release of the performance reserve, the values reached at the end of 2018 for these indicators must be between 75 % and 85 % of the milestone value.71
As shown in a previous ECA report45, the vast majority of the indicators used in the performance framework under the cohesion policy related to output (57 %), key implementation steps (9 %) and financial indicators (33 %), with the use of result indicators remaining marginal. Thus, the release of the reserve will to a large extent remain input- and outputs-oriented, and not focused on results (see also paragraph 62).
Most reporting at EU level concerns output indicators72
The main objective of the Commission’s strategic report 201746, which encompasses the 2017 annual synthesis report drawn up based on the 2016 AIRs, is to report on the achievements of the ESI Funds towards attaining the EU2020 objectives. However, the report mainly presents the level of implementation as well as the values reached for the main output indicators by the end of 2016. It presents no information on the achievement of results, except for the following ESF result indicators: the number of participants that found a job immediately after the training, and the number of participants that gained a qualification immediately after training. For the ERDF, this is also due to the fact that there are no common result indicators; those used at Member State level cannot, therefore, be aggregated.73
As we have reported recently, there is a fundamental problem affecting result indicators in particular. In fact, ERDF and ESF result indicators measure different things in different ways47. Furthermore, the concept of a “result” is interpreted differently in the fund-specific regulations. The ERDF result indicators cannot be used to measure the immediate results of the OPs. In addition, the high number of different performance indicators pose a real challenge to the collection and reporting of performance information48.
Conclusions and recommendations74
Although the set-up of OPs is now more results-oriented, with a stronger intervention logic and an extensive set of indicators, overall we conclude that the selection of projects under the ERDF and ESF is not fully results-oriented and that the monitoring in place is still more outputs-oriented.
Selection procedures were not results-oriented in important aspects75
For the OPs we visited, selection procedures were used to support the selection of projects relevant to the OPs’ objectives. However, they do not systematically include selection criteria requiring the definition of quantified result indicators at project level corresponding to those at OP level. Consequently, result indicators were rarely included in applications; even when this was the case, they did not necessarily correspond to the OP indicators, or had not been quantified. Furthermore, only one-third of the assessment reports included a specific assessment of the expected results in relation to OP objectives (see paragraphs 18 to 27).76
We found that MAs had provided comprehensive information on the OPs, on access to EU funding, and on the selection procedures through several channels. Beneficiaries were generally well informed, or could rely on support from specialised organisations (see paragraphs 28 to 31).77
However, only 1 of the 20 procedures examined involved a scoring and a ranking of applications against each other. Projects were generally approved on a first-come first-served nature. As a result, funding may not have been allocated to the best projects (see paragraphs 32 to 34).
There are risks to the quality of monitoring data, which remains more outputs-oriented rather than results-oriented78
Following the late adoption of the legislative framework, we found that some of the OPs’ monitoring systems were not operational at the start of the programme nor fully operational at the time of the audit. Consequently, data was input manually and at a later stage. Data collection also occurred outside the IT systems, creating errors, which, in some cases, impacted upon the data presented in the AIRs (see paragraphs 35 to 52).79
The late adoption of the regulatory framework led, in most OPs, to a late start of the implementation. Until the end of 2016, only a limited amount of data was collected in IT systems. Consequently, the audits necessary to provide assurance on the monitoring systems and the reliability and availability of the data mostly started in 2017. The results of these audits only became available at the end of 2017, four years into the seven-year programme period. This means that assurance obtained upon the monitoring data is, up to now, only partial. The Commission thus does not have an overview of the functioning of the monitoring systems, including IT systems. The performance review will take place in 2019 and, in the remaining time available, there is a risk that any corrective measures cannot be completed in time and that the performance review will not be based on correct information (see paragraphs 53 to 60).80
We found that a number of milestones of the performance framework are no longer realistic. In case milestones need to be revised, there is no certainty that the necessary OP amendments can be made in time for the performance review (see paragraphs 61 and 62).81
Regarding result indicators, ERDF and ESF indicators measure different things in different ways. In fact, the concept of a “result” is interpreted differently in the fund-specific regulations. For both funds, we identified the risk that a meaningful aggregation of performance data (in particular regarding results) may not be feasible at EU level49. Furthermore, we recommended50 that the Commission should define a common definition of “outputs” and “results” (see paragraph 73).82
For the ERDF, many result indicators often use other sources, such as data from national statistics offices. Evaluations should measure achievements against the OPs’ objectives. However, given the late start of implementation, few evaluations were available at the time of our audit. Consequently, for the ERDF, it is difficult to monitor to what extent the projects contribute to the achievement of the OP objectives (see paragraphs 63 to 69). This problem also applies to the ESF, as many grant agreements do not include quantified result indicators.83
Similarly, output and financial indicators predominantly form the basis of the performance framework. This means that the release of the performance reserve will be mainly based on these types of indicators rather than on the results achieved and the actual progress made towards the OP objectives. Also, the information published in the Commission’s reporting is mostly outputs-related, except for the ESF where the aggregated value for some result indicators is provided (see paragraphs 70 to 72).
Recommendation 1 – Results-oriented selection
In order to ensure a consistent and truly results-oriented approach to the selection of projects, Member States should:
- have selection criteria requiring the beneficiaries to define at least one genuine result indicator based on a common definition of what constitutes a “result” for their project, including a baseline and a target. This result indicator should contribute to the result indicators set at OP level;
- include an assessment of these expected results and indicators in the assessment report for the applications;
- when deciding on the selection procedures to be used, ensure that a comparison between project applications takes place.
Target implementation date: starting in 2019 for upcoming calls for proposals.
Recommendation 2 - Results-oriented monitoring
- In order to allow the monitoring of projects’ contributions to OP objectives, Member States should include quantified result indicators in the grant agreement, contributing to the results indicators set at OP level.
Target implementation date: starting in 2019 for upcoming calls for proposals.
- In order to make the ERDF monitoring framework more results-oriented, and in particular to allow the aggregation of performance information, the Commission should define common result indicators for this fund based on a common definition of “results”.
Target implementation date: preparation of the next MFF.
Recommendation 3 - Reporting on performance and preparation of the performance review in 2019
To improve the Commission’s reporting on performance and to carry out a meaningful performance review with reliable data and realistic milestones, the Commission should:
- have an overview of the main weaknesses in and uncertainties about the OPs’ monitoring systems based on audits by both the Commission and the AAs (the necessary systems audits/performance-reliability audits);
- ensure that OP amendments requested by Member States relating to justified revisions of milestones of the performance framework are processed in time for the performance review.
Target implementation date: before the finalisation of the performance review.
This Report was adopted by Chamber II, headed by Mrs Iliana IVANOVA, Member of the Court of Auditors, in Luxembourg at its meeting of 27 June 2018.
For the Court of Auditors
List of projects selected for this audit
(amounts in million euros)
|No||Project Description||Grants received previously||Total amount of investment||Amount of EU Grant||Fund||TO (a)||PA (b)||IP/SO (c)||Start & end date||Outputs and results achieved||Selection procedure No (see Annex III)|
|Czech Republic - OP Employment (OPE) 2014CZ05M9OP001 (amounts in million CZK)|
|1||”New working opportunities” (Call Instruments of Active labour market policy)||Yes||4 011||3 304.6||ESF||8||1.1||1.1.1||1.4.2015 ongoing||Participants in employment: 14 304||1|
|2||Child care centre for preschool children #1 (Call Support of creation and operation of day care services for children of pre-school age for companies and public outside the City of Prague)||Yes||4.9||4.2||ESF||8||1.2||1.2.1||1.1.2016 - 31.12.2017||Places in kindergarten: 24||2|
|3||Child care centre for preschool children #2(a) (Call Support of creation and operation of day care services for children of pre-school age for companies and public outside the City of Prague)||Yes||4.1||3.4||ESF||8||1.2||1.2.1||1.1.2016 - 31.12.2017||Places in kindergarten: 20||2|
|4||Child care centre for preschool children #2(b) (Call Support of creation and operation of day care services for children of pre-school age for companies and public outside the City of Prague)||Yes||4.9||4.2||ESF||8||1.2||1.2.1||1.1.2016 - ongoing||Places in kindergarten: 20
Number of persons using a childcare facilities: 30
|5||“Support of selected social services (asylum houses, daily centres, etc) in one region”||Yes||411.8||350||ESF||9||2.1||2.1.1||1.1.2016 - ongoing||Actual realisation: NA yet, project in start up phase||3|
|Czech Republic - OP Entrepreneurship and Innovation for Competitiveness (OPEIC) 2014CZ16RFOP001 (amounts in million CZK)|
|6||Support to exports for the companies||Yes||1.2||0.6||ERDF||3||2.1||3b
|3.7.2015 - 10.5.2016||Number of participations in exhibitions and fairs abroad 3 (but the same company): one company supported
Grants previously received:
12 projects OPPI 265.2 million CZK,
4 Fair grants 53 million CZK
|7||Participations in trade fairs abroad in 2016||Yes||3.3||1.6||ERDF||3||2.1||3 b
|9.9.2015 - 21.12.2016||Number of participations in exhibitions and fairs abroad 3 (but the same company): one company supported
Grants previously received:
8 projects from OPPI 117.3 million CZK
1 OPLZZ 5.3 million CZK
|Czech Republic - OP Entrepreneurship and Innovation for Competitiveness (OPEIC) 2014CZ16RFOP001 (amounts in million CZK)|
|8||Building of a production hall for woodworking division||Yes||8.6||3.8||ERDF||3||2.1||3 c
|1.11.2015 - 25.1.2016||Revitalisation of the company’s site (844 mq)
Number of companies using the revitalised infrastructure: 1
Grants previously received:
7.2 million CZK
|9||Reconstruction of a Building||Yes||21.2||7.4||ERDF||3||2.1||3 c
|7.9.2015 - 6.9.2016||Revitalisation of the company’s site (2721 mq)
Number of companies using the revitalised infrastructure: 1
Grants previously received:
OPPI 12 projects 123.7 million CZK,
OPLZZ project 2.2 million CZK
|France - OP ERDF/ESF Lorraine et Vosges - 2014FR16M0OP015|
|10||Take over a printing activity (due to retirement of the previous owner)||No||2||0.04||ERDF||3||2||2.3A||18.9.2014 - 12.10.2015||Number of enterprises receiving support: 1
Jobs: + 2
|11||Acquisition of equipment for production (e.g.: presse 140 tons with bi-injection system)||No||0.6||0.05||ERDF||3||2||2.3A||1.1.2014 - 31.12.2016||Number of enterprises receiving support: 1
Jobs: + 2
|12||Investments for the development of a ski resort||No||6.7||1.1||ERDF||3||9||9.3A||1.1.2015 - 31.12.2016||Number of enterprises receiving support: 1
Number of enterprises receiving subsidy: 1
|13||Acquisition of equipment for production (hydraulic tools)||No||1.6||0.02||ERDF||3||2||2.3A||2.7.2014 - 31.2.2016||Number of enterprises receiving support: 1
Number of enterprises receiving subsidy: 1
Jobs: + 6
|France - National OP ESF (PON FSE) - 2014FR05SFOP001|
|14||Support to people benefitting from the minimum income in the department||Yes||10.7||5.3||ESF||3||9||1.1||1.1.2014 - 31.12.2016||Number of participants expected: 6360 / year or 19 080 in total||8|
|15||Accompany/ guiding people in an insertion process, including learning French language||Yes||2||1||ESF||3||9||1.1||1.1.2014 - 31.12.2016||Number of participants to the action (2014-2016): 657
Positive outcome within 3 months after the project participation: 40 %
|France - National OP ESF (PON FSE) - 2014FR05SFOP001|
|16||Professional training of employees from the construction sector (Paris and neighbouring region)||Yes||9.5||4.8||ESF||2||8||5.3||1.1.2014 - 31.12.2015||Number of participants to the action (2014-2016): 2 464||10|
|17||Professionalization of networks about the creation of activities||Yes||5.2||2.6||ESF||1||8||3.2||1.1.2014 - 31.12.2016||No participants||11|
|18||Training activities for unemployed people||Yes||80||30.7||ESF||2||8||5.4||1.1.2014 - 31.12.2016||Number of participants to the training: 21 860 (targeted: 22 734)||12|
|Italy - OP ESF Piemonte - 2014IT05SFOP013|
|19||Training courses to help insertion in the labour market (Technical e-commerce)||Yes||0.69||0.69||ESF||8||1||8.x||1.10.2015 - 15.6.2016||Technical e-commerce course (600 hours)||13|
|20||Training courses to help insertion in the labour market (hairdresser)||Yes||0.69||0.69||ESF||8||1||8.x||1.10.2015 - 15.06.2016||Hairdresser course (600 hours)||13|
|21||Vocational training to back up compulsory education (Carpenter)||Yes||0.99||0.99||ESF||8||2||8.x||1.10.2015 - 15.06.2016||Carpenter course (1 050 hours)||14|
|22||Vocational training to back up compulsory education (Agro-food processor)||Yes||0.99||0.99||ESF||8||2||8.x||1.10.2015 - 15.6.2016||Agro-food course (1 050 hours)||14|
|Italy - OP ERDF/ESF Puglia - 2014IT16M2OP002|
|23||Acquisitions of 10 new buses for passenger transport||Yes||4.12||1.41||ERDF||3||3||3a||5.5.2015 - 31.12.2015||none||15|
|24||Acquisition of equipment for production of gluten-free pasta||Yes||2.7||0.8||ERDF||3||3||3a||16.6.2015 - 31.12.2015||- Turnover (2017 o/s)
- Increase 4 work units
|Italy - OP ERDF/ESF Puglia - 2014IT16M2OP002|
|25||Restructuring and adaptation (building) of an existing school into a crèche||Yes||0.5||0.44||ERDF||9||9||9a||1.1.2016 - 31.12.2016||57 new places created in crèche||16|
|26||Restructuring and adaptation (building) of an existing school into a crèche||Yes||0.35||0.31||ERDF||9||9||9a||1.1.2016 - 31.12.2016||42 new places created in crèche||16|
|Finland - OP Sustainable growth and work - Finland’s structural funds programme - 2014FI16M2OP001|
|27||Investment in a powder painting line, acoustic wall element line and steel bending machine. Production reorganized according to LEAN||Yes||0.85||0.32||ERDF||3||1||3d 2.1||16.2.2015 - 30.6.2016||Turnover: + €0.3 million, + 10 % of current one
Direct export: + €0.07 million, + 100 % from the current level
Jobs: + 2 (of which 0 are women’s jobs)
Other: Significant energy savings per produced unit, at least 10 % of energy use
|28||Investment for undertaking concrete production line development, internal logistics re-organisation as well as develop a new range of products||Yes||2.45||0.619||ERDF||3||1||3d 2.1||2.3.2015 - 30.9.2016||Right after completion:
Turnover: + €0.41 million
Jobs: + 1
2 years after completion (not known yet):
Turnover: + €2.5 million, + 20 % of current turnover
Jobs: + 7
Project contributes to low carbon economy
|29||To develop and confirm the service model used in one city’s Labour Service Center for young adults that have a background of substance abuse to encourage them to normalize their life so that they may envisage to start looking for a job||Yes||0.24||0.17||ESF||9||5||8i||2.1.2015 -31.12.2016||Beneficiaries: 39 persons (target: 80 persons)||18|
|Finland - OP Sustainable growth and work - Finland’s structural funds programme - 2014FI16M2OP001|
|30||To offer training for people employed intermittently to increase their chances of finding jobs and their choice of jobs (social economy)||Yes||0.31||0.22||ESF||8||3||9i||1.1.2015 - 31.12.2016||Beneficiaries: 82 persons (target 100 persons)||18|
|31||Development of a user interface software for medical ultrasound scanner||Yes||0.34||0.17||ERDF||3||1||3a 1.1||30.1.2015 - 30.4.2016||At project completion / estimate 2 years after:
Turnover:+ €0.25 million + 100 % / + €10 million + 100 %
Direct exports: + € 0.25 million + 100 % / + €9.8 million + 100 %
Jobs: + 3 / 15 (of which women 0 / 5) (target 35) Jobs in R&D: + 3 / 10 (of which women 0 / 3)
Other intangible rights 0 / 3 (target 4)
|32||Development of new wooden cladding products coating line||Yes||1.4||0.56||ERDF||3||1||3d 2.1||27.1.2015 - 27.1.2017||At project completion / estimated 2 years after completion:
Turnover: + €1 million +12 % / + €2 million + 24 % (target €3 million)
Direct exports: NA / + €0.3 million + 33 % (target €1 million)
Jobs: + 3 / + 6 of which female: + 1 / + 2 (target 2)
|Finland - OP Sustainable growth and work - Finland’s structural funds programme - 2014FI16M2OP001|
|33||Extension of a factory for the development of production and logistics operations||Yes||1.17||0.35||ERDF||3||1||3d 2.1||1.6.2014 - 31.12.2014||At project completion / estimated 2 years after completion:
Turnover: + €10 million + 150 % (target €11 million) / + €35 million + 350 %
Direct exports: + €9 million + 643 % (target € 0.9 million)/ + €26 million + 999 %)
Jobs: + 10 / + 50 of which women + 3 / + 5 (target 50)
R&D jobs: + 2 /+ 2 of which women + 0/ + 1 (target 4)
Other intangible rights 1 / 3
Project contributes to low carbon economy
|34||Extension of the machinery park for modern technology for piping and steel structure production and re-organise the lay-out of the production site||Yes||1.43||0.5||ERDF||3||1||3d 2.1||13.2.2015 – 31.12.2016||At project completion / estimated 2 years after completion:
Turnover: 0 / + €1.4 million + 34 %
Direct exports: 0 / + €0.46 million + 136 % (target €0.75 million)
Jobs: + 5 / + 8 of which female 0 / 0
Other : energy savings NA / + 1 MWh/a
Notes: (a) TO: thematic objective. (b) PA: priority axis. (c) IP: investment priority and SO: specific objective.
|1||Strengthening research, technological development and innovation|
|2||Enhancing access to, and use of quality of, information and communication technologies|
|3||Enhancing the competitiveness of SMEs|
|4||Supporting the shift towards a low-carbon economy in all sectors|
|5||Promoting climate change adaptation, risk prevention and management|
|6||Preserving and protecting the environment and promoting resource efficiency|
|7||Promoting sustainable transport and removing bottlenecks in key network infrastructures|
|8||Promoting sustainable and quality employment and supporting labour mobility|
|9||Promoting social inclusion, combating poverty and any discrimination|
|10||Investing in education, training and vocational training for skills and lifelong learning|
|11||Enhancing institutional capacity of public authorities and stakeholders and an efficient public administration|
Source: Article 9 and Annex XI Part I of Regulation (EU) No 1303/2013.
List of selection procedures examined in this audit
|No||Name / identification||Remarks||Fund||TO (a)||PA/IP/SO (b)||Type of selection procedure (c)||Competitive||1st in 1st serve||Projects audited (see Annex II)|
|Czech Republic OPE|
|1||Instruments of Active labour market policy
|Selection of actions to increase employment of supported persons, especially older, low-qualified and disadvantaged ones
The beneficiary (CZ’s Labour office) set in the Partnership Agreement
29.7.2015 - 30.4.2016
|2||Childcare services outside the City of Prague
|To decrease the differences in the situation of women and men
Support of creation and operation of day care services for children of pre-school age for companies and public outside the City of Prague (“Childcare services”)
|ESF||8||1.2.1||Temporary call for projects: 11.11.2015 - 8.1.2016||YES||2, 3, 4|
|2.1.1 - To increase employability of persons in danger of social exclusion or socially excluded persons in the labour market and
2.1.2 - Development of social economy sector
Support of selected social services linked to the regional mid-term plans of development of social services (“Social services”)
The beneficiaries (all CZ’s regions) as well as the envelope per region, were set in the Partnership Agreement
|Direct award launched: 26.6.2015
Applications accepted between: 20.7.2015 - 13.12.2019
|Czech Republic OPEIC|
|4||Marketing I||Objective SO 2.2: To increase internationalization of SMEs
- Services for SMEs focused on international competitiveness enabling entry to foreign markets;
- Sophisticated consultancy of experts in international markets and consultancy for strategic management and innovation management, incl. mentoring and coaching;
- Services to support SME networking in international research cooperation (Horizon 2020, COSME).
|ERDF||3||PA2, SO 2.2||Temporary call
2.6.2015 - 30.11.2015
|YES provided that the applications score a minimum of 60 points out of 100||6, 7|
|5||Real Estate I||Objective SO 2.3: To increase the use of entrepreneurial infrastructure
- Modernization of production facilities and reconstruction of existing obsolete infrastructure, and
- Reconstruction of brownfields (excluding costs for the removal of ecological burdens) and their transformation into modern business premises and newly reconstructed areas.
|ERDF||3||PA2, SO 2.3||Temporary call in two phases:
1.6. till 31.8.2015
Eligible applications only:
1.12.2015 till 31.1.2016
|YES provided that the applications score a minimum of 60 points out of 100||8, 9|
|France OP Lorraine et Vosges|
|6||Entreprenariat et entreprises||Objective: To select projects with a view to support the investment in SMEs, at all stages of their development, to generate sustainable jobs and added-value||ERDF||3||PA 2 - SO 2.3.A||Permanent call
1 year revolving
|YES provided that the criteria are met||10, 11, 13|
|7||Développement économique et touristique du massif||Objective: to increase the number of tourists in the Vosges area||ERDF||3||PA 9 - SO 9.3.A||Permanent call,
1 year revolving
|YES provided that the criteria are met||12|
|France PON FSE|
|8||Call for project||Call launched by the Intermediate Body||ESF||9||PA3 - 18.104.22.168||Temporary call 16.12.2014 - 31.3.2015||YES provided that the criteria are met||14|
|9||Call for projects - AAP interne 2014 - 2017||Call launched by the Intermediate Body||ESF||9||PA3 - 22.214.171.124||Temporary call
9.6.2015 - 30.7.2015
|YES provided that the criteria are met||15|
|10||Call for project - 2014IDF-AXE2-01||Call launched by the Delegated Managing Authority||ESF||8||PA2 - 126.96.36.199||Temporary call
11.8.2014 - 17.11.2014
|YES provided that the criteria are met||16|
|11||Call for project||Call launched by the Intermediate Body||ESF||8||PA1 - 188.8.131.52||Permanent call revolving
1.1.2014 - 31.12.2016
|YES provided that the criteria are met||17|
|12||Call for projects - CSP 2014/15||Call launched by the Managing Authority||ESF||8||PA2 - 184.108.40.206||Direct award revolving
1.1.2014 - 31.12.2017
|YES provided that the criteria are met||18|
|Italy OP Piemonte|
|13||“Mercato del Lavoro” in the region Piedmont (except Turin province)||To select trainings to shorten the access to the labour market of mainly unemployed youths and adults (19-25 years sometimes older) with low levels of education and exposed to a range of factors that put them at a greater risk of long-term unemployment (Formazione professionale finalizzata alla lotta contro la disoccupazione)
Call managed by the Managing Authority
10.7.2015 - 29.7.2015
|YES scoring and ranking||19, 20|
|14||“Obbligo d’Istruzione” CMT 2015/2016 in Turin province||To select courses to allow young people that have difficulties to integrate and have been excluded from school mainstream education, or even dropped out of school /compulsory education/instruction to fulfil their right/duty to 10 years education and training||ESF||8||PA1
|NA This call was not a selection procedure as such but consisted in the prolongation of the contracts for beneficiaries selected previously via a similar call 2011/2012. For simplicity and time saving, the MA decided to reiterate the same training activity for the period 2015-2018.||21, 22|
|Italy OP Puglia|
|15||D.D. No 2487 of 22/12/2014||Objective: Facilitate the creation of new production units; the extension of existing production units; the diversification of the production to new, additional products; fundamental change in the overall production process of an existing unit||ERDF||3||PA3 - 3.6||Permanent call||YES provided that the criteria are met.||23, 24|
|16||DD 367 of 6.8.2015||To select projects aiming at improving and upgrading the network of social welfare, educational and healthcare facilities across Puglia region. Its purpose is to fill in the gaps in the supply of social-care services to people, families and communities in the Region, by co-financing social investment plans or structural measures in the social, healthcare or educational fields. Eligible beneficiaries are e.g. public institutions, mainly municipalities, and private bodies offering social care services, which were formerly public welfare and charity institutions or other public utilities (e.g. local health services, chamber of commerce)||ERDF||9||9.10 and 9.11||Permanent call||YES as long as applications score a minimum of 70 points out of 100||25, 26|
|Finland - OP Structural Funds|
|17||Enterprise development and support schema||Complementary ERDF support schema executed in four Centres for Economic Development, Transport and the Environment||ERDF||3||3 / all, discretional focussing on regional targeted actions||Permanent, valid for the whole MFF 2014-20. Annual national budgetary appropriation||Yes, as long as applications score a minimum threshold||27, 28, 31, 32, 33, 34|
|18||Centre for Economic Development, Transport and the Environment South Savo Finland, Mikkeli||ERDF: focus on Intelligent specialization
ESF: focus on Support for youth actions and Integration of actors and enhanced cooperation
|ERDF - PA2
ESF - PA3
ESF - PA4
ESF - PA5
|Yes, as long as applications score a minimum threshold||29, 30|
|19||Centre for Economic Development, Transport and the Environment, North Ostrobothnia, Oulu||ESF ERDF||all||All PA/IP/SO||Temporary call:
|Yes, as long as applications score a minimum threshold||N/A
The projects selected were not advanced enough, thus not audited
|Finland - OP Structural Funds|
|20||Centre for Economic Development, Transport and the Environment, North Ostrobothnia, Oulu||Under ESF: special focus on the integration of the immigrants into society
Under ERDF: focus on environmental and natural resources in particular bio-recycling economy as well as energy and material efficiency and
on operations/projects linked to research-and development
|ERDF - PA1
ERDF - PA2
ESF - PA3
ESF - PA4
ESF - PA5
ESF: 12.12.2015 -5.2.2016 ERDF: 12.12.2015 -12.2.2016
|Yes, as long as applications score a minimum threshold|
(a) TO: thematic objective.
(b) PA: priority axis, IP: investment priority, SO: specific objective.
(c) Type of selection procedures:
Temporary Calls: Calls that have a duration inferior of 12 months, usually few weeks to a few months.
Permanent Calls: Calls open with a duration superior of 12 months, sometimes for the whole duration of the programme period.
Direct awards: Involve the earmarking of funding to an organisation, usually a public body, who then redistributes the money to external beneficiaries.
Source: Documentation of the selection procedures.
List of OPs examined in this audit
|ERDF||ESF||Total EU funds||Other funds||Total EU + other funds||TOs covered||Partnership Agreement approval date||EC approval date of the OP||Latest version and approval date||Designation of National Authorities by EC/ MS||Managing authority (MA)
If applicable: Intermediate Body (IB) or Delegated Managing Authority (DMA)
|Amounts in million euros except for CZ|
|Czech Republic - OP Employment (OPE) - 2014CZ05M9OP001 (amounts in million CZK)|
|NA||2 119||2 119||450||2 569||8,9,11||13.4.2014||6.5.2015||21.7.2017||EC: 16.9.2016
|MA: Ministry of Labour and Social Affairs
|Czech Republic - OP Entrepreneurship and Innovation for Competitiveness (OPEIC) - 2014CZ16RFOP001 (amounts in million CZK)|
|4 331||NA||4 331||3 611||7 942||1,2,3,4,7||13.4.2014||29.4.2015||1.9.2017||EC: 13.3.2017
|MA: Ministry of Industry and Trade
IB: Agency for Entrepreneurship and Innovation
|France - OP ERDF/ESF Lorraine et Vosges - 2014FR16M0OP015|
|MA (since 1 January 2016): L’autorité régionale pour la nouvelle région Grand Est|
|France - National OP ESF for employment and social inclusion (PON FSE) - 2014FR05SFOP001|
|NA||2 894||2 894||2 577||5 471||8,9,10,11||8.8.2014||10.10.2014||NA||EC : 12.1.2017
|MA: Ministère du travail, de l’emploi, de la formation professionnelle et du dialogue social
Several IB: of which Conseil départemental de la Seine-Saint-Denis, Conseil départemental du Val-de-Marne, Association France active, Fonds paritaire de la sécurisation des parcours professionels
|Italy - OP ESF Piemonte - 2014IT05SFOP013|
|MA: Region Piemonte (unit “Social Cohesion”)
IB: Citta Metropolitana Torino
|Italy - OP Puglia - 2014IT16M2OP002|
|1 394||386||1 780||1 780||3 560||1,2,3,4,5,6,7,8,9,10,11,12||29.10.2014||17.7.2014||13.8.2015||EC: 21.2.2018
|MA: Region Puglia (unit “Servizio Attuazione del programma”)
IB: Puglia Sviluppo
|Finland - OP Sustainable Growth and work - 2014FI16M2OP001|
|766||513||1 279||1 279||2 558||1,3,4,8,9,10||7.10.2014||11.12.2014||24.5.2016||EC: 20.12.2016
|MA: The Department of Enterprise and Regional Development at the Ministry of Economic Affairs and Employment
IB: ELY-Centers in Mikkeli and Oulu
Priority axis, investment priorities and specific objectives to which the selected projects in Annex I correspond
|Priority Axis||Thematic Objective||Investment priority||Specific Objective|
|Czech Republic - OPE|
|PA1||8||8i - Access to employment for job-seekers and inactive people, including the long-term unemployed and people far from the labour market, also through local employment initiatives and support for labour mobility of workers||1.1 - Increase employment of supported persons, especially older, low-qualified and disadvantaged ones|
|8iv - Equality of women and men in all areas, including access to employment and career promotion, harmonisation of working and private life and promotion of equal pay the same work||1.2 - Reduce the differences in the situation of women and men at the labour market|
|PA2||9||9i - Active inclusion, including with a view to promoting equal opportunities and active participation, and improving employability||2.1 - Increase the assertion of persons in danger of social exclusion and socially excluded persons in the society and at the labour market|
|Czech Republic - OPEIC|
|PA2||3||3 (b) - Developing and implementing new business models for SMEs, in particular with regard to internationalisation||2.2 - Increase the internalisation of small and medium sized enterprises|
|3 (c) - Supporting the creation and the extension of advanced capacities for product and service development||2.3 - Increase business infrastructure utilization|
|France - OP Lorraine et Vosges|
|PA2||3||A - Promotion of entrepreneurship spirit||To support investment in SMEs at all stages of their development|
|PA9||3||A - Promotion of entrepreneurship spirit||To increase tourism in the Vosges mountains|
|France - PON FSE|
|PA1||8||8iii - To support self- employment, entrepreneurship and creation of businesses, including innovating micro- and SMEs||2 -0 1. Reinforcement and sharing of the service offer within networks and or between the different actors supporting the creation or take over of activities and the strengthening of activities|
|1 - The professionalisation of employees and volunteers from the support networks to the creation of businesses and supporting organisations for strengthening the activities|
|PA2||8||8v - Employees’, businesses’ and entrepreneurs’ adaptation to change||3 - Actions allowing to meet the conditions and prerequisites for an effective access to training|
|PA3||9||9i - Active inclusion, including the promotion of equal rights, the active participation and a better ability to work||1 - a) Implementation of personalised paths, strengthening employability, with regard to the different obstacles to overcome, in a global approach|
|Italy - OP Piemonte|
|PA1||8||8i - Access to employment for job-seekers and inactive people, including the long-term unemployed and people far from the labour market, also through local employment initiatives and support for labour mobility||ER 8.5 - Promote the entry onto the labour market and the employment of the long-term unemployed and those with the greatest difficulties in finding work, and support for people at risk of long-term unemployment|
|8.ii - Sustainable integration into the labour market of young people, in particular those not in employment, education or training, including young people at risk of social exclusion and young people from marginalised communities, including through the implementation of the Youth Guarantee||ER 8.1 - Increase youth employment|
|Italy - OP Puglia|
|PA3||3||3.a - Promoting entrepreneurship, in particular facilitating the economic exploitation of new ideas and fostering the creation of new businesses||ER 3.5 - Promoting the set-up and strengthening of micro-enterprises and SMEs|
|PA9||9||9.a - Investing in health and social infrastructure, which contributes towards development, reduced health inequalities and better service accessibility||ER 9.3 - Increasing/ strengthening/ improving the quality of socio-educational services and infrastructure for children and welfare services and infrastructure for people of restricted autonomy, and upgrading the infrastructure network and local healthcare and welfare services|
|Finland - OP Sustainable growth and work|
|PA1||3||3a - Promoting entrepreneurship, in particular by facilitating the economic exploitation of new ideas and fostering the creation of new firms, including through business incubators||1.1 - Creating new business activities|
|PA1||3||3d - Supporting SMEs so that they can be part of the growth in the local, national and international markets and innovative processes||2.1 - Promoting the growth and expansion abroad of SMEs|
|PA3||8||8i - Access to employment for job-seekers and inactive people, including the long-term unemployed and people far from the labour market, also through local employment initiatives and support for labour mobility of workers||6.1 - Promoting the employment of young people and those in a weak position in the job market|
|PA5||9||9i - Active inclusion, including with a view to promoting equal opportunities and active participation, and improving employability||10.1 - Improving the ability of those outside employment to work and function|
OP’s output and result indicators for the priority axis described under Annex V funded with ERDF
|PA/IP/SO||Output indicators||Less developed||Transition||Developed||Total||Result indicators and source||Transition||Developed||Call||Project|
|CZECH REPUBLIC - OP Entreprise and Innovation for competitiveness (OPEIC) 2014CZ16RFOP001|
|PA 2||CO1: Number of entreprises receiving support||4 600||4 600||SR: percentage of exports in the total turnover of companies (source: MPO)||2011: 21.3 %||
22.3 % - 23.8 %
|3b.2.2||CO2: Number of enterprises receiving grants||2 100||2 100|
|CO4: Number of entreprises receiving a non financial support||3 000||3 000|
|CO6: Private investment matching public support to enterprises (grants) (in million CZK)||84||84|
|PA 2||CO1: Number of entreprises receiving support||695||695||SR: total area of regenerated territory in ha (source: national data base on regenerated borwnfields)||2011: 25 875||
25 900 – 26 050
|3c.2.2||CO2: Number of enterprises receiving grants||695||695|
|CO6: Private investment matching public support to enterprises (grants) (in million CZK)||335||335|
|CO22: Total area of regenerated territory in hectares||150||150|
|France - OP GRAND EST FR16M0OP015|
|PA 2||CO1: Number of entreprises receiving support||1 100||1 100||SR4: evolution of the investment of the region's industrial SMEs (source: national statistical office)||2012: - 8.0 %||
||10,11 and 13
|3.A||CO2: Number of enterprises receiving grants||805||805|
|CO3: Number of entreprises receiving a support other than grants||530||530|
|CO4: Number of entreprises receiving a non financial support||520||520|
|CO8: Employment increase in supported enterprises||960||960|
|CO1: Number of entreprises receiving support||46||33||79||SR22: Number of expected visits and of skiers days
in millions (sources: Observatoires régionaux du Tourisme and Domaine skiable de France)
|2011: 1 692||
|2011: 1 468||
|CO2: Number of enterprises receiving grants||34||24||58|
|CO4: Number of entreprises receiving a non financial support||12||9||21|
|CO9: Increase of number of expected visits||11 760||8 240||20 000|
|ITALY - OP PUGLIA 2014IT16M2OP002|
|CO1: Number of entreprises receiving support||2 000||SR3002: company net turnover rate (source: ISTAT)||2012: - 0.6 %||
|15 & 16
||23, 24, 25 and 26|
|CO5: Number of start ups benefiting from support||500|
|CO8: Employment increase in supported enterprises||50||1||2|
|FINLAND - OP Sustainable growth and work (2014FI16M2OP001)|
|PA 1||CO1: Number of enterprises receiving support||2 465||2 465||Business dynamics index (source: Statistics Finland)||2010: 1||1.1||17||31|
|3a||CO2: Number of enterprises receiving grants||2 455||2 455|
|CO3: Number of enterprises receiving financial support other than grants||10||10||SME locations||2010: 16 761||18 437|
|CO5: Number of new enterprises supported||1 200||1 200|
|CO6: Private investment matching public support to enterprises (grants in million euros)||51.697||51.697|
|CO7: Private investment matching public support to enterprises (non-grants in million euros)||10||10|
|CO8: Employment increase in supported enterprises||7 800||7 800|
|6: Number of companies that start a new business operation after receiving funding||850||850|
|7: Number of companies with a significant increase in turnover or staff||1 010||1 010|
|8: Number of companies promoting low-carbon solutions||615||615|
|10: Number of SMEs with improved accessibility as a result of the project (Outermost or northern sparsely populated regions)||220||220|
|9: Other investments promoting entrepreneurship as a result of the project (Outermost or northern sparsely populated regions - in million euros)||13.7||13.7|
|PA 1 3d||CO2: Number of enterprises receiving grants
CO3: Number of enterprises receiving financial support other than grants
|Business dynamics index (source: Statistics Finland)||1||1.10||17||27, 28, 32, 33, 34|
|PA 1 3d||CO6: Private investment matching public support to enterprises (grants) in million euros||337.15||337.15||Growing companies (employs min. 3 people and average growth of staff exceeds 10 %) (source: Statistics Finland)||4 326||4 759|
|CO7: Private investment matching public support to enterprises (non-grants) in million euros||10||10|
|CO8: Employment increase in supported enterprises||4 900||4 900|
|Number of companies in projects where the main objective is to promote growth and international business operations||5 170||5 170|
|Number of companies that are starting to export or that expand their exports to a new market area||1 330||1 330|
|Energy saved (MWh)||460 435||460 435|
|Companies with a significant increase in turnover or staff||1 340||1 340|
|Companies promoting low-carbon solutions||650||650|
Legend: CO: common output indicator; CR: common result indicator, SR: specific result indicator (under the ERDF, all result indicators are specific to the Member State / OP).
Output and result indicators for the priority axis described under Annex V funded with ESF
|PA/IP||Output indicators||M||F||Total||Result indicators and source||Baseline||Target (2023)||Call||Project|
|CZECH REPUBLIC - OP Employment 2014CZ05M9OP001|
|CO06: Participants below 25 years of age||38 571||CR01: Inactive participants who started seeking a job again after termination of their participation (IS ESF 2014+)||2013: 1 210||3 500||1||1|
|CO07: Participants above 54 years of age||51 429||CR03: Participants gaining a qualification after termination of their participation||13 268||67 000|
|CO20: Number of projects fully or partially implemented by social partners or NGOS||100||CR04: Participants employed, including the self-employed, after termination of their participation||2 256||58 740|
|CR05: Disadvantaged participants seeking a job after termination of their participation, involved in education/ training, improving their qualification or already employed, including the self-employed||4 089||42 000|
CESF0: Total number of participants
|178 000||CR06: Participants employed 6 months after termination of their participation, including the self-employed||1 805||82 000|
|Unemployed participants, including the long-term unemployed persons||172 493||CR007: Participants whose position on the labour market has improved 6 months after termination of their participation||903||1 700|
|Participants with completed primary ISCED) or lower-secondary (ISCED 2) education||8 900||CR008: Participants at the age of 54+ employed 6 months, including self-employed, after termination of their participation||1 292||12 300|
|Inactive participants||5 687||CR09: Disadvantaged participants employed 6 months after termination of their participation, including the self-employed||268||41 000|
|PA1 1.2||CO20: Number of projects fully or partially implemented by social partners or NGOS||90||CR01: Inactive participants who started seeking a job again after termination of their participation (IS ESF 2014+)||134||1 220||1 354||100||1 400||1 500||2||2, 3, 4|
|CO21: Number of projects targeting sustainable employment of women and sustainable career progression of women||410||CR03: Participants gaining a qualification after termination of their participation||49||440||489||50||450||500|
|CO22: Number of projects targeting public authorities or public services at national, regional and local levels||60||CR04: Participants employed, including the self-employed, after termination of their participation||26||245||271||48||432||480|
|CO23: Number of micro, small and medium sized enterprises supported (incl. cooperatives and social businesses)||130||CR05: Disadvantaged participants seeking a job after termination of their participation, involved in education/training, improving their qualification or already employed, including the self-employed||69||622||691||131||1 184||1 315|
|CO35: Capacity of supported child-care facilities or educational facilities||4 000||CR06: Participants employed 6 months after termination of their participation, including the self-employed||48||432||480||55||495||550|
|Specific output indicators:
CESF0: Total number of participants
|CR007: Participants whose position on the labour market has improved 6 months after termination of their participation||50||243||105||105||495||600|
|CR008: Participants at the age of 54+ employed 6 months, including self-employed, after termination of their participation||4||34||38||5||45||50|
|80500: The number of analytical and strategic documents written and published (incl. evaluation ones)||35||CR09: Disadvantaged participants employed 6 months after termination of their participation, including the self-employed||336||150|
|50100: Number of supported child-care facilities or educational facilities||333||50110: Number of persons using a childcare facility for pre-school children||400||6 000|
|50105: Number of employers supporting flexible forms of work||70||50130: Number of persons using a flexible form of work||50||500|
|CO20: Number of projects fully or partially implemented by social partners or NGOS||415||CR01: Inactive participants who started seeking a job again after termination of their participation (IS ESF 2014+)||2 527||3 326||3||5|
|CO22: Number of projects targeting public authorities or public services at national, regional and local levels||14||CR04: Participants employed, including the self-employed, after termination of their participation||1 010||1 386|
|CO23: Number of micro, small and medium sized enterprises supported (incl. cooperatives and social businesses)||231||CR05: Disadvantaged participants seeking a job after termination of their participation, involved in education/training, improving their qualification or already employed, including the self-employed||934||5 636|
|CR06: Participants employed 6 months after termination of their participation, including the self-employed||665||860|
CESF0: Total number of participants
|167 244||CR007: Participants whose position on the labour market has improved 6 months after termination of their participation||3 326||16 724|
|67001: Capacity of supported services||32 063||CR008: Participants at the age of 54+ employed 6 months, including self-employed, after termination of their participation||143||138|
|67101: Number of supporting institutions supported||5||CR09: Disadvantaged participants employed 6 months after termination of their participation, including the self-employed||455||832|
|80500: The number of analytical and strategic documents written and published (incl. evaluation ones)||18||Specific indicators:
67010: Use of supported services
|16 000||74 000|
|67401: New or innovated social services related to housing||247||67110: Number of ancillary institutions which operate even after termination of the support||2||5|
|10210: Number of social enterprises established thanks to the support||138||62910: Participants employed including self-employed 12 months upon leaving||333||430|
|67300: Number of participants provided with consulting on social entrepreneurship||80||67310: Former participants of projects in which intervention by means of social work served its purpose
including the self-employed
|27 914||62 558|
|10211: Number of social enterprises established thanks to support, operating even after termination of support||100||92|
|FRANCE - NATIONAL OP ESF 2014FR05SFOP001|
|CO01: Unemployed, including long-term unemployed||1 400 000||CR02: Participants following a qualification after termination of their participation||91 322||180 000||8 & 9||14 & 15|
|CO03: Participants inactive||675 000|
|CR03: Participants gaining a qualification after termination of their participation||25 961||52 500|
9i3: Number of women
|1 000 000||CR04: Participants employed, including the self-employed, after termination of their participation||281 063||575 000|
|9i4: Number of participants from priority districts from the City policy||230 000|
|9i5: Number of projects aiming at coordinating and facilitating the offer related to social insertion||Specific indicators :
R91.4: Number of actions for coordinating and facilitating implemented
|9i7: Number of projects aiming at mobilising employers from the commercial and non-commercial sectors||R91.6: Number of organisations with social utility and of employers that received support|
|Specific indicators :
8iii1: Number of entrepreneurs or buyers that received support
|340 400||Specific indicators (no common indicators for this specific objective) the values for different categories of regions are cumulated:
R83.1: Number of businesses created
|8iii2: Number of female entrepreneurs that received support||129 200||R83.2: Number of businesses created priority districts from the City policy||9 000|
|8iii3: Number of entrepreneurs from priority districts from the City policy that received support||34 000||R83.3: Number of sharing activities achieved|
|R83.4: Number of businesses created by women||36 000|
|ITALY - OP PIEMONTE 2014IT05SFOP013|
|PA 1 1.8i||CO01: Unemployed, including long-term unemployed||9 900||10 600||20 500||CR06: Disadvantaged participants in employment, including self-employment, 6 months after leaving training||34||36||35||45||45||45||13||19 & 20|
|CO23: Number of supported micro and SMEs||2 700|
|PA 1 1.8i||CO01: Unemployed, including long-term unemployed|
|CO23: Of micro, small and medium sized enterprises supported (incl. cooperatives and social businesses)|
|PA 1 1.8ii||CO01: Unemployed, including long-term unemployed||8 500||8 500||17 000||CR06: Disadvantaged participants in employment, including self-employment, 6 months after leaving training||30.5||30.5||30.5||40.5||40.5||40.5||14||21 & 22|
|CO03: Participants inactive||13 300||7 700||21 000|
|CO06: Participants below 25 years of age||18 150||12 550||30 700|
|FINLAND – OP Sustainable growth and work (2014FI16M2OP001)|
|PA 3 8i||Specific indicators:
Number of participants under 30 years of age who are unemployed or outside the labour force
|26 000||Specific indicator:
Participants under 30 in employment when leaving the measure
(source: statistics Finland)
|Number of participants over 54 years of age who are unemployed or outside the labour force||8 600||Participants over 54 in employment when leaving the measure||17||23|
|CO02: The long-term unemployed||11 250||Specific indicator:
Improved work and functional capacity of the participants (Separate review)
|CO04: Those outside employment and not in training||4 100|
|CO20: Number of projects realised or partly realised by social partners or civic organisations||55|
Legend: CO: common output indicator; CR: common result indicator; M: male; F: female.
Outputs and results in the selection procedures and applications examined
|Are there quantified indicators in the applications?||Is there an assessment of the indicators and the values provided?|
|(a)||(b)||Is there a selection criteria about result indicators / expected results?||Output indicators corresponding to those at OP level and quantified||Result indicators corresponding to those at OP level and quantified||Output||Results|
|Czech Republic OPE 2014CZ05M9OP001|
|1||1||Expected results to be outlined, result indicators mentioned and corresponds to the OP indicators but no target / quantification required||NA, no information on results|
|3||3||Use of “unit costs” (c)|
|5||NA, no information on results|
|Czech Republic OPEIC 2014CZ16RFOP001|
|4||6||Expected results to be outlined, result indicators mentioned and correspond to the OP indicators but no target / quantification required||Applications outlining expected results and their contribution to the OP objectives, no quantification of the result indicator related to the OP||They are binding and are mentioned in the grant agreement||Assessment used for the scoring|
|5||8||Assessment used for the scoring|
|France OP Lorraine et Vosges 2014FR16M0OP015|
|6||10||Expected results to be outlined (project have to prove how they contribute to the achievement of the indicators)||Applications outlining expected results and their contribution to the OP objectives||Limited assessment of the expected results in the assessment reports|
|7||12||Limited assessment of the expected results in the assessment reports (result indicator not assessed)|
|France PON FSE 2014FR05SFOP001|
|8||14||Not explicitely requested but this is required in the standard format application||But result indicators different from the OP result indicators|
|9||15||(Indicators different from the OP’s were also provided)|
|10||16||But result indicators different from the OP result indicators|
|11||17||Output indicators not related to OP indicators and not quantified||Only qualitative information on the expected results||Only the relevance of the indicators used was assessed||Only relevance of the indicators used was assessed|
|12||18||(Indicators different from the OP’s were also provided)||Consistency with amount of grant|
|Italy OP Piemonte 2014IT05SFOP013|
|13||19&20||Use of unit costs (c)||NA, no info on results|
|Italy OP Puglia 2014IT16M2OP002|
|15||23&24||Request for expected results only through the business plan||Applications outlining expected results and their contribution to the OP objectives||Assessment of the described expected results but with weaknesses|
|Finland OP Sustainable growth and work 2014FI16M2OP001|
|17 ERDF||27, 28 31, 32 33, 34||But they do not correspond to the OP indicators||No information on output indicators but data may be obtained directly from the IT system||NA, no info on outputs||But indicators do not correspond to the OP indicator|
|18 ESF||29||Assessment used for the scoring|
|30||NA, no info on results|
|19 ESF/ERDF||But for the ERDF they do not correspond to the OP indicator||NA, no projects examined under these selection procedures|
|YES||Partially||NO||NA: not applicable|
Note: (a) See Annex III list of selection procedures. (b) See Annex I list of projects examined. (c) Unit costs is a methodology to determine the amount of eligible costs where this amount is equal to the output to be reached (for instance the number of places to be created in a crèche, here number of places created) multiplied by the unit cost per place.
Source: (1) Documentation for the selection procedures. (2) Applications submitted for the projects examined during this audit. (3) Assessment reports for these applications
Europe 2020 Strategy: The EU’s growth strategy for the 2010–2020 period, aimed at allowing it to recover from the crisis. The strategy is split into five headline targets covering employment; research and development; climate and energy; education; and social inclusion and poverty reduction.
European Code of Conduct on Partnership: A set of principles set out in Commission Delegated Regulation (EU) No 240/2014 to support Member States in organising partnerships for drawing up and implementing partnership agreements and operational programmes.
European Structural and Investment Funds (ESIF or ESI funds): The ESI funds are five separate funds that aim to reduce regional imbalances across the Union, with policy frameworks set for the seven-year MFF budgetary period. The funds include the European Regional Development Fund (ERDF), the European Social Fund (ESF), the Cohesion Fund (CF), the European Agricultural Fund for Rural Development (EAFRD) and the European Maritime and Fisheries Fund (EMFF).
Ex-ante conditionalities: Conditions which Member States must meet before they can receive any funding under the European Structural and Investment Funds. When preparing operational programmes in the 2014-2020 programme period, Member States were required to assess whether these conditions have been fulfilled. If they have not, action plans were required to be drawn up to ensure fulfilment by 31 December 2016.
Financial indicators: Values used to monitor progress in terms of the (annual) commitment and payment of the funds available for any project, measure or programme in relation to its eligible cost.
Indicator: A measurable proxy for an objective, providing information to support an assessment of the degree to which an objective has been met.
Intervention logic: The link between the assessed needs, objectives, inputs (planned and allocated), outputs (targeted and achieved) and results (intended and actual).
Investment priorities: For each priority axis, Member States must set out in their operational programmes the investment priorities and corresponding specific objectives. They are listed in the specific Regulations governing the ERDF, ESF and CF.
Managing authority (MA): A managing authority is a public or private body which has been designated by a Member State to manage an operational programme. Its tasks include selecting projects to be funded, monitoring how projects are implemented, and reporting to the Commission on financial aspects and achieved results.
Monitoring: The regular examination of expenditure, outputs and results, which provides up-to-date information on whether projects/programmes are progressing as intended.
Monitoring committee: A committee which monitors the implementation of an OP. It consists of “representatives of the relevant Member State authorities” (e.g. representatives of the managing, certifying and audit authorities, implementing bodies, employer or employee organisations, and civil society). The Commission also has an observer role.
Operational Programme (OP): A statement of a Member State’s priorities and specific objectives which describes how funding (EU and national public and private co-financing) will be used over a given period (currently seven years) to finance projects. The projects within an OP must contribute to a certain number of objectives specified at the level of the OP’s priority axes. OP funding may come from the ERDF, CF and/or ESF. The OP is drawn up by the Member State and has to be approved by the Commission before any payments can be made from the EU budget. OPs can only be modified during the programme period if both parties agree.
Outputs: Something that is produced or accomplished with the resources allocated to an intervention (e.g. training courses delivered to unemployed young people, number of sewage plants, or a length of road built).
Output indicators: Values used to measure the outputs of the operations supported or the outputs at operational programme level.
Performance framework: A set of milestones and targets defined for each priority axis in an Operational Programme, forming an important pillar of the performance-orientation approach.
Performance reserve: Resources constituting 6 % of the resources allocated to the ERDF, ESF and CF or to the EAFRD and EMFF, to be made available after the 2019 performance review if certain requirements are met or exceeded.
Priority axis: One or more investment priorities associated with one thematic objective. The funding in an operational programme is organised by priority axis.
Results: Immediate changes that arise for beneficiaries at the end of their participation in an intervention (e.g. improved accessibility to an area due to the construction of a road, trainees who have found a job).
Result indicators: Values used to measure the results generated by supported projects, or the results achieved at operational programme level.
Selection/award criteria: Predefined criteria used in selection procedures to assess applicants’ ability to complete the proposed action or work programme.
Specific objective: The envisaged result to which EU action aims to contribute.
List of abbreviations
AA: Audit Authority
AIR: Annual Implementation Report
CA: Certifying authority
CF: Cohesion Fund
CPR: Common Provision Regulation
CSF: Common Strategic Framework
DMA: Delegated Managing Authority
EC: European Commission
ERDF: European Regional Development Fund
ESF: European Social Fund
ESI funds or ESIF: European Structural and Investment Funds
IB: Intermediate body
IP: Investment priority
MA: Managing Authority
MS: Member State
OP: Operational Programme
PA: Priority axis
SMEs: Small and medium-sized enterprises
SO: Specific objective
TO: Thematic objective
1 Investment priorities and specific objectives are established based on the Member State’s partnership agreement, which is an agreement between the Member State and the Commission outlining the country’s strategic goals and investment priorities.
2 COM(2010) 700 final of 19 October 2010 “The EU Budget Review”.
3 All these aspects were dealt with in our Special Report No 2/2017 “The Commission’s negotiation of 2014-2020 Partnership Agreements and programmes in Cohesion: spending more targeted on Europe 2020 priorities, but increasingly complex arrangements to measure performance” and Special Report No 15/2017 “Ex ante conditionalities and performance reserve in Cohesion: innovative but not yet effective instruments”.
4 Article 19 of Regulation (EU) No 1303/2013 of the European Parliament and of the Council of 17 December 2013 laying down common provisions on the European Regional Development Fund, the European Social Fund, the Cohesion Fund, the European Agricultural Fund for Rural Development and the European Maritime and Fisheries Fund and laying down general provisions on the European Regional Development Fund, the European Social Fund, the Cohesion Fund and the European Maritime and Fisheries Fund and repealing Council Regulation (EC) No 1083/2006 (OJ L 347, 20.12.2013, p. 320).
5 General Ex ante conditionality 7, Annex XI Part II of Regulation (EU) No 1303/2013.
6 Articles 20, 22 and 96 and Annex II of Regulation (EU) No 1303/2013.
7 Articles 9 and 96 of Regulation (EU) No 1303/2013.
8 Annex I of Regulations (EU) No 1300/2013, No 1301/2013 and No 1304/2013, Article 5 of Regulation (EU) No 1301/2013, article 3 of Regulation (EU) No 1304/2013 and Article 96(2)(b)(ii) of Regulation (EU) No 1303/2013.
9 Special Report No 2/2017.
10 Article 48(3) of Regulation (EU) No 1303/2013.
11 Article 53(2) of Regulation (EU) No 1303/2013.
12 Article 9 of Regulation (EU) No 1303/2013.
13 See ECA’s SR No 2/2017.
14 Article 96(2)(b) of Regulation (EU) No 1303/2013.
15 Article 125(3)(a)(i) of Regulation (EU) No 1303/2013.
16 Article 125(3)(a) of Regulation (EU) No 1303/2013.
17 Projects Nos 27, 28, 30, 31, 32, 33 and 34.
18 Article 116 and Annex XII(3) of Regulation (EU) No 1303/2013.
19 Article 132 of Financial Regulation (EU, Euratom) No 966/2012, Articles 34 and 125 of Regulation (EU) No 1303/2013.
21 OP Piemonte, Italy, Call “Obbligo d’istruzione” 2015/2016.
22 Articles 72, 73 and 74 of Regulation (EU) No 1303/2013.
23 Article 125(2)(d) of Regulation (EU) No 1303/2013, this requirement is also part of the Management and control system, Article 72(d) of Regulation (EU) No 1303/2013.
24 Ex ante conditionality 7, Annex XI Part II of Regulation (EU) No 1303/2013.
25 Article 19 of Regulation (EU) No 1303/2013.
26 Article 122 of Regulation (EU) No 1303/2013.
27 Article 122(3) of Regulation (EU) No 1303/2013, their actual use by the beneficiaries is not obligatory.
28 EGESIF_14-0010-final 18/12/2014, Guidance for the Commission and Member States on a common methodology for the assessment of management and control systems in the Member States.
29 For instance information on the participant’s background: migrants, foreign background, minorities including marginalised minorities such as the Roma, participants with disabilities, other disadvantaged.
30 On the basis of 20 Commission’s audits on the reliability of performance data carried out mostly in 2017.
31 The MA managed to settle this later during 2017.
32 Article 50(2) of Regulation (EU) No 1303/2013.
33 Article 50(4) of Regulation (EU) No 1303/2013.
34 As well as on the Member States progress reports for 2017. Article 52 of Regulation (EU) No 1303/2013, to be submitted by 31 August 2017 and 31 August 2019 on the implementation of the Partnership Agreements as at 31 December 2016 and 31 December 2018. Progress reports present, namely, the “progress made towards the achievement of the Union strategy for smart, sustainable and inclusive growth, as well as of the Fund-specific missions referred to in Article 4(1), through the contribution of the ESI Funds to the thematic objectives selected, and in particular with regard to the milestones set out in the performance framework for each programme, and to the support used for climate change objectives”.
35 See Articles 75 and 127(4) of Regulation (EU) No 1303/2013 and Annex VII of Commission Implementing Regulation (EU) 2015/207 of 20 January 2015 laying down detailed rules implementing Regulation (EU) No 1303/2013 of the European Parliament and of the Council as regards the models for the progress report, submission of the information on a major project, the joint action plan, the implementation reports for the Investment for growth and jobs goal, the management declaration, the audit strategy, the audit opinion and the annual control report and the methodology for carrying out the cost-benefit analysis and pursuant to Regulation (EU) No 1299/2013 of the European Parliament and of the Council as regards the model for the implementation reports for the European territorial cooperation goal (OJ L 38, 13.2.2015, p. 1) and EGESIF_14-0010-final 18.12.2014, Guidance for the Commission and Member States on a common methodology for the assessment of management and control systems in the Member States - key requirement 6 and 15.
36 Articles 123 and 124 of Regulation (EU) No 1303/2013.
37 Article 124(2) and Annex XIII of Regulation (EU) No 1303/2013.
38 Performance indicators are a set of indicators defined for each priority axis in an OP, which values at the end of 2018 will be compared by the Commission with that of set milestones in order to decide upon the release of the performance reserve in 2019.
39 Milestones are the values set for the indicators of the performance framework to be reached by the end of 2018.
40 See Article 30(1) CPR, CPR Annex II and Article 5(6) of Regulation (EC) No 215/2014.
41 Article 56 of Regulation (EU) No 1303/2013.
42 Projects 27, 28, 31, 32, 33 and 34.
43 Projects 6, 7, 8 and 9.
44 E.g. AR 2015 - Chapter 6 paragraph 6.86 “Result indicators were not defined for 38 % of the projects”. Or AR 2016 - Chapter 6 paragraph 6.56 “For 42 % of the projects it was not possible to identify and measure a specific contribution to the overall programme objectives, since no result indicators or targets were defined at project level”.
45 SR No 15/2017.
46 COM(2017) 755 final of 13 December 2017 “Strategic report 2017 on the implementation of the European Structural and Investment Funds”.
47 See SR No 2/2017, paragraph 150.
48 See SR No 2/2017, paragraph 131.
49 See SR No 2/17, paragraph 150.
50 See SR No 2/17, recommendation 3.
|Adoption of Audit Planning Memorandum (APM)/Start of audit||18.1.2017|
|Official sending of draft report to Commission (or other auditee)||3.5.2018|
|Adoption of the final report after the adversarial procedure||27.6.2018|
|Commission’s (or other auditee’s) official replies received in all languages||23.7.2018|
The ECA’s special reports set out the results of its audits of EU policies and programmes, or of management-related topics from specific budgetary areas. The ECA selects and designs these audit tasks to be of maximum impact by considering the risks to performance or compliance, the level of income or spending involved, forthcoming developments and political and public interest.
This performance audit was carried out by Audit Chamber II Investment for cohesion, growth and inclusion spending areas, headed by ECA Member Iliana Ivanova. The audit was led by ECA Member Ladislav Balko, supported by Branislav Urbanič, Head of Private Office and Zuzana Frankova, Private Office Attaché; Myriam Cazzaniga, Principal Manager; Pekka Ulander Head of Task; Michaela Binder, Seconded National Expert.
EUROPEAN COURT OF AUDITORS
12, rue Alcide De Gasperi
More information on the European Union is available on the internet (http://europa.eu).
Luxembourg: Publications Office of the European Union, 2018
|ISBN 978-92-847-0548-1||ISSN 1977-5679||doi:10.2865/390387||QJ-AB-18-018-EN-N|
|HTML||ISBN 978-92-847-0523-8||ISSN 1977-5679||doi:10.2865/439457||QJ-AB-18-018-EN-Q|
© European Union, 2018.
For any use or reproduction of photos or other material that is not under the European Union copyright, permission must be sought directly from the copyright holders.
GETTING IN TOUCH WITH THE EU
All over the European Union there are hundreds of Europe Direct Information Centres. You can find the address of the centre nearest you at: https://europa.eu/european-union/contact_en
On the phone or by email
Europe Direct is a service that answers your questions about the European Union. You can contact this service:
- by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls),
- at the following standard number: +32 22999696, or
- by email via: https://europa.eu/european-union/contact/write-to-us_en
FINDING INFORMATION ABOUT THE EU
Information about the European Union in all the official languages of the EU is available on the Europa website at: https://europa.eu/european-union/index_en
You can download or order free and priced EU publications: https://publications.europa.eu/en/publications. Multiple copies of free publications may be obtained by contacting Europe Direct or your local information centre (see https://europa.eu/european-union/contact_en)
EU law and related documents
For access to legal information from the EU, including all EU law since 1951 in all the official language versions, go to EUR-Lex at: http://eur-lex.europa.eu
Open data from the EU
The EU Open Data Portal (http://data.europa.eu/euodp/en/home?) provides access to datasets from the EU. Data can be downloaded and reused for free, both for commercial and non-commercial purposes.