Special Report
No28 2018

The majority of simplification measures brought into Horizon 2020 have made life easier for beneficiaries, but opportunities to improve still exist

About the report The simplification of EU research and innovation funding has been on the agenda for many years, with industry, academia and researchers across Europe seeking simpler administrative rules, better communication with the European Commission, legal certainty and consistency. With Horizon 2020, the Commission introduced changes designed to simplify the rules in this area. Our audit examined whether these changes have been effective in reducing the administrative burden for beneficiaries. We concluded that the majority of the simplification measures have been effective, although not all actions produced the desired result and opportunities to improve still exist. Beneficiaries need more user-friendly guidance and tools, and the Commission has to further test the appropriateness and usability of new funding schemes. Stability in the rules is also important and, while beneficiaries are able to adapt to complexity, frequent modifications to guidance can cause confusion and uncertainty.

This publication is available in 23 languages and in the following format:
PDF
PDF General Report

Executive summary

I

Horizon 2020 is the eighth EU framework programme for research and innovation. With a budget of 76.4 billion euro for the period 2014 to 2020, it ranks as the world’s biggest public research and innovation programme.

II

Horizon 2020 has an impact on a wide range of EU policies and is managed by a number of Commission’s Directorates General, which adds to its complexity. In Horizon 2020, the Commission has simplified its rules and procedures, speeded up the “time to grant”, streamlined IT systems, reduced the number of funding schemes and provided clearer guidance and more legal certainty to beneficiaries.

III

We assessed whether the Commission’s simplification measures had reduced the administrative burden of beneficiaries. We examined whether the design of the measures was based on previous evaluations and feedback from stakeholders. We carried out a survey of beneficiaries of Horizon 2020 grants to assess their perceptions of the positive and negative impact of those measures in terms of reducing the administrative burden on them.

IV

We found that the majority of the Commission’s simplification measures have been effective in reducing the administrative burden for beneficiaries in Horizon 2020, although not all actions produced the desired result and opportunities to improve still exist. Stability in the rules is also important; beneficiaries are able to adapt to complexity but frequent modifications to guidance can cause confusion and uncertainty.

V

We found that the Commission drew on its experience of managing previous framework programmes to identify where simplification was needed. New organisational and horizontal structures, notably the creation of the Common Support Centre (CSC), were a major contribution to simplification. This led to more coherent implementation of the programme.

VI

The Research Enquiry Service, providing advice and support to applicants and participants, was not incorporated into the CSC. Although improved since FP7, there are other channels answering queries, meaning that consistent treatment of cases could not be guaranteed. National Contact Points also offer varying levels of support and guidance.

VII

Support tools, such as the Participant Portal, were improved. The introduction of electronic signatures led to simplified grant award and management for applicants and beneficiaries. However, some technical improvements are still required. Similarly, although the grants manual offers comprehensive and detailed information, it is difficult to navigate, especially for inexperienced beneficiaries.

VIII

The “time to grant”, from application to signature of a grant agreement, has been reduced significantly thanks to electronic management and withdrawal of the negotiation stage, but the opportunities to reduce the administrative burden have not been fully exploited. Only a fraction of calls for proposals employs two-stage evaluation. This has an impact in particular on unsuccessful applicants. Moreover, the Seal of Excellence, intended to help the best of the unsuccessful proposals find funding elsewhere, has not yet been effective.

IX

Attempts to simplify rules on personnel costs have not had the intended results and these remain complex for beneficiaries, leading to errors in cost declarations. Some negative side-effects emerged from the new approaches adopted, and subsequent adjustments introduced have caused some confusion and legal uncertainty. Increasing reliance on usual cost accounting practices, in particular for personnel costs, could reduce the administrative burden. New simplified cost options, such as lump sums and inducement prizes, have the potential to reduce the administrative burden, but they have not yet been sufficiently tested by the Commission.

X

The creation of a Common Audit Service and a new audit strategy have reduced the audit burden compared to FP7, but variable quality in outsourced audits can lead to frustration and confusion for beneficiaries.

XI

We recommend that the Commission should:

  1. better communicate with applicants and beneficiaries;
  2. intensify testing of lump sums;
  3. explore greater use of two-stage proposal evaluations;
  4. re-examine remuneration conditions for expert evaluators;
  5. increase recognition of the Seal of Excellence;
  6. ensure stability for rules and guidance for participants;
  7. improve quality of outsourced audits;
  8. further simplify tools and guidance for SMEs.

Introduction

What is Horizon 2020?

01

Horizon 2020 is the eighth EU framework programme for research and innovation. With a budget of 76.4 billion euro1 for the period 2014 to 2020, it ranks as the world’s biggest public research and innovation programme.

02

Horizon 2020 has its origins in the priorities of the Europe 2020 strategy. It is built around three main pillars and two specific objectives covering a large range of topics (see Figure 1). It provides funding to researchers, research institutes, universities, private companies (both large companies and SMEs) and public bodies either individually or in consortia, engaged in collaborative research projects.

03

The programme has a particular focus on SMEs. The European Commission targets the participation of SMEs, both in collaborative projects and through the new SME Instrument, which was designed specifically for highly innovative smaller companies.

04

The cross-cutting nature of research and innovation and its impact on a wide range of EU policies shapes the Horizon 2020 mode of management. The Commission’s Research and Innovation Directorate General (DG RTD) manages the programme together with eight other Directorates General2. Parts of the budget can be the responsibility of more than one DG. In all, 22 different bodies3 implement the Horizon 2020 budget.

Figure 1

Horizon 2020 architecture

Source: ECA.

A history of EU research and innovation funding

05

European funding for research activities began under the first Community treaties and was extended in 1983 with the establishment of the “first Community framework programme for research” (FP1).

06

Since then, successive framework programmes have become a major part of research cooperation in Europe, growing progressively in size, scope and ambition. Figure 2 shows the evolution of European research funding since the first programme.

Figure 2

Evolution of research funding 1984-2020

Source: ECA.

07

As the eighth programme, Horizon 2020 is an important implementing tool for the Innovation Union, one of seven flagship initiatives of the Europe 2020 strategy4. Horizon 2020 combines all research and innovation funding previously provided through the Seventh Framework Programme (FP7), the innovation-related activities of the Competitiveness and Innovation Framework Programme (CIP) and the European Institute of Innovation and Technology (EIT).

Summary of the most recent simplification initiatives

08

The simplification of EU research and innovation funding has been on the agenda for many years, with industry, academia and researchers across Europe seeking simpler administrative rules, better communication with the European Commission, legal certainty and consistency. The Commission has been criticised for its unclear guidance to beneficiaries, lengthy time to grant, complex funding rules and ineffective grant management procedures.

09

With Horizon 2020, simplification became a central aim5. The European Commission proposed changes designed to simplify the rules governing EU research and innovation funding.

10

These changes included: establishing a single set of rules, harmonising the grant management processes and streamlining the supporting IT systems, reducing the number of programmes, increasing coherence and clarity of rules, setting clearer priority objectives and indicators, providing more coherent cost-eligibility rules, simplifying forms of grants and reduced times to grant and pay (see Figure 3).

Figure 3

Principal Horizon 2020 simplification measures

Source: ECA.

Audit scope and approach

Audit scope

11

This special report is the latest in a series of ECA publications concentrating on the Framework Programme for research and innovation. This includes a forward-looking briefing paper “A contribution to simplification of EU research programme beyond Horizon 2020” published in March 2018. The ECA decided to carry out this audit in time for the preparation of the next innovation and research Framework Programme.

12

In this audit, we asked “Have the Horizon 2020 simplification measures taken by the Commission been effective in reducing the administrative burden of beneficiaries?”. To answer this question, we examined whether the Commission’s measures were designed on the basis of previous evaluations and feedback from stakeholders. We surveyed beneficiaries of the Horizon 2020 programme to assess the positive and negative impact of those measures.

Approach

13

We examined information from a wide range of sources:

  1. We reviewed and analysed legal bases, guidelines, evaluation and monitoring reports, position papers and other documentation with a bearing on simplification;
  2. We discussed the simplification measures with the relevant Commission departments6 and representatives of three umbrella organisations7; we also attended a meeting of the national contact points and a stakeholders’ meeting organised by the Common Support Centre.
14

In February 2018 we sent an online survey to 32 918 contacts from 20 797 organisations granted Horizon 2020 funding. The survey covered the period from the start of the programme in 2014 to January 2018 and comprised 59 questions. We asked the beneficiaries for their views on the effectiveness of the simplification measures, including, where applicable, how the programme compared with FP7. We received 3 598 replies. To obtain more detailed information to support the survey results, we conducted meetings with eight final beneficiaries (two SMEs, two universities, one large private enterprise and three RTOs).

Observations

The Commission drew on its experience of previous programmes to identify which simplification measures were needed

15

It is important, when preparing a new Framework Programme, to make a thorough analysis and evaluation of the predecessor programmes. The Commission should evaluate whether policies and spending activities are fit for purpose and have delivered the desired changes to European business and citizens. The results of the evaluation should help the Commission to decide whether EU actions can be continued unchanged or need modification.

16

The legal base for FP7 required the Commission to continually and systematically monitor implementation of the Programme8. We found that the Commission had prepared annual monitoring reports during the implementation of FP7 in which it regularly analysed, in detail, participation patterns, implementation issues and the situation with regard to the simplification process.

17

In line with the interinstitutional agreement on better law-making9, the Commission also sought feedback from key stakeholders via a range of consultation channels (see Box 1).

18

The Commission summarised stakeholders’ input to a wide public consultation in its “Green Paper on a Common Strategic Framework for EU Research and Innovation Funding”. The green paper identified simplification as a “top priority in order to make EU research and innovation funding generate more impact and be more attractive to participants”10.

19

In the impact assessment accompanying the Horizon 2020 set of legislative proposals11, the Commission defined the need for further simplification in research and innovation funding, set specific and operational objectives for the next programming period, analysed four policy options and eventually proposed Horizon 2020 as the most appropriate option to achieve the objectives set.

20

We conclude that the simplification measures undertaken by the European Commission in Horizon 2020 are the result of an analysis of previous framework programmes, in particular its immediate predecessor FP7.

Box 1

Consultations with stakeholders and interested parties in preparation for Horizon 2020

  • FP6 ex-post evaluation, February 2009;
  • FP7 interim evaluation, November 2010;
  • Public consultation on Green Paper describing the Common Strategic Framework for Research and Innovation, June 2011;
  • Public consultation on the successor to the Competitiveness and Innovation Framework Programme (CIP), November 2011;
  • CIP: interim and final evaluations, ex-ante evaluations and impact assessment studies for the ICT-PSP, IEE and innovation-related parts of the Entrepreneurship and Innovation Programme;
  • Large stakeholder conferences on the successor to the CIP (January 2011) and the Common Strategic Framework (June 2011);
  • Expert panels and stakeholder conferences on the European Research Council, Marie Curie, EIT, etc.;
  • EU Presidencies: Lund conference on the future of EU research (Sweden, July 2009); FP7 interim evaluation conference (Hungary, February 2011);
  • Wide range of position papers on future EU research and innovation funding during EU budget preparations;
  • Thematic stakeholder consultations: ICT, transport, health, biotechnology, space;
  • Discussions with representatives of national administrations (CIP Joint Management Committees meeting, meetings of EIP Management Committee).

Source: European Commission.

The main changes to achieve simplification were through design, organisation and supporting tools

The Common Support Centre is a major step towards coherent implementation of Horizon 2020

21

Programme management is defined as the centralised coordinated management of a programme to achieve the programme’s strategic objectives and benefits12. The shared services concept is one way to gain such centralised coordinated management.

22

In our special report on FP713 we criticised the Commission’s management and found that the Research Clearing Committee, established by the managing Directorates-general to enhance legal certainty and equal treatment of beneficiaries, failed to identify all divergent practices in FP7 implementation.

23

The Commission streamlined the management approach for Horizon 2020 by setting up a new directorate, the Common Support Centre (CSC), in 2014. By centralising horizontal support services in research and innovation, the CSC ensures that all bodies in the Research and Innovation family (Commission DGs, executive agencies and joint undertakings) apply the Horizon 2020 legislation in a coherent manner.

24

The CSC’s tasks are presented in Figure 4.

Figure 4

Common Support Centre tasks

Source: ECA.

25

The Commission put in place a variety of tools to facilitate participation in Horizon 2020, increase awareness and strengthen consistent application of its rules. Some of these tools, such as the Participant Portal, the Annotated Model Grant Agreement (AMGA), the Horizon 2020 online manual and FAQs, fall under the direct responsibility of the CSC; others, such as the National Contact Points (NCPs), are for the Member States to appoint. The Research Enquiry Service and IT Helpdesk are mainly run by the Research Executive Agency (REA) and DG DIGIT respectively. The CSC only provides the second-level helpdesks for the subjects under its responsibility.

26

We asked the beneficiaries we surveyed whether they were satisfied with the support tools. Overall, respondents showed a high level of satisfaction, ranging from 45 % of respondents considering the Enterprise Europe Network to be fit for purpose, up to 86 % for the Participant Portal (see Figure 5).

Figure 5

Survey respondents’ satisfaction with support tools

Source: ECA survey, March 2018.

27

The CSC started its activity with clear objectives (see Figure 6)14. Since its establishment in 2014, it has made great efforts to improve the Participant Portal and has frequently updated and expanded the AMGA. A legal and financial helpdesk can now answer questions from beneficiaries (via the Research Enquiry Service) and other Commission departments. Another step towards simplification was the adoption of the Common Audit Strategy and a common ex ante control strategy.

28

The introduction of an electronic grant management workflow was among the most significant simplification achievements in Horizon 2020. Another positive element, especially appreciated by the NCPs, was the practice retained from FP7 of organising roadshows in the Member States, at which the CSC makes presentations to beneficiaries, NCPs and certifying auditors on various legal and financial issues.

Figure 6

Common Support Centre: objectives vs. achievements

Source: ECA.

29

Thus the CSC took the lead in the process of simplification. In the areas under its remit, it has harmonised the rules for participation, developed a new audit strategy and improved the IT tools for grant management and reporting. However, not all support services have been brought under CSC control, leading to a risk of inconsistent advice and interpretation being provided to beneficiaries. This issue is discussed below in respect of two communication tools.

Beneficiaries appreciate the communication and feedback channels but some still report inconsistent treatment and varying levels of service

30

Effective communication and a functioning feedback system are prerequisites for coherent application of the rules for participation and the consistent treatment of beneficiaries. A failure in communication can negatively impact a project. The CSC has several channels, outlined below, for communicating with, and receiving feedback from, beneficiaries on the functioning and implementation of the Framework Programme.

Research Enquiry Service
31

The Commission put the Research Enquiry Services (RES) in place during FP7 to facilitate communication with and effective feedback from the beneficiaries of research and innovation funding. RES replaced a system of 60 functional mailboxes and is accessible from the Participant Portal, but also through other channels, such as the Horizon 2020 Europa website. The tool plays the role of a helpdesk, where potential and current Horizon 2020 beneficiaries can seek methodological, technical and legal support throughout the life-cycle of their projects.

32

Currently, the Research Executive Agency manages RES. Part of the service is executed by the Europe Direct Contact Centre (EDCC)15. EDCC has a wider scope, answering queries about a range of EU policies, not only Horizon 2020.

33

The EDCC answers questions using information which can be found in publicly available sources, and more complicated questions are answered with input from the appropriate Commission services.

34

In our survey, we asked beneficiaries to evaluate the quality of RES. More than half of the respondents who expressed an opinion considered that RES is generally a useful and fit-for-purpose tool.

35

On the subject of RES, respondents highlighted two main issues: on one hand, the timeliness of the replies to beneficiaries, and on the other, the lack of thoroughness (some replies are taken directly from the Commission guidance).

36

We found that despite the CSC’s general aim to centralise common services, RES has a decentralised structure. In practice RES is supported by 36 different helpdesks hosted by a range of DGs and executive agencies. Questions that cannot be answered by the external contractor (56 % in 2017) are forwarded to the REA team or directly to one of the 36 helpdesks. Every month the REA team of three check all responses provided by the external contractor and the decentralised helpdesks for both quality and categorisation.

37

We also found that, in addition to the service provided by RES, certain executive agencies (e.g. EASME) use other functional mailboxes. Other departments (e.g. DG CNECT) reply via social media. In such cases, it is hard for either REA or the CSC to exercise any control over replies.

38

We used the survey to ask beneficiaries whether, if they had participated in more than one Horizon 2020 project, identical or equivalent situations had been treated inconsistently. 36 % of respondents replied that this was the case. The most common inconsistencies related to the interpretation of financial rules (20 %), reporting requirements (18 %) and project evaluations (17 %).

39

The Commission set up a dedicated RES back-office to handle notifications of possible inconsistencies. However, we found that 82 % of the respondents who had experienced cases of inconsistent treatment were not aware of this service. As a result the Commission is rarely informed about inconsistencies and cannot always prepare the appropriate remedial measures.

40

Although RES was improved with the programming of Horizon 2020, the alternative channels through which queries and replies can be exchanged means that consistency of information still cannot be guaranteed.

National Contact Points
41

The network of NCPs serves as an important conduit between the Commission and beneficiaries. The Commission considers it to be “the main structure for providing practical information and assistance to potential participants”16. It also enables the Commission to obtain feedback from beneficiaries. NCP support can be especially helpful for SMEs and newcomers to the programme, whose relative lack of expertise may threaten the success of their applications.

42

In order to ensure consistency in the support which NCPs offer to applicants and beneficiaries of research funding, in 2013 the Commission produced the “Minimum standards and guiding principles for National Contact Points”17. This document provides a common reference for all participating countries and defines the NCPs` core functions and the mechanisms for cooperation between them and the Commission.

43

The Member States and associate countries appoint NCPs. Although NCPs are not under the direct responsibility of the Commission, the Commission DGs in charge of different parts of the programme organise meetings with them to discuss policy developments, procedural aspects, and the content of calls for proposals. The CSC also organises meetings with legal and financial NCPs to present the latest developments.

44

In our survey, we asked beneficiaries about their experience of the NCPs. The overall opinion of the NCPs’ work was positive (71 % of respondents considered the NCP support to be fit-for-purpose), but the level of appreciation differed from one country to another. Dissatisfaction with the NCPs’ quality of service ranged from 4 % to 25 % in different Member States; most concerns related to their degree of preparedness and availability.

45

We conclude that, although beneficiaries are generally satisfied with their NCPs, there are significant differences between participating countries in terms of the technical support and guidance they provide, which reduces the usefulness of the network for beneficiaries.

The Participant Portal simplifies grant management for beneficiaries

46

The Participant Portal is the main interface between the Commission and the beneficiaries of research and innovation funding. It should facilitate grant application and management for applicants and beneficiaries. It represents a gateway to the various systems (proposal submission tool, project management tool, notification system) facilitating stakeholders’ participation and is intended to be the one-stop shop for efficient paperless grant management.

47

The Commission made its first release of the Participant Portal during FP7. However, at that time only a limited number of services were available for beneficiaries.

48

Since the start of Horizon 2020, the CSC has introduced new tools and functionalities which have improved the quality of Portal services. These include “partner search service” (for finding partners for collaboration on future projects), the possibility of filling out the forms in any browser, and a platform that is HTML5-compliant.

49

More than 85 % of the respondents to our survey were satisfied with the Portal and recognised the efforts made by the CSC to enhance its functions. In terms of accessibility, the current Portal is a considerable improvement over FP7, where each tool required separate access and log-ins. It has become a single reference point covering a large number of services. Beneficiaries also praised the introduction of an electronic signature function, which has considerably reduced the administrative burden in Horizon 2020.

50

However, we found that, some beneficiaries had experienced difficulties with system slowdown, navigation between a confusing or excessive number of screens, pop-ups, etc. These beneficiaries reported that such problems generated additional work and took time to resolve.

51

Moreover, newcomers to EU funding schemes, in particular SMEs, found it difficult to deal with the complexity of the Commission’s IT tools. For example in one of our visits, the beneficiary SME reported having experienced difficulties with the Portal and needing an external consultant to input all required information and documentation in the tool.

Commission guidance (AMGA) is comprehensive but difficult to use and frequent changes have led to uncertainty

52

Regulation (EU) No 1290/2013 establishing the rules for participation18 in Horizon 2020 requires the Commission to provide sufficient guidance and information to all potential participants at the time of publication of the call for proposals, in particular the applicable model grant agreement.

53

The Annotated Model Grant Agreement (AMGA) has one clear strength with significant impact on simplification: it focuses all guidance related to Horizon 2020 in a single document. In 750 pages, using examples of best practice, specific cases and exceptions, the AMGA explains each article of the General Model Grant Agreement (GMGA) and specific model agreements. In contrast, FP7 guidance was dispersed among separate rulebooks for each part of the programme. Some 77 % of the respondents to our survey considered the AMGA to be a fit-for-purpose tool.

54

We asked for views on the AMGA in our survey. Respondents said that the guidance had the advantage of being very comprehensive and detailed. However, they expressed concern about its excessive length and complexity and pointed to the difficulty of navigating between the different sub-sections of a document in pdf format.

55

Since the release of the initial version in December 2013, the MGA has been modified six times and the AMGA has been through 18 updates. Although informed about these changes, beneficiaries expressed a number of concerns about the consequences of this (see Figure 7). Nevertheless, most respondents to the survey and beneficiaries interviewed stated that they would find it easier to cope with such updates if they were scheduled at fixed intervals to cater for needs emerging during the implementation of Horizon 2020.

Figure 7

Impact of the frequent updates of MGA and AMGA

Source: ECA.

New initiatives with potential for simplification have not yet been fully tested and evaluated

Lump sums and prizes
56

Encouraged by the European Parliament19 and the Council20, before launching MFF 2014-2020 and Horizon 2020 the Commission started exploring the possibility of simplifying grant management through innovative, simplified cost options. The legal basis for the use of lump-sum funding and inducement prizes is given by the 2012 Financial Regulation21.

57

The use of simplified cost options under Horizon 2020 has been limited, with prizes being used to support a small number of initiatives22 and lump sums being used to support: (i) SME Instrument 1 projects, (ii) a restricted number of accompanying measures and (iii) two pilot calls23 for multi-beneficiary projects and the special needs support under the Marie Skłodowska-Curie Actions included in the 2018-2020 work programme.

58

As of September 2018, the two lump sum pilots were still ongoing and conclusive evidence was not yet available from the analysis of full project life cycles.

59

Through our survey and during on-site visits, we asked Horizon 2020 beneficiaries and umbrella associations if they would welcome a more extensive use of these simplified cost options and what they perceived to be the major advantages and drawbacks.

60

The answers to our survey reveal that these simplified cost options are seen as a viable alternative to traditional cost reimbursement. Figure 8 shows that 74 % of respondents would welcome the wider use of lump sums, while 49 % favour the introduction of inducement prizes.

Figure 8

Opinions on the more extensive use of lump sums and inducement prizes

Source: ECA survey, March 2018.

61

The perception of simplified cost options varies according to the type of beneficiaries and is in particular dependent on their level of experience24. More experienced beneficiaries are less inclined towards the wider adoption of both lump sums and inducement prizes (see Figure 9).

Figure 9

Opinion on a more extensive use of lump sums and inducement prizes, by level of experience (number of Horizon 2020 projects)

Source: ECA survey, March 2018.

62

Notwithstanding the generally positive view, beneficiaries expressed some concerns about lump sums and prizes, highlighting perceived risks and pointing to the fact that these simplified cost options might not be suitable for all types of projects. These concerns were shared by the beneficiaries and NCPs we interviewed (see Annex I).

63

Simplified cost options could reduce the administrative burden and the occurrence of errors for Horizon 2020 grants, enabling beneficiaries to focus on scientific goals while managing projects in a more flexible way. However, they may not be suitable for all types of research and innovation projects, and the Commission does not yet have sufficient evidence from its pilot testing.

64

As stated in our March 2018 briefing paper, funding schemes based on lump-sum payments, if properly designed, could also encourage a greater participation by all types of beneficiary groups (including SMEs and new entrants).

Cascade funding
65

Cascade funding works by providing a mechanism for grant beneficiaries to distribute Horizon 2020 funds to third parties in the form of sub-grants (following a call for proposal) or prizes. This shifts the administrative burden from the Commission to the beneficiary responsible for managing the calls for sub-grants.

66

The Horizon 2020 beneficiary or consortium publishes its own calls for proposals to attract specific groups of potential beneficiaries, especially start-ups and SMEs, and provides grants ranging from 50 000 euro to 150 000 euro per third party. Cascade funding was piloted under FP7; under Horizon 2020 the European Commission is making increasing use of this mechanism.

67

In our survey, we asked whether using cascade funding under Horizon 2020 had resulted in a lower administrative burden compared to FP7. A small majority (52 %) of respondents who expressed an opinion replied that cascade grants had little or no effect in this regard, whereas 39 % agreed that the burden was lower compared to FP7. Figure 10 shows how the perception of cascade funding varied with experience: 23 % of respondents involved in more than ten Horizon 2020 projects considered that cascade funding had reduced their administrative burden, whereas for 26 % administration had increased.

Figure 10

Cascade grants reduced the administrative burden compared to FP7, by level of experience (number of Horizon 2020 projects)

Source: ECA survey, March 2018.

68

We found that the guidance issued by the Commission25 has not provided enough clarity to beneficiaries on how to manage the calls for cascade grants. In addition, the interviewed beneficiaries were uncertain about the type of regulation they had to apply when designing their call for proposals and when providing grants to third parties.

69

The Commission has not assessed the cascade funding mechanism ex post to determine whether it has effectively and efficiently achieved the intended outcomes.

Obtaining a grant is faster but opportunities to reduce administrative burden have not been fully exploited

70

A balance needs to be struck between, on the one hand, minimising the time, effort and money spent in submitting a proposal and, on the other, presenting enough information to allow the Commission to compare applications and identify the best proposals for funding. Simplification of the proposal procedure is especially important given the low success rate for applications, where only one in eight proposals receives funding26.

Figure 11

Proposal preparation effort in Horizon 2020 compared to FP7, by funding scheme and role in project

Source: ECA survey, March 2018.

71

In our survey we asked beneficiaries to compare FP7 and Horizon 2020 in terms of the administrative workload of preparing proposals. Among the respondents able to make a comparison (see Figure 11), 30 % reported a greater workload, 20 % a lower workload, and around half reported no difference at all. SMEI-1 beneficiaries reported the greatest improvement, with half of these respondents considering their current burden to be lower or much lower than for FP7. However, SMEI-1 was a new instrument for Horizon 2020, and in FP7 the “Research for the benefit of SMEs” action appeared under a different funding scheme. Project coordinators reported a larger increase in workload than other beneficiaries; this is a consequence of the move to centralise project coordination and interaction with the Commission at their level under Horizon 2020.

Withdrawal of negotiation stage has speeded up the time to grant

72

One of the changes introduced under Horizon 2020 was the withdrawal of the ‘negotiation stage’, which had previously taken place between the selection of a proposal for funding and the signing of the grant agreement. In Horizon 2020, projects are to be implemented in the form proposed.

73

The respondents to the survey and the interviewed beneficiaries were broadly in favour of the no-negotiation approach. At the same time, a minority considered that removing negotiations limited the possibility of improving projects and increased the likelihood of amendments to a grant agreement after signature.

74

The purpose of removing the negotiation stage was to accelerate the launch of funded projects. The indicator “time to grant” is defined as the time elapsed between the close of a call and the signing of the grant agreement, marking the official start of the project. The Horizon 2020 rules for participation provide for a maximum time to grant of eight months27. Reducing the length of this process is important both for unsuccessful applicants, which will need to make alternative arrangements as quickly as possible, and for successful ones, which may be trying to bring a product to market ahead of competitors.

Figure 12

Average time to grant in days, by programming period

FP6 (2002–2006)1 FP7 (2007–2013)2 Horizon 2020 (2014–2017)3
Time to grant 347 309 191

1 Average number of days from call deadline to grant signature.

2 Average number of days from call deadline to grant signature (except ERC).

3 Average number of days from call deadline to grant signature (except ERC).

Source: European Commission.

75

Withdrawal of the negotiation stage, together with the increased use of electronic document transmission and signature, has significantly shortened the time to grant compared to FP7 and FP6 (see Figure 12).

Wider use of the two-stage approach could reduce costs for the large number of unsuccessful applicants

76

In accordance with Article 15(5) of the rules for participation28, Horizon 2020 employs both one and two-stage evaluation processes. In the two-stage approach, coordinators submit short proposals for initial evaluation, with successful applicants then invited to submit a full proposal. After a second evaluation, a ranked list is forwarded to the Commission for selection. In Horizon 2020, around 10 % of all full proposals have arrived via the two-stage approach29.

77

The two-stage approach can help applicants avoid devoting unnecessary effort to developing detail in ultimately unsuccessful project proposals. However, for some projects the two-stage process may be too slow, with the risk that products arrive on the market later than those of competitors. The Commission estimates that a second stage of evaluation adds around three months to the process, as well as creating additional administrative costs for the Commission.

78

The respondents to our survey generally considered that a two-stage evaluation would reduce the overall workload of submission, with public bodies showing strongest support (see Figure 13). Some of the beneficiaries we interviewed felt that much of the work of preparing a proposal lay in setting up a consortium and developing the basic idea in enough detail to minimise the risk of rejection, rather than in writing the proposal itself; for them, a two-stage approach would not make much difference to the workload. Others expressed the concern that stage one approval would be delayed, making preparations for the second stage necessary in any case and negating any potential resource-saving.

Figure 13

Preference for one- or two-stage evaluations, by type of respondent

Source: ECA survey, March 2018.

79

Many applicants invest a lot of time and money in developing ultimately unsuccessful proposals. The beneficiaries we consulted generally agreed that this problem could be reduced by expanding the use of two-stage evaluations, but only where any associated delays in time to grant did not create other disadvantages.

Concerns remain about the quality of evaluations and feedback to unsuccessful applicants

80

High-quality evaluation of proposals made against appropriate criteria, although only indirectly linked to simplification, is essential to help ensure that the best projects are funded and the aims of Horizon 2020 achieved. Unsuccessful applicants should be given informative feedback with reasons for the outcome to assist them with future proposals.

81

Admissible and eligible proposals are evaluated against set criteria by at least three independent experts. Each evaluator awards scores for excellence, impact and quality and efficiency of implementation, based on award criteria published in the Horizon 2020 work programme. The “panel of evaluators” produces a summary report on all the proposals received; the Commission then uses this to draw up a list for funding.

Evaluations
82

The Research Executive Agency (REA) manages and contracts all independent experts involved in Horizon 2020, though selection is made by individual DG units (or Agency) for each call, drawing from the database of 120 000 registered experts (experts register themselves in the database and no endorsement, approval or quality check is implied). This database is established from an open-ended call for experts. The Commission is obliged to publish annually the list of evaluators used. Refreshment and rotation rules apply. For example, at least 25 % of experts for a call should not have been involved in the previous three years. REA also monitors the number of completely new evaluators.

83

In a DG RTD survey of evaluators30, 44 % of evaluators replied that they had been compensated for less time than they had actually needed and spent. Allocating insufficient time to an evaluation can substantially compromise its quality.

84

Most respondents to our survey of beneficiaries considered the evaluation criteria to be suitable, with only 17 % expressing a negative opinion. However, some beneficiaries questioned whether the experts had sufficient technical knowledge in the field. The Commission’s own mid-term evaluation of Horizon 202031 concluded that the quality of the evaluation process at the time should be improved.

85

Adequate remuneration is needed in order to recruit qualified evaluators so that proposals are correctly assessed and that the best proposals are selected and funded.

86

Experts are paid based on a rate per working day which has not been updated since 2007. They spend an average of 5.6 hours per single evaluation32, which we consider insufficient for an evaluator to read thoroughly and make a high-quality evaluation of a proposal.

87

As a result, there is a risk that insufficient remuneration dissuades qualified individuals from taking part in the evaluation process. Allocating insufficient time for the evaluation can have the same effect and/or compromise the quality of the evaluation.

Feedback
88

Feedback to applicants on the outcome of unsuccessful proposals should simplify their work when developing subsequent proposals and contributes to the overall quality of the Horizon 2020 programme.

89

Coordinators and participant contacts are informed about the evaluation outcome through a results letter. Though 46 % of survey respondents stated that quality of feedback had improved from FP7 to Horizon 2020 (see Figure 14), interviews with beneficiaries and comments received in our survey expressed concerns about the quality of feedback on evaluations. One in five survey respondents felt there had been a decline in quality between FP7 and Horizon 2020. The Commission’s interim evaluation of Horizon 202033 also reported that 34 % of respondents assessed feedback quality to be “poor” or “very poor”.

Figure 14

Opinions on evaluation feedback in Horizon 2020 compared to FP7

Source: ECA survey, March 2018.

90

A common complaint from beneficiaries, both to the Commission and in the survey comments, was that, having first been unsuccessful, they had resubmitted a proposal for a subsequent call, including changes based on evaluator feedback, but had received a lower score than before. However, the call may have been different from the previous one, or the original idea may have become less innovative over time.

The Seal of Excellence has not met expectations

91

The “Seal of Excellence” is a quality label awarded to project proposals which, having satisfied the selection and award criteria for Horizon 2020 funding and been ranked above a predefined quality threshold, could not ultimately be funded due to budget constraints. The aim is to help Seal of Excellence holders obtain funds for these proposals from other national, European or international programmes, without having the additional administrative burden of resubmitting proposals. The Commission launched the Seal of Excellence initiative in October 2015. The initiative was first piloted in the SME Instrument and in 2016 extended to Marie Skłodowska-Curie Individual fellowships.

92

In our survey, we asked recipients of a Seal of Excellence whether the label had helped them to obtain funds from other sources. We found that the Seal of Excellence was not universally recognised: only 15 % of respondents replied that it had helped them obtain another type of funding (see Figure 15).

93

We found that, despite the information campaign undertaken by the Commission, the lack of success of the Seal of Excellence is due to: (i) the lack of recognition by other funding institutions; (ii) the lack of clear guidance on how to use the label; and (iii) the lack of alignment between national and other EU programmes and Horizon 2020 regarding topics, selection and award criteria or evaluation processes.

94

In addition, at the time the Commission launched the Seal of Excellence, effective mechanisms of cooperation with other funding schemes had not yet been established and there was insufficient clarity on the application of state aid rules to the public funds supporting Seal of Excellence projects. It did not clarify the applicability of state aid rules until 201734.

95

The Commission collects country data on Seal of Excellence support schemes (see Figure 15). In our survey, we enquired about recipients’ access in each country to alternative sources of funding. The responses show that, all in all, access to other sources is still limited in the EU Member States.  The Commission does not have data on the number of Seal of Excellence projects funded by other programmes in each country.

Figure 15

Usefulness of the Seal of Excellence

Source: Eurostat.

96

As stated in our March 2018 briefing paper, the absence of a proper coordination mechanism between Horizon 2020 and other European and national programmes limited the impact of the Seal of Excellence in helping beneficiaries to find alternative sources of funding. It therefore failed to ease the burden of applying for research funding.

Reporting and auditing project costs

The rules on personnel costs were simplified, but some changes have created difficulties for beneficiaries and personnel costs remain principal major source of error

97

Personnel costs are a key cost category, accounting on average for approximately 45 % of the total costs of Horizon 2020 research projects.

98

In our survey and on-site visits, we asked Horizon 2020 beneficiaries whether they appreciated the simplification introduced under Horizon 2020 for the calculation and reporting of personnel costs, and whether there was still a need and scope to reduce the administrative burden in this area.

99

The majority of respondents approved of the changes made in Horizon 2020 regarding personnel costs. Figure 16 summarises the opinions they expressed about:

  • the changes to the overall methodology for calculating personnel costs;
  • the easing of time-recording requirements (e.g. abolition of time-recording obligations for staff working exclusively on a Horizon 2020 project);
  • the acceptance of monthly hourly rates alongside annual hourly rates;
  • the use of average personnel costs calculated by beneficiaries;
  • the use of unit costs for SME owners and natural persons without a salary;
  • the acceptance of supplementary payments of up to 8 000 euro per person per year for non-profit organisations.

Figure 16

The simplification measures introduced under Horizon 2020 reduced the administrative burden compared to FP7

Source: ECA survey, March 2018.

100

A closer look at the results of our survey shows that:

  • opinions depend on the experience of respondents with Horizon 2020 projects35, with more experienced beneficiaries generally showing less appreciation for the changes introduced;
  • when asked which measures introduced under Horizon 2020 have increased the administrative burden for the reporting of project costs, most of the respondents pointed to measures linked to the calculation and reporting of personnel costs;
  • when asked where simplification was most needed for FP9, respondents named the reporting of personnel costs.
101

Furthermore, 41 % of the respondents to our survey declared that they needed to run a specific time-recording system to manage their Horizon 2020 projects, with no significant reduction compared to FP7.

102

The most frequent complaints expressed by beneficiaries and other stakeholders (such as NCPs and associations) related to:

  • the use of detailed timesheets showing a split by work package;
  • the cumbersome recalculations that some beneficiaries have to make to reconcile staff salaries with Horizon 2020 rules;
  • the frequent introduction of changes throughout the implementation of Horizon 2020;
  • difficulties in applying the concept of additional remuneration.
103

Following the Commission’s first audits of Horizon 2020 projects, 68 % of all subsequent adjustments were linked to the improper reporting of personnel costs36. The vast majority (65 % of the total) concerned the calculation of personnel costs per se (notably, incorrect calculation of productive hours, incorrect remuneration costs, incorrect time claimed for a given activity, missing timesheets, incorrect additional remuneration), while the remainder (3 % of the total) were linked to personnel costs being reported as unit costs37.

104

70 % of respondents who expressed an opinion consider that, compared to FP7, Horizon 2020 rules on reimbursement of project costs rely more on the acceptance of usual cost accounting practices. However, the share of beneficiaries who do not see a difference in the level of acceptance of the usual cost accounting practices between FP7 and Horizon 2020 remains high.

105

Increasing reliance on usual cost accounting practices, in particular for personnel costs, could reduce the administrative burden.

The audit burden has decreased but beneficiaries face inconsistent treatment in outsourced ex post audits

106

The principles of control and audit of Horizon 2020 expenditure are defined in the Horizon 2020 regilation38. They aim to ensure (i) an appropriate balance between trust and control and (ii) an audit process in line with the principles of economy, efficiency and effectiveness in order to minimise the audit burden on participants.

Figure 17

The Commission’s financial control and audit strategy in Horizon 2020

Source: ECA.

107

In line with the ECA’s recommendations39, the Commission has centralised the design and implementation of the Horizon 2020 audit strategy through the Common Audit Service (CAS) using a single representative sample across Horizon 2020, and has taken a more risk-based audit strategy40.

108

In our survey, we asked beneficiaries to give their views about the Horizon 2020 control and audit burden. 53 % of respondents replied that the overall audit burden in Horizon 2020 had decreased compared to FP7, and only 14 % stated that it had increased. Two thirds of respondents also replied that the audit burden should be further reduced.

109

The beneficiaries we interviewed appreciated the new Common Audit Service as a means of ensuring consistency. However, some respondents to the survey complained that there was still no consistency in the control and audit process. In their view, inconsistencies may result from frequent changes to the AMGA, as auditors were sometimes unaware of retroactive amendments or did not know which version of the AMGA to apply.

110

The Commission has a framework contract with independent external audit firms for around 80 % of Horizon 2020 ex post audits, with the Commission itself performing the remaining 20 %. The Commission concluded the first ex post audits of Horizon 2020 projects in 2017.

111

Beneficiaries we met during our on-the-spot visits and at workshops criticised the work of some external audit firms. The most common complaints concerned the quality of work, the auditors’ limited knowledge of the programme and the time taken to produce audit reports. NCPs made the same criticisms at an event organised by the Commission41 and in interviews.

112

We found that the creation of a common audit service and the new audit strategy have reduced the audit burden for beneficiaries. However, we also found that there is room for improvement in the quality of outsourced ex post audits.

SME participation has increased but barriers remain

113

Supporting SMEs is one of the Commission’s priorities in the current programming period, and it has encouraged SMEs to participate in all areas of Horizon 2020. It designed the SME Instrument42, recently incorporated into the European Innovation Council pilot, to facilitate SME access to research and innovation funding.

114

Despite increased participation by SMEs in Horizon 2020, we found that they have experienced difficulties both during the application process and when implementing their research and innovation projects. SMEs that were new to EU funding schemes found it difficult to understand the regulatory requirements with regard to the grants. The length and comprehensiveness of the AMGA is particularly burdensome for SMEs who do not have dedicated staff to deal with the interpretation of EU guidelines and reporting.

115

We found that SMEs rely heavily on external consultants to overcome the difficulties they encounter in the preparation of their proposals. This is especially the case for single-beneficiary projects in SMEI phases 1 and 2 (see Box 2).

Box 2

Use of external consultants in proposal preparation and project reporting

  • 36 % of the respondents to our survey said that they required help from external consultants during the proposal preparation phase. The figure varied according to the respondents’ experience (less experienced beneficiaries used external support more often), the type of respondent and the funding scheme. SMEs operating alone are particularly reliant on external consultants, especially in SMEI phases 1 and 2 (see Figure 18).
  • The main reason for using external consultants to prepare proposal was to increase the chances of success, followed by the need to manage the administrative complexity of the process. More than half of respondents preferred to use external consultants rather than internal resources for this purpose, with a minority turning to consultants for scientific and technical support.
  • External consultants were less used during project implementation than for proposal preparation. Nevertheless, almost one third of SMEs stated that they made some use of external consultants for project management and reporting. The figure increased to approximately 50 % for single-beneficiary SMEs in SMEI phases 1 and 2.
  • Most respondents believed that the need for external consultants either remained the same or increased from FP7 to Horizon 2020, both for proposal preparation and for project implementation.
  • When asked about the fees paid to external consultants (as a percentage of total funding), the median value reported by the respondents was 5 % for proposal preparation and 5 % for project implementation.
  • The need for external consultants during the proposal preparation phase is a barrier to entry to Horizon 2020 for SMEs which cannot afford the related costs. This is particularly serious for newcomers since, given their lack of experience, they are especially in need of external help.

Figure 18

Use of external consultants to prepare proposals, by funding scheme

Source: ECA survey, March 2018.

Conclusions and recommendations

116

The majority of simplification measures taken by the Commission have been effective in reducing the administrative burden for beneficiaries in Horizon 2020, although not all actions produced the desired result and opportunities to improve still exist. Replying to our survey, beneficiaries expressed the need for more user-friendly guidance and tools and for further testing of the appropriateness and usability of new funding schemes. Stability in the rules is also important; beneficiaries are able to adapt to complexity but frequent modifications to guidance can cause confusion and uncertainty.

Organisational efforts to simplify

117

The simplification measures introduced by the Commission for Horizon 2020 were a result of its analysis of previous Framework Programmes (in particular FP7) and feedback from key stakeholders. The creation of the Common Support Centre was a major contribution to simplification, notably through the harmonisation of the rules for participation, a new audit strategy and IT solutions for grant management and reporting (see paragraphs 15 to 29).

118

Many applicants and beneficiaries still consider that they are treated inconsistently during the application process and the implementation of their projects. Despite the Commission’s efforts to improve communication, there is limited awareness of the dedicated back-office tool, established by the Commission for the reporting of inconsistent treatment. Moreover, we found that the work of the Research Enquiry Service (RES), in parallel with the other existing channels, is too fragmented to guarantee consistency (see paragraphs 30 to 40).

119

NCPs provide useful assistance to applicants and beneficiaries, helping to ensure consistent application of Horizon 2020 legislation, but we found significant differences in the level of technical support and guidance provided by the NCPs in the various Member States (see paragraphs 41 to 45).

120

The support tools developed by the Commission to facilitate the implementation of research and innovation programmes have undergone significant improvements since the beginning of Horizon 2020 and are appreciated by beneficiaries. However, there were some technical issues with the Participant Portal (see paragraphs 46 to 51), and the Commission manual on research and innovation (AMGA), which aims to provide comprehensive guidance, has evolved into a complex and long document. Frequent modifications of the AMGA added to the overall administrative burden and sometimes increased legal uncertainty for the beneficiaries (see paragraphs 52 to 55).

Recommendation 1 – better communication with applicants and beneficiaries

The Commission should improve its channels for communication with grant applicants and beneficiaries by:

  1. establishing better procedures and controls with regard to the performance of the helpdesk functions, and in particular of RES, and raising awareness of the tools through which beneficiaries can report inconsistent treatment during the application process or during the implementation of their projects.
  2. resolving the remaining technical issues affecting the Participant Portal, improving its design and facilitating navigation and the search function.
  3. work with Member States to improve the methodological and technical guidance to the NCPs so that they provide the necessary quality of service to potential beneficiaries of research and innovation funding.

These measures should be in place by the start of 2021.

121

Simplified cost options – lump sums and inducement prizes – have potential for reducing the administrative burden linked to the full life cycle of a Horizon 2020 project. A large majority of beneficiaries support further use of lump sums. However, they may not suit all types of research and innovation projects and evidence from pilot testing is necessary before deploying them on a larger scale (see paragraphs 56 to 69).

Recommendation 2 – intensify testing of lump sums

The Commission should intensify the testing of simplified cost options, and in particular lump sums, by:

  1. analysing and reporting on the outcome of the calls already launched under Horizon 2020 as soon as the first results are available;
  2. launching new pilot initiatives on a larger scale to identify the most suitable types of project, assess possible drawbacks and design appropriate remedies.

These measures should be put in place immediately.

Launching a project

122

Only 20 % of respondents to our survey considered that the Commission’s simplifications had actually reduced the administrative burden associated with applying for funding. Half of the respondents reported that the effort involved in developing a proposal was unchanged (see paragraphs 70 to 71).

123

Most respondents were satisfied that withdrawal of the negotiation stage had eased the burden, though a significant minority took the opposing view. The lack of formal negotiation was seen as speeding up the process, and fine-tuning remains possible in the grant agreement. Those unhappy with the removal of this stage considered that amendments to the grant agreement after signature were now more likely (see paragraphs 72 to 75).

124

Applicants regret the effort spent in developing long proposals which are found to be of good quality but not good enough to be funded. Wider use of two-stage calls would reduce the burden, according to respondents, but would only be suitable for those scientific fields and market sectors where the additional stage would not delay the arrival of project results on the market (see paragraphs 76 to 79).

Recommendation 3 – explore greater use of two-stage proposal evaluations

The Commission should identify a greater number of topics where the use of two-stage proposal evaluations could reduce the administrative burden for unsuccessful applicants, while maintaining the shortest possible time to grant if speed in reaching the market is critical.

This measure should be in place by the start of 2021.

125

Most respondents considered the evaluation criteria for assessment of proposals to be appropriate. However, some beneficiaries were concerned that experts might not have sufficient technical knowledge to ensure quality in evaluations, especially in multi-disciplinary projects. Furthermore, evaluators may not be given sufficient time to evaluate lengthy and challenging proposals and the daily remuneration rate has remained unrevised since 2007. A significant number of beneficiaries reported dissatisfaction with the quality of evaluation feedback, which could frustrate their efforts to improve subsequent proposals (see paragraphs 80 to 90).

Recommendation 4 – re-examine remuneration conditions for expert evaluators

The Commission should update the daily remuneration rate and reassess the time needed for experts to carry out reliable evaluations of project proposals.

This measure should be in place by the end of 2019.

126

The Seal of Excellence has had only limited impact at national level, as it is not universally recognised in the Member States. It has not adequately reduced the burden on applicants with good proposals (see paragraphs 91 to 96).

Recommendation 5 – increase recognition of the Seal of Excellence

The Commission should:

  1. establish proper mechanisms in the design of the next Framework Programme to facilitate the recognition of excellent research projects by the various EU and national funding schemes.
  2. work to build synergies between programmes so as to increase the likelihood that projects awarded the Seal of Excellence can more easily access other funding sources;
  3. produce appropriate guidance on how to use the Seal of Excellence.

These measures should be in place by the start of 2021.

Reporting and auditing project costs

127

The majority of respondent beneficiaries appreciated the measures introduced in Horizon 2020 to simplify the calculation and reporting of personnel costs. However, some negative side-effects arose, and adjustments made during the Programme’s implementation have caused some confusion and legal uncertainty. The calculation and reporting of personnel costs remains the main area in which further simplification would help beneficiaries by reducing administrative requirements. Increasing reliance on usual cost accounting practices, in particular for personnel costs, could reduce the administrative burden (see paragraphs 97 to 105).

Recommendation 6 – stability for rules and guidance for participants

The Commission should:

  1. maintain continuity in the rules for participation between Framework Programmes wherever possible;
  2. minimise adjustments to the guidance during implementation of the Framework Programme;
  3. simplify time-sheets to avoid unnecessary reporting of effort by work package;
  4. explore the possibility of more widely accepting the usual cost accounting practices, notably for personnel costs.

These measures should be in place by the start of 2021.

128

For Horizon 2020, the Commission put in place an audit strategy based on a more risk-driven approach than for previous framework programmes. We found that the new strategy has reduced the audit burden for beneficiaries. However, we also found that the quality of the ex post audits which are carried out by external audit firms could be improved. The ex post audit process is still considered too long (see paragraphs 106 to 112).

Recommendation 7 – improve quality of outsourced ex-post audits

The Commission should:

  1. improve its mechanisms for examining the quality of outsourced ex post audits, and;
  2. speed up such audits.

These measures should be in place for by the start of 2021.

SME participation

129

SME participation has increased in Horizon 2020 compared to FP7. However, we found that some SMEs have experienced difficulties both during the application process and when implementing their research and innovation projects. The practice of using external consultants is common in the preparation of proposals, notably for the SME Instrument and, to a lesser extent, project management and reporting. This practice has increased, moreover, from FP7 to Horizon 2020. One of the main reasons for seeking external support is to manage the complexity of the programme. The lengthy guidance created an additional burden for SMEs (see paragraphs 113 to 115).

Recommendation 8 – further simplify tools and guidance for SMEs

The Commission should further simplify its tools and guidance in such a way that they impose a minimal burden on SMEs, and especially on start-ups without the resources and staff to deal with their complexity. In particular, the Commission should consider issuing an abridged version of the guidance (AMGA) for SMEs and newcomers.

These measures should be in place by the start of 2021.

This Report was adopted by Chamber IV, headed by Mr Neven MATES, Member of the Court of Auditors, in Luxembourg at its meeting of 2 October 2018.

For the Court of Auditors

Klaus-Heiner LEHNE
President

Annex

The risks of simplified cost options, as perceived by beneficiaries

  • Lump sums and prizes might be more suitable for smaller, high-TRL (technologically mature) projects with clearly defined deliverables, less suitable for larger, more basic research projects, which are inherently riskier.
  • The evaluation of project achievements will be critical and the Commission will need to rely upon skilled sector-competent reviewers (often difficult to find).
  • Linking payments to the achievement of scientific results could mean beneficiaries receiving no funding for research work actually done.
  • Unless lump sums are indexed to national labour costs, project coordinators might be tempted to select project partners based on labour cost considerations rather than scientific excellence. This could create geographic imbalances in favour of countries where labour costs are lower.
  • Project and consortium management will be more complex. Tensions could emerge in project consortia (especially in large multi-beneficiary projects) due to higher financial interdependencies among project partners (e.g. the entire consortium might not receive funding if one partner underperforms). If not addressed properly, this issue could make well-established consortia more reluctant to involve new, potentially unreliable partners, in particular SMEs.
  • Prizes might favour certain players. Smaller ones will not have the resources to perform research projects without pre-financing or a reasonable assurance of being financed ex-post. Public bodies, such as many universities and research centres, cannot take the risk associated with failing to win a prize (after having carried out the research project) and cannot launch a research project without financial backing.

Commission’s replies

Executive summary

VIII

The Seal of Excellence has up to now more than 30 seal support schemes in place in 15 countries. The Commission accepts, however, that further development is necessary. See for details the Commission’s reply in paragraph 91.

X

The Common Audit Service has stepped up its efforts to ensure that all audits are carried out effectively. The external firms are closely monitored and controlled via meetings, training, daily instructions and guidance.

XI

The Commission accepts all the recommendations.

Observations

Common Commission reply to paragraphs 35 and 36.

The contractor replies in 3 working days and the reports 2016/2017 contain information on the time to reply per helpdesk (with average at 7.5 days).

The decentralised structure allows the most competent service to reply to the specific enquiry directly. The 36 thematic helpdesks ensure input from staff with the appropriate professional expertise in a particular area, depending on technical specificities and characteristics of the schemes. This set-up adds consistency and coherence.

In a very limited number of cases where incorrect information has been given to participants, these are corrected by the RES back-office in REA with a corrective e-mail sent to the participant.

66

On the basis of the Financial Regulation1, cascading grants may not exceed EUR 60.000 for each third party, unless it is necessary to achieve the objectives of the action. The Commission emphasises that the figures quoted in the text refer to a specific area of the work programme (ICT).

Common Commission reply to paragraphs 92 to 96.

The Seal of Excellence was a first concrete attempt to create concrete synergies with the ESIF. It has had a certain level of success, with 35 schemes put in place in 15 Member States. However, the Commission accepts that the impact has been limited.

The Commission has taken a number of steps in Horizon 2020 to improve the potential impact of the scheme:

  • Extension in 2016 to MSCA
  • Adoption in early January 2017 of an Explanatory note on the application of State Aid Rules relating to the Seal of Excellence
  • Establishment of a Community of practice, bringing together more than 200 funding bodies
  • Publication of guidance notes

Nevertheless, the constraints identified by the ECA, especially the freedom for national and ESIF programmes to choose whether to recognise or not the Seal, remains a problem.

The proposals for the next generation of programmes will further facilitate synergies between the EU Research Framework Programme, the ESIF and other EU programmes, and national and Regional Programmes. The experience from the Seal of Excellence has been a major contributor to these new provisions.

109

The Common Audit Service put in place a Quality Control Cell that review all the audit reports, at draft and final levels, and ensures consistency with the rules in force and with previous audit opinions on the same issues.

111

The Common Audit Service has put great emphasis on ensuring that audits are carried on a consistent basis and to a high quality. The external firms are closely monitored and controlled via meetings, trainings, daily instructions and guidance.

Common Commission reply to paragraphs 113 and 114.

The Commission notes that there has been a considerable increase in participation by SMEs, which shows the success of Horizon 2020 in attracting and supporting them.

In FP7, SME participation was 17% of relevant expenditure, with €4.9bn of EU contribution over the seven years. In H2020, by July 2018, SME participation was 24% of the relevant expenditure, well above the target of 20%, with an EU contribution of €5.6bn in just four years.

Conclusions and recommendations

118

On the Participant Portal, participants can report on an inconsistency issue through the web submit form of the RES.

The decentralised structure allows the most competent service to reply to the specific enquiry directly. The 36 thematic helpdesks ensure input from staff with the appropriate professional expertise in a particular area, This set-up adds consistency and coherence.

There is also a quality control operated by the Research Enquiry Service (RES) back-office at REA.

Recommendation 1- better communication with applicants and beneficiaries

The Commission accepts this recommendation.

Recommendation 2 – intensify testing of lump sums

The Commission accepts this recommendation.

123

In FP7 the average number of amendments per grant was 0.99. The preliminary figure for H2020 is 0.60. There is therefore no evidence that the “no-negotiation” approach has increased the likelihood of amendments to a grant agreement after signature.

Recommendation 3 - explore greater use of two-stage proposal evaluations

The Commission accepts this recommendation.

Recommendation 4 – re-examine remuneration conditions for expert evaluators

The Commission accepts this recommendation.

126

Today, there are 35 Seal of Excellence schemes operating in 15 countries. The Commission accepts that there is further potential to be exploited. Additional provisions have been included for the next generation of programmes. However, Member States will still have no obligation to recognise the seal of excellence, this is their own choice.

Recommendation 5 – increase recognition of the Seal of Excellence

The Commission accepts this recommendation.

Recommendation 6 – stability for rules and guidance for participants

The Commission accepts this recommendation.

128

See Commission replies to paragraphs 109 and 112.

Recommendation 7 – improve quality of outsourced ex post audits

The Commission accepts this recommendation.

Recommendation 8 – further simplify tools and guidance for SMEs

The Commission accepts this recommendation.

Glossary

Annotated Model Grant Agreement (AMGA): a user guide for applicants and beneficiaries that summarises and explains the terms of the General Model Grant Agreement (GMGA) and the various specific Model Grant Agreements for Horizon 2020.

Applicant: Person or organisation (individually or grouped) requesting EU funding.

Beneficiary: Person or organisation (individually or grouped) awarded EU funding in the form of a grant.

Briefing Paper: ECA’s Briefing Paper of March 2018, “A contribution to simplification of EU research programme beyond Horizon 2020”.

Call for proposals: Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding. There are several types of calls (with special submission or evaluation schemes): 2-stage calls; calls with multiple cut-off dates; and invitations to submit a proposal.

Cascade funding: A European Commission mechanism through which grant beneficiaries can distribute funds via sub-grants to third parties outside the consortium.

Common Support Centre: A Commission directorate centralising all support services in research and innovation.

Europe 2020: A ten-year strategy proposed by the European Commission on 3 March 2010 to advance the economy of the European Union (replacing the Lisbon Strategy for 2000-2010). The main objective is “smart, sustainable, inclusive growth” with greater coordination of national and EU policy.

Ex ante controls: Relate to preventive checks carried out either before grant signature or before payment.

Ex post controls: Relate to checks carried out after payment has been made.

Framework Programme (FP): The Framework Programme is the EU’s main instrument for research funding in Europe. The FP is proposed by the European Commission and adopted by Council and the European Parliament following co-decision procedure.

FP7: The Seventh Framework Programme.

H2020: The Eight Framework Programme.

Horizon 2020 dashboard: online presentation of implementation data of Horizon 2020, accessible via the Participant Portal.

Inducement prizes: a type of simplified cost option with financial contribution given as a reward following a contest.

Innovation Union: One of the seven flagship initiatives of the Europe 2020 strategy.

Lump sums: a type of simplified cost option with pre-determined amounts to be reimbursed when pre-defined project goals or milestones are reached, with no need for reporting of actual costs incurred by the participant.

Multi-annual financial framework (MFF): Multi-year EU budget, fixing the priorities and parameters for EU spending for consecutive periods of 5-7 years.

National Contact Point (NCP): A national structure set up in an EU Member State or another country participating in Horizon 2020 to give applicants personalised support on the spot and in their own languages.

Participant Portal: The Participant Portal is an internet platform for electronic administration of EU-funded research and innovation projects, which hosts the services for managing grant proposals and projects throughout their lifecycle.

Project officer: An EU official with responsibility for managing the EU’s interest in a project; the main contact point for beneficiaries. Normally follows a project from beginning to end, with a key role in all relevant decisions affecting the project (payments, amendments, reviews, etc.).

Research and Innovation family: Commission DGs, Executive Agencies (EAs) and Joint Undertakings (JUs).

Research Enquiry Service: An online service provided by the Commission to answer questions from applicants and beneficiaries about European research in general and about the validation process of legal entities.

Rules for participation: Specific funding rules for Horizon 2020, set out in Regulation (EU) No 1290/2013.

Small and medium-sized enterprises (SMEs): companies employing fewer than 250 staff and with an annual turnover not exceeding 50 million euro and/or an annual balance sheet total not exceeding 43 million euro (defined in Article 2 of the annex to Recommendation 2003/361/EC).

Technology readiness level: A measurement scale developed by NASA to assess the maturity of a particular technology. Used mainly to assess the readiness of individual technological components to operate in a larger technology system. On a scale from 1 to 9, TRL 1 corresponds roughly to basic research, TRLs 2-4 to applied research, TRLs 5-6 to applied research/development, TRLs 7-8 to demonstration and TRL 9 to full-scale deployment.

Time to grant: defined as the administrative period between submission of a proposal and signature of the grant agreement, marking the official start of a project.

Abbreviations and acronyms

AMGA: Annotated Model Grant Agreement

CIP: Competitiveness and Innovation Framework Programme

CSC: Common Support Centre

DG CNECT: Commission Directorate-General for Communications Networks, Content and Technology

DG RTD: Commission Directorate-General for Research and Innovation

EASME: Executive Agency for Small and Medium-Sized Enterprises

EARTO: European Association of Research and Technology Organisations

ECA: European Court of Auditors

EIT: European Institute of Innovation and Technology

EUA: European University Association

EP: European Parliament

FP7: Seventh Framework Programme

FP9: Ninth Framework Programme

H2020: Horizon 2020 (the Eighth Framework Programme)

IA: Innovation Action

JRC: Joint Research Centre

MGA: Model Grant Agreement

NCP: National Contact Point

REA: Research Executive Agency

RES: Research Enquiry Service

RTO: Research and Technology Organisation

SME: Small and medium-sized enterprises

SMEI: Small and medium-sized enterprise Instrument

TRL: Technology readiness level

UEAPME: European Association of Craft, Small and Medium-sized Enterprises

Endnotes

1 Updated H2020 budget after the establishment of the European Fund for Strategic Investments (EFSI) and including the Euratom Research & Training Programme.

2 DG RTD, DG CNECT, DG GROW, DG EAC, DG AGRI, DG MOVE, DG HOME, DG ENER and the JRC.

3 Commission executive agencies, public-public partnerships between the EU and Member States, public-private partnerships with industry, the European Investment Bank and the European Institute of Innovation and Technology (EIT).

4 http://ec.europa.eu/research/innovation-union/index.cfm

5 See recital 20 to Regulation No 1291/2013 of the European Parliament and of the Council of 11 December 2013 establishing Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) (OJ L 347, 20.12.2013, p. 104).

6 DG RTD, DG CNECT and REA.

7 EARTO, UEAPME, EUA.

8 Article 7 of Decision No 1982/2006/EC of the European Parliament and of the Council of 18 December 2006 concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007-2013) (OJ L 412, 30.12.2006, p. 1).

9 2003/C 321/01 in force until 12/04/2016 and replaced by 32016Q0512(01) (OJ L 123, 12.5.2016).

10 European Commission, “Green Paper on a Common Strategic Framework for EU Research and Innovation Funding”, June 2011.

11 SEC(2011) 1427 final of 30.11.2011.

12 Project management Institute, “A Guide to the Project Management Body of Knowledge (PMBOK Guide)”, Fourth edition.

13 Special Report No 2/2013 “Has the Commission ensured efficient implementation of the seventh framework programme for research?”.

14 Commission Decision C(2014) 2656 final on the operating rules for the Common Support Centre for Horizon 2020, the Framework Programme for Research and Innovation (2014-2020).

15 Under a contract with DG COMM.

16 http://ec.europa.eu/research/participants/data/support/20131125_NCP%20Minimum%20standards.pdf.

17 http://ec.europa.eu/research/participants/data/support/20131125_NCP%20Minimum%20standards.pdf.

18 Article 5 of Regulation (EU) No 1290/2013 of the European Parliament and of the Council of 11 of December 2013 laying down the rules for participation and dissemination in Horizon 2020 (OJ L 347, 20.12.2013, p. 81).

19 P7_TA (2010)0401, European Parliament resolution of 11 November 2010 on simplifying the implementation of the Research Framework Programmes (2010/2079(INI)).

20 Conclusions of the 3016th Competitiveness Council meeting, Brussels, 26 May 2010.

21 Regulation (EU, Euratom) No 966/2012 of 25 October 2012 on the financial rules applicable to the general budget of the Union (OJ L 298, 26.10.2012, p. 1).

22 The total value of prizes is approximately 0.11 % of the Horizon 2020 budget.

23 Call DT-NMBP-20-2018 (one-stage). Call SC1-BHC-15-2018 (two-stage).

24 Measured here as the number of Horizon 2020 projects that the beneficiary has been involved in.

25 Guidance note on financial support to third parties under Horizon 2020.

26 European Commission, “Horizon 2020 in full swing - Three years on - Key facts and figures 2014-2016”, December 2017.

27 Article 20 of Regulation (EU) No 1290/2013: “up to five months to inform applicants whether a proposal has been accepted, followed by a maximum of three months to sign the grant agreement”.

28 ”The Commission shall take into account the possibility of a two-stage submission procedure […], where appropriate and consistent with the objectives of the call for proposals.”

29 SWD(2017) 220 final– “In-depth interim evaluation of Horizon 2020”.

30 Evaluation of the operation of REA (2012-2015) Final Report, 2016.

31 SWD(2017) 221 final, “In-depth interim evaluation of Horizon 2020”, Annex 1.

32 SWD(2017) 220 final, “In-depth Interim Evaluation of Horizon 2020”, May 2017.

33 SWD(2017) 220 final “In-depth Interim Evaluation of Horizon 2020”, May 2017.

34 Commission Staff Working Document “Explanatory note of the Commission services on the application of State Aid Rules to national and regional funding schemes that offer alternative support to SME Instrument project proposals with a Horizon 2020 ‘Seal of Excellence’”, January 2017.

35 Measured as the number of Horizon 2020 projects that the beneficiary has been involved in.

36 The ECA also finds errors of this nature and reports on them in its Annual Reports.

37 Presentation Audits and Financial issues, NCP meeting, 14-15 March 2018.

38 Article 29 of Regulation (EU) No 1291/2013 establishing Horizon 2020.

39 See our special report 2/2013.

40 Horizon 2020 Ex post Audit Strategy, DG RTD 9.11.2015.

41 11th Meeting of the Horizon 2020 NCPs for Legal and Financial Matters, Brussels 14-15 March 2018.

42 Under the SME instrument companies can apply for two distinct non-related phases, depending on the technological maturity of their innovative projects. In phase 1 each project receives 50 000 euro for a feasibility study; if selected, in phase 2 each project can receive up to 2.5 million euro to finance innovation activities.

 

1 Regulation (EU, Euratom) No 966/2012 of 25 October 2012 on the financial rules applicable to the general budget of the Union (OJ L 298, 26.10.2012, p. 1).

Event Date
Adoption of Audit Planning Memorandum (APM) / Start of audit 12.12.2017
Official sending of draft report to Commission (or other auditee) 19.7.2018
Adoption of the final report after the adversarial procedure 2.10.2018
Commission’s (or other auditee’s) official replies received in all languages 31.10.2018

Audit team

The ECA’s special reports set out the results of its audits of EU policies and programmes, or of management-related topics from specific budgetary areas. The ECA selects and designs these audit tasks to be of maximum impact by considering the risks to performance or compliance, the level of income or spending involved, forthcoming developments and political and public interest.

This performance audit was carried out by Audit Chamber IV Regulation of markets and competitive economy, headed by ECA Member Neven Mates. The audit was led by ECA Member Alex Brenninkmeijer, supported by Raphael Debets, Head of Private Office and Di Hai, Private Office Attaché; Paul Stafford, Principal Manager; Daniela Hristova, Head of Task; Wayne Codd, Juan Antonio Vázquez Rivera and Marco Montorio, Auditors.

From left to right: Raphael Debets, Di Hai, Wayne Codd, Alex Brenninkmeijer, Paul Stafford, Daniela Hristova, Marco Montorio, Juan Antonio Vázquez Rivera.

Contact

EUROPEAN COURT OF AUDITORS
12, rue Alcide De Gasperi
1615 Luxembourg
LUXEMBOURG

Tel. +352 4398-1
Enquiries: eca.europa.eu/en/Pages/ContactForm.aspx
Website: eca.europa.eu
Twitter: @EUAuditors

More information on the European Union is available on the internet (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2018

PDF ISBN 978-92-847-0754-6 ISSN 1977-5679 doi:10.2865/198822 QJ-AB-18-020-EN-N
HTML ISBN 978-92-847-0718-8 ISSN 1977-5679 doi:10.2865/00954 QJ-AB-18-020-EN-Q

© European Union, 2018.

For any use or reproduction of photos or other material that is not under the European Union copyright, permission must be sought directly from the copyright holders.

GETTING IN TOUCH WITH THE EU

In person
All over the European Union there are hundreds of Europe Direct Information Centres. You can find the address of the centre nearest you at: https://europa.eu/european-union/contact_en

On the phone or by e-mail
Europe Direct is a service that answers your questions about the European Union. You can contact this service

FINDING INFORMATION ABOUT THE EU

Online
Information about the European Union in all the official languages of the EU is available on the Europa website at: https://europa.eu/european-union/index_en

EU Publications
You can download or order free and priced EU publications from EU Bookshop at: https://publications.europa.eu/en/web/general-publications/publications. Multiple copies of free publications may be obtained by contacting Europe Direct or your local information centre (see https://europa.eu/european-union/contact_en)

EU law and related documents
For access to legal information from the EU, including all EU law since 1951 in all the official language versions, go to EUR-Lex at: http://eur-lex.europa.eu/homepage.html?locale=en

Open data from the EU
The EU Open Data Portal (http://data.europa.eu/euodp/en/data) provides access to datasets from the EU. Data can be downloaded and reused for free, both for commercial and non-commercial purposes.