Special report
15 2022

Measures to widen participation in Horizon 2020 were well designed but sustainable change will mostly depend on efforts by national authorities

About the report:To tackle the innovation divide, Horizon 2020 introduced specific widening measures to support Member States lagging behind in research and innovation. We assessed whether these measures were fit for purpose.

We conclude that the widening measures were well-designed to address the limited participation of widening countries in R&I framework programmes, but sustainable change requires efforts at national level. The ability of the Policy Support Facility to induce these changes was limited.

The participation in the widening measures was uneven and the funded projects, whilst starting to show promising results, face challenges in terms of complementary funding and sustainability.

We recommend that the Commission strengthen the use of the Policy Support Facility, aim for a more balanced participation in widening measures, facilitate the availability of complementary funding, enhance their capacity to exploit project results and improve monitoring.

ECA special report pursuant to Article 287(4), second subparagraph, TFEU.

This publication is available in 24 languages and in the following format:
PDF
PDF Special report: Horizon 2020 tackling EU research - innovation divide

Executive summary

I In its Europe 2020 Strategy, the Commission highlighted the role of research and innovation as a key driver of social and economic prosperity and environmental sustainability in the EU. The EU has provided support for research and innovation since 1984 through consecutive “framework programmes”. Horizon 2020, the 8th Framework Programme, covered the 2014-2020 period (budget of €76.4 billion). It has now been followed by Horizon Europe (budget of €95.5 billion) for 2021‑2027.

II There are significant differences among EU Member States in terms of research and innovation performance. The divide has also been reflected in differing levels of participation in the framework programmes, for which the allocation of funding is based on excellence. Low participation in turn limits the potential of under-performing countries to overcome the shortcomings in their research and innovation systems. To address the innovation divide and improve participation, Horizon 2020 earmarked €935 million of funding for “widening measures” specifically targeting low-performing Member States. The aim was to enhance the capacity of research institutions in these Member States, for example by helping them to network, partner with leading institutions and attract highly-skilled staff. These measures have been reinforced under Horizon Europe and now have a budget of €2.95 billion.

III We decided to carry out this audit to inform policymakers about issues affecting the design and implementation of the widening measures, in particular as these measures are continuing under Horizon Europe. This special report is the latest in a series of ECA publications examining the support provided by Horizon 2020.

IV We assessed whether the Horizon 2020 widening measures were fit for purpose. To this end, we examined the measures’ overall design as well as the implementation and monitoring arrangements for two of them. These are Teaming (support for centres of excellence) and ERA Chairs (bringing outstanding academics to research institutions), both of which were designed to generate long-lasting effects at beneficiaries.

V We concluded that the widening measures were well-designed to target many of the factors that have limited the participation of widening countries in successive framework programmes. However, genuine sustainable change depends to a large degree on research and innovation investments and reforms at national level.

VI The Policy Support Facility is part of a set of instruments used by the EU to foster reforms in national ecosystems. Not all widening countries made use of the Facility in the 2014-2020 period, and the Commission had to delay some projects due to a lack of resources. We found that, while it was the Member States´ responsibility to react to the recommendations coming out of Policy Support Facility activities, the Commission also used the knowledge gained on national research and innovation systems, mainly for its country assessments in the context of the European Semester, which lead to country-specific recommendations.

VII We found that the widening measures have benefitted a group of countries with uneven levels of performance in research and innovation and in the framework programme. Although not necessarily fully correlated, these differences also reflect in the countries’ participation in the widening measures.

VIII While it is too early to assess the measures’ full impact, we found that Teaming and ERA Chairs projects are starting to produce promising results, specifically in terms of number of publications, networking and access to more grant funding. However, they faced a number of challenges in implementation, such as securing complementary funding from sources other than Horizon 2020, recruiting international researchers and, most importantly, ensuring sustainability by generating income from the exploitation of research results.

IX We found that there was room for improvement with regard to the monitoring of Teaming and ERA Chairs projects, in particular with regard to measuring the long-term impact of both instruments.

X We recommend that the Commission:

  • strengthen the use of the Policy Support Facility;
  • aim for a more balanced participation of widening countries in widening measures;
  • facilitate the timely availability of complementary funding;
  • enhance the capacity of project beneficiaries to exploit their research results; and
  • enhance its monitoring of the widening measures.

Introduction

Importance of research and innovation in the EU

01 In its Europe 2020 Strategy, the Commission set smart, sustainable and inclusive growth as an objective and highlighted the role of research and innovation (R&I) as a key driver of social and economic prosperity and environmental sustainability. The continued importance of R&I for the EU is also reflected in the Commission’s six priorities for 2019-2024 as R&I plays a key role in at least four: a European Green Deal, an economy that works for people, a Europe fit for the digital age and a stronger Europe in the world.

02 The EU 2020 Strategy also set the goal of increasing spending on research and development in order to attract private investment of up to two thirds of total investments, thereby reaching at EU level a cumulative total of 3 % of gross domestic product (GDP) by 2020. Member States established their own national targets ranging from 0.5 % to 4.0 %.

03 The EU budget has been supporting specific funding programmes to foster R&I since 1984. The 8th Framework Programme (FP) for R&I, Horizon 2020 (H2020), covered the 2014-2020 period and had a budget of €76.4 billion. The 9th FP, Horizon Europe, covers 2021-2027 and has a budget of €95.5 billion. Other EU funds also support R&I, notably the European Regional Development Fund (ERDF), one of the funds implementing cohesion policy.

04 The final evaluation of the 6thFP (2002-2006 period) and the interim evaluation of the 7th FP (2007-2013) both pointed to recurrent low levels of participation by certain Member States in the FPs. This low participation limits the involvement of their R&I actors in EU and international collaborative projects and communities.

05 The H2020 regulation recognised in 2013 that, “despite a recent tendency for the innovation performances of individual countries and regions to converge, sharp differences among Member States still remain”. As the allocation of funding under the FPs is based on excellence (i.e. only the best proposals are selected), lower performing Member States faced difficulties in competing with the stronger ones. The Commission first identified some reasons for this in 2011, such as insufficient national R&I investments and poor access to existing networks. The regulation therefore introduced a specific strand “spreading excellence and widening participation” (for the period 2014-2020). It was specifically earmarked for the low-performing countries now dubbed “widening countries”. The aim was to tackle uneven R&I performance among Member States and unlock excellence (i.e. activate existing excellence pockets and connect them to networks of excellence) in low-performing countries, thereby expanding participation in Horizon 2020.

06 The widening measures consisted of a range of instruments to support capacity-building, create links between leading research institutions and low-performing countries and regions and provide expert policy support (see Figure 1).

Figure 1 – Widening measures under Horizon 2020

Source: ECA.

07 A total of €935 million was committed to the widening measures (1.2 % of all H2020 commitments). The biggest share went to Teaming projects (€390 million, or 41 %, see Figure 2). The measures are continuing under Horizon Europe, but include new features and have a significantly larger allocation of €2.95 billion (3 % of the Horizon Europe budget) (see Annex I). On the basis of its interim evaluation of Horizon 2020 and impact assessment of Horizon Europe, the Commission had proposed increasing the budget to €2.1 billion; this was then further increased by the co-legislators as a result of the legislative process.

Figure 2 – “Spreading excellence and widening participation” commitments under H2020 (in million euros)

Source: ECA, based on Commission data.

08 Under H2020, the widening measures targeted: the thirteen Member States that have joined the EU since 2004 (the EU-13) plus Luxembourg and Portugal (Figure 3). The same countries are targeted by Horizon Europe, except that Luxembourg has been replaced by Greece. The outermost regions of France and Spain have also joined the group. The Horizon Europe regulation specifies that this group will be valid throughout the duration of Horizon Europe.

Figure 3 – Member States selected to benefit from the H2020 widening measures

Source: ECA.

Roles and responsibilities

09 The Commission's Directorate-General for Research and Innovation (DG RTD) is the main DG responsible for EU policy on research, science and innovation (including the design and management of the FPs). H2020 was implemented through multiannual work programmes, setting out the Commission’s priorities, which were subsequently translated into calls for project proposals.

10 The European Research Executive Agency (REA) manages all the widening measures (in particular the selection, implementation and monitoring of projects) except the PSF, which is managed by DG RTD and COST implemented by the European Cooperation in Science & Technology (COST) association.

11 Projects funded in the context of the widening measures may benefit from complementary funding from other sources, such as the ERDF. The ERDF is implemented on the basis of operational programmes prepared by national/regional authorities and approved by the Commission. The managing authorities in charge of the programmes concerned assess and select the projects eligible for complementary funding and monitor their implementation by beneficiaries.

Audit scope and approach

12 The objective of this audit was to shed light on the design of the widening measures and highlight issues which may impair their successful implementation, particularly as they have been reinforced under Horizon Europe. As the widening measures were new with H2020 and most of the widening projects are still ongoing, the audit did not focus on assessing the impact of the measures as it is only expected to fully unfold from the mid-2020s.

13 Our previous Special Reports on H20201 already pointed to the different levels of participation of EU Member States in the FPs. We decided to carry out this audit to address the widening measures newly introduced in the 8th FP and to complement our set of audits on H2020.

14 We focused on Teaming and ERA Chairs, the two widening measures that target long-lasting effects and on the role of the Policy Support Facility. This report does not assess synergies between H2020 and the funds implementing cohesion policy, currently planned to be covered in a later report.

15 We asked whether the H2020 widening measures were fit for addressing the R&I divide. To answer this question, we assessed whether:

  • the widening measures were well designed;
  • the widening measures were on course to achieve their objectives;
  • the Commission has an effective system for monitoring the impact of the widening measures.

16 We examined evidence from a range of sources:

  • a desk review of relevant documents;
  • questionnaires and interviews with the Commission, REA as well as managing authorities and national contact points for the widening measures in Croatia, Poland, Portugal, Romania, and Slovenia;
  • statistical analysis of H2020 data and data on R&I investments by Member States;
  • in-depth analysis of a judgemental sample of five ERA Chairs projects and of six Teaming projects – four financed under the first call (2016-2017 work programme) and two under the second call (2018-2020). The analysis of projects was based on data collected until mid-2021. See Annex II for further details of our methodology;
  • detailed analysis of complementary funding for all Teaming projects.

Observations

While the design of the widening measures was mostly appropriate they can only kick-start progress in R&I performance

17 The objective of the widening measures was to address the R&I divide and the recurrent low participation in the FPs by unlocking excellence in countries defined as being low-performing (see paragraph 05).

18 The Commission identified structural issues (e.g. weak governance and institutions and/or fragmentation of national R&I ecosystems and/or missing links between education and research) as one of the reasons why certain Member States lag behind in innovation performance. The EU has a set of instruments to spur reforms, amongst them the PSF (see paragraph 06).

19 We assessed whether:

  • the widening measures were designed to address the causes of low R&I performance;
  • the widening measures were coherent with other EU measures spurring reforms in Member States; and
  • the widening measures targeted the low-performing countries.

The causes of low R&I performance were mostly addressed by the measures but national commitment is necessary to increase investments in R&I

20 Several research papers2 have analysed the causes of low R&I performance of widening countries and their limited participation in H2020. The causes identified include such issues as: inadequate R&I ecosystems (fragmented governance or limited investments), weaker links to international collaboration networks, deficits in human capital, brain drain, limited collaboration between public and private bodies involved in innovation and limited internationalisation of R&I institutions.

21 We assessed whether the widening measures specifically targeted these causes. Our results are laid out in Table 1.

Table 1 – Causes of low R&I performance addressed by widening measures

Causes of low R&I performance Teaming ERA Chairs Twinning COST PSF
1. Ecosystem 1.1. Low national R&I investment          
  1.2. Limited private R&I investment          
  1.3. Structural issues affecting national R&I ecosystems (e.g. fragmentation of R&I governance) X X     X
  2. Limited access to international networks X X X X  
  3. Limited international visibility of universities and research centres X X X X  
  4. Limited collaboration between research centres and business X        
  5. Salary disparities / brain drain X X      
  6. Limited experience in previous FPs X X X    
  7. Limited synergies between the FPs and the funds implementing cohesion policy X       X

Source: ECA, based on H2020 regulation and work programmes.

22 All causes other than the levels of national (public and private) R&I investments were addressed by one or more of the widening measures. Nevertheless, widening measures did not focus on innovation and offered only limited support to the network of national contact points, which provide information and assistance with the FP to beneficiaries. In our special report 28/2018, we concluded that this support was needed. We take note that under Horizon Europe, the measures now include this type of support as well as support to address brain drain and foster excellence in universities and place-based innovation ecosystems (see Annex I).

23 Box 1 explains on how the two measures on which we focused our analysis of project implementation (Teaming and ERA Chairs) addressed the causes of the R&I divide.

Box 1

Explanations on how Teaming and ERA Chairs addressed the causes of the R&I divide

Teaming

Teaming projects (see Figure 1) involve a research organisation (“centre of R&I excellence”) in a widening country, acting as coordinator, and one or more institutions of renowned international excellence, which participate as project partners. This should provide the basis for overcoming weaknesses 2, 3 and 6 (see Table 1).

Supported centres of excellence are expected to commit to a strategic growth path in terms of economic development; they therefore have to include in the business plans submitted with their project proposal, how they will create links with business and exploit research results (weakness 4). They are also expected to contribute to a significant improvement in the national R&I ecosystem (weakness 1.3).

Each Teaming project can obtain a maximum of €15 million in H2020 funding, mainly to cover the cost of salaries and overheads (weakness 5). Public authorities or private entities must support the project through complementary funding at least equal to that provided by H2020. This funding can be provided via operational programmes, thus creating synergies between H2020 and the funds implementing cohesion policy (weakness 7).

ERA Chairs

ERA Chairs projects (see Figure 1) involve the provision of H2020 funding so that a research organisation in a widening country (the Chair “host”) can appoint a high-profile researcher (the Chair “holder”). The funding mainly covers the salaries of the Chair holder and a team of researchers established by them. The aim is to increase the host organisation’s attractiveness for international researchers, while also helping the host to implement structural changes, achieve excellence on a sustainable basis and apply more successfully for competitive funding. ERA Chairs thus address weaknesses 1.3, 2, 3, 5 and 6.

24 The H2020 widening measures did not specifically address the levels of national (public and private) R&I investments. In fact, the size of the budget of the widening measures (see paragraph 07) implies that these measures were primarily intended to have a catalytic effect on R&I performance in general and the participation in H2020 in particular.

25 Nevertheless, other EU measures exist to boost investment levels, both under H2020 or other EU programmes or through the European Semester process, i.e. the annual cycle of economic and fiscal policy coordination within the EU (see also our special report 16/2020 on the European Semester).

26 Moreover, Member States are primarily responsible for the level of public investments. R&I performance and participation in H2020 are strongly influenced by the level of national R&I investment (public and private) and the strength of the national R&I ecosystems. For more information on this correlation, see Annex III.

27 In 2020, the EU invested 2.3 % of its GDP in R&I. R&I investment rates vary greatly among the EU Member States (see Figure 4) but none of the widening countries invested as much as the EU average and only two (Slovenia and Czechia) invested more than 2.0 %. Moreover, at the end of 2020, most Member States, and more especially the widening countries, were still far from achieving their targets (see paragraph 02Error! No bookmark name given.), which in most cases were below the EU average (see Figure 4).

Figure 4 – National R&I investment as a percentage of GDP, end 2020, vs. Europe 2020 Strategy targets

Source: ECA, based on Eurostat data for actual figures, Europe 2020 Strategy for targets.

The Policy Support Facility’s ability to induce needed changes in national R&I systems was limited as it also depends on the degree of Member States’ commitment but feeds into other EU measures addressing reforms

28 For the 2014-2020 period, in response to the recognised need for reforms in Member States (see paragraph 18), the Commission adopted new instruments to encourage the reform process and help Member States tackle the underlying issues. Some of those instruments were introduced in the fields of research and cohesion policy. Under H2020, this was the aforementioned PSF (see paragraph 06).

29 We examined whether the PSF was coherent with other EU support measures and whether the Commission reacted appropriately to the results of its 2019 evaluation of the PSF.

30 The various services offered under the PSF and the number of projects financed in widening countries are shown in Box 2. The total budget was relatively small (just €5.7 million) and some projects started late owing to staff shortages at the Commission.

Box 2

PSF services and implementation of projects (2014-2020 period)

  • Peer review of national R&I systems: panels of experts and policy peers made in-depth assessments of national R&I systems and provided operational recommendations on the reforms needed to strengthen it. Eight projects implemented (including five in widening countries).
  • Mutual learning exercise: multiple countries exchanged good practices, lessons learned and success factors. 12 projects implemented (all but three widening countries participated).
  • Specific support activity: pools of experts provided tailored advice to individual countries and helped them to design or implement specific reforms relating to R&I (such as the start-up ecosystem, R&I governance or links between research and business). 14 projects implemented (including eight in widening countries).

31 Member States participated in the PSF activities on a voluntary basis. We found that, during 2014-2020, four widening countries (Croatia, Czechia, Luxembourg and Portugal) did not participate in the activities directly targeting reforms in the R&I systems (peer review and specific support).

32 In line with the subsidiarity principle, the participating countries decide themselves whether and how to take forward recommendations made in the context of these activities. The Commission’s 2019 evaluation of the PSF concluded that the Commission was putting “not enough emphasis on following-up and monitoring the actual policy changes”. It is therefore important to ensure that insights regarding the necessary national reforms are not lost but are used to good effect in the context of the European Semester. The European Semester work results in the Council annually approving country-specific recommendations based on a Commission proposal.

33 We analysed the annual country assessments that form the basis for the country-specific recommendations and found that the Commission did not start explicitly referring to the insights gained from PSF activities3 until 2019. All such references were selective, i.e. they contained the most important points emanating from the PSF.

34 Similarly, it was not until 2019 that the Commission started more systematically to address country-specific recommendations on R&I to Member States. Our analysis of the 2019 country-specific recommendations on R&I showed that these were general in nature and focused on investment rather than on reforms. Between 2015 and 2018 some widening countries received no country-specific recommendations on R&I (for more details, see Annex IV). In our special report 16/2020 on the European Semester, we have already concluded that the Commission did not make sufficient recommendations to address research.

35 That said, it is the Member States that are responsible for the state of their R&I systems, which includes taking the necessary steps to implement necessary reforms. The Commission’s analysis of the implementation of the 2019 country-specific recommendations showed that one year after the recommendation were issued, more than half of the widening countries showed limited or no progress.

36 The implementation of reforms is also a key aspect of the Recovery and Resilience Facility (RRF). The RRF regulation states that the Facility should contribute “to effectively address all or a significant subset of challenges identified in the relevant country-specific recommendations”. The Commission must assess this condition before approving a recovery and resilience plan submitted by each Member State. At the time of the audit, it was too early to assess to what extent the implementation of reforms planned under the RRF will be successful in practice, i.e. converted from paper commitments (such as the adoption of a legal act) to real progress.

37 The increased importance of reforms in the context of the RRF, and the limited progress made on the implementation of country-specific recommendations (see paragraph 32) highlights the significance of policy support by the Commission.

38 The Commission’s 2019 evaluation of the PSF reported that the vast majority of stakeholders replying to a survey were very positive about the benefits of the scheme. Nevertheless, the evaluation included a number of recommendations regarding:

  1. upgrading the PSF’s role as a driver of reforms (e.g. using it also to encourage more systematic and extended dialogue among Member States and between them and the Commission);
  2. expanding the PSF’s design specifications (e.g. extending its period of validity and introducing a framework for follow-up to support implementation and monitor impact);
  3. improving the mechanics of the PSF (e.g. putting more effort into dissemination and communication to enhance visibility and ensure results have a greater impact in the countries involved).

39 However, three years later, the Commission has still not fully implemented these recommendations. The Commission informed us that, building on existing experiences with selected Member States, it intends to further develop in-depth bilateral dialogue on R&I with Member States that are willing to engage in the process (point a) and that efforts are currently ongoing to improve dissemination, communication and visibility (point c).

40 The Commission’s follow-up of recommendations for Member States resulting from the PSF (point b) currently occurs mainly through the Semester process, and in the future also via the monitoring of recovery and resilience plans (see paragraph 36). Also, in response to this point of the 2019 evaluation, under Horizon Europe PSF, a new feature, “PSF Open”, has been introduced. This feature allows Member States that have already benefitted from a PSF activity to receive support with implementing PSF recommendations. However, the PSF remains a demand-driven facility. We were not in a position to assess the appropriateness of PSF Open as the practical aspects related to its implementation have not yet been defined.

41 The staff of the Joint Research Centre that we interviewed highlighted the importance of accompanying measures to help Member States reform their R&I systems. They mentioned the provision of ongoing support, investment guidance and advice on the most suitable tools and actions, none of which was provided by the PSF under H2020.

Some of the targeted countries benefitted significantly less than others from the widening measures

42 For H2020, the decision on the countries that were to benefit from the widening measures was taken in the context of the adoption of the work programmes. This is done by the Commission through the “comitology” procedure, i.e. it involves a committee of Member States’ representatives. For Horizon Europe, the widening countries were decided by the co-legislators and named in the regulation itself.

43 Although it had no direct remit for the selection of beneficiary countries, the Commission provided analysis to allow evidence-based decision-making in line with its better regulation principles. This analysis was based on a number of indicators chosen to reflect the causes of low performance.

44 Having reviewed the indicators used for H2020, the Commission applied a different set for its Horizon Europe analysis (see Annex V for details of the indicators for both periods). This analysis, provided in form of a non-paper to the co-legislators for the selection of the countries for Horizon Europe, did not include information on a country’s:

  1. participation in previous FPs. This indicator is relevant given that the immediate objective of the widening measures is to increase participation (see Article 7.5 of the Horizon Europe regulation); and
  2. position in transnational and multidisciplinary research networks. This is a relevant indicator because central position in the networks and hence strong connectivity reflects a country’s level of participation in the FP. The majority of the FPs’ budget is spent on collaborative R&I projects to promote, amongst others, the creation of transnational and multidisciplinary networks seen by the Commission as a key added value of the FPs.

45 We therefore analysed these two indicators. With regard to the participation in H2020, all widening countries show a lower level of participation in H2020 compared to non-widening countries (see Figure 5).

Figure 5 – H2020 funding committed by country in absolute and relative terms (at the end of the programme)

Source: ECA, based on H2020 database.

46 However, this statistic is largely influenced by the size of each country. A different, more meaningful picture emerges if account is taken of country size. We normalised the data by population (funding committed per capita) and by number of researchers (in full time equivalents). To avoid overlooking any distortions linked to different salary scales, we also normalised it by GDP (see Figure 6).

Figure 6 – H2020 funding by country normalised by population (per capita), by full-time equivalents of researchers and by GDP (at the end of the programme)

Source: ECA, based on Eurostat data and H2020 database.

47 Figure 6 shows important differences among the widening countries. Most of them are ranking low. However, the stronger of the widening countries have a level of participation similar to that of the best performing non-widening countries. Despite this, all the H2020 widening countries except Luxembourg retained their status under Horizon Europe (see paragraph 08).

48 Similar differences are visible in terms of country participation in research networks (see Figure 7). The Commission’s H2020 monitoring flash report (2018) noted that “[...] several EU-13 countries are consistently found at the bottom of the ranking over the period.”

Figure 7 – Network indicator normalised by size of country (at the end of the programme)

Source: H2020 dashboard.

49 Differences also emerge within the group of widening countries in terms of the commitment of widening funds (see Figure 8). Around half of the H2020 widening funding went to just three of the 15 countries. However, this variable is largely influenced by the size of the country. Therefore, a more meaningful picture emerges when normalised by population (see Figure 9). The less connected of the widening countries (see paragraph 47) also benefit less from the widening measures.

Figure 8 – H2020 widening funding by country (at the end of the programme)

Source: ECA based on H2020 database.

Figure 9 – H2020 widening funding per capita (at the end of the programme)

Source: ECA based on H2020 database.

50 When putting the spotlight on the two measures we analysed (see paragraph 14) we found that:

  • More than 60 % of the Teaming projects funded under H2020 are being implemented by just four widening countries. Cyprus leads the way with six (24 %) projects. Five countries have not had any Teaming project (see Annex VI for more details).
  • Some 51 % of ERA Chairs projects are concentrated in just two widening countries. Three countries have not hosted any ERA Chairs project and four had just one (see Annex VI for more details).

51 The same reasons that explained low participation of countries in the past (see paragraphs 05, 20-21) explain the divide within the group of widening countries. While experience in the FPs and in the widening measures in particular is meant to help the countries in improving their capacity to prepare competitive project proposals and to establish collaborations, this effect will only unfold in the years to come (see paragraph 12).

The implementation of the widening measures faced challenges but shows first results

52 We identified aspects that are of importance for a smooth implementation. These aspects were:

  • the timely arrival of complementary funding for Teaming projects as it usually covers infrastructure and equipment costs. Moreover, state-of-the-art infrastructure and equipment are key factors in attracting international researchers;
  • the recruitment of international researchers to bring R&I excellence to the country;
  • achieving sustainability. To achieve a lasting impact, Teaming centres of excellence need to become self-sustainable in the long term, and are required to demonstrate this aim in a business plan attached to their project proposal. ERA Chairs projects need to induce structural changes in the hosting institution if they are to achieve excellence on a sustainable basis4.

53 We carried out interviews with project beneficiaries, managing authorities and national contact points, and analysed supporting documentation (grant agreements and monitoring reports) to assess whether these aspects were handled successfully by the projects we audited. No Teaming projects had been completed at the time of our audit, but four ERA Chairs projects had just ended. We also assessed whether the widening measures had started contributing to closing the innovation divide.

Complementary funding often arrived with delay

54 According to the grant agreements, the value of complementary funding for the 24 ongoing Teaming projects was €784 million, more than twice the H2020 allocation. ERDF operational programmes have provided 44 % of all complementary funding. National resources (both public and private), in-kind contributions, grants and revenue generated by the centres of excellence provide the remaining 56 %.

55 Our documentary review and detailed analysis of all Teaming projects, showed that 55 % of beneficiaries from the first call and at least 28 % from the second call, had experienced delays in obtaining complementary funding, in particular from operational programmes (ERDF plus national co-financing). Information about disbursements of complementary funding from the second call is limited since all projects had been running for less than a year and a half when we gathered the data.

56 These lengthy delays – in one case almost two years (see Box 3) – as well as cumbersome administrative procedures in obtaining complementary funding (ERDF and/or national funds) were among the reasons given for the delays in building or updating premises which were experienced by four of the six projects. A survey of beneficiaries of the 24 Teaming projects which REA carried out in support of its 2021 policy impact report reached similar conclusions on delays. The survey also revealed that, at the beginning of 2021, the premises of 60 % of beneficiaries from the 2016 call had not yet been fully constructed or renovated.

57 In addition to the delays discussed above, four of the six Teaming projects in our sample have had difficulties with managing complementary funding, mainly because of timing or regulatory issues. Examplesof such issues we identified through different analyses are:

  • Time-wise, ERDF calls for the provision of complementary funding were not coordinated with the corresponding Teaming calls. This issue has already been highlighted in a 2018 technical report by the Commission’s Joint Research Centre.
  • H2020 projects can run until 2026, and costs are accepted until that date. However, only costs incurred before the end of 2023 are eligible for ERDF funding. Beneficiaries complain that it will be difficult to spend their ERDF complementary funding by this deadline. This is a particular concern for projects funded under the second Teaming call (grant agreements signed in 2019-2020) (see Box 3). We found that eight out of ten projects funded under the 2016 call have already experienced delays and requested extensions. We found that the COVID‑19 pandemic has also impacted the timeline. Therefore, even more projects funded under the second call are likely to face the same problem, putting their implementation at risk, particularly if national funds are not provided to make up for the lost funding from the ERDF.

Box 3

Example of difficulties faced by a Teaming project with ERDF complementary funding

The beneficiary signed the H2020 grant agreement in October 2019. The agreement included a letter of commitment by a managing authority to provide ERDF funding. In February 2020 the managing authority launched an ad hoc call for this project, requesting a large number of documents. Since then it extended the deadline for submitting the documents three times and requested additional documents on six occasions (most recently in May 2021) to complete its extensive evaluation process. After the heavy and long-lasting evaluation process of the managing authority, the ERDF grant agreement was signed in June 2021, almost two years after the H2020 grant agreement.

The ERDF call included the possibility of using 20 % of the grant for buildings renovation. The managing authority revoked this clause in January 2021, considering that infrastructure costs could not be funded under the operational programme. The proposed solution was to reduce the grant and launch a new call under another measure to cover infrastructure costs. As of the end of June 2021, the second call had yet to be launched.

Costs relating to the ERDF grant have to be incurred by the end of September 2023, even though, according to the business plan annexed to the grant agreement counted on ERDF funding to be used throughout the project implementation period (until end 2026). The beneficiary has stated that it is unlikely that the funding will be absorbed in such a short period of time (from June 2021 to September 2023).

58 ERA Chairs projects may also benefit from additional ERDF funding. Two of the five projects in our sample received such funding, but again it was also with considerable delay – in one case four years after the award of the H2020 grant.

59 The REA, which is responsible for selecting projects and for their implementation, was not in a position to perform (i) a thorough ex-ante assessment of complementary funding committed to Teaming projects or (ii) ex-post monitoring that funding was actually disbursed. This limits REA´s capacity to take corrective actions if needed. As it highlighted in its 2021 policy impact report:

  • “Teaming projects do not always report on complementary funding in a uniform and harmonized way”, making it difficult to compare funding received with commitments indicated in the project proposals, and
  • “the delivery of the complementary funding […] requires a stricter monitoring”.

60 We analysed the data on complementary funding in the grant agreements for all Teaming projects and found that it is not presented clearly or consistently. The REA policy impact report arrived at similar conclusions. In particular, we found that:

  • 28 % of all project proposals included both secured complementary funding and unsecured funding (competitive grants, income from research projects, etc.), without distinguishing between them.
  • 56 % of all project proposals included complementary funding in the form of in-kind contributions, but failed to describe adequately how in-kind contributions were evaluated.
  • 36 % of projects included complementary funding extending beyond the seven-year Teaming project period. Project proposals did not always provide sufficient information to calculate what proportion of the commitment fell within the seven years. Moreover, it is not possible for the REA to check whether complementary funding is actually disbursed after the end of a project.

61 We note that the REA and/or the Commission have taken initial action in response to the conclusions in the REA’s 2021 policy impact report. In particular, they have standardised (i) the information on complementary funding to be provided with the project proposal and (ii) reporting on the progress of complementary funding (so that commitments can be compared with amounts actually received), and in-kind contributions can no longer be considered complementary funding.

62 With regard to the difficulties affecting monitoring (see paragraph 59), we found that this is partly due to the fact that national authorities’ and other contributors’ letters of commitment to provide complementary funding are sometimes vague and unenforceable. For example:

  • no indication of the amount committed;
  • use of expressions like “we will consider contributing […] up to €[…] per year";
  • an overall commitment to all the projects competing from a given country, with the risk of not being able to support all of those ultimately selected.

The recruitment of international staff was time-consuming

63 Based on our interviews with project beneficiaries and analysis of periodic progress reports on the projects in our sample (see paragraph 15) we found that recruiting international staff has proved to be difficult and time-consuming (see Table 2). REA’s June 2021 policy impact report reached similar conclusions.

Table 2 – Recruitment challenges encountered by our sampled projects

  Teaming ERA Chairs
    Percentage of projects Comment Percentage of projects Comment
Recruitment 75 % Recruited at least one researcher from abroad. 80 % Recruited a high-profile international researcher for the position of Chair holder.
  50 % Issues in recruiting international staff. 80 % Issues in recruiting international staff.
Retention -- -- 60 % The Chair holder changed during the funding period.
      50 % Did not manage to retain the Chair holder after the end of EU funding.

Source: ECA own elaboration.

64 The reasons explaining these difficulties include the need to relocate and the perceived low attractiveness of a career in research in certain widening countries. Salary unattractiveness can also play a role, in particular where research centres are public bodies, as legal restrictions on the terms of employment they can offer their staff may hamper their ability to recruit top international researchers. The REA has confirmed that Teaming centres of excellence have often leaned towards recruiting national applicants.

Ensuring sustainability remains a challenge

65 Sustainability means projects having to seek further competitive grant funding, but also to generate income through the exploitation of their research results. Two prerequisites to this end are good visibility for their research work and links with business to bring innovation to the market. Ensuring sustainability remains primarily a task of the project beneficiaries and national public authorities. Nevertheless, the Commission can provide tools to facilitate the path towards sustainability.

66 Our analysis of periodic progress reports on the most advanced projects in our sample (see paragraph 16) and the income they have generated revealed that most projects are not assured of long-term sustainability. The main issues in this regard are shown in Table 3.

Table 3 – Issues affecting the long-term sustainability of projects

Issues Teaming, number of projects ERA Chairs, number of projects
Difficulties with the commercial exploitation of research results 50 % 50 %
No competitive research grants secured in addition to Teaming / ERA Chairs funding 25 % 50 %
No links established with business 50 % 50 %
Beneficiaries perception of low visibility in the EU research arena 100 % 75 %

Source: ECA.

67 We analysed the grant agreements in our sample of Teaming projects and found that 75 % included performance indicators for commercialisation. However none of them were anywhere near their target at the time of the audit. Moreover, 50 % of the projects have generated no income so far.

68 The results of our analysis on issues affecting the sustainability of Teaming projects are consistent with those obtained by the REA:

  • In the survey underlying its 2021 report, 80 % of the respondents from the first Teaming call said they had been able to attract grant funding, and 50 % said they were generating other types of revenue. It is however unclear whether the available sources of revenue will be enough to guarantee the financial sustainability of the centres of excellence. Moreover, 40 % of the respondents stated that they were not yet in a position to generate any revenue other than competitive research funding.
  • According to another analysis by REA in October 2020, only three of the ten ongoing Teaming projects from the 2016 call had created links with business.

69 Regarding Teaming projects, to improve commercialisation, the Council decision implementing H2020 envisaged the possibility of “building links with innovative clusters and recognising excellence in low performing RDI Member States and regions, including through peer reviews and awarding labels of excellence to those institutions that meet international standards”. However, the Commission has not established peer review practices and only to a limited extent labels of excellence.

70 We note that, under the widening strand of Horizon Europe, the Commission has introduced a new measure to support “excellence hubs”. One intention, among others, is to create better links between science and business. However, there is no guarantee that current Teaming projects will benefit from this initiative: for example, these excellence hubs are topic-oriented, so Teaming projects focusing on topics other than those covered by the hubs may miss out. Moreover, the initiative does not address the beneficiaries’ lack of awareness and expertise on making the most of their research results.

71 Under H2020 and Horizon Europe, there are further initiatives available to all Member States seeking to create links between the research community and business – such as knowledge innovation communities. However, the Commission has not actively promoted the establishment of links between projects and innovation communities.

72 In order to disseminate research results and promote their exploitation, the Commission maintains a specific database known as the Community Research and Development Information Service (CORDIS). This provides information on all EU-supported research and development activities, including project results and publications, which project beneficiaries are required to record in CORDIS. However, we found that the information in CORDIS is outdated and only partially reliable (especially in respect of ERA Chairs projects).

There is early evidence of concrete results

73 Considering that one the objectives of the widening measures (see paragraph 05) was to expand widening countries’ participation in H2020, and bearing in mind that the measures are only expected to have an impact in the medium and long term (see paragraph 12), we looked at early trends in terms of participation. We also assessed whether projects were already showing first concrete results.

74 The share of funding secured by widening countries increased between the 7th FP and H2020 (+ 1.7 percentage points and a jump of + 31 % in relative terms) (see Table 4). All the widening countries except Croatia and Hungary increased their participation in both percentage points and relative terms (for detailed information on each Member State see Annex VI).

Table 4 – Share of 7th FP and H2020 committed funding by group of countries (at the end of both programmes)

  7th FP committed funds (a) H2020 committed funds (b) Variation
(b-a)
Variation (in %)
(b-a)/a
EU widening countries 5.5 % 7.2 % + 1.7 + 31
EU non-widening countries 68.4 % 71.5 % + 3.1 + 4.5
United Kingdom 15.5 % 11.5 % - 4.0 - 25.8
Non-EU countries (excluding United Kingdom) 10.6 % 9.7 % - 0.9 - 8.0

Source: ECA, based on the 7th FP and H2020 databases.

75 After widening commitments are deducted, the increase in funding committed to widening countries was less marked, at 1.1 in percentage points and 20 % in relative terms. The countries with a stronger participation in the widening measures (see Figure 9) were also among those with the largest increase in participation between the 7th FP and H2020 in terms of funding (see Table 10 in Annex VI ) and networking (see Figure 7).

76 We found that the projects in our sample have already delivered a number of positive intangible and tangible outcomes. For Teaming projects, these include the following:

  • According to beneficiaries and managing authorities, projects attracted much political interest from national and regional authorities, both in individual projects and, more generally, in R&I in widening countries.
  • Projects have been a source of employment for researchers and innovators (three of the four new centres we analysed from the first call have created up to 70 research jobs each).
  • Some outputs are already emerging, mainly in the form of scientific publications (four projects from the first call, but still too early for those from the second call).
  • Projects have raised more than required in complementary funding (see paragraph 11).

77 Some early positive outcomes of ERA Chairs projects are:

  • Beneficiaries and managing authorities report international recognition for Chair holders and host institutions.
  • 80 % of the sampled projects have benefited from increased networking.
  • 60 % of the sampled projects have generated structural changes at the host institution.

78 Box 4 provides two examples of projects showing promise and concrete results.

Box 4

Examples of positive results

Teaming project: early promising results

The project was launched in 2017 for a period of five years. After four years the newly established centre of excellence already employed 68 staff from 17 countries. The more advanced partners provided support with setting up connections and recruiting international researchers.

In its first four years, the centre has secured funding for 47 research projects (including eight H2020 projects), produced a significant number of scientific publications, established connections with business and filed one patent application.

ERA Chairs project: positive results

The project started in July 2015 and ran for six years. The Chair holder delivered concrete results and is still employed at the host institution now the project has ended. In particular, the project prompted the host institute to (i) endorse the EU Charter for Researchers and the Code of Conduct for the Recruitment of Researchers; (ii) improve international networking; (iii) increase by 20 % the number of peer‑reviewed publications in the research area of the ERA Chair; and (iv) set up a technology transfer office, which has contributed to an increase in collaboration with industry and the exploitation of research results (new inventions, patents, business licences and spin-offs).

In addition, the team has secured competitive grants, including international grants. It has also taken a very active role in COVID-19 research.

The Commission has made insufficient arrangements for monitoring the impact of widening measures

79 The EU's financial regulation requires the Commission to set objectives for its programmes and monitor their achievement by means of performance indicators. We assessed whether the arrangements for monitoring the widening measures were appropriate.

80 In our 2015 annual report, we already pointed to weaknesses in the design of H2020, particularly as regards performance indicators, and at overly general and high-level objectives.

81 Among the indicators defined for H2020, only one was specific to the “widening” strand (but limited to ERA Chairs and Twinning activities): “Evolution [in percentage] (compared to a reference period [3 years] prior to the signature of the grant agreement) of the publications in high impact journals in the given research field of the research organisation funded”.

82 Since 2016, several documents prepared for the Commission or the REA have stressed the importance of introducing data collection for specific and measurable key performance indicators (KPIs)5, especially for Teaming projects. For example, the REA has suggested KPIs measuring "structural changes" in an organisation, or indicators for the information given in project proposals, such as the number of patents, licence agreements, researchers hired and new research projects funded.

83 In response to these suggestions, the Commission’s “widening” work programme for 2018-2020 contained the following details relevant to monitoring:

  • under Teaming, the potential impact of new/upgraded centres of excellence was to be reinforced through measurable KPIs, and improvements in the R&I ecosystem in the medium to long term should also be shown through indicators (e.g. research intensity);
  • under ERA Chairs, the work programme provided examples of indicators for measuring the increase in research excellence (e.g. publications in peer-reviewed journals, collaboration agreements with business, intellectual property, new innovative products and services).

84 However, the Commission did not set any detailed KPIs for the Teaming measure and for the ERA Chairs measure indicators lacked baselines and targets. Instead, indicators were to be defined and detailed in the grant agreement for each project. The absence of common indicators makes it difficult to draw conclusions about impact.

85 Project beneficiaries are required to report regularly on progress. In addition, the REA carries out regular surveys to assess actual progress, in particular for Teaming projects. These surveys have successfully identified the main issues affecting projects (see paragraph 56).

86 Widening projects are expected to have a medium to long-term impact in the host countries. To that end, grant agreements also include a “dissemination and exploitation plan” that beneficiaries are expected to implement after the end of the EU funding period. However, the Commission has not envisaged any follow-up or monitoring of projects after the end of the funding period.

Conclusions and recommendations

87 Our overall conclusion is that the widening measures introduced under Horizon 2020 were fit for purpose in that they addressed many of the causes of the low participation of some Member States in the Framework Programmes. However, disparities in the R&I ecosystem among the widening countries meant that they did not all benefit to the same extent. The full impact of the measures is only expected to unfold fully over the coming years. While these measures together with other EU measures spurring reforms can kick-start progress in R&I in the recipient countries, genuine sustainable change is dependent on national governments fully playing their part in making R&I a priority, both in terms of investment and reforms.

88 The Horizon 2020 widening measures addressed many of the issues identified as affecting R&I performance. Nevertheless, they mainly act as a catalyst for progress as national R&I investment levels as well as reforms, and hence strategic national decisions, play a decisive role in a country’s performance (see paragraphs 20-27).

89 The Policy Support Facility is part of a basket of instruments that the EU put in place to spur reforms in Member States, the latest of which being the Recovery and Resilience Facility. The Commission has used the insights gained from the Policy Support Facility, particularly from 2019 onwards, to feed selectively into its European Semester work. However, the Policy Support Facility’s ability to induce needed changes in national systems was limited as (i) the resources available for the Policy Support Facility were limited, (ii) not all widening countries requested support from the Policy Support Facility and (iii) Member States were free to decide to what extent they would implement the reforms identified in the context of the Policy Support Facility (see paragraphs 28-37).

90 The Commission has not yet fully implemented the recommendations of its own 2019 evaluation of the Policy Support Facility and stated that it has the intention to (i) further develop dialogue on national strategies both between itself and the Member States that are willing to engage in that process and among the Member States, and (ii) put more effort into dissemination and communication. Under a new feature of the demand-driven Policy Support Facility, “PSF Open” (2021-2027 period), Member States can seek for support in implementing reforms in their R&I systems, but the Commission still has to define all practical aspects (see paragraphs 38-41).

Recommendation 1 – Strengthen the use of the Policy Support Facility

To make effective use of the instruments at its disposal to induce the changes in national R&I systems which are needed for progress in widening countries, the Commission should:

  1. Translate into concrete action its intentions with regard to (i) better disseminating and communicating Policy Support Facility results and (ii) improving dialogue (e.g. decide on the purpose, tools, timing and regularity of dialogue), the outcome of which should also feed into the Commission’s work under the European Semester and its monitoring of the milestones set out in recovery and resilience plans;
  2. Design “PSF Open” in such a way that it can provide continuous support to national authorities implementing reforms, either as a result of recommendations emanating from Policy Support Facility activities or requested in the context of the European Semester or committed to in the recovery and resilience plans.

Target implementation date: end 2022 for (a) and mid-2023 for (b).

91 We found that most widening countries ranked low on participation in Horizon 2020 (amount received per capita) and in terms of connectivity with partners across the EU. Nevertheless, a few countries had levels of participation and connectivity similar to the best-performing non-widening countries (see paragraphs 45-51).

92 The decision on which countries were to benefit from the Horizon Europe widening measures was taken by the co-legislators. The decision is not open for review at mid-term. The Commission provided supportive analyses of a few indicators selected as proxies for the causes of low performance. The analysis did not look at a country’s (i) degree of participation in previous Framework Programmes and (ii) degree of connectivity with partners across the EU (see paragraphs 42-44).

Recommendation 2 – Aim for a more balanced participation of widening countries in widening measures

To avoid situations in which the bulk of widening projects go to just a few countries, the Commission should closely monitor participation levels in widening measures under Horizon Europe and, if continuous significant imbalances emerge, introduce measures to achieve a wider pattern of participation.

Target implementation date: during implementation of the widening measures.

93 Beneficiaries of Teaming and ERA Chairs projects have experienced a number of issues which have led to delays and/or may adversely affect project results (see paragraphs 52-53). These related in particular to:

  • Delays in receiving funding complementary to Horizon 2020 grants, the largest share of which, in particular for Teaming projects, comes from the ERDF. Moreover, although Teaming projects can run until 2026, costs to be covered by ERDF grants need to be incurred by the end of 2023, so some beneficiaries may have difficulties in absorbing the funds (see paragraphs 54-62).
  • Recruitment of international researchers (see paragraphs 63-64). It proved time-consuming but eventually a majority of projects managed to do so.
  • Assuring the long-term sustainability of projects, for example by focusing on visibility and establishing links with business to promote the exploitation of research results (see paragraphs 65-72).

94 Complementary funding is crucial, in particular for financing infrastructure and equipment at the centres of excellence supported by Teaming. However, REA was not in a position to properly evaluate and monitor this funding, mainly due to a lack of harmonised data or, in some cases, the vague nature of the commitments made by funding providers (see paragraphs 59-62).

Recommendation 3 – Facilitate the timely availability of complementary funding

To reduce the risk of projects facing difficulties with complementary funding from ERDF operational programmes the Commission should encourage managing authorities to follow a simplified grant approval process, for example by taking full account of the evaluation made in the context of Horizon Europe funding.

Target implementation date: end 2022.

95 Teaming and ERA Chairs projects aim at creating long-lasting effects. However, projects analysed showed limited capacity to exploit their research results. While ensuring sustainability remains the primary responsibility of beneficiaries and national public authorities, the Commission also provides support in this regard. It takes the form of initiatives (under Horizon 2020 and Horizon Europe) aiming at creating linkages between research and business. However, there is no guarantee that current Teaming and ERA Chairs projects will benefit from these (see paragraphs 65-71).

96 The CORDIS database, which provides the results of projects supported by the various Framework Programmes, is not used effectively to promote the visibility and facilitate the exploitation of research results from Teaming and ERA Chairs, as it is not systematically updated (see paragraph 72).

Recommendation 4 – Enhance the capacity of project beneficiaries to exploit their research results

To boost the projects’ prospects of sustainability, the Commission should:

  1. Provide support (such as training and coaching) to widening projects, especially Teaming projects, to enable them to take better care of the valorisation of their research results;
  2. Promote contacts between project beneficiaries and potential industrial partners, in particular through existing EU initiatives aiming at creating links between research and business (such as knowledge innovation communities);
  3. Further support the visibility of projects by encouraging beneficiaries to provide regular updates of project results and making them publicly available on the EU platforms established for that purpose.

Target implementation date: end 2023.

97 At the time of the audit, it was too early to assess whether the widening measures have given the expected impetus to the performance of widening countries, as any effects will not unfold until part way through the implementation of Horizon Europe. It is therefore unsurprising that, by the end of the Horizon 2020 period, there was only an incremental increase in the widening countries’ participation rate since the 7th Framework Programme (in particular after discounting the impact of ring-fenced funding for widening) (see paragraphs 73-75).

98 Nonetheless, even at this early stage, and in spite of the implementation difficulties described above, we found a number of first positive outputs of Teaming and ERA Chairs projects. Teaming ventures in particular have attracted significant amounts in complementary funding, started publishing scientific results and provided employment for researchers; and most ERA Chairs projects have helped to bring structural change to their host institutions (see paragraphs 76-78).

99 Monitoring of the success of the “spreading excellence and widening participation” objective, as well as of individual measures (such as Teaming and ERA Chairs), has been hampered by a lack of performance indicators or, where they exist (ERA Chairs), by a lack of baseline and target values. Moreover, the Commission does not envisage monitoring individual projects after closure even though (i) one of the main objectives is the continuity of actions and (ii) many results and impacts will not be visible until then (see paragraphs 79-86).

Recommendation 5 – Enhance monitoring of the widening measures

To monitor the widening measures and their impacts effectively, the Commission should:

  1. Set specific objectives and translate them into appropriate KPIs at measure level;
  2. Follow up Teaming projects also after closure to be able to conclude on their mid to long-term impacts.

Target implementation date: end 2024 for (a); two years after the closure of the projects for (b).

This Report was adopted by Chamber IV, headed by Mr Mihails Kozlovs, Member of the Court of Auditors, in Luxembourg on 3 May 2022.

 

For the Court of Auditors

Klaus-Heiner
Lehne President

Annexes

Annex I – Widening measures under Horizon Europe

The widening measures under Horizon Europe have a budget of €2.95 billion (3 % of the Horizon Europe budget). The scheme has been expanded, and now includes four additional measures:

  • Excellence initiative for universities aims at the transformation of higher education sector and their surrounding ecosystems, including non-university research centres.
  • Excellence hubs aims at fostering innovation ecosystems in Widening countries and beyond, creating better linkages between academia, business and government, in line with regional or national smart specialisation strategies.
  • Brain circulation consisting in a set of different grants aiming at increasing the attractiveness for researchers of entities in widening countries, by providing competitive grants and spreading attractive working and employment practices
  • Hop-on facility consisting in grants provided to entities from low R&I performing countries that join already selected collaborative R&I actions.

Table 5 shows the budget allocation to these various measures under Horizon Europe.

Table 5 – Widening measures under Horizon Europe

Measure/instrument Available funds (in million euros) Share of widening budget
Teaming 743.6 26 %
Twinning 486.2 17 %
ERA Chairs 343.2 12 %
COST 400.4 14 %
Excellence initiative for universities 286.0 10 %
Excellence hubs (innovation ecosystems) 257.4 9 %
Brain circulation 143.0 5 %
Support package (incl. “PSF Open”) 57.2 2 %
Hop-on facility 143.0 5 %

Source: ECA, based on information provided by the European Commission.

Annex II – Methodology

Our audit was based on the following evidence:

  1. Desk review of public documents and internal Commission documents such as legal texts, guidelines, impact assessments, evaluation and monitoring reports communications;
  2. Questionnaires and fact-checking videoconferences with managing authorities and National Contact Points for widening measures from five Member States (Croatia, Poland, Portugal Romania and Slovenia) selected on the basis of their R&I performance, their overall participation in the FP and in widening measures in particular;
  3. Statistical analysis of data from a range of sources: CORDA, Business Objects, Innovation Scoreboard, Eurostat and CORDIS;
  4. Detailed analysis of a sample of Teaming projects and ERA Chairs projects selected on a judgemental basis, in particular among projects at an advanced stage of implementation (see Table 6). The detailed analysis consisted of (i) a review of project related documents (proposals, grant agreements, monitoring reports,...), (ii) questionnaires and fact-checking videoconferences with beneficiaries;
  5. Detailed analysis of complementary funding for all Teaming projects.

Table 6 – Sample of Teaming and ERA Chairs projects

Call Number of projects financed Sample Coverage
Teaming 2016-2017 10(1) 4 40 %
Teaming 2018-2019 14 2 14 %
Total Teaming 24 6 25 %
ERA Chairs 59 5 8 %

Note: (1) 11 projects initially obtained a Teaming grant, but one was stopped in 2020 ( before the end of its implementation).

Source: ECA.

Annex III – Relation between R&I investments and R&I performance

We found a positive correlation between:

  • the level of national R&I investment per capita and performance as measured by the European Innovation Scoreboard’s summary innovation index. The higher the national R&I investment was per capita in the 2014-2020 period, the higher the ranking on the 2021 Scoreboard (see Figure 10);
  • the level of national R&I investment per capita (over the 2014-2020 period) and the level of participation in H2020 (see Figure 11).

Figure 10 – Correlation between national R&I investment per capita (2014-2020) and 2021 European Innovation Scoreboard

Source: ECA, based on Eurostat and 2021 European Innovation Scoreboard data.

Figure 11 illustrates the correlation between national R& I investments per capita (2014-2020) and participation in H2020. The coefficient of determination, (R2) explaining the proportion of the variation in one variable that is predictable from the other variable, is 0.45. However, if Cyprus (an outlier) is excluded the R2‑ increases to 0.65. Most widening countries are concentrated in the bottom left corner, which shows both low national investment in R&I and low H2020 funding per capita. All widening countries except Czechia and Luxembourg are below the trend line.

Figure 11 – Correlation between national R&I investment per capita (2014-2020) and participation in H2020 (funds committed at the end of the programme)

Source: ECA, based on Eurostat data and H2020 database.

Annex IV – Country-specific recommendations on R&I

The following country-specific recommendations on R&I were made to widening countries during 2014-2020:

  • 2014: 10 out of 15 widening countries received a recommendation, mainly for links to be established between R&I and industrial policy.
  • 2015-2018: each year, recommendations were made to between two and five countries. Five did not receive any recommendations (Bulgaria, Croatia, Hungary, Slovenia, Romania).
  • 2019: all but one of the widening countries received a recommendation to focus investment-related economic policy on R&I.
  • 2020: 10 out of 15 countries received a recommendation, mainly to focus on investment in R&I.

Annex V – Indicators used in the analysis provided by the Commission to the co-legislators for the selection of widening countries

  1. H2020:

    A Member State had to score below 70 % of the average EU-27 composite indicator on research excellence, which is composed of four variables:

    • share of highly cited publications where at least one of the authors has an affiliation in the country;
    • number of top scientific universities and public research organisations in the country;
    • patent applications;
    • total value of European Research Council grants received.
  2. Horizon Europe:

    • gross national income below 90 % of the EU average;
    • for each country, the percentage of scientific publications of its researchers which rank amongst the top 10 % of citations (i.e. quoted by other research papers); and
    • number of patent applications per million inhabitants below the EU average.

Annex VI – Statistics

Table 7 and Table 8 provide an overview of widening countries’ participation in the two measures analysed. They show how many of the project proposals submitted were selected for funding at the end of the competitive selection process. The main factor influencing a country’s success rate is its overall R&I performance. Other factors include for example, experience gained from previous participation, the combined efforts by national actors to improve success in participation and existing network connections. This is in line with the observations in our special report 02/2020 (paragraph 43) on the SME Instrument.

Table 7 – Teaming proposals submitted and funded by country (at the end of the programme)

Country Proposals submitted (a) Projects funded (b) Success rate (b/a) Share of all funded projects
Cyprus 41 6 15 % 24 %
Portugal 21 3(2) 14 % 12 %
Czechia 28 3 11 % 12 %
Poland 32 3 9 % 12 %
Latvia 14 2 14 % 8 %
Hungary 17 2 12 % 8 %
Bulgaria 28 2 7 % 8 %
Estonia 11 1 9 % 4 %
Slovakia 18 1 6 % 4 %
Slovenia 20 1 5 % 4 %
Serbia(1) 30 1 3 % 4 %
Lithuania 9 - 0 % -
Romania 44 - 0 % -
Croatia 16 - 0 % -
Malta 9 - 0 % -
Luxembourg 2 - 0 % -

Notes:

(1) H2020 associated countries can also participate in the widening measures.

(2) It includes a project that was stopped in 2020 (i.e. before the end of its implementation).

Source: ECA based on H2020 database.

Table 8 –ERA Chair proposals submitted and funded by country (at the end of the programme)

Country Submitted proposals (a) Funded projects (b) Success rate (b/a) Share of all funded projects
Portugal 61 16 26 % 27 %
Estonia 43 14 33 % 24 %
Poland 46 6 13 % 10 %
Cyprus 27 5 19 % 8 %
Czechia 23 4 17 % 7 %
Slovenia 39 3 8 % 5 %
Turkey(1) 7 2 29 % 3 %
Romania 12 2 17 % 3 %
Croatia 14 2 14 % 3 %
Luxembourg 5 1 20 % 2 %
Bulgaria 15 1 7 % 2 %
Lithuania 15 1 7 % 2 %
Serbia(1) 18 1 6 % 2 %
Slovakia 16 1 6 % 2 %
Latvia 20 - 0 % -
Malta 6 - 0 % -
Hungary 6 - 0 % -

Note: (1) H2020 associated countries can also participate in the widening measures.

Source: ECA based on H2020 database.

Table 9 – Distribution by country of commitments of H2020 widening funds (at the end of the programme)

Member State/country block Commitments (in million euros) (excluding COST)
Portugal 102.3
Cyprus 88.2
Poland 59.4
Estonia 52.9
Czechia 50.1
Bulgaria 30.1
Latvia 28.3
Slovenia 26.0
Hungary 24.8
Slovakia 18.0
Romania 10.2
Croatia 9.9
Lithuania 5.3
Luxembourg 4.7
Malta 4.1
   
Non-widening countries 157.5
Non EU (including United Kingdom) 82.3

Source: ECA based on Horizon 2020 Dashboard.

Table 10 – Comparison by country between commitments under the 7th FP and H2020 (at the end of both programmes)

Member State / Country block 7th FP (bln euros) H2020 (bln euros) Share of 7th FP
(a)
Share of H2020
(b)
Variation
(b-a)
Variation (in %)
(b-a)/a
Germany 7.13 9.97 15.70 % 14.74 % -0.96 -6.1 
France 5.21 7.34 11.48 % 10.86 % -0.62 -5.4
Spain 3.30 6.34 7.27 % 9.37 % 2.10 28.9
Italy 3.63 5.62 8.00 % 8.31 % 0.31 3.9
Netherlands 3.42 5.33 7.52 % 7.88 % 0.36 4.8
Belgium 1.84 3.39 4.05 % 5.01 % 0.96 23.7
Sweden 1.75 2.29 3.85 % 3.39 % -0.46 -11.9
Austria 1.19 1.92 2.62 % 2.84 % 0.22 8.4
Denmark 1.08 1.76 2.38 % 2.60 % 0.22 9.2
Greece 1.02 1.70 2.25 % 2.51 % 0.26 11.6
Finland 0.87 1.52 1.93 % 2.25 % 0.32 16.6
Ireland 0.63 1.19 1.38 % 1.77 % 0.39 28.3
Portugal 0.53 1.15 1.16 % 1.70 % 0.54 46.6
Poland 0.44 0.74 0.97 % 1.10 % 0.13 13.4
Czechia 0.29 0.50 0.63 % 0.74 % 0.11 17.5
Slovenia 0.17 0.38 0.38 % 0.56 % 0.18 47.4
Hungary 0.29 0.37 0.64 % 0.55 % -0.09 -14.1
Cyprus 0.09 0.32 0.21 % 0.47 % 0.26 123.8
Romania 0.15 0.30 0.34 % 0.44 % 0.10 29.4
Estonia 0.10 0.27 0.21 % 0.41 % 0.20 95.2
Luxembourg 0.06 0.20 0.13 % 0.29 % 0.16 123.1
Bulgaria 0.10 0.16 0.22 % 0.24 % 0.02 9.1
Slovakia 0.08 0.14 0.17 % 0.21 % 0.04 23.5
Croatia 0.09 0.14 0.20 % 0.20 % 0.00 0.0
Latvia 0.05 0.12 0.11 % 0.17 % 0.06 54.5
Lithuania 0.05 0.10 0.11 % 0.14 % 0.03 27.3
Malta 0.02 0.04 0.05 % 0.06 % 0.01 20.0
Non EU countries (United Kingdom) 7.03 7.75 15.48 % 11.46 % -4.02 -26.0
Non-EU countries (Other) 4.80 6.58 10.56 % 9.72 -0.84 -8.0
TOTAL 45.41 67.62        

Source: ECA based on Horizon 2020 dashboard.

Acronyms and abbreviations

CORDIS: Community Research and Development Information Service

COST: European Cooperation in Science & Technology

DG RTD: Commission’s Directorate-General for Research and Innovation

ERDF: European Regional Development Fund

FP: Framework programme

GDP: Gross domestic product

H2020: Horizon 2020

KPI: Key Performance Indicator

PSF: Policy Support Facility

REA: European Research Executive Agency

RRF: Recovery and Resilience Facility

R&I: Research and innovation

Glossary

Allocation of funding on the basis of excellence: Allocation on the basis of competitive calls for proposals and through independent and merit-based peer review, selecting only the best projects without any consideration of geographical distribution.

Commitment: An amount earmarked in the budget to finance a specific item of expenditure, such as a contract or grant agreement.

Comitology procedure: Comitology applies when the Commission has been granted implementing powers in the text of a law. It implies that the Commission is to be assisted by a committee composed of representatives from all EU countries. The committee provides a formal opinion, usually in the form of a vote, on the Commission's proposed measures. Depending on the procedure, committee opinions can be more or less binding on the Commission.

Country-specific recommendations: Annual guidance which the Commission issues, as part of the European Semester, to individual Member States on their macroeconomic, budgetary and structural policies.

Europe 2020 strategy: The EU’s ten-year strategy, launched in 2010, to boost growth and create jobs.

European Regional Development Fund: EU fund that strengthens economic and social cohesion in the EU by financing investments that reduce imbalances between regions.

European Semester: Annual cycle which provides a framework for coordinating the economic policies of EU Member States and monitoring progress.

Financial instrument: Financial support from the EU budget in the form of equity or quasi-equity investments, loans or guarantees, or other risk-sharing instruments.

In-kind contributions: Non-financial resources made available free of charge by third parties to the beneficiary.

Joint Research Centre: The Commission’s science and knowledge service, providing scientific advice and support for EU policy.

Managing authority: The national, regional or local public (or private) authority designated by a Member State to manage an EU-funded programme.

National Contact Point: An entity established and financed by government of an EU Member State or other participating country to provide in-country support and guidance to applicants and beneficiaries under Horizon 2020 or Horizon Europe.

Operational programme: The basic framework for implementing EU-funded cohesion projects in a set period, reflecting the priorities and objectives laid down in partnership agreements between the Commission and individual Member States.

Place based innovation: This innovation is fostered by the close interaction of innovation actors such as firms, research entities and local/regional government in a close territorial context where geographic proximity facilitates knowledge exchange and collaboration.

Recovery and Resilience Facility: The EU’s financial support mechanism to mitigate the economic and social impact of the COVID-19 pandemic and stimulate recovery in the mid-term, while promoting green and digital transformation.

Subsidiarity: Principle whereby the EU takes action only when doing so is more effective than action taken nationally, regionally or locally.

Audit team

The ECA’s special reports set out the results of its audits of EU policies and programmes, or of management-related topics from specific budgetary areas. The ECA selects and designs these audit tasks to be of maximum impact by considering the risks to performance or compliance, the level of income or spending involved, forthcoming developments and political and public interest.

This performance audit was carried out by Audit Chamber IV Regulation of markets and competitive economy, headed by ECA Member Mihails Kozlovs. The audit was led by ECA Member Ivana Maletić, supported by Sandra Diering, Head of Private Office and Tea Japunčić, Private Office Attaché; Marion Colonerus, Principal Manager; Juan Antonio Vazquez Rivera, Head of Task; Marco Montorio and Katja Mravlak, Auditors. Thomas Everett provided linguistic support.

 

From left to right: Marco Montorio, Tea Japunčić, Juan Antonio Vazquez Rivera, Ivana Maletić, Sandra Diering, Katja Mravlak.

Endnotes

1 Special report 02/2020 (paragraph 41) and special report 04/2016 (paragraph 86).

2 Final report of the MIRRIS project, June 2016; “MLE on National Practices in Widening Participation and Strengthening Synergies: Summary Report”, European Commission, 2018; “Overcoming innovation gaps in EU-13 Member States”, European Parliament Research Service, March 2018; “Mobilising European Structural and Investment Funds and Horizon 2020 in support of innovation in less developed regions”, Joint Research Centre, November 2018.

3 “Research and Innovation analysis in the European Semester 2020 Country Reports”, European Commission, May 2020.

4 The aim (achieving excellence on a sustainable basis) was stated in the H2020 Work Programme on Spreading Excellence and Widening Participation.

5 Final Report, H2020 Advisory Group on Spreading Excellence and Widening Participation June 2016; Brainstorming on Future Widening Actions in Horizon Europe: Contribution from REA B5, October 2019; Policy Impact Report on Teaming, REA, June 2021.

Contact

EUROPEAN COURT OF AUDITORS
12, rue Alcide De Gasperi
1615 Luxembourg
LUXEMBOURG

Tel. +352 4398-1
Enquiries: eca.europa.eu/en/Pages/ContactForm.aspx
Website: eca.europa.eu
Twitter: @EUAuditors

More information on the European Union is available on the internet (https://europa.eu).

Luxembourg: Publications Office of the European Union, 2022

PDF ISBN 978-92-847-8321-2 ISSN 1977-5679 doi:10.2865/359822 QJ-AB-22-015-EN-N
HTML ISBN 978-92-847-8291-8 ISSN 1977-5679 doi:10.2865/01855 QJ-AB-22-015-EN-Q

COPYRIGHT

© European Union, 2022

The reuse policy of the European Court of Auditors (ECA) is set out in ECA Decision No 6-2019 on the open data policy and the reuse of documents.

Unless otherwise indicated (e.g. in individual copyright notices), ECA content owned by the EU is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) licence. As a general rule, therefore, reuse is authorised provided appropriate credit is given and any changes are indicated. Those reusing ECA content must not distort the original meaning or message. The ECA shall not be liable for any consequences of reuse.

Additional permission must be obtained if specific content depicts identifiable private individuals, e.g. in pictures of ECA staff, or includes third-party works.

Where such permission is obtained, it shall cancel and replace the above-mentioned general permission and shall clearly state any restrictions on use.

To use or reproduce content that is not owned by the EU, it may be necessary to seek permission directly from the copyright holders:

Icons Figure 1: © Freepik Company S.L. All rights reserved.

Software or documents covered by industrial property rights, such as patents, trademarks, registered designs, logos and names, are excluded from the ECA’s reuse policy.

The European Union’s family of institutional websites, within the europa.eu domain, provides links to third-party sites. Since the ECA has no control over these, you are encouraged to review their privacy and copyright policies.

Use of the ECA logo

The ECA logo must not be used without the ECA’s prior consent.

GETTING IN TOUCH WITH THE EU

In person
All over the European Union there are hundreds of Europe Direct Information Centres. You can find the address of the centre nearest you at: https://europa.eu/european-union/contact_en

On the phone or by e-mail
Europe Direct is a service that your questions about the European Union. You can contact this service

FINDING INFORMATION ABOUT THE EU

Online
Information about the European Union in all the official languages of the EU is available on the Europa website at: https://europa.eu/european-union/index_en

EU Publications
You can download or order free and priced EU publications at: https://op.europa.eu/en/web/general-publications/publications. Multiple copies of free publications may be obtained by contacting Europe Direct or your local information centre (see https://europa.eu/european-union/contact_en)

EU law and related documents
For access to legal information from the EU, including all EU law since 1951 in all the official language versions, go to EUR-Lex at: https://eur-lex.europa.eu/homepage.html?locale=en

Open data from the EU
The EU Open Data Portal (https://data.europa.eu/en) provides access to datasets from the EU. Data can be downloaded and reused for free, both for commercial and non-commercial purposes.