Synthesis Report on the coordinated audit on
Joint Assistance to Support Projects in European Regions (JASPERS)

Common statement by the Heads of the State Audit Office of Croatia, the Supreme Audit Office of Poland and the Reporting Member of the European Court of Auditors

About Jaspers

In 2005, the European Commission decided to engage in a new initiative together with the European Investment Bank (EIB), known as ‘Joint Assistance to Support Projects in European Regions’ (JASPERS), to provide the Member States that joined the EU in 2004 or later with independent technical advice. The initiative was established by the Commission, the EIB, the European Bank for Reconstruction and Development (EBRD) and later the Kreditanstalt für Wiederaufbau (KfW). The aim of the JASPERS initiative was to help the Member States, free of charge, to prepare high-quality proposals for large investment projects for funding through the EU’s Cohesion and European Regional Development Funds. The JASPERS initiative is managed by a dedicated department within the EIB and mainly financed by the Commission through the EU budget. The actual cost of JASPERS between commencing operations in 2006 and the end of 2016 was around 284 million euro.

Between commencing operations in 2006 and the end of 2016, JASPERS completed 1 147 individual tasks (known as ‘assignments‘). Around two thirds of these concerned major projects. Poland accounted for around 27 % of all completed assignments (307) and Croatia for around 4 % (46). Between the same period, the Commission approved 963 major projects from the 2007-2013 programme period. Of these, around 53 % were supported by JASPERS. The total amount invested in these projects was around 77.6 billion euro; the total EU contribution was 46.2 billion euro.

Memorandum of Understanding

The Agreement for conducting the coordinated audit was signed between the State Audit Office of Croatia, the Supreme Audit Office of Poland as well as the European Court of Auditors (ECA) in March 2016. The audit approach was consistent with the International Standards of Supreme Audit Institutions (ISSAIs).

From left to right: Ivan Klešić, Oskar Herics and Krzysztof Kwiatkowski.

This co-operation project aimed at planning and conducting a performance audit of the effectiveness of the JASPERS initiative in the field of shared management in Structural/Cohesion funds. The participants benefited from the mutual exchange of experience, views and ideas. In particular, the ECA took advantage of the relevant working knowledge of the partner Supreme Audit Institutions (SAIs), based also on their decentralised organisation, particularly in the field of national (co-)financing to projects, whilst the SAIs benefited from the experience of the ECA auditors on EU co-financed projects.

Audit objectives

The audit objectives of the three SAIs had a similar focus from an EU respectively national perspective. The ECA assessed whether JASPERS had made a positive contribution to the development and overall quality of the EU-financed projects in those Member States it supported, as well as improved their administrative capacity. The SAI of Croatia assessed whether JASPERS’ impact on the project proposals submitted by Croatia for co-financing from EU-funds was efficient. The SAI of Poland assessed whether JASPERS resulted in an increased quality of grant applications and, as such, a better use of the European funds in Poland, as well as boosted the country’s administrative capacity, namely through increased competence, knowledge and experience.

The SAIs of Croatia and Poland conducted the audit in Croatia respectively Poland at different ministries and bodies responsible for projects receiving JASPERS assistance and at the level of beneficiaries. In addition to Croatia and Poland, in which the ECA conducted the audit in close cooperation with their SAIs, the ECA performed audit activities in Malta and Romania as well as at the Commission and the EIB. The audits covered the period from when JASPERS began operations in 2006 until the end of 2016.

Within the respective audit mandates of the three SAIs, the main audit topics and audit criteria were established in a coordinated way between the three audit teams. The audit work was carried out in full cooperation and a spirit of trust, while maintaining the independence of the three SAIs. Each SAI produced its own audit report and published it individually.¹ Summaries of the key messages of the three audit reports which have been prepared individually by each SAI are attached to this Synthesis Report (Annexes I-III).

Summary of key results

Overall, JASPERS was a good initiative to support new Member States in the development and approval of project proposals. Because of missing needs assessments before lancing the initiative and no clear measurable objectives it was however not possible to determine whether JASPERS’s purpose had been achieved. Better definition of objectives, milestones and criteria would allow JASPERS to focus on those activities which add the most value.

In general, the assistance provided by JASPERS was relatively comprehensive, regardless of the precise point at which JASPERS became involved. JASPERS had a positive impact on the quality of the underlying project documentation and thereby contributed to quicker project approval at Commission level. The impact of JASPERS was more visible when it was involved at an early stage of project preparation.

Over time, JASPERS increased its focus on building Member States’ administrative capacity. National authorities and project beneficiaries both stated in general that JASPERS had a positive impact on their administrative capacity. Still, JASPERS’s impact did not yet result in higher degrees of independence from JASPERS’s assistance.

Shortcomings were identified in the monitoring and evaluation of JASPERS activities. Insufficient records and information were available to allow determining possible improvements to the assistance provided by JASPERS. This resulted in missing potential gains in efficiency and effectiveness of the operation of JASPERS.

The coordinated audit showed that complex issues with several players at EU and national level can be examined more comprehensively and efficiently by international cooperation. The three SAIs will make this Synthesis Report available to the public on their own websites and disseminate it to national and international stakeholders.

Signatures

For the State Audit Office of Croatia	For the European Court of Auditors	For the Supreme Audit Office of Poland
Ivan Klešić Auditor General	Oskar HERICS Member	Krzysztof Kwiatkowski President

Annex I

European Court of Auditors’ report on Joint Assistance to Support Projects in European Regions (JASPERS) - time for better targeting

Upgrade Ten-T road Network, Malta

Source: Office of the Deputy Prime Minister of Malta

Key findings

There were shortcomings in the definition of JASPERS’s main objectives and roles and responsibilities. As a result, JASPERS did not target its assistance sufficiently. While JASPERS was conceived as an initiative for the 2007-2013 programme period, it became a longer-term initiative, as it was extended to the 2014-2020 programme period. It has no clear measurable objectives to signal that its purpose has been achieved. At the start of the 2014-2020 programme period, JASPERS also began to support phased projects and further encouraged Member States to increase their use of its free-of-charge assistance during the project implementation stage. Neither of these was a priority.

There were significant weaknesses in the setting-up of the new Independent Quality Review (IQR) function for the 2014-2020 programme period. The functional independence of JASPERS’s IQR service was detracted because the same person was responsible for signing off the work of both IQR and advisory divisions. Moreover, by October 2017, no procedures defining the IQR processes in JASPERS had been adopted. We noted a high risk of a lack of impartiality in relation to JASPERS’s advisory function. Furthermore, although IQRs were a crucial part of the Commission’s approval process for major projects, the Commission had no direct input into how IQRs were conducted in practice, opening itself to the risk that they would not be delivered as intended.

JASPERS contributed to quicker project approval and had an impact on the quality of projects. Moreover, JASPERS had a positive impact on the quality of the underlying project documentation. We also found that assistance provided by JASPERS was relatively comprehensive. In the 2007-2013 programme period, the Commission overall needed less time to approve major projects if they had been assisted by JASPERS. However, we observed that JASPERS could generally not impact on the absorption of EU funds. For the 2014-2020 programme period, we noted that the overall time needed to approve major projects could be significantly reduced. Lastly, we observed that major projects assisted by JASPERS had been less frequently affected by legality and regularity errors during our 2014 and 2015 compliance audit exercises.

JASPERS’s impact on administrative capacity did not yet result in higher degrees of independence from JASPERS’s assistance. In general, national authorities and project beneficiaries both stated that JASPERS had a positive impact on their administrative capacity, although some of them stated that they remained strongly reliant on JASPERS assistance. We found no evidence to confirm whether this improvement in administrative capacity had actually materialised. We noted that, over time, JASPERS had increased its focus on building Member States’ administrative capacity.

This, in combination with significant shortcomings in the monitoring of JASPERS activities, puts at risk the successful operation of the initiative, particularly in terms of its efficiency and effectiveness. The EIB was unwilling to provide information on JASPERS’s real costs, and the Commission was only partially able to demonstrate the plausibility of the standard costs of JASPERS used up to 2014 for staff members provided by the EIB.

Rehabilitation of the railway line Brasov – Simeria, component part of the IVth Corridor for the train speed of maximum of 160 Km/h, section Coslariu – Sighisoara, Romania

Source: National Railway Company „CFR” SA”

Recommendations

The Commission should:

1. take more control over the strategic planning of JASPERS, including all types of JASPERS activities, keeping in mind that JASPERS was originally conceived as a temporary initiative, and therefore allowing it to be phased out if its main objectives have been met;
2. take immediate actions to mitigate the high risk of a lack of impartiality when JASPERS carries out IQRs to appraise projects which have received JASPERS advisory support;
3. obtain full access to the relevant documentation to verify the quality of JASPERS’s IQR procedures;
4. target JASPERS assistance according to the project’s stage of development and maintain its focus on providing advisory services to major projects;
5. integrate JASPERS activities into its own technical assistance strategy, with the aim of improving coordination with JASPERS on carrying out ongoing (as opposed to ad hoc) activities for developing Member States’ administrative capacity; adjust the role, nature and intensity of JASPERS’s capacity-building activities in Member States over time, to provide incentives for them to reach a sufficient and sustainable level of administrative capacity;
6. introduce a comprehensive system for monitoring the extent to which JASPERS’s long- and short-term objectives have been met; ensure that future evaluations of JASPERS are sufficiently comprehensive, and draw conclusions on the achievement of its main objectives; take action to optimise JASPERS’s efficiency and effectiveness; and ensure that JASPERS’s costs are reasonable, and that they reflect the actual costs incurred.

Annex II

State Audit Office of Croatia’s report on Impact of JASPERS programme initiative on the submitted project proposals of the Republic of Croatia for co-financing from EU funds

The main objective of the audit was to assess the efficiency of JASPERS’ impact on the project proposals submitted by Croatia for co-financing from EU funds. The auditees were Managing Authorities responsible for implementation of OPs in Croatia for the programming period 2007–2013 and the programming period 2014–2020 as well as other bodies responsible for the implementation of the projects, selected by sampling method (six projects)².

Based on the conducted audit procedures and identified facts, the State Audit Office of Croatia assess, following the established criteria and taking into account that the auditees do not act entirely in accordance with the prescribed roles and responsibilities, that JASPERS activities have a positive impact on the preparation and approval of project proposals, but that in certain audit areas minor weaknesses and gaps in the implementation of JASPERS activities are determined, and that an impact of JASPERS programme initiative on the submitted project proposals of Croatia for co-financing from EU funds is partially efficient.

Construction of the bridge - Čiovo island in Trogir with access roads, Croatia

Source: Feasibility study, Croatian Roads ltd.

Key findings

Competent bodies responsible for JASPERS activities didn’t conduct “needs assessment” for using JASPERS meaning there are no written procedures for the selection of projects for using JASPERS technical assistance. Also, on the basis of collected documents, it was identified that data in the Action Plans are not regularly updated, status and progress of the implementation of assignments are incorrect or include assignments that are completed or cancelled and also some of the projects that JASPERS work on are not recorded in the Action Plans.

Competent bodies do not analyse areas and topics which they can prepare independently and do not continuously carry out specific programs for their employees with a view to transfer knowledge and experience gained through work with JASPERS. Also, they have not developed a systematic approach in monitoring the impact of JASPERS activities meaning there are no records or detailed information on duration of JASPERS assignments nor the number and type of recommendations.

There were shortcomings in following procedures concerning the ways of cooperation with JASPERS which makes it difficult to monitor the impact of JASPERS and negatively affects the transparency of implementation of JASPERS activities. Shortcomings were also identified within the planning of horizontal activities meaning that a large number of assignments was cancelled, there was an informal ad hoc approach identified in choosing and conducting horizontal tasks, as well as the lack of official Guidance Notes.

Through the review of six projects in the sample and analysis of the records of the total number of JASPERS assignments in Croatia, it was identified that the need for JASPERS assistance is not decreasing. It was identified that the project documentation in specific projects, before JASPERS involvement in its preparation, had certain shortcomings related to specific areas and topics because the institutions were not prepared well enough for the work on preparation of the project documentation. Also, it was identified that the impact of JASPERS is more visible in projects where it was involved at an early stage of project preparation (project concept and programming or pre-feasibility study) than when it was involved at a later stage and worked only on review of already prepared documentation (feasibility study and project application).

Although auditees are not yet fully prepared to independently prepare project documents without JASPERS help, it was identified a positive impact of JASPERS on the transfer of knowledge through the work on the preparation of projects, the implementation of horizontal tasks and providing presentations.

Development of the research infrastructure on the campus of the University of Rijeka

Source: University of Rijeka

Recommendations

In order to contribute to the elimination of the identified weaknesses and gaps, and thus to increasing the efficiency of impact of JASPERS programme initiative on the submitted project proposals of Croatia for co-financing from EU funds, the SAO gave the following recommendations to auditees:

1. Conduct a detail needs assessment before including projects in the Action Plan and adopt written procedures to determine priorities in choosing projects for JASPERS technical assistance, regularly update Action Plans, determine the actual status of all JASPERS assignments, harmonize data with all competent bodies and regularly forward such updated data to JASPERS;
2. Clearly define the areas in which necessary knowledge is acquired for independent work, so that JASPERS help can be directed to topics that cannot be prepared independently and also regularly document the implementation of JASPERS co-operation in order to efficiently monitor the impact of JASPERS;
3. Keep the official records about the meetings and requesting from JASPERS to issue the official Guidance Notes with recommendations or provide other written evidence on communication, in order to monitor the impact of JASPERS recommendation on projects;
4. Systematically monitor the impact of JASPERS activities on projects and on strengthening administrative capacities, report in more detail on the activities of JASPERS and keep records of the duration of JASPERS assignments and the number and type of recommendations;
5. Clearly define the priorities among the projects that need JASPERS assistance, with an aim to use JASPERS in an efficient way and to consider the involvement of JASPERS in an early stage of project development. Also, continuously transfer experiences and knowledge gained through JASPERS activities to larger number of employees working on the preparation of projects for co-financing from EU funds.

Based on the contradictory procedure all auditees have fully accepted the findings and recommendations given by the State Audit Office, they stated which measures will be taken with the aim to eliminate the identified weaknesses and gaps. On the basis of a statement made by the auditees within the legal deadline of 60 days, auditees have confirmed that some audit recommendations have already been implemented (for example the Draft of need assessments which identifies needs, criteria and priorities for inclusion in the Action Plans has already been prepared, Action Plans were corrected and updated, data and internal documents were mutually harmonized, etc.), while for those recommendations that haven’t been implemented, they stated deadlines and responsibilities for their implementation.

Supreme Audit Office of Poland’s report on the functioning of the JASPERS initiative in Poland

Introduction

The audit was conducted at the Ministry of Development, the General Directorate for National Roads and Motorways in Warszawa, the Government of the City of Warszawa, the Marshal’s Office of the Mazowieckie Voivodeship, the Government of the City of Poznań, the Marshal’s Office of the Wielkopolskie Voivodeship and the Government of the City of Kraków. Direct audit procedures were supplemented with information sought under Article 29(1)(2)(f) of the NIK Act from all 16 Marshals’ Voivodeship Offices and from beneficiaries of JASPERS assistance.

Centrum Kongresowe (Grunwaldzkie)

Source: Krakowskie Biuro Festiwalowe W.Wandzel, wandzelphoto.com

Annex III

Key findings

The advisory assistance of JASPERS experts contributed to the improved quality of grant applications of particular projects. Those applications that obtained JASPERS advisory assistance received a smaller number of observations by the European Commission and a shorter processing time compared to the projects in the case of which applications were prepared without such assistance. In some projects, the assistance also contributed to the creation of added value.

The average processing time for applications submitted to the European Commission for major projects (the implementation period 2007-2013 and 2014-2020) was 314 days for the projects receiving JASPERS assistance and 473 days for the others. As a result, JASPERS advisory assistance resulted in shortening the processing time for the proceeding on the proposal, cutting it by one third on average. The shortest period of application examination was 65 days for the project using JASPERS assistance and the longest 1 837 days for the non-assisted project.

According to the audited beneficiaries, value added in terms of the implementation of the projects receiving JASPERS advisory assistance consisted in the following: shortening the project approval decision time by the European Commission, ensuring acceptance of the proposal by the European Commission, strengthening the justification of the suitability of the project for the development of the region and drawing attention to the need to create a budget reserve to enable the implementation of tasks unforeseen at the planning stage. Based on the responses received from those beneficiaries of JASPERS assistance not subject to the direct audit, it was found that for 30 projects (38 % of 78 projects implemented by them) JASPERS expert assistance contributed to the value added of the project.

The Ministry of Development prepared general principles for cooperation with JASPERS which were used for preparing JASPERS action plans. These principles were informational material for institutions involved in the implementation of the EU funds in Poland. The advisory assistance of JASPERS experts was given to projects and horizontal tasks included in the action plans, following consultations between the representatives of the Ministry of Development and JASPERS. NIK reported an irregularity: key documents were prepared in English only.

The functioning of JASPERS was based on annual Action Plans, including a list of projects and horizontal tasks receiving assistance in a given year. Since 2014, the Action Plan has taken the form of a so-called rolling plan. Projects and horizontal tasks that were unfinished as of 31 December 2013 were added to it, and its content was supplemented on a regular basis by the addition of new projects and horizontal tasks after prior arrangements made by JASPERS and the Ministry of Development. The plan was drawn up in English.

The list of projects and horizontal tasks in the JASPERS Action Plan was created based on proposals collected from the Managing Authority or from the coordinator of regional operational programmes. In the event of any objection expressed by JASPERS with regard to assistance for a particular project, the coordinator of regional operational programmes conveyed this information to the competent Implementing Authorities or coordinator of regional operational programmes. For example, the 2009 Action Plan did not finally include three of the submitted projects and horizontal tasks and the 2010 Action Plan omitted seven projects and horizontal tasks. To justify the rejection of individual projects and horizontal tasks, JASPERS indicated issues such as a lack of sufficient own human resources or the fact that the proposed project did not meet the criteria for a major project.

The scope of advisory assistance provided as part of the JASPERS initiative varied and was determined by the specifics of the project and the expectations of the beneficiary. Advisory assistance basically focused on the verification of the project documentation, including primarily the grant applications, feasibility study, environmental documentation and an analysis of costs and benefits. JASPERS cooperation with beneficiaries of the eight audited projects proceeded in an appropriate manner and in accordance with their expectations. Beneficiaries’ expectations of JASPERS assistance were compliant with the actual assistance received, whereas the cooperation period and the quality thereof was regarded positively. Any potential barriers that might have impeded communication with experts in a foreign language were remedied by providing translations. Except for one project, the resulting assistance helped to improve the quality of applications and project documentation.

An example of effective cooperation between the Polish authorities and JASPERS was the project, ‘The purchase of rolling stock for regional passenger rail transport’, implemented by the Government of Wielkopolskie Voivodeship. The project involved the purchase of 22 multi-segment electric multiple units to support regional rail passenger traffic on three rail routes in the Wielkopolska Voivodeship. The period of cooperation between the beneficiary and JASPERS experts was relatively short at 6 months. JASPERS assistance helped to shorten the processing time of the grant application. The European Commission did not include any more comments and JASPERS addressed all the issues raised by the Commission. The time from submission of the JASPERS Completion Note to the European Commission to the date of the decision was only 84 days. JASPERS advisory assistance made it possible to develop, among other things, model contract provisions for the provision of public services, for which the assumptions, particularly in terms of permissible public aid and control mechanisms, were used in other projects in the field of public transport.

Based on the responses received from the beneficiaries of assistance from JASPERS, which were not subject to immediate inspection measures, it was found that they received assistance in the following areas: cost-benefit analysis (95 %), verification of general proposals and feasibility study (88%), risk analysis/sensitivity (79 %), environmental issues (78 %), review/preparation of the application (75 %). The average cooperation between JASPERS and beneficiaries lasted 22 months.

The audit did not provide any direct proof confirming the improved administrative capacity of Polish entities working with JASPERS. The positive evaluation in the field was a result of an opinion expressed by interested parties that emphasised that contacts with experts from JASPERS yielded additional quality for developing other projects. Besides acquiring the knowledge needed to prepare better quality projects, the experience gained in course of the cooperation with experts from JASPERS yielded additional effects in terms of creating framework documents. An example here can be the Managing Authority of the Operational Programme Infrastructure and Environment, which, based on the sectoral guidelines for the environmental sector that were co-written by JASPERS experts for the 2007-2013 framework, in cooperation with the Ministry of Environment and the National Fund for Environmental Protection and Water Management, developed for the years 2014-2020 model solutions for the cost-benefit analysis of investment projects applying for funding in the environment sector (including the application of the ‘polluter pays’ principle and financial performance indicators).

The NIK believes that although improved administrative capacity of the audited institutions related to JASPERS advisory assistance may be evaluated only in relation to future independently developed projects, the prior cooperation and acquired experience must be perceived positively in terms of preparing projects and, as such, increasing a chance for effective use of European funds.

In the whole period when JASPERS was present in Poland, Polish authorities conducted no complex evaluative research for the JASPERS initiative, whereby results could be used for determining possible improvements to the advisory process. The NIK believes that the starting point for conducting this type of research should be a complete, reliable and up-to-date database of projects and horizontal tasks receiving JASPERS assistance.

Recommendations

In connection with the audit findings, the NIK requests the Minister of Development to consider the need to create a complete database of projects and horizontal tasks aided by JASPERS in Poland, and to carry out ongoing evaluative studies, as well as an assessment of the current contribution of JASPERS into the use of European funds in Poland.

Furthermore, given the results of the audit, the NIK recommends that beneficiaries of JASPERS advisory support conduct an in-depth analysis of the experience gained from cooperation with JASPERS experts in various projects and tasks.

Budowa trasy tramwajowej Os. Lecha-Franowo

Source: Wyniki kontroli NIK.

Audit team

This coordinated audit was performed by three dedicated teams of the State Audit Office of Croatia, the Supreme Audit Office of Poland and the European Court of Auditors (ECA).

The audit of the State Audit Office of Croatia was headed by Auditor General Ivan Klešić and coordinated by Assistant Auditor General Lidija Pernar. The audit was led by Anita Materljan, head of department for EU funds, Martina Jurjević and Kord Kordić, auditors.

The audit of the Supreme Audit Office of Poland was performed by Regional Branch in Kraków, headed by Director Jolanta Stawska. The coordinating team consisted of Marta Pankowska and Wojciech Dudek. Also auditors from Regional Branches in Warszawa and Poznań participated in the audit.

The ECA audit was produced by Audit Chamber II – headed by ECA Member Iliana Ivanova – which specialises in structural policies, transport and energy spending areas. The audit was led by ECA Member Oskar Herics, supported by the head of his private office, Margit Spindelegger, Emmanuel Rauch, principal manager, Rafal Gorajski, head of task, and Thierry Lavigne, Mircea-Christian Martinescu, Maciej Szymura, and Michaela Binder, auditors.

From left to right: Marta Pankowska, Mircea-Christian Martinescu, Jolanta Stawska, Emmanuel Rauch, Oskar Herics, Maciej Szymura, Ivan Klešić, Rafal Gorajski, Margit Spindelegger, Anita Materljan, Wojciech Dudek, Lidija Pernar and Thierry Lavigne.

Contact

THE STATE AUDIT OFFICE
Tkalčiċeva 19
10000 ZAGREB
CROATIA

Tel. +385 1 4813 302

Enquiries: revizija@reivzija.hr
Website: www.revizija.hr

EUROPEAN COURT OF AUDITORS
12, rue Alcide De Gasperi
1615 Luxembourg
LUXEMBOURG

Tel. +352 4398-1

Enquiries: eca.europa.eu/en/Pages/ContactForm.aspx
Website: eca.europa.eu
Twitter: @EUAuditors

NAJWYŻSZA IZBA KONTROLI
ul. Filtrowa 57, 02-056 Warszawa
Tel. +48 22 444 00 00, F +48 22 444 00 00

Skr. poczt. P-14, 00-950 Warszawa 1

More information on the European Union is available on the internet (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2018

© European Union, Heads of the State Office of Croatia, Supreme Audit Office of Poland. Reproduction is authorized provided the source is acknowledged. For reproduction of the photos, please contact the copyright owners; exception: photos on the cover page (source: Pixabay, CCO license).