EU-wide stress tests for banks: unparalleled amount of information on banks provided but greater coordination and focus on risks needed

Executive summary

I

The EU-wide stress test is an evaluation of impacts that a common shock could impose on the financial position of large European banks. The 2010 Regulation founding the European Banking Authority (EBA) tasked the EBA with initiating and coordinating EU-wide stress tests, in cooperation with the European Systemic Risk Board (ESRB).

II

Stress tests have been carried out since 2011, and all of them have been conducted according to the bottom-up approach where banks produced the results yielded by the shock scenario based on the methodology approved by the EBA. Verification of the quality of the results was largely entrusted to the relevant competent authorities (CAs) (national authorities or the European Central Bank). In other jurisdictions with large financial systems, such as the United States, supervisory authorities rely on a top-down approach, which gives them a much larger degree of control over the results produced by banks.

III

To ensure that methods, practices and the results projected by banks are comparable and reliable, the regulation specifically states that the EBA has authority to request information directly from banks and CAs to conduct specific reviews and on-site inspections, with the EBA participating in these activities.

IV

We focused on the EU-wide stress test run by the EBA in 2018. In particular, we assessed whether the stress test was fit for purpose, whether the EBA had sufficient assurance about the robustness of the figures calculated by the individual banks, and whether the publication of the results allowed stakeholders to conclude whether the system was resilient.

V

We reviewed relevant documentation and interviewed staff from the EBA, the ESRB, and the European Central Bank. We also conducted surveys among banks and CAs and visited two national CAs.

VI

With very limited staff resources and a lot of effort the EBA coordinated the exercise involving many stakeholders and within tight deadlines.

VII

We found that - as key decisions at the EBA are taken by representatives of national supervisors - an EU-wide perspective was insufficiently taken into account in the design and implementation of the stress test. The impact is visible in the various phases of the stress test exercise.

VIII

At the start of the process, the EBA did not specify the risks nor the level of severity it deemed relevant for the stress-testing procedure. In turn, the ESRB who developed the stress scenario obtained substantial input from the ECB and national central banks and authorities. We found that, as a result, the EBA lacked control over important stages of the process and thus:

• important systemic risks were subject to a low level of stress, or none at all;
• the shock was not triggered by events from within the EU financial system but by an economic downturn;
• the intensity of the economic shocks varied significantly from country to country, with the shock often being less severe where the economy was weaker and the financial system was more vulnerable. A minimum level of severity to generate stress was not ensured.

IX

Secondly, although the regulation entrusted the EBA with ensuring the reliability and comparability of methods, practices and results, the EBA did not exercise authority beyond initiating, providing methodology for and broadly coordinating the stress-test activities. It decided to fully rely on the CAs for verifying the way banks implement the methodology and estimate stress impacts. The EBA did not challenge the CAs’ quality control using the powers conferred on it by the regulation. In effect, the EBA does not currently have the resources it needs to exercise full oversight. Therefore it did not request specific reviews, nor did it participate in any on-site inspections, and - except for defining methodology - little other activities took place to ensure the comparability and reliability of results. The EBA manual describing the quality assurance to be carried out by CAs was not binding, thus leaving CAs with a large degree of discretion.

X

Thirdly, EBA’s publications showed an unprecedented level of transparency as a large amount of bank data was made accessible. However, in its reports the most critical information was missing, namely the capital requirements for each bank and how many banks would have breached them under stress. Moreover, while the intensity of the stress/shocks varied significantly between countries, the EBA report did not explain that low impacts (small capital depletion) resulting from the stress for banks in certain countries were not necessarily due to the healthy situation of a bank but to a low level of stress.

XI

Given this situation, and in order to achieve its objective of detecting (the build-up of) systemic vulnerabilities, we make the following recommendations to ensure that the stress test is a more meaningful exercise:

1. The EBA should use its legal powers to enhance its control over the stress-test process.
2. The EBA should develop a top-down approach for stress tests to complement the current bottom-up approach.
3. The selection of banks for the stress test should be based not only on size but also on risk and systemic relevance and ensure appropriate geographical coverage.
4. The EBA should ensure that the stress test fulfils its purpose of assessing resilience against adverse market developments. In particular, it should vary the stress scenarios from one exercise to another, take due account of risks emanating from within the EU financial system, and ensure a minimum level of stress.
5. The EBA should publish banks’ institution-specific minimum capital requirements, and present the results in a way that allows users to put them into perspective.
6. The EBA should request the additional resources needed to fully carry out its role as specified in the regulation.
7. The European Commission should address the appropriateness of EBA’s governance structure with the next review of the EBA Regulation.

Introduction

Stress testing of banks

01

A stress test is an evaluation of a bank’s financial position if placed under severe pressure. The basic idea of a stress test is to project what would happen to the main parameters of a bank’s viability in the event of one or more large negative shocks. These shocks can be triggered by (i) market-wide events such as a severe recession, a stock market crash, or a loss of confidence in banks; (ii) “idiosyncratic” events, i.e. a shock that is bank-specific and not necessarily correlated to the overall economic situation; or (iii) a combination of the two.

02

Originally, stress testing was a tool used by banks themselves as part of their internal risk management, but later also by their supervisors. Generally speaking, this stress testing was micro-prudential in nature as it focused on the resilience of individual institutions, the scenarios did not assume system-wide spillovers, and the process focused mainly on investor and depositor protection.

03

The financial crisis highlighted weaknesses in micro-prudential stress-testing practices. According to the 2009 ‘de Larosière Report’, “stress-testing too often was based on mild or even wrong assumptions”¹. In the aftermath of the financial crisis, policymakers recognised the need also to focus on systemic shocks affecting the whole financial system simultaneously. This involves the use of economic and financial shocks, the examination of spillovers, and the impact of shocks on the financial system as a whole.

The EBA’s mandate and other actors involved

04

The EBA, which was established in 2010, was given the authority in cooperation with the European Systemic Risk Board² (ESRB) of initiating and coordinating Union-wide stress tests. The EBA Regulation also stipulates that the EBA should “ensure comparability and reliability of methods, practices and results”³.

05

These stress tests seek to impose a common, consistent shock on a large share of European banks to assess the resilience of the EU financial system as a whole. To do this, a baseline scenario (best estimate of future macroeconomic conditions) and an adverse (stress) scenario (significantly more negative estimate) are developed.

06

The EBA initiated and coordinated EU-wide stress tests for banks in 2011, 2014, 2016 and 2018. In general, a stress-testing approach can be top-down or bottom up. Under a top-down approach, it is the supervisor who generates the adverse scenario and calculates the impacts on banks, as is the case for example in the UK, the US, and Japan. Early in the process, the EBA opted for a bottom-up approach, where the supervisor generates the scenario but the banks produce estimates of impacts resulting from shocks on their main financial parameters. The option of a top-down approach was discussed by the EBA on several occasions – the last time was in December 2016 – but was rejected by a substantial majority of its Board of Supervisors’ members.

07

The roles of the various stakeholders (see Annex I) can be described as follows, using the context of the 2018 stress test:

• the EBA prepared the methodology and, as general non-binding guidance, a quality assurance manual, collected the results from the banks after they had undergone a quality assurance process by the relevant supervisory authorities (the competent authorities, or “CAs”), did numerical and plausibility checks, and published the results;
• the ESRB General Board⁴ approved the adverse scenario. In advance of this, the adverse scenario was developed and discussed by the ESRB Task Force on Stress Testing and by the ESRB Advisory Technical Committee, respectively⁵. The Task Force relied heavily on ECB resources (see Annex I);
• the ECB and national central banks provided the macroeconomic projections that served as the baseline scenario;
• the CAs (the ECB for euro area banks and national authorities for non-euro area banks) were responsible for quality assurance of the banks’ projections of stress impacts. Limited resources and its complex governance have prevented the EBA from activating its powers conferred by the EBA Regulation⁶.

08

The EBA’s governance structure has been based on considerable involvement by national authorities. Its Board of Supervisors comprises representatives of national supervisors. Under the current legal framework, the representatives also select a candidate to serve as Chairperson, whose selection may however be objected to by the Parliament. Although the regulation stated that members of the Board of Supervisors should “act independently and objectively in the sole interest of the Union as a whole”⁷, their appointment is not subject to any approval by EU bodies; they remain as officials of CAs and can be replaced at any time.

09

This can give rise to tensions as members of the Board of Supervisors may defend purely national interests without taking sufficient account of the wider European interest. Already in 2014, the Commission recognised⁸ that these tensions mean that at times decisions are not taken, in particular in the area of regulatory and supervisory convergence, or promote decisions that are more geared towards national rather than broader EU interests. Therefore, in 2017, it submitted a proposal to amend the regulations of the European Supervisory Authorities (including the EBA)⁹ dealing amongst others with funding and governance issues. In particular, one of the aims was to ensure that decision-making was more EU-oriented than at present. In particular, the Commission proposed that the EBA should have an executive board with full-time members, based on a shortlist drawn up by the Commission, and appointed by the Council, which would not have voting rights on the Board. However, the co-legislators did not reach an agreement on this proposal. Nevertheless, with the intention of enhancing the governance of the European Supervisory Authorities by other means, the political agreement of 21 March 2019 strengthens the position of the Chairperson.

Audit approach

10

Considering that there was some criticism of past EBA stress tests, we decided to carry out an audit on the 2018 stress test while also considering aspects of the 2016 stress test. The audit aimed at assessing whether the governance and implementation of the EBA stress tests were sufficient to provide a clear answer as to whether the EU financial system is resilient. To that end, we asked:

1. Was the stress test fit for purpose?
2. Did the EBA have assurance that banks’ calculations were correct?
3. Did the publication of the results allow stakeholders to assess the resilience of the system?

11

The audit criteria were derived from relevant legal instruments, the international standards set by the Bank of International Settlements (BIS) and the EBA itself, and best practice at other stress-testing authorities such as the Bank of England and the US Federal Reserve. Further details of the criteria are provided in the various sections of this report.

12

Audit evidence was collected on the basis of:

1. a review of relevant documentation from the EBA, ESRB and ECB;
2. an online survey of staff from competent authorities and supervised banks;
3. visits to competent authorities;
4. interviews with EBA, ESRB and ECB staff.

13

The audit is expected to provide input into the debate on the benefits and costs of stress test exercises as well as the pros and cons of different methodological approaches.

Observations

The EBA stress test had shortcomings for assessing resilience against systemic risks

14

The EU-wide stress-testing regime is designed¹⁰ to assess the resilience of EU financial institutions to adverse market developments, and to evaluate the potential for systemic risk to increase in situations of stress. Adverse market developments are defined in the EBA Regulation as micro-prudential trends, potential risks and vulnerabilities¹¹.

15

Stress testing is expected to identify those institutions that may themselves pose systemic risk, in particular in times of stress, and to ensure that these risks are mitigated¹². It is up to the EBA Board of Supervisors to decide which banks should undergo the stress test. Relevant benchmarks for stress testing are produced by the Basel Committee on Banking Supervision¹³ and the EBA itself¹⁴. They conclude that stress tests should have clearly articulated and formally defined objectives, and that scenario design should be aligned with them. The stress test should be sufficiently severe but plausible.

16

Therefore, we examined whether:

1. the sample of participating banks was appropriate for the purpose of the exercise;
2. the risks identified were appropriate for the purpose of the exercise; and
3. the stress imposed was sufficiently severe to assess resilience.

Certain risky banks were excluded from the stress test

17

The EBA had the objective of achieving wide coverage of EU banking assets and also capturing the biggest banks. In its sample, it therefore included only banks with a minimum of € 30 billion in consolidated assets.

18

The number of participating banks has fallen since the first round of stress tests. In 2011, 90 banks in 21 countries participated, while by 2018 this had fallen to 48 banks in 15 countries: nine countries where the ECB is the main supervisor¹⁵ and six countries where the ECB is not the main supervisor¹⁶.

19

Not all banks that exceeded the threshold of € 30 billion were included in the final sample: the largest banks were included until the sample covered roughly 70 % of euro area banks in terms of total consolidated assets as well as roughly 70 % of non euro-area banks. This meant that the actual threshold for banks in the euro area was € 100 billion which led to the exclusion of some countries with weaker banking systems.

20

We also found that the Board of Supervisors eventually excluded seven banks with assets above € 30 billion, as they were either undergoing restructuring¹⁷ or merging with another bank, or their consolidated assets had dropped below the minimum threshold by the time the sample was adopted. However, banks that are undergoing restructuring and that have received State aid are amongst the most vulnerable. Lastly, amongst the excluded banks were banks in which capital gaps eventually emerged.

21

We found no evidence that the EBA Board of Supervisors had discussed the pros and cons of using additional risk-based criteria to select banks. The EBA’s decision to use size as the sole criterion had drawbacks such as:

• none of those banks with a high share of non-performing loans (NPLs), based in five countries, were included, nor were banks with high exposure to domestic sovereign and other public debt¹⁸;
• information on banks that are valued by markets at a rate much lower than their book equity was not used in the selection process, either.

Significance of some relevant systemic risks not appropriately reflected in the adverse scenario

22

In line with the EBA Regulation, the EBA is obliged to initiate and coordinate the EU-wide stress test exercise in cooperation with the ESRB. However, there are no formal arrangements for the respective roles of the parties. Over the years, a practice has been developed which involves the EBA asking the ESRB to prepare the adverse scenario with the aid of various committees, in which EBA representatives participate (see Figure 1 of Annex I).

23

The 2018 adverse scenario was developed by the ESRB Task Force on Stress Testing, which comprised staff from the ESRB Secretariat and ESRB Members¹⁹, and included input from the ECB. The Task Force was chaired by an ECB representative (the deputy Director-General from the Directorate for Macro-Prudential Policy and Financial Stability) and relied heavily on ECB resources (staff, models and data). It made extensive use of these ECB resources²⁰ for calibrating the models that produced the variables that banks were obliged to use for their calculations.

24

Considering that the aim of the stress test is to evaluate how systemic risk would increase in a situation of stress (see paragraph 14), the starting point for the development of a stress test scenario should be to identify the most important risks both in terms of their systemic nature and their relevance. A meaningful level of stress should then be applied to the key risk drivers.

25

Therefore, we examined whether:

1. the most important systemic risks had been identified;
2. these risks were the drivers of the stress that was applied.

Not all systemic risks were taken into account

26

The EBA Board of Supervisors did not formally communicate to the ESRB its views on the risks that should be stressed (i.e. serve as a trigger) at the start of the process, be they risks emanating from the EU financial sector, country-specific risks or risks originating from individual banks or groups of banks that could seriously harm the financial system. While there is some overlap in the membership of committees that designed the adverse scenario, the risks were only approved by the EBA Board towards the end of the process (December 2017).

27

Risk identification was thus mainly left to the discretion of the ESRB, after discussion at constituent committees, including its Task Force on Stress Testing. The mandate of this Task Force explicitly states that it should reflect on how systemic risks – including those affecting specific sectors or a few countries – can be captured, and that the approach adopted should aim to create a sufficient level of stress on banks.

28

The ESRB runs a regular risk assessment exercise. This is also used for the purpose of the stress test. Other input factors for the stress test were (i) the result of a bottom-up survey (BUS) of ESRB members (mainly macro-prudential authorities); (ii) ESRB members’ direct input; (iii) discussions and presentations at ESRB expert groups; (iv) a range of data included in the ESRB risk dashboard²¹; and (v) ECB internal analysis.

29

In the bottom-up survey, one of the input factors to the risk assessment, macro-prudential authorities are asked on a quarterly basis to identify and rank a number of financial stability risks pertinent to their own economies and to the EU. However, the outcome of the survey had shortcomings which could impact on the risk identification and aggregation process and have the potential to bias it, for example:

• macro-prudential authorities are systematically more positive in their assessment of their own countries than of the situation in the EU as a whole. This may lead to risks which stem from one country or sub-set of countries being under-weighted;
• there is considerable variation in the tone and substance of the qualitative assessments made by the macro-prudential authorities. In terms of substance, some authorities stress the downside in their answers, while others stress the opposite.

30

The ESRB General Board chose four risks as drivers of the adverse scenario as explained in the scenario’s narrative (see Annex II). While the risks stemming from asset quality in the banking sector, e.g. issues related to non-performing loans, were ranked as important in the bottom-up survey they were not included as a main risk or shock in the adverse scenario²², although non-performing loans were the cause of most bank bail-outs after the financial crisis (for further details see also paragraph 37 f.).

31

Liquidity risks for banks themselves were not within the scope of the exercise as it focused on the solvency of banks. The issue of covering liquidity risks was last discussed by the EBA Board of Supervisors in 2011²³. In comparison, the IMF, which also runs stress tests to assess the resilience of the euro area banking system, used a two-pronged approach involving both liquidity and solvency testing.

32

The ESRB did not identify risks possibly stemming from individual banks or groups of banks despite a clear mandate²⁴ to also identify risks that individual institutions pose to the financial system. Rather, the risk identification process involved looking at aggregates at national, euro-area or EU level.

33

The EBA regularly publishes a risk dashboard²⁵, which consists of identifying and monitoring systemic risks. For each risk, the EBA dashboard describes the risk drivers and risk level, i.e. the probability of risk factors materialising and the likely impact on banks, including statistical trends without commentary. However, these dashboard indicators did not have an explicit role in the development of the stress test, nor did the EBA request that they be used as a key input.

The stress imposed resulted from an economic downturn rather than from a shock originating in the EU financial sector

34

Based on the risks identified, the adverse scenario itself should “determine the intensity of the shocks, the transmission channels and time horizon over which the stress factors can affect the banks”²⁶. However, there was no formal discussion or decision by the EBA Board of Supervisors on the type of shocks to be imposed on banks in advance of the risk identification process. This was left to the discretion of the ESRB. Only towards the end of the process did the Chair of the Task Force present the scenario and motivating risks to the EBA’s Board of Supervisors.

35

The approach chosen by the ESRB in 2018, as for the previous stress tests, was a sequence of adverse macroeconomic and financial events, with an impact on variables such as GDP, unemployment, house prices and interest rates that would materialise over a three-year period. The baseline corresponds to the most current Eurosystem and ECB staff macroeconomic projections for the EU²⁷, and the adverse scenario consists of a set of deviations from the baseline over the same period for the main parameters.

36

The EBA stress test assessed the vulnerability of the system and of banks to joint macro-financial stress (an economic downturn) rather than to a severe financial shock, which would then generate a certain level of stress. However, according to a 2009 Working Paper by the Bank for International Settlements, empirical evidence does not confirm the implicit assumption of past macro stress tests, namely that it is a severe macroeconomic shock that breaks down a fragile financial system²⁸.

37

The choice of an economic downturn scenario did not reflect the significance of some risks, including their uneven distribution across countries. Also, the choice of a downturn scenario with financial risks being stressed implicitly made it impossible to establish sensitivities to specific systemic risks. In other words, the scenario was not based on a financial shock triggered by failures of large financial institutions or by systemic risks as identified in the EBA risk dashboard (see paragraph 33), such as:

• a sharp rise in central bank rates or a sharp rise in credit spreads for sovereign bonds of certain Member States which would further fuel a sovereign debt crisis;
• the remaining high NPL stocks in the light of possibly increasing impediments to NPL reduction and the risk from elevated levels of indebtedness.

38

Moreover, the downturn used was triggered by events outside the EU. Neither the 2018 adverse scenario nor any of the previous adverse scenarios used an event or a risk within the EU as a trigger for the adverse scenario. Nor was consideration given to an event or a risk from within the banking sector²⁹ for use as trigger, in spite of the fact that the bottom-up survey (see Annex II) indicated two of the four most important risks as originating within the banking sector. Amplification effects (e.g. firesales of portfolios, one bank failing and affecting the credit spreads of others) and the varying degree of legacy issues across countries were not included in the scenario, either, although this was achieved by proxy to some extent in the scenario variables (e.g. a rise in interest rates).

2018 adverse scenario did not ensure minimum severity for all countries and was less severe than the financial crisis

39

When generating the 2018 scenario, the relevant ESRB Task Force reflected on criticism from the previous exercise, such as that the scenario (i) had not been sufficiently forward-looking or (ii) had been less relevant for some countries, with uneven severity.

40

Neither the EBA nor the ESRB clarified ex ante what they consider “severe”. Although the Task Force made comparisons at Member State level, the main focus of the severity comparisons, both internally at the ESRB and in publications, concerned the level of severity at aggregate EU and euro-area level. This was at the expense of a focus on the path of the parameters at Member State level, which is where many of the vulnerabilities might be pronounced.

41

Therefore, we examined the process for developing the scenario (including modelling aspects) and the proposed path for a number of parameters.

42

We found that national central banks and supervisory authorities were heavily involved in generating the adverse scenario through their presence in the Task Force and the ESRB General Board. For example, much of the modelling outputs were driven by what are known as basic model elasticities (BMEs) which are supplied by the national central banks as part of the ECB’s regular forecasting process.

43

Such a prominent role for national authorities and the limited ability of the EBA to control the process was not conducive to achieving an unbiased, objective adverse scenario that would take due account of banks’ and countries’ vulnerabilities in an EU-wide consistent manner. As a result of the discussions within the ESRB structures, a number of ad hoc decisions were taken with regard to the adverse scenario (see Box 1).

44

Next to the national central banks, the ECB was also heavily involved in scenario development, particularly Directorates from its monetary policy rather than its supervisory function. As a result, the ESRB, on the one hand, did not consider possible consequences of future monetary policy decisions as a trigger for an adverse scenario³⁰. On the other hand, it assumed that monetary policy would (i) limit the increase of long-term interest rates in the EU under the adverse scenario and (ii) prevent a substantial widening of credit spreads for sovereign debt³¹ (i.e. the difference in yield between bonds issued by different national governments). Such a widening occurred during the European debt crisis. Hence, the relatively muted interest rate increases in the adverse scenario.

45

All in all, countries were subject to very different shock levels and thus banks were exposed to very different shock levels depending on their geographical exposure. For example, Sweden had the largest GDP shock in absolute terms, more than twice as large as Italy’s (see graphic presentation for all countries in Figure 4 later in this report).

46

In order to assess the severity of the 2018 exercise, we looked at the scenario along a range of dimensions, by benchmarking aspects of the scenario against previous EBA stress tests, historical events (the 2008 financial crisis), and the scenario(s) used by other stress testing authorities as described in the following paragraphs.

47

As mentioned in paragraph 35, the adverse scenario was defined relative to the baseline over the forecast period, and the EBA stated in its communication that the 2018 stress test was the most severe compared to previous EBA stress tests in terms of the deviation for the GDP variable (deviation of 8.3 %). However, it is the absolute decline between the starting point (2017 GDP) and the end point under the adverse scenario that is the most relevant (see Glossary). In other words, even a large deviation can mean a very low stress if the baseline is strong, and revised up as it was the case.

48

In fact, the 2018 adverse scenario was essentially finalised in late November 2017. However, a new baseline was completed in December 2017 and the adverse scenario had to be attached to it. As the baseline had become more positive, the absolute levels of stress relative to the starting point were lower for many countries. The EBA therefore sought to ensure that minimum levels of stress were imposed (see Box 1). This came very late and meant a lot of changes very close to the end of the process.

49

By comparison with the 2014 and 2016 EBA stress tests, the 2018 adverse scenario was more severe, for example for euro-area GDP and unemployment in terms of deviation from the baseline. However, given favourable projections for the baseline, the situation was different in absolute terms:

• for euro-area GDP, the 2018 exercise appeared stronger, albeit only slightly. However, this was not the case for the majority of Member States (see Table 1). The absolute decline was largest in 2018 – compared to previous exercises – in several large economies, most of which actually did quite well during the last recession. For other Member States, which suffered significantly during the last recession, the absolute decline in the scenario was relatively mild. This specific distribution of adverse impacts on GDP across Member States was not well explained;
• for unemployment, the 2018 exercise was slightly softer, because the baseline for unemployment was much more positive in the 2018 exercise.

50

When examined from the perspective of individual economies and variables, the stress imposed is not consistently severe. The decline in GDP in absolute terms is not as severe as the financial crisis and its aftermath for most Member States, and the increase in unemployment is considerably less severe, in some cases by a very large margin (see Box 2).

51

The most relevant external benchmarks for the EBA exercise are the Bank of England stress testing exercise, the stress test run by the US Federal Reserve, and the stress test run by the IMF for the euro area. The 2018 EBA adverse scenario was equal or more severe than others in terms of GDP, but was weaker than the others for unemployment, long-term interest rates, credit spreads and real-estate prices (see Box 3 for examples).

52

The relevant international benchmarks suggest that stress tests can involve one or more adverse scenarios³². The minutes of the meetings of the EBA Board of Supervisors do not refer to any discussion or decision on whether one or more scenarios should be used. All EBA stress tests since 2011 have been run with one adverse scenario. By comparison the practice of the US Federal Reserve and the Bank of England is to generate two scenarios. Also, the IMF in its recent stress test for the euro area complemented its macro-financial scenario with a range of sensitivity tests³³.

Banks’ calculations remain a black box for the EBA to some extent

53

The EBA developed a methodology which defines how banks should calculate the stress impacts of the baseline and adverse scenarios.

54

The methodology involves the use of caps and floors to ensure a level of supervisory conservatism, specifically so that banks cannot benefit from the stress imposed in certain cases. For example, the interest income from non-performing loans, as projected by the banks, is subject to a cap to avoid it being overly optimistic.

55

Since it is the banks that calculate the results, these are subject to a quality assurance process.

56

We examined whether:

1. the methodology was appropriate;
2. the EBA gained sufficient assurance on the robustness of the banks’ calculations.

Choices made had an impact on the plausibility of results

57

The methodology is adopted by the EBA’s Board of Supervisors. It can also grant exemptions to the methodology. Although such exemptions concern individual banks, there is no mechanism to ensure that voting members who have a potential conflict of interest do not participate in the vote. One such exemption occurred in the 2016 exercise when the Board approved a deviation from the methodology (to the benefit of one large bank in spite of EBA staff concerns). There were no such exemptions under the 2018 stress test.

58

Overall, we found the EBA methodology to be very comprehensive. However, for certain aspects the methodology lacked detail in terms of constraints and guidance, such as for the area of credit risk (unsecured parts of credits, bail-out purchases and treatment of non-mortgage collateral).

59

When applying the methodology, banks must translate the baseline and the adverse scenario (i.e. macroeconomic parameters) into risk parameters. This is often done via models³⁴. These model-based calculations result in changes to the balance sheet, the profit and loss account as well as to the risk-weighted assets (and consequently capital requirements³⁵). Box 4 provides an illustration, in very simplified terms, for credit risk.

60

For banks that were using their own models, their accuracy is crucial for the credibility of the stress-test results. However, being developed by banks themselves, such models can be biased³⁶ which demonstrates the need for rigorous quality assurance.

61

When banks do not have appropriate models for estimating credit risk parameters (i.e. PDs and LGDs, see Figure 3), they are expected to rely on the credit risk benchmarks generated by the ECB. These benchmarks were modelled by the ECB’s Directorate for Macro-Prudential Policy and Financial Stability (DG MF), a directorate within the ECB’s monetary function.

62

In addition to the resulting model uncertainty, we also identified shortcomings in some of the assumptions used.

63

The benchmarks should consist of a coefficient or set of coefficients for each risk parameter (PD, LGD, etc.) which measure the responsiveness of each risk to the individual macroeconomic variables. The methodology used for the generation of benchmarks and the benchmarks per se (including the coefficients) were discussed with national CAs. The benchmarks were approved by the EBA Board of Supervisors³⁷. However, the ECB does not divulge these precise coefficients to the banks concerned, nor does it publish them. Instead, a result for each portfolio type for each country is generated based on the macroeconomic variables in the baseline and adverse scenarios, and only this outcome is communicated to the banks. Altogether, this has reduced transparency³⁸, and prompted banks to comment in our survey that the generation of the credit risk benchmarks was a ‘black box’-type process.

64

The bottom-up approach was constrained by imposing a number of caps and floors (see paragraph 54). The EBA did some ad-hoc assessments (also including bank data) to estimate the impact of these caps and floors on the banks’ results. However, given its limited resources, the EBA has not been able to produce a comprehensive overview of the impact of these caps and floors on the results.

65

To enable banks to do their projections, assumptions need to be made. Some of these assumptions are imposed by the methodology; others can be made by the banks themselves. One key methodological choice was to work with the assumption of a static rather than a dynamic balance sheet, specifically an unchanged business model throughout the projection period, meaning that maturing assets and liabilities are replaced by items with similar characteristics in the banks’ projections.

66

The static balance sheet assumption has drawbacks:

• banks strongly impacted in a stress situation are likely to lose business opportunities and access to funding (notably wholesale or interbank funding). However, this cannot be reflected in the banks’ calculations;
• to some extent, banks would be able to counter the negative effects of the adverse scenario by using recovery options³⁹ such as asset sales. However, the static balance sheet assumption does not allow banks to project such actions in the adverse scenario. While we recognise that the EBA/CAs would have to assess the credibility and feasibility of such actions, we note that the 2018 Bank of England stress test included the use of ‘strategic’ management actions that a bank could realistically take in a stress scenario. Not taking this aspect into account hampers the assessment of the actual resilience of a bank that would suffer under the stress.

The EBA has limited insight into the robustness of banks’ calculations

67

The EBA does have the legal base⁴⁰ to become directly involved in checking the quality of banks’ models and results. Specifically, the EBA has the power to (i) request information directly from banks; (ii) require CAs to conduct specific reviews; (iii) ask CAs to carry out on-site inspections; and (iv) ask CAs to require banks to have relevant information independently audited.

68

However, due to the current governance structure and limited resources, the EBA relied fully⁴¹ on the CAs to do the quality assurance. Only for the 2011 exercise was quality assurance carried out directly by EBA staff, assisted by a team of national and ECB/ESRB experts.

69

The EBA produced standard templates that were to be filled in by the banks with all required data (up to 900 000 data points for the larger banks). Related template guidance was also provided by the EBA.

70

The banks had to submit their completed templates to the CAs, which had to perform checks to ensure the quality of the data. Thereafter, CAs had to submit the templates to the EBA.

71

After having accepted the files, the EBA ran automated data quality checks on the submitted templates. Any data quality issues identified were transmitted to the CAs for clearance.

72

Therefore, we examined whether:

1. the EBA had sufficient assurance on the quality checks carried out by the CAs;
2. the EBA’s own checks were such as to ensure the quality of the results provided by the banks.

The EBA had limited information on the quality checks done at CA level

73

With regard to the quality checks that had to be carried out by the CAs, the methodology note made reference to certain actions that the CAs should have taken (i.e. CAs are requested to “review”, “require”, “challenge”, etc.). The methodology note was a binding document. On the other hand, the EBA produced a specific manual for quality assurance. It provided guidelines to CAs for their review of banks’ calculations and for challenging the results submitted by the banks. The manual was approved by the Board of Supervisors as a non-binding document, thus leaving the CAs with considerable power of discretion.

74

CAs were expected to ask banks to provide them with an explanatory note or other documents in a format to be defined by the CA, including the qualitative information listed in Annex III of the methodological note. This explanatory note was expected to be used in the quality assurance process by the CAs to help them to carry out a meaningful analysis of the submitted data. The quality and level of detail of these notes varied across CAs⁴².

75

The EBA did not regard itself as being in a position to systematically question or monitor the CAs’ work. We found that the EBA had no detailed information on the depth or extent of the checks carried out by the CAs, in particular:

• whether CAs assess the functioning of banks’ stress-testing models against minimum standards in terms of econometric soundness and responsiveness of the risk parameters. In fact, according to our survey of CAs and the interviews that took place, supervisors did not perform in-depth, on-the-spot checks of these models. Models were only challenged by the supervisors when they found figures to be implausible;
• how much the results were influenced by the application of constraints (caps and floors), i.e. how far results would have differed positively or negatively in the absence of constraints. Neither the EBA nor the CAs have systematically collected such information;
• how much the results were influenced by other data such as ECB credit risk benchmarks. In addition, it did not have information on whether the benchmarks were used because of weak/deficient bank internal models or because the benchmark figures were more favourable than banks’ own calculations. It is left to the CAs’ discretion whether or not to use the ECB credit risk benchmarks where they assess banks’ figures as being overly optimistic, or where they deem banks’ own models to be inappropriate.

76

The quality assurance manual had suggested that CAs carry out a self-assessment of their application of the manual. However, no CA actually provided such an assessment to the EBA.

The EBA’s own checks are insufficient

77

The EBA had limited staff – around seven full time equivalents – to handle the 2018 exercise. The EBA ran automated checks on the data submitted by CAs. These included (i) numerical checks (wrong signs, subtotals should not exceed totals, etc.); (ii) verification that the constraints of the methodology were applied correctly; and (iii) statistical plausibility checks. It communicated the results of these checks to the CAs, which were responsible for follow-up.

78

For the plausibility checks, various data from the banks were compared. Data that deviated from the normal distribution were regarded as having a potential data quality issue (outliers).

79

For these checks all banks were considered in the same basket. The small sample size meant that peer groups (in the form of banks with similar geographical exposure and related stress levels, banks with similar business models or banks with a similar degree of financial health) could not be created. Therefore, the EBA’s identification of outliers was of very limited use for verifying the results as there are many valid explanations for outliers. In fact, looking only at outliers may even deter consideration of more critical cases, namely banks that should have been outliers but were not.

80

Despite its role as coordinator, the EBA was not systematically informed of – nor had the resources to systematically request information resulting from – the CAs’ supervisory activities (such as the SREP scores⁴³ by bank) which would be relevant for judging the validity of the stress test results.

81

The EBA had limited information on the extent to which the CAs had followed up the quality issues it had raised. Where the EBA raised concerns about the conservatism of the results, it depended on the CAs’ good will for the response. Indeed, the quality of feedback for the 2018 exercise differed significantly from one CA to another. Where the EBA received explanations, it did not have the time or resources to challenge them in depth; where it received no explanations, in many cases it did not follow up on the assumption that the relevant CA had valid reasons for not commenting any further.

82

There were a few instances of capital gaps emerging in banks soon after the EBA had published stress-test results. The gaps emerged under normal economic and financial conditions (i.e. not under a stress scenario). The CET 1 ratio⁴⁴ under the baseline scenario did not reflect such situations. In practice, the EBA relied on the starting-point data provided by the banks.

83

Thus, as the EBA relied on the CAs for quality assurance, it played a limited role in the quality assurance process. The Board of Supervisors never decided to make use of its powers as conferred by the EBA Regulation, in particular the powers to require CAs to conduct specific reviews or to carry out on-site inspections (see paragraphs 67-68).

Publications by the EBA and competent authorities showed a varying degree of quality

84

Results and key relevant data from the EU-wide stress test exercise were published in November 2018. In particular, the impact of the stress test is shown by the variation in CET 1 capital and in the leverage ratio⁴⁵. The EBA’s website provides documents that analyse the results and also grants access to a database with the information collected from banks in the course of the procedure.

85

As well as the EBA, some CAs and participating banks also published results of the EBA stress test.

86

Therefore, we examined:

1. the relevance of the EBA’s publications;
2. the accuracy of communication by CAs and banks.

The EBA published an unparalleled amount of data but certain key information missing

87

The quantity of information published by the EBA goes well beyond the information published by other authorities for their own stress tests. This results in greater transparency, in particular for supervisors, bank analysts and other expert readers.

88

In order to assess the informative value of the data published by the EBA, we examined whether the information it did publish provided answers to the following questions:

1. What were the drivers of the results?
2. Are the results comparable?
3. Is it clear whether EU banks and the EU financial system are resilient to stress?

Information on the drivers

89

On an aggregate level, the EBA report analyses the main drivers of the impact by risk type (credit risk, market risk, operational risk) and the impacts on main balance sheet items. In addition, there is a more detailed description of the impact of specific risk types and methodological assumptions (e.g. it states that credit risk losses are the main contributor to the stress impact).

90

The following important information is missing from the EBA report on results:

• To what extent credit losses, by far the largest driver of negative results, were driven by new defaults or by the old stock of defaulted assets⁴⁶.
• To what extent the EBA methodology including the EBA assumptions and constraints (see paragraph 64) or the use of the ECB credit risk benchmarks (see paragraphs 60-63) had an impact on the results. Currently, the EBA itself has not evaluated these aspects.

Comparison of results

91

The EBA report refers to the fact that there are large variations between banks. Indeed, the results for individual banks are not easily comparable for a number of reasons, for example:

• the static balance sheet assumption (see paragraphs 65-66) and other prescriptive elements (see paragraph 64) of the methodology have varying impacts on individual banks;
• banks’ results (i.e. variation in the CET 1 capital ratio) are affected by the approach they use to calculate their regulatory capital requirements, i.e. it affects the denominator of the ratio⁴⁷ (see paragraph 59);
• each bank translates the parameters of the baseline and adverse scenario in its own way (mostly via bespoke models) (see paragraphs 59-60);
• there was no homogeneous quality assurance approach across CAs (see paragraphs 73-83).

92

As acknowledged in paragraph 87, the EBA publishes a wide array of useful information which would not be available otherwise. However, the EBA report did not provide certain explanations to help the reader to put figures into perspective. For example, the report did not present banks-per-country analysis⁴⁸ comparing the results of banks to the relative severity of the adverse scenario for their country. Examples of missing explanations concern:

• the extent to which stress levels differed from country to country;
• the extent to which the stress levels differed from the financial crisis (a similar comparison was made by the Bank of England);
• the approach used by each bank for calculating its capital requirements and how it affected the results⁴⁹.

93

Box 5 provides examples of analyses that we performed to put the 2018 results into perspective. However, such information and the implications for resilience were not included in the EBA publications.

Resilience of banks and the financial system

94

For each bank, the EBA report presents the evolution of several capital ratios (such as the CET 1 capital and leverage ratio), i.e. it gives the value for 2017, the value after the stress (in 2020) and the difference between the two. One improvement when compared with the past was the fact that the 2018 report ordered banks for the first time according to the size of the CET 1 capital ratio.

95

To get a grasp of a bank’s resilience, i.e. to understand whether a bank would respect or fail to respect its capital requirements under adverse conditions, a crucial piece of information is lacking from the EBA report: the capital requirements for each of the banks (i.e. pillar 1, pillar 2 and the combined buffer requirements).

96

While EBA staff had proposed publishing the capital requirements, this proposal was rejected by the Board of Supervisors. However, in its opinion from 2015, the EBA made a strong case for publishing own funds requirements. It also pointed out that the Market Abuse Directive⁵⁰ requires publication of capital requirements for institutions that have publicly traded securities⁵¹. The then-Chairperson of the EBA Board of Supervisors has repeatedly made public claims that CAs should publish this information. By comparison, the Bank of England includes this information in the publication of its own stress-test results⁵². Practice in this area varies between CAs (which have the authority to publish these requirements) and banks (which can publish them at their own initiative).

97

As the EBA did not publish the banks’ capital requirements, readers are obliged to obtain this information from other public sources, which often disclose data inconsistently and in different forms. With regard to the 2016 stress test, we searched for this information for a sample of eight banks and found that they would have breached not only the combined buffer requirements but also the minimum capital requirements (i.e. pillar 1 and pillar 2). The EBA 2016 report remained silent on this important information.

98

For 2018, based on the information at the EBA’s disposal, we conclude that no bank would have breached the minimum requirements (pillar 1 and pillar 2 on a CET 1 basis). However, four banks would have breached the threshold mentioned in Article 27.1 of the Bank Recovery and Resolution Directive⁵³ (own funds requirements plus 1.5 percentage points) which can be a trigger for early intervention measures. We also identified nine banks that would have breached their combined buffer requirements.

99

Also, the fact that a number of banks were at risk of breaching the 3 % threshold for the leverage ratio in a downturn is a cause of concern that would have deserved more attention than a single line in the 60-page EBA report⁵⁴.

100

Bearing in mind that the stress test uses a static balance sheet assumption, this means that the EBA does not collect information on management actions which banks could take, in particular their options for recovery, and is therefore not in a position to publish such information. Without this additional information, the capacity of banks to mitigate the negative impacts of the adverse scenario – and thus their resilience – cannot be assessed⁵⁵.

101

The adverse scenarios change from exercise to exercise in terms of severity and risks covered. This makes it hard to assess whether banks’ resilience has improved or worsened over time.

102

The EBA’s overall conclusion in 2016 was that the EU banking system was resilient. The EBA did not make the same explicit statement in its 2018 report, but a number of CAs claimed that the results demonstrated the resilience of the banks they supervised. Furthermore, a clean bill of health cannot be given either for all individual banks or for the financial systems in all countries:

• a comparison with the 2008 financial crisis would show whether EU banks are better equipped than ten years ago to withstand such a severe stress. However, as already explained, the adverse scenario was less severe than the financial crisis;
• for individual banks, there were breaches or near-breaches of the regulatory minimum requirements (see paragraphs 97-98), and for the system as a whole no analysis (including amplification effects, common patterns, etc.) was performed;
• there are a few examples of banks which, shortly after the publication of a stress test, faced a significant capital gap.

103

Lastly, we note that the former head of the ECB’s Supervisory Board repeatedly highlighted that “in conducting the rigorous balance sheet review in combination with a stress test, [the ECB] substantially enriched [its] knowledge of the actual financial situation of the banks.”⁵⁶. Having regard to the 2018 stress test, she noted that it is very cost-intensive for the financial supervisor and does not deliver enough new insights⁵⁷.

104

The then-Chairperson of the EBA emphasised the positive role played by the “significant supervisory pressure coordinated by the EBA via several stress tests and recapitalisation exercises”⁵⁸ in strengthening banks’ capital ratios and that “detailed disclosure of the results had also reinforced market discipline and contributed to restore confidence”⁵⁹. But he also stated that⁶⁰ “regardless of the amount of data we publish […] the informative value of the results [is] limited” due to the “decoupling of stress test results and supervisory actions and the inconsistency between the transparency of the former and the opaqueness of the latter”.

Communication by some CAs and banks paints an overly positive picture

105

In advance of publication, the Board of Supervisors had approved an EU-wide stress test communication protocol which was designed to promote coordination, consistency and coherence in external communication. It focuses on the interaction with and external communication by CAs, and sets minimum standards for publications by CAs and banks.

106

Nevertheless, the stress-test press releases published by CAs and participating banks differ, in terms of both quantity and substance. The ECB referred in its communication to the EBA report. However, we have identified cases, including in the euro area, where national CAs or banks painted a benign picture of the resilience of a bank or the financial system at country level, even though there were breaches of the regulatory minimum capital requirements under the adverse scenario (see Box 6). This shows that the EBA does not have the power to prescribe what individual CAs and participating banks publish. This is more significant as all CAs are requested to share their draft communication material with EBA staff in advance of publication.

Conclusions and recommendations

107

The 2010 EBA Regulation gave the EBA, in cooperation with the ESRB, the task of initiating and coordinating Union-wide stress tests. To ensure “comparability and reliability of methods, practices and results”, the regulation also gave the EBA the authority to request information directly from financial institutions, and to ask competent authorities to conduct specific reviews, carry out on-site inspections and participate in these activities (see paragraphs 04, 07 and 67).

108

In practice, the EBA has decided to limit its role to initiating, providing methodology for and broadly coordinating stress-test activities, without making genuine efforts to ensure reliability and comparability of results produced by the banks under the bottom-up approach (see paragraphs 67-68 and 73-76).

109

The EBA’s manual describing the quality assurance actions to be carried out by CAs was not binding. In effect, it had little or no knowledge of the content of the checks carried out by the CAs and the models used by banks. In particular, in line with its interpretation, the EBA did not request specific reviews or participate in any on-site inspections; nor did any other activities take place that would ensure comparability and reliability of results (see paragraphs 73-83).

Recommendation 1 – The EBA needs to enhance its control over the stress-test process

In order to ensure that the results that are published are meaningful, comparable and reliable, the EBA should:

1. Use its legal powers to obtain from CAs all information that it deems necessary, and participate in their on-the-spot visits where relevant to obtain assurance on (i) the reliability of the methods and models used by banks and (ii) banks’ results. The selection of banks subjected to specific surveillance by the EBA should be risk-based.
2. Provide clear and binding guidance for CAs and develop its own quality assurance procedures accordingly.
3. In duly justified cases, reject the results of the stress test, i.e. in cases where CAs and banks do not follow its guidance and where results have not passed its quality checks.
4. Request from the budgetary authorities the resources it needs to deliver fully on its obligations as specified in the EBA Regulation.

Timeframe: The 2022 stress test

Recommendation 2 – Complement the current bottom-up procedure with top-down elements

The EBA should test the resilience of financial institutions to adverse market developments by introducing a top-down approach as a complement to the current bottom-up approach. This would ensure greater consistency and more control over the process, while at the same time providing a benchmark for the stress tests conducted by the competent authorities and individual financial institutions. The selection of financial institutions subject to either approach can vary.

Timeframe: The 2022 stress test

110

The EBA’s starting point for selecting banks was the amount of assets, but the EBA then took an ad hoc decision to exclude some banks (see paragraph 19-20). Furthermore, the EBA did not consider the systemic risk that banks may pose to the financial system. As a result, not all vulnerable banks were included. Some of the banks that were not included had recently been subject to restructuring, were from countries where banks have considerable exposure to their own sovereign bonds, or have a high concentration of non-performing loans (see paragraph 21).

Recommendation 3 – Select banks based on risk rather than just size

In order to ensure that the sample of participating banks is relevant to cover the risks it has identified as relevant for the exercise, the EBA should increase the geographical coverage and add risk-based criteria to the size criterion for selecting banks for the stress test.

Timeframe: The 2022 stress test

111

We found that the EBA exercise tested banks against an economic downturn rather than a shock stemming primarily from failures in the financial system, even though it was this kind of shock that was the main factor in triggering the last major recession (see paragraphs 35-36).

112

Moreover, we found that significant systemic risks – and certain countries and variables – were subject to a low level of stress, or indeed none at all. Although the stress should be ‘severe but plausible’, neither the EBA nor the ESRB set out ex ante measures of severity for the process. For a number of variables and a number of Member States, the stress imposed by the adverse scenario was significantly milder than during the financial crisis (see paragraphs 37-52 and Box 2).

Recommendation 4 – Introduce alternative stress scenarios

To ensure that the stress is severe enough to evaluate the potential for systemic risk to increase in a situation of stress, and to assess the bank’s resilience against systemic vulnerabilities in the EU, the EBA should:

(1) Make the risk identification and aggregation process more EU-oriented by:

• Taking due account of risks from within the EU that could trigger an adverse event with implications for the financial system.
• Requiring that the risks identified within the EBA’s risk dashboards are the key input for modelling the adverse scenario.

(2) Try different types of scenarios from one exercise to another (such as stressing different risks), and consider adding additional, more country-specific shocks or sensitivity analyses.

(3) Indicate what level of overall severity it is seeking for key parameters, and define criteria for assessing minimum levels of severity in absolute terms for all countries.

Timeframe: The 2020 stress test

113

The dominant role played by national supervisory and macro-prudential authorities in the design of the stress test was not conducive to ensuring comparable and unbiased scenarios for Member States (see paragraphs 29, 42, 43 and Box 1) as the EU-wide perspective was insufficiently taken into account.

Recommendation 5 – The governance structure should ensure that EU interests are duly taken into account

In September 2017, the European Commission presented a series of proposals “to pave the way for further financial integration and a full capital markets union”, which included governance and funding of the European supervisory authorities, one of which is the EBA. However, the political agreement reached in March 2019 between the Council and the European Parliament does not envisage such important changes.

The Commission should, in the context of the next three-year review of the EBA Regulation, address the appropriateness of EBA’s governance structure.

Timeframe: the next review in 2022

114

The EBA published a wide range of data on the stress-test results, thereby enhancing transparency (see paragraphs 84 and 87). However, pillar 2 capital requirements, and therefore the overall capital requirements were not published. Thus, the most crucial information for understanding the implications of the stress tests was not available (see paragraphs 94-99).

115

The EBA report does not make the link between the results and the adverse scenario. In addition, it does not contain important information on the drivers of the banks’ results, which would put them into perspective (see paragraphs 90, 92-93 and Box 5).

116

Banks’ and CAs’ current practices as regards publishing additional capital requirements vary but in many Member States these data are publicly available (see paragraph 96).

117

In several cases, national authorities (mainly central banks) and banks published a skewed picture of the impact of the stress on banks’ financial position (see paragraph 106 and Box 6).

Recommendation 6 – Increase the informative value of publications

In order to enable readers of the EBA publication to understand the implications of the data that are published, the EBA should:

1. Include the banks’ institution-specific minimum capital requirements in the information it publishes and present the results in a way that enables users to put the results into perspective (e.g. grouping results by country, by the stress level banks were exposed to, and by type and size of bank).
2. Make clear assertions about the resilience of the EU financial system as a whole in comparison to the previous stress test, and clearly indicate which factors have the greatest impact on resilience.

Timeframe: The 2022 stress test

This Report was adopted by Chamber IV, headed by Mr Neven MATES, Member of the Court of Auditors, in Luxembourg at its meeting of 4 June 2019.

Annexes

Annex I

Governance structure and roles of actors in the stress test

EBA governance structure

The EBA is an EU regulatory agency.

Board of Supervisors: This consists of the EBA's Chairperson and heads of the national banking supervisors from the 28 Member States of the European Union. The Board takes all EBA policy decisions, including the adoption of draft technical standards, guidelines, opinions and reports. It also takes the final decision on the EBA's budget.

Management Board: This consists of the EBA's Chairperson and six members who are elected from the Board of Supervisors. It takes decisions on EBA operational matters and is responsible for implementing the EBA’s work programme. Its role is to ensure that the EBA carries out its mission and performs the tasks assigned to it in accordance with its regulation.

ESRB governance structure and working groups

The ESRB is an independent EU body.

GB (General Board): The General Board, chaired by the President of the ECB, is the ESRB’s decision-making body. It is mandated with identification and prioritisation of systemic risks, and, where necessary, issues recommendations and warnings. The President and the Vice-President of the European Central Bank (ECB), the Governors of the national central banks of the Member States, one member of the European Commission, the Chairpersons of the EBA, EIOPA and ESMA, the Chair and the two Vice-Chairs of the Advisory Scientific Committee (ASC) and the Chair of the Advisory Technical Committee (ATC) are the members of the General Board with voting rights. Non-voting members include high-level representatives of national CAs, the President of the Economic and Financial Committee, governors of the national central banks or high-representatives from Iceland, Norway and Liechtenstein.

The stress test scenarios are prepared and discussed at the technical level and approved by the General Board.

ATC (Advisory Technical Committee): The ATC provides advice and assistance on issues relevant to the work of the ESRB. The membership of the ATC mirrors the full membership of the General Board and comprises representatives from national central banks, national supervisory authorities, the three ESAs, the European Commission (EC), the ASC and non-EU EEA Member States. Stress test scenarios are prepared and discussed by the ATC.

TFST (Task Force on Stress Testing): The TFST was established under the auspices of the ATC; it liaises with the appropriate ESA structures and relies heavily on technical and modelling support from the ECB. The TFST prepares the draft scenarios that are discussed by the ATC and subsequently submitted for discussion and approval by the General Board. For this reason, its Chair reports to the ATC, Steering Committee and General Board. It is made up of experts from national central banks and national CAs, as well as from the ECB, EIOPA, EBA, ESMA and the European Commission.

Actors and their roles in the stress test

The roles of the various actors in the EU-wide stress test are displayed in Figure 1.

Annex II

Risks identified in the bottom-up survey and risks chosen by the ESRB General Board

01

Table 1 compares the four risks with the highest scores identified in the bottom-up survey (see paragraph 29) with the four risks eventually chosen by the ESRB General Board to drive the scenario as explained in the narrative.

02

The first three risks chosen by the ESRB General Board broadly correspond to three of the main risks identified by the bottom-up survey.

03

By contrast, the fourth risk in the narrative (liquidity risks in the non-bank financial sector) was included, even though the pre-defined risks related to it were given relatively low priority in the bottom-up survey⁶¹.

Annex III

Key variables of the EBA 2018 adverse scenario in comparison to the financial crisis

Endnotes

¹ Report of the High-Level Group on Financial Supervision in the EU (de Larosière Report), 2009, paragraph 14.

² The European Systemic Risk Board (ESRB) was established as an independent body by Regulation (EU) No 1092/2010 of the European Parliament and of the Council of 24 November 2010 to oversee the financial system of the European Union (EU) and prevent and mitigate systemic risk, (OJ L 331, 15.12.2010, p. 1).

³ Article 32.3(a) of Regulation (EU) No 1093/2010 of the European Parliament and of the Council establishing a European Supervisory Authority (European Banking Authority) (known as the ‘EBA Regulation’), (OJ L 331, 15.12.2010, p. 12).

⁴ Members of the ESRB include: the European Commission, the ECB, EBA, EIOPA and ESMA, and national macro-prudential authorities (central banks and competent supervisory authorities).

⁵ The ESRB Task Force on Stress Testing was established under the auspices of the ESRB Advisory Technical Committee and comprises staff from the ESRB Secretariat and ESRB Members, including the EBA.

⁶ Regulation 1093/2010, Article 32(3a): “For the purpose of running the Union-wide assessments of the resilience of financial institutions under this Article, the Authority may, in accordance with Article 35 and subject to the conditions set out therein, request information directly from those financial institutions. It may also require competent authorities to conduct specific reviews. It may request competent authorities to carry out on-site inspections, and may participate in such on-site inspections in accordance with Article 21 and subject to the conditions set out therein, in order to ensure comparability and reliability of methods, practices and results”.

⁷ Article 42 of the EBA Regulation.

⁸ Report from the Commission to the European Parliament and the Council on the operation of the European Supervisory Authorities (ESAs) and the European System of Financial Supervision (ESFS), COM(2014) 509 final of 8.8.2014.

⁹ COM(2017) 536 final of 20.9.2017 as amended by COM(2018) 646 final of 12.9.2018.

¹⁰ Article 21 (2) (b) of the EBA Regulation.

¹¹ Recital 43 of the EBA Regulation, and ‘de Larosière report’, paragraph 4.

¹² Article 22(2) and Article 23 of the EBA Regulation.

¹³ Basel Committee on Banking Supervision, Bank for International Settlements, Stress testing principles, last updated in October 2018.

¹⁴ EBA/GL/2018/03 of 19 July 2018: Guidelines on the revised common procedures and methodologies for the SREP and supervisory stress testing.

¹⁵ Austria, Belgium, Germany, Spain, Finland, France, Ireland, Italy, and the Netherlands.

¹⁶ Denmark, Hungary, Norway, Poland, Sweden, and the United Kingdom.

¹⁷ In 2011 and 2014, the stress test sample included banks undergoing restructuring.

¹⁸ According to the EBA’s risk dashboard of December 2017 which ranked countries’ financial systems by their ratios of NPLs, there were five countries (GR, CY, PT, BG and SI) with an NPL ratio of about 10 %. None of these five were included in the stress test sample.

¹⁹ Members of the ESRB include the European Commission, the ECB, EBA and the national macro-prudential authorities (central banks and competent supervisory authorities).

²⁰ Staff from the Macro-Prudential Policy and Financial Stability, International and Economics Directorates.

²¹ By law, the ESRB is required to produce regular risk dashboards. The ESRB does not assess or comment on the risks, but provides country-specific information for a specific point in time (i.e. there are no historical data for assessing any changes).

²² Regarding asset quality, e.g. non-performing loans, there is only an indirect link as a factor for bank income and a general-type scenario description with an impact on real-estate prices and non-performing loans, but not a scenario tailored specifically to hit banks with weak portfolios.

²³ In 2019, the ECB ran a stand-alone liquidity stress test.

²⁴ Article 22 (2) of the EBA Regulation.

²⁵ A risk dashboard is required by Article 22(2) of the EBA Regulation.

²⁶ Bank for International Settlements, Financial Stability Institute, FSI Insights on policy implementation No 12, stress-testing banks – a comparative analysis, November 2018.

²⁷ Before 2018, the baseline scenario was based on projections from the European Commission.

²⁸ Rodrigo Alfaro/Matthias Drehmann, BIS quarterly Review December 2009, page 34 as well as Borio, Drehmann, Tsatsaronis, BIS Working Papers No 369, p. 8.

²⁹ Such as risks related to the quality of the assets in banks’ portfolios or concerns with regard to banks’ profitability.

³⁰ It should be stressed that the “de Larosière” report concluded inter alia that the fundamental underlying factor which made the crisis possible was the ample liquidity and related low interest rate conditions which prevailed globally.

³¹ One modelling assumption was to calibrate the interest rate shocks based on the post-2012 period, in contrast to the economic shocks which were calibrated on a much longer period. The rationale for this was the policy readiness of monetary authorities since 2012 to intervene to keep rates down in times of stress.

³² Basel Committee on Banking Supervision, Bank for International Settlements, Stress testing principles, last updated in October 2018, p. 6.

³³ IMF Country Report No 18/228 “A wide range of sensitivity tests was conducted to further explore the resilience of the euro area banking system to wider shifts to risk factors. A reverse stress test on opaque complex assets assessed the valuation shock from soft mispricing required to exhaust capital buffers […]”.

³⁴ Banks have different possibilities for establishing their capital requirements. They can use either the so-called standardised approach or the IRB (Internal Ratings Based) approach (at foundation or advanced level). In the latter case, models are likely to be used. By law, these models need to be approved by the banks’ supervisor.

³⁵ Capital requirements are expressed as a percentage of risk-weighted assets.

³⁶ Due to errors in the development, implementation and/or use of these models, risks can be over-or underestimated, even significantly so. This is generally referred to as ‘model risk’. Banks can also calibrate the models to minimise the impact of the stress on their results.

³⁷ The methodology for generation of benchmarks was also presented to the banking industry (euro-area banks only) in the context of the stress-test industry workshop in advance of the stress test.

³⁸ While the EBA believes that greater transparency around the credit risk benchmark methodology may make it easier for banks to game the results, gaming is possible in all parts of the process.

³⁹ This can include the application of measures as set in a bank’s recovery plan. Recovery plans outline the measures that banks would take if their financial situation were to deteriorate seriously and have to be updated every year.

⁴⁰ Article 32 (3a) and (3b) of the EBA Regulation.

⁴¹ See for example EBA’s quality assurance manuals.

⁴² For example, our survey showed that the comprehensiveness of data provided to the CAs with regard to Annex 3 in the 2016 exercise varied significantly (documents ranged from 11 to 180 pages).

⁴³ The overall SREP score ranges from grade 1 (no discernible risk) to grade 4 (high risk).

⁴⁴ CET 1 ratio = CET 1 capital / ∑ Risk Exposure Amounts (for credit risk, market risk, operational risk etc.).

⁴⁵ The ratio was introduced after the financial crisis to constrain the build-up of excessive leverage in the banking sector (it does not take banks’ risks into account). It corresponds to: Tier 1 capital/sum of exposure values of assets and off-balance sheet items.

⁴⁶ The latter aspect is of particular relevance as such losses could stem from the EBA methodology being stricter than a bank’s accounting. If this were the case, credit losses in the stress test would highlight current underprovisioning rather than additional future losses in an adverse scenario.

⁴⁷ For example, the capital requirements for credit risk remain largely unchanged for the banks (the least sophisticated ones) that use the standardised approach, but would rise for the banks that use the IRB foundation approach since the probability of default would increase. However, only the capital requirements of the (most sophisticated) banks that use the IRB advanced approach would reflect the adverse scenario to a large extent.

⁴⁸ For example, unlike the 2016 report, the 2018 report includes one table that compares the aggregate transitional and fully loaded CET 1 capital ratios by jurisdiction but without providing a breakdown for the Euro area countries (showing just six countries, the Euro area and EU). It should be noted that in 2016 the country-by-country data were excluded from the final publication at the request of the ECB, in its role as CA.

⁴⁹ The EBA report (p. 22) merely stated that “when comparing the bank-by-bank fully loaded and transitional results, the evolution of the banks’ capital ratios and the impact with and without transitional arrangements differs across banks”. It did not shed light on how banks benefited from these different arrangements, although this is visible in the interactive tools online.

⁵⁰ Directive 2003/6/EC of the European Parliament and of the Council of 28 January 2003 on insider dealing and market manipulation (market abuse), (OJ L 96, 12.4.2003, p. 16).

⁵¹ EBA, opinion on interaction of pillar 1, pillar 2, combined buffer requirements and restrictions on distributions, 16.12.2015, paragraphs 10-13, 15: https://www.eba.europa.eu/documents/10180/983359/EBA-Op-2015-24+Opinion+on+MDA.pdf.

⁵² Stress testing the UK banking system: 2017 results, both on an aggregate and a bank-by-bank basis, p. 10.

⁵³ Directive 2014/59/EU of the European Parliament and of the Council, (OJ L 173, 12.6.2014, p. 190).

⁵⁴ Christian Stiefmueller, Banks stress-tests 2018: Trying too hard to reassure, Finance Watch, 7 November 2018.

⁵⁵ Assessing the credibility and feasibility of these recovery options is part of ongoing supervision.

⁵⁶ Speech by Danièle Nouy, A year of the SSM – résumé and outlook, at the European Supervisor Education Initiative Conference 2015, Prague October 2015.

⁵⁷ Handelsblatt, “EZB fordert Banken-Stresstest 2.0” by A Kröner, J Deters, 27.11.2018.

⁵⁸ Speech by Andrea Enria, then-Chairperson of the European Banking Authority, given at Danmarks Nationalbank, 14.06.2017.

⁵⁹ Introductory remarks by Andrea Enria, given at the EBA-IMF Colloquium in London, 1.3.2017.

⁶⁰ Speech by Andrea Enria, then-Chairperson of the European Banking Authority, given at the National Bank of Romania, 15.11.2018.

⁶¹ Specifically, ‘market liquidity shortages’, ‘pension schemes’, and ‘investment funds and other financial institutions.

Replies of the Commission

Introduction

09

The Commission had recognised the risks of the ESAs decision making body being influenced by national interests and not sufficiently taking into account the wider European interest. Therefore, in 2017 the Commission submitted a proposal to amend the regulations of the European Supervisory Authorities (including the EBA) dealing, amongst others, with funding and governance issues. In particular, one of the aims was to ensure that decision-making was more EU-oriented than at present. In particular, the Commission proposed that the EBA should have an executive board with full-time members, based on a shortlist drawn up by the Commission, and appointed by the Council, which would not have voting rights on the board.

However, the co-legislators were not able to reach agreement on this solution and the proposal for such changes has been dropped with the latest political agreement of 21 March 2019. Nevertheless, with the intention of enhancing the governance of the European Supervisory Authorities by other means, the political agreement of 21 March 2019 strengthens the position of the Chairperson. In particular, as of the entry into force of these modified provisions, the Chairperson shall be appointed by the Council, after confirmation by the European Parliament, based on a short list of qualified candidates prepared by the Board of Supervisors with the assistance of the Commission. He or she will also have voting power in the main decision body of the Board of Supervisors and the explicit right to put forward draft decisions for adoption.

Conclusions and recommendations

Recommendation 5: The governance structure should ensure that EU interests are duly taken into account

The Commission accepts the recommendation and agrees that the governance structure of the European Banking Authority (and the other European Supervisory Authorities) should ensure that EU interests are duly taken into account.

As required by Article 81 of Regulation 1093/2010 (and in line with the political agreement of 21 March 2019 between the co-legislators), by 2022 the Commission will again review the operation of the EBA. This review will also take into account the governance of EBA. Based upon the outcome of that review, the Commission will determine whether further improvements to the governance of the EBA would be warranted.

Replies of EBA

General comments

The EBA welcomes the ECA report and acknowledges the efforts made by the ECA in providing valuable insights to improve the efficiency of the EU-wide stress test in the future.

Since 2011, the EBA has conducted four stress test exercises. The announcement of the stress test facilitated a considerable degree of pre-emptive action. The EBA exercises have helped in a significant strengthening of the capital position of European banks, appropriate identification of non-performing loans and ongoing action to reduce them and significantly improved market’s understanding of the EU banking system.

While the bank-by-bank supervisory assessment remains under the responsibility of the national supervisors, the EBA bases its analysis on statistical tools that allow banks’ own results to be benchmarked against the others. Conducted over a large cross-border sample of banks and pooling information previously only available at national level, the EBA established the first comprehensive database of this kind in Europe.

Similarly to the US, the results of the EU-wide stress are disclosed on a bank-by-bank basis, but the granularity of the information provided in the EU has been unprecedented, as acknowledged by the ECA report. This represents an additional benefit in a fragmented supervisory community as in the EU. The disclosed figures complement the results, and are used by market participants for running their own stress tests or general analysis.

The exercise involves many actors and it is run under tight deadlines. This necessarily leads to challenges in the governance structure, which is difficult in any stress test but particularly in a region-wide setting. The role of the EBA as coordinator, developer and guardian of the methodology needs to be reconciled with a legal framework in which the EBA has a limited role in assuring the quality of banks’ results and scarce resources.

As the report points out, the EU setting requires extra effort to ensure consistency across banks belonging to different jurisdictions, and subject to not-fully harmonised rules and supervisory practices. The EBA is committed to consider the ECA recommendations as part of the ongoing discussion on possible longer-term changes to the EU-wide stress test.

The EBA has liaised with the ESRB and ECB regarding these replies where relevant.

Detailed comments

Executive summary

VII

The EBA does not comment on its own governance beyond what was published in the “Opinion of the European Banking Authority on the public consultation on the operation of the European Supervisory Authorities”.

VIII

The constituency of the EBA Board of Supervisors and ESRB General Board and subgroups is in many cases the same and therefore collaboration between these two organisations takes place during the entire process.

IX

The EBA role reflects its governance and resources. This aspect is key in many decisions on the setting up and implementing the exercise.

The currently implemented division of tasks related to stress tests between the EBA and the CAs whereby the CAs bear the full responsibility for the quality assurance is a prudent and efficient way to execute the exercise, given the current legal setting, governance and resources. The EBA welcomes the close and successful collaboration with the CAs, including the ECB-Banking Supervision, that has taken place on stress testing over the past years which has resulted in a stronger and more credible stress testing approach

X

The EBA would like to stress that this observation needs to be read in conjunction with the debate on Pillar 2 disclosure among CAs. Before the very recent amendments to the capital requirements legislation, the obligation to publish banks’ levels of Pillar 2 requirements was not explicit. This will change with the amendments to the capital requirement legislation (regulation 575/2013 and directive 2013/36/EU) (see art. Article 447: Disclosure of key metrics), coming into force in July 2019.

The EBA report is a factual report, but the EBA stressed that the results should be read in conjunction with the macro scenario.

Introduction

07

The role of the EBA reflects its legal mandate, which does not give the EBA any specific quality assurance responsibilities, along with limited resources.

Observations

14

On this section, the EU-wide stress test has been pivotal for identifying pockets of vulnerabilities and triggered measures for recapitalizing the EU-banking sector. The unprecedented disclosure of banks exposures also enhanced market discipline. However, as any stress test, the EU-wide exercise does not have the ambition to cover all possible risks. The EBA has been always clear in explaining and communicating this limitation in its reports.

20

Banks under restructuring are excluded from the sample because the assessment of “the DG COMP’s viability assessments in the context of State aid procedures” is regularly carried out by the European Commission itself. In addition, the static balance sheet assumption would be suboptimal for banks under restructuring, particularly when deleveraging is part of the conditions for receiving State aid.

21

The EBA would like to note that the purpose of the stress test is to provide a forward-looking view on potential risks for banks. Therefore, the starting point should be a neutral selection of banks, without any a priori belief. Otherwise, there could be selection bias with supervisors selecting only “weaker” banks according to backward-looking risk indicators and missing banks with potential, but yet unknown, vulnerabilities.

Most of the banks from the countries mentioned in the first bullet formed part of a sample stressed by the ECB. A report of this stress test was published in February 2019 and disclosed results in aggregate format (with the exception of Greece where bank-by-bank data were published).

22

On this section, the EBA notes that its stress test uses a single adverse scenario and, thus, cannot cover all systemic risks, but focuses on the most important ones. The inclusion of additional scenario would make the exercise more burdensome and, thus, costs, benefits and resource implications would have to be assessed.

26-33

On this section, the EBA notes that, as any stress test, the EU-wide exercise does not have the ambition to cover all possible risks. The EBA has been always clear in explaining and communicating this limitation in its reports.

27

While the process is not documented, the EBA regular work on risk assessment, including the EBA Risk Dashboard, helps EBA staff to develop its views on the scenario. Furthermore, in ECA’s following paragraph, the contribution of ESRB members to the ESRB risk assessment is acknowledged, which also includes the EBA’s.

30

The EBA stress test is a scenario analysis, where macroeconomic variables and stressed according to a risk narrative (input) and bank specific variables (output) under stress are then estimated by banks, in line with the common methodology. Therefore, a significant increase of non-performing loans is the outcome, not the starting point of the exercise, and indeed the provisions for credit risk increase in the stress test. It is also noted that the adverse macro-financial scenario for the 2018 EU-wide banking sector stress test states that “Overall, the increase in borrower credit risk would result in banks facing higher levels of non-performing loans and a concomitant rise in forgone interest income”.

Furthermore, the ESRB would like to point out that the risks identified by the General Board tend to encompass the more granular terminology used within the bottom-up survey, such that the risks relating to asset quality fall under Risk 2 in Annex II of this report.

31

Although this sentence is correct, it does not acknowledge that a liquidity stress test and a solvency stress test are methodologically different. In this regard, the EU-wide stress test is a solvency exercise and not a liquidity test whereas liquidity risk should be captured in other ways such as liquidity coverage ratio and net stable funding ratio. A liquidity stress test would require a different exercise (as the one ECB is running in 2019) and, as the time horizons are different, a liquidity stress test can be hardly included in the current EU-wide stress test given: i) its time horizon; ii) the static balance sheet assumption; and iii) the detailed disclosure of bank by bank results.

32

The ESRB risk assessment includes analyses of individual bank-level data from multiple sources whenever it deems appropriate, in line with its mandate. The ESRB is entrusted with the macroprudential oversight of the EU financial system and the prevention and mitigation of systemic risk, and not with microprudential supervision of banks.

33

The EBA risk dashboard is used as a source of information helping EBA staff to develop their thinking on the ESRB scenario. In addition, as mentioned above, the ESRB members’ input to the risk assessment exercise is acknowledged, which also includes the EBA.

34

The EBA stress test is a scenario analysis, where macroeconomic variables are stressed according to a risk narrative (input) and bank specific variables (output) under stress are then estimated by banks, in line with the common methodology.

As mentioned above the narrative of the scenario is a joint effort of the EBA and the ESRB. The membership of the EBA Board of Supervisors (and substructures) and ESRB General Board (and substructures) is largely overlapping and therefore the same CAs are represented in both organisations which also means that exchanges of views and discussions on the scenario design with all the actors involved are taking place during all the process.

36

The scenario included shocks from both the real and the financial sectors.

37

It is important to differentiate the triggers of the scenario and their interaction with other risks and vulnerabilities, which nonetheless are reflected in the narrative and in the calibration of the scenario. Furthermore, the ESRB highlights that the threats to financial stability that underlie the adverse scenario do not represent any unintended consequences of future monetary policy decisions.

38

Risks triggered within the EU play an important amplification role in the scenario. An adverse feedback loop between weak bank profitability and low nominal growth together with public and private debt sustainability concerns are explicitly mentioned in the narrative and play a key role in the calibration of shocks. The scenario design ensures that those banks which are overly-exposed to these risks are identified by the exercise, including the possibility of bank failures if the evidence supports it. Furthermore, given the broader implications of amplification effects, they are usually beyond the microprudential scope of the EBA stress test.

43

The ECB and the ESRB find that the ECA’s statement that the involvement of national authorities in the process was not conducive to an objective scenario does not provide a full description of the scenario development process.

Box 1

The ESRB and ECB note that, as regards the ECA’s observations on the use of additional severity, any model based on historical data has limitations and that policy-makers’ judgement is an input in the overall process. Many discussions took place in the whole process and the CAs had different views on the appropriate severity for their country but these views and discussions did not automatically lead to actions. Each decision was taken in a transparent way in collaboration with the entire task force.

45

With reference to the ECA’s observation that the countries were subject to very different shock levels, it should be noted that the cross country variation of the shocks has been shown to follow from the narrative – see Bianchi (2019), “The role of country factors in the 2018 EBA stress test”, No. 1/FS/19, Central Bank of Ireland.

In Sweden, the large house price shock, which reflected real estate vulnerabilities in this country, explains the overall severity of the scenario.

Box 3

The US and the EU-wide stress test scenarios are different from the Bank of England one since they indicate low inflation through the whole three-year period rather than a sharp rise in prices level as it is assumed for the UK. In addition, there are differences between the application of shock on the long term interest rates. The EU-wide stress test scenario assumes a relatively mild increase in 10y interest rates, while the Bank of England scenario simulates a high increase. On the other hand, the Comprehensive Capital Analysis and Review scenario assumes a drop in long-term rates.

The unemployment rate shock in the adverse EU-wide stress test is lower compared to all of the other stress test scenarios, however, the final level of unemployment for the EU is the highest among other stress tests.

58

The methodology cannot explain every single case. Potential improvements should provide enough details without overcomplicating the methodology.

60

While it is true that models can have shortcomings, the bottom-up exercise is complemented with:

1. Constraints included in the methodology
2. Quality assurance process (and quality assurance manuals);
3. Descriptive statistics;
4. Credit risk benchmarks;
5. A wide set of tools to ensure comparability in the results.

62

The assumptions taken are based on the findings of the related economic literature, or on state-of-the-art stress testing methodologies.

63

The precise coefficients are not shared with the banks to ensure the effectiveness of the EU-wide Stress Test. Importantly, it should be noted that providing full transparency about the EBA/ECB benchmarks and challenger models would defeat the purpose of conducting a (constrained) bottom-up stress test as one of its key objectives is to help foster banks’ risk management and modelling capacity. In a bottom-up exercise the participating institutions should focus on developing new or improving existing models rather than trying to anticipate supervisors’ expectations. Moreover, providing the banks with the full set of elasticities underlying the EBA/ECB benchmarks would make it too easy for them to tailor their stress test projections and required model documentation in a way that would make it practically impossible for the CAs to properly challenge the banks’ results.

66

The static balance sheet assumption serves the purpose of analysing the impact of the stress test under a ceteris paribus assumption. The possible mitigating managerial actions are considered as part of the SREP. Changing this approach would require significant more resources for the quality assurance and for assessing the credibility of the management actions.

68

The EBA is not in charge of the quality assurance but did provide some tools to help CAs (descriptive statistics, quality assurance reports). See also EBA reply on the subject of on-site inspections.

75

The EBA agrees with the ECA that the limited resources did not allow to systematically collect comprehensive information on how far results would have differed positively or negatively in the absence of constraints. Nevertheless, on the functioning of banks’ stress-testing models against minimum standards, the EBA provided the descriptive statistics to CAs and quality assurance reports to help CAs in the quality assurance process in challenging banks’ estimations under the stress test.

Regarding the credit risk benchmarks, the credit risk template included in 2018 fields to be filled in by banks in case credit risk benchmarks are used, so that this information was in a way requested to banks. Nevertheless, the EBA recognised that the way banks were flagging the use of benchmarks was identified as inconsistent, and for that reason changes are expected for the next stress test exercise. Further guidance should be provided to banks on the conditions in which the use of benchmarks should be flagged.

The ECB, in its function as CA, notes that there is a large number of stress-testing models in use at bank level which are not subject to supervisory monitoring or approval. Therefore, with the resources and time available during the stress test process, the ECB is not in a position to systematically subject all models to quality control. However, when figures were found to be implausible, the related models were challenged.

76

The EBA would like to point out that even though the Quality Assurance manual suggests that CAs should carry out such self-assessment, this does not include any requirement to share this assessment with the EBA.

79

The EBA would like to point out that where possible (credit risk and NII), it was decided to provide statistics by country of the counterparty for all EU, SSM and Nordic countries.

82

In a non-pass fail stress test the goal is not to identify banks that fail but discern which ones are weaker and deserve more supervisory scrutiny. In fact, the bank that this sentence seems to be referring to had one of the highest capital depletion under the stress test. Additionally, any verification of the starting point data is carried out by the CAs.

83

The planning, preparation of on-site inspections is a process that is usually planned several months in advance. In terms of resource needs, it should be highlighted that for instance a single on-site inspection can require more than 50 person-weeks to be spent on-site, depending on the scope of the investigation and the complexity of the subject matter under review.

87

The EBA welcomes the finding on the transparency of the exercise, which provides a large amount of information, and acknowledges that the only missing information is in the level of Pillar 2 requirements, which reflects the debate on its disclosure in the EU.

See also the comment below on the paragraph related to the disclosure of capital requirements.

90

It should be noted that the stress test template includes information on migration between stages, so that new defaulted assets are captured.

91

The EBA would like to point out that all bullets in this paragraph highlight clear features that are typical of a bottom-up stress test.

The comparison between internal models and standardised approach mentioned in the second bullet is mandated by the Capital Requirements Regulation and not by the EBA methodology.

92

The footnote to this paragraph is factually correct but does not mention that this comparison can also be seen on a bank-by-bank level in the single spreadsheets provided.

93

The EBA report on the stress test result is a very neutral summary of the results and aims at being informative but impartial. While full transparency is provided on results and underlying exposures, the judgement is left to CAs and other stakeholders, also considering that the stress test is the starting point not the end point of the supervisory review process.

95

The EBA would like to stress that this needs to be read in conjunction with the debate on Pillar 2 disclosure. Before the very recent amendments to the capital requirement legislation, the publication of banks’ levels of Pillar 2 requirements was not mandatory and there have been some different views on the benefits of this full transparency.

The EBA nevertheless agrees that some improvements in this area are needed, especially in line with the speech of the former Chairperson of the EBA at the National Bank of Romania (November 2018).

97-100

The SREP Guidelines highlight the important role of mitigating actions in potentially enabling the institution to fully meet its applicable capital requirements within an appropriate timeframe. This is indeed recognised in this report. Since the EBA stress test is based on a static balance sheet assumption, a simple comparison of stress test outcomes and capital requirements requires caution.

101

The EBA would like to point out that this sentence can only be understood as focusing on the assessment of banks’ sensitivity to the scenario. Resilience also depends on banks’ capital starting points, which can be compared over time.

In fact, the EU-wide stress test contributed to the gradual increase of capital in banks, as confirmed looking at the evolution of CET1 ratio starting point over time.

102

The EBA does not claim to deliver a clean bill of health nor this is implied in its reports, but it always stresses that the results need to be read in conjunction with its follow-up actions and assumptions, and especially they need to be taken as a starting point for discussions with the CAs.

The EBA does not assess if banks could currently withstand the 2008 financial crisis nor any other crisis, since only a scenario that is exactly the same as those could do that.

Conclusions and recommendations

108

The role of the EBA reflects its legal mandate, which does not give the EBA any specific quality assurance responsibilities, along with limited resources. The only way to have a higher involvement of the EBA on the Quality Assurance process is through more resources. In addition, it should be thought what implication this has in terms of cost of the stress test and possibility of duplication with the work carried out by supervisors (such as the SSM).

The EBA highlights that comparability is provided by the publication of very granular data and it provides transparent explanations of possible differences across banks. The EBA also provides benchmarking tools to CAs for their quality assurance.

109

The EBA agrees with the ECA that, regardless of legal mandates, these activities are resource intensive and for this reason they cannot be carried out with the seven full-time equivalents, including statisticians, currently working on the stress test. In addition, the specific reviews and inspections should be organised, procured for, etc, which is almost impossible to do in the stress test timeframe, particularly with little or no budget allocated to these tasks.

Recommendation 1 – The EBA needs to enhance its control over the stress-test process

The EBA accepts the recommendation. It must be stressed that its success is conditional on: i) getting the substantial additional resources as indicated already and recommended by the ECA itself; ii) the governance is fit for purpose as decided by the Commission based on the ECA’s recommendation.

It is also important to avoid overlaps with the tasks already carried out by CAs.

Recommendation 2 – Complement the current bottom-up procedure with top-down elements

The EBA accepts the recommendation. It must be stressed that its success is conditional on receiving additional resources.

Also in this case, dedicated resources and budget would be needed. Setting up from scratch a top down model requires specific expertise and potentially the support of consultants, but a gradual implementation of top-down elements may be feasible with an early preparation from the side of the EBA and the actors involved, in order for such elements to be implemented by 2022.

110

Decisions on the sample are not arbitrary but based on the criteria included in the methodology and the formal feedback received from the CAs.

The EBA would like to note that, since the very purpose of the stress test is to identify potentially weak banks, the selection of the sample should be neutral, i.e. there could be banks considered safe based on backward looking risk indicators, but very sensitive to an adverse shock.

Recommendation 3 – Select banks based on risk rather than just size

The EBA partially accepts the recommendation. The EBA will reconsider the geographical coverage. However, the EBA has a different view from the ECA on the use of risk-based criteria to complement the size criterion when setting the sample for the EU-wide stress test.

111

The EBA notes that, in both cases, the final impact would be measured in terms of deterioration of macro and financial variables.

Recommendation 4 – Introduce alternative stress scenarios

The EBA partially accepts this recommendation.

For sub-recommendation 4.1, since risks can be global in origin, the EBA retains the discretion (in coordination with the ESRB) to select the most relevant risks.

For sub-recommendation 4.2, this would increase the burden of the exercise for CAs, banks and the EBA. Without additional resources, multiple scenarios can be non-manageable in the EU-wide stress test.

The EBA also notes, on sub-recommendation 4.3, that it can improve the criteria for assessing the severity of the scenario, but it would not be possible to quantify levels of severity of each and every parameter that is rather the result of the overall scenario.

Beyond resource concerns, implementation for the 2020 stress test would be challenging. According to the usual timeline of the stress test, the ESRB would have only very few months to discuss, implement and decide on this significant adjustment in the design of the scenario.

Recommendation 5 – The governance structure should ensure that EU interests are duly taken into account

It is not for the EBA to comment on its own governance and we would accept any Commission’s determination.

114

Pillar 2 and therefore the Overall Capital Requirement are not published, because of debate among CAs on the disclosure of Pillar 2 requirements.

The EBA aims to deliver a compact, factual and accessible report summarising the results of the stress test. The report complements the rich data dissemination that accompanies the release of stress test results. The EBA stands ready to consider possible improvements for enriching further the informative value of the report.

Recommendation 6 – Increase the informative value of publications

The EBA accepts sub-recommendation 6.1.

The EBA partially accepts sub-recommendation 6.2. It notes that the stress test is the starting point of the SREP process and it is based on the static balance sheet assumption, any assertions on resilience could be misleading since it is for CAs to provide an assessment of the resilience at the end of the SREP process. The EBA will increase the information content of the summary report.

Audit team

The ECA’s special reports set out the results of its audits of EU policies and programmes, or of management-related topics from specific budgetary areas. The ECA selects and designs these audit tasks to be of maximum impact by considering the risks to performance or compliance, the level of income or spending involved, forthcoming developments and political and public interest.

This performance audit was carried out by Audit Chamber IV Regulation of markets and competitive economy, headed by ECA Member Neven Mates. The audit was led by ECA Member Neven Mates, supported by George Karakatsanis, Head of Private Office and Marko Mrkalj, Private Office Attaché; Marion Colonerus, Principal Manager; Mirko Gottmann, Head of Task; Karolina Beneš, Giuseppe Diana, Shane Enrigh, Jörg Genner, Helmut Frank, Athanasios Koustoulidis, Violeta Radu and Julio Cesar Santin Santos, Auditors.

From left to right: Marion Colonerus, Giuseppe Diana, Mirko Gottmann, Julio Cesar Santin Santos, Neven Mates, Marko Mrkalj, Jörg Genner, Athanasios Koustoulidis, Shane Enright

Contact

EUROPEAN COURT OF AUDITORS
12, rue Alcide De Gasperi
1615 Luxembourg
LUXEMBOURG

Tel. +352 4398-1
Enquiries: eca.europa.eu/en/Pages/ContactForm.aspx
Website: eca.europa.eu
Twitter: @EUAuditors

More information on the European Union is available on the internet (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2019

PDF	ISBN 978-92-847-2189-4	ISSN 1977-5679	doi:10.2865/6310	QJ-AB-19-008-EN-N
HTML	ISBN 978-92-847-2201-3	ISSN 1977-5679	doi:10.2865/27110	QJ-AB-19-008-EN-Q

© European Union, 2019.

For any use or reproduction of photos or other material that is not under the European Union copyright, permission must be sought directly from the copyright holders.

Getting in touch with the EU

In person
All over the European Union there are hundreds of Europe Direct information centres. You can find the address of the centre nearest you at: https://europa.eu/european-union/contact_en

On the phone or by e-mail
Europe Direct is a service that answers your questions about the European Union. You can contact this service:

• by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls),
• at the following standard number: +32 22999696 or
• by email via: https://europa.eu/european-union/contact_en

FINDING INFORMATION ABOUT THE EU

Online
Information about the European Union in all the official languages of the EU is available on the Europa website at: https://europa.eu/european-union/index_en

EU Publications
You can download or order free and priced EU publications at: https://publications.europa.eu/en/publications. Multiple copies of free publications may be obtained by contacting Europe Direct or your local information centre (see https://europa.eu/european-union/contact_en).

EU law and related documents
For access to legal information from the EU, including all EU law since 1952 in all the official language versions, go to EUR-Lex at: http://eur-lex.europa.eu

Open data from the EU
The EU Open Data Portal (http://data.europa.eu/euodp/en) provides access to datasets from the EU. Data can be downloaded and reused for free, for both commercial and non-commercial purposes.