The NBS Head Office Building was built from 1888 – 1890, on the basis of blueprints designed by Konstantin Jovanovic (Vienna 1849 – Zurich 1923), son to distinguished artist Anastas Jovanovic...
Stress test design and implementation should ideally be based on “best practice” principles. They draw on the experience from the recent financial crisis. In addition, it is extremely important that stress tests are sufficiently operational. Such principles should be used as operational guidance for all participants in the stress testing process. All testing phases should be adjusted to specific circumstances while ensuring standardisation, which will enhance comparability, protect against obvious pitfalls, and contribute to the appropriate interpretation of test results.
Principles presented here are mainly focused on macroprudential stress tests, although they are also applicable to other types of stress tests1. This is recognized in “Principles for Sound Stress Testing Practices” (BCBS, 2009) which are focused on the internal stress tests conducted by financial institutions.
This principle targets the selection of the institutions to be included in the tests. In case of macroprudential stress tests, the best results are obtained when the entire banking system is covered, but if that is not possible, the test should include systemically important financial institutions (i.e., capable of triggering or amplifying systemic risk).
Size, substitutability, complexity, and interconnectedness are the criteria that are used to assess systemic importance2. A distress or failure in one institution can cause damage to other financial institutions, financial system, or the entire economy if its activities comprise a large share of financial intermediation. Great interconnectedness among financial institutions increases the probability of problems spilling over. The systemic impact of a financial institution’s failure depends on the degree of its substitutability (in the case where specific services are provided). Also, the systemic impact of a financial institution's failure is higher for more complex institutions, as the costs and time needed to resolve these are greater. While size, degree of substitutability, and complexity are observable features, an assessment of interconnectedness requires the use of sophisticated network approach. Financial institutions hold claims against each other, which can be presented as a matrix of bilateral claims. Network models can also be used to measure the impact of simultaneous defaults by several institutions (stress testing of individual institution’s solvency can lead to underestimated systemic risk).
Behaviour of nonbank institutions may trigger or propagate systemic risk, which must be taken into account in network modelling.
Ultimately, stability of the financial system as a whole depends on the reliable functioning of the financial infrastructure (payments systems, central securities depositories, securities settlement systems).
In addition to network effects among financial intermediaries, there are other channels of shock propagation.
Key examples of propagation channels:
The feedback between liquidity and solvency risks. This includes the non-linear relation between funding costs and risk perception by bank creditors; and the relation between fire sales and further declines in asset values, that in turn aggravate solvency problems.
The feedback from financial instability to the real economy, for example, deleveraging of banks can have negative effects on the real economy that in turn degrade bank asset values further (second-round effects).
The bank-sovereign link. Traditionally, the key source of risk in the relation between banks and the sovereign is related to the funding of sovereign debt from the credits granted by the banking sector or through the purchase of government securities by banks. Policy reactions (or lack of them) could have a significant impact on risk transmission and on the duration of a crisis.
Proper stress test design requires a careful examination of the transmission channels and an understanding of the range of possible responses of financial institutions and capital markets. There are still gaps in our understanding of the interaction of the real economy with the financial sector — the macro-financial framework — and of the role of the financial architecture and business practices in amplifying and transmitting negative shocks.
Capturing all risks is key to obtaining reliable stress test results. Until the global financial crisis, stress tests typically focused on credit risk and market risk. The crisis revealed that this coverage was inadequate and other sources of risk, such as sovereign risk or liquidity shortage risks should also be included. For complex financial institutions, incorporating cross-border activities through cross-ownership, liquidity, credit and market risk exposures is important for both home and host country supervisors.
Incorporating certain sources of risk in stress testing is not simple. Some risks, e.g. sovereign risk, can be so large that independent protection against them is debatable. It is, thus, important to create an international financial stability infrastructure. Similar problems can also occur with a system-wide liquidity shock. Incorporating all risks in the stress testing remains critical for gaining a comprehensive insight into the situation, which would enable timely reaction at both national and international level.
Stress tests conducted by central banks, as well as FSAPs, have a two-year or even longer time horizon, and the profit projections can have a significant impact on the test results. Ignoring these profit buffers could thus exaggerate the impact of the shocks.
The application of this principle in practice requires the stress testers to verify banks’ reports and have a good understanding of all risks, including those that might be too big to mitigate, as well as business polices and different models of bank behaviour, before designing the stress tests.
Observance of this principle is equally important for microprudential and macroprudential supervision. However, microprudential supervision is better at identifying bank-specific risks, while macroprudential surveillance is more effective at recognising macroeconomic risks and macrofinancial dependencies. This is why both stress tests are necessary, particularly given the synergy effect on the quality of results.
When modelling stress tests, it is important to incorporate different business models of financial institutions. In the past decade, a large number of global financial institutions started to rely more on uninsured short-term funding, instead on deposits. During the latest crisis, investors' concern about the adequacy of asset valuation triggered confidence shocks that had an adverse effect on some major banks. Delays in recognising that the crisis was motivated by solvency concerns, as well as political difficulties in finding solutions to address these concerns, made the crisis deeper.
All this has led to the conclusion that stress testing based on regulatory and accounting standards should be complemented with the market perspective.
Application of macroprudential stress testing has resulted in the changes of legislation requiring additional recapitalisation and setting higher benchmarks for the quality of bank capital. From a macroprudential standpoint, financial institutions have to be sufficiently capitalised not only to ensure their own viability in the event of a system-wide shock but also to prevent them from becoming transmission channels. This requires higher capital than is necessary for a single bank, when it is considered on a stand-alone basis, separately from the financial system as a whole. This principle also has implications on the approach to be taken when it comes to publishing stress test results. Publication of stress test results during periods of uncertainty can help remove false information and restore market confidence. In the case of stress tests undertaken for surveillance purposes during stable periods, communication of their results could create awareness of risks and enhance market discipline, which, in turn, could reduce the probability of sudden reversals of banks’ policies. Of course, communication of test results for individual institutions also entails great responsibility, since stress tests need to be objective assessments of risk, explicit about the coverage and limitations, and the announcement of their results needs to be accompanied by the measures aimed at removing or mitigating vulnerabilities, including but not necessary limited to necessary recapitalisation.
This principle is related to the main purpose of stress testing — to estimate the resilience of the system to extreme but plausible shocks. The size of shocks should be calibrated on past experience: “worst-in-a-decade” events, low-probability event (e.g. one percent probability), tail event, or an “n standard deviation shock”. A problem with this approach is that historical experience changes over time: a “worst-in-a-decade” scenario would look very different today. Another problem is that the history of many financial products was too short to provide a reasonable amount of volatility, and the shocks calibrated on this experience were too mild. It is not easy to select the scenarios appropriately. Therefore, some central banks may be reluctant to use excessively negative tail scenarios but instead rely on baseline scenarios. In order for the test results of individual institutions or the system as a whole to be valuable, it is not enough to take into account the influence of a single risk, but the combination and interdependence among risks, as well as interdependence among affected institutions. Modest but correlated shocks can generate extreme outcomes. Also, risk factors may be weakly correlated under normal economic circumstances but highly correlated in times of distress. Finally, the joint default risk within a system varies over time and depends on the individual institutions’ likelihood to cause or propagate shocks through the system.
Network modelling as a technique shows interdependence among financial institutions, enables quantification of network’s vulnerability to shocks, as well as the analysis of network's resilience to risks.
Since the onset of the global crisis, public communication of stress test results for individual financial institutions has gained momentum. Before the crisis, only a few central banks disseminated stress test results on an aggregate basis in their Financial Stability Reports. However, the crisis has brought about a shift in policy regarding the publication of test results. Economic policy-makers saw it as a good way to shore up market confidence. In the US, publication of stress test results is stipulated by law3, which has boosted the public interest in this area.
Public disclosure of stress test methodologies, assumptions, individual risks, and scenarios, in addition to the publication of test results can help raise public awareness of risks, strengthen market discipline, and encourage a discussion about financial stability policies. Public communication can have a positive impact even if the results are unfavourable, if it is accompanied by credible contingency plans and support measures for financial institutions that fail the tests, measures aimed at solving problems and measures dedicated to financial stability maintenance. Otherwise, if assumptions and scenarios are not in tune with the proposed measures, there is a great risk of misinterpretation. This could diminish the confidence of the public and encourage financial institutions to "cheat" on the test (in case of bottom-up tests), or falsify reports (top-down). Also, as publication of stress test results becomes common, and assumptions and scenarios are constantly changing, the results can cause confusion.
Both advantages and disadvantages of test result disclosure should be considered:
Regardless of the comprehensive coverage of risk factors, the quality of analytical models, the assumed severity of shocks, and the communications strategy, there is always the risk that the “unthinkable” will materialise. Stress tests do not predict the future. They provide an assessment of the resilience of a financial institution or a system to the shocks assumed. Future shocks will likely arise from completely new products, unexpected events (e.g., the breakup of currency unions), factors that have historically shown little impact on system stability, or risks that have not materialised for such a long time that they have been forgotten (e.g., advanced country sovereign defaults).
What practical ways are there to incorporate these factors into stress tests?
One approach is to supplement the traditional stress tests with possible shocks based on expert judgment and new information. Some central banks published a “guide” for the choice of the scenario, bearing in mind its significance.
The Federal Reserve Board typically uses two scenarios for stress tests: one is unique to each institution and chosen by it; the other is common. In this way, the institutions are assessed under specific scenarios that they themselves created on the basis of common assumptions.
It is important to strike a balance when projecting shocks. It is extremely important not to let “weak banks” show good results (pass the test), but it is not useful to design shocks that are too large and fail many banks.
Reverse stress testing – receiving shock assumptions based on the expected results of individual institutions - could help anticipate a wider range of risks.
Another approach is the application of distribution theory to the scenarios themselves, as opposed to the practice of choosing just one adverse scenario. This approach is based on the recognition that the future is stochastic and can be represented by a number of event combinations, each of which with a probability of realisation. Scenarios are prepared on the basis of a historic distribution of inputs, using Monte Carlo simulations. Each scenario is represented by a combination of inputs. The final outcome is represented by a distribution of capital adequacy ratios for each bank, in which each point of the distribution is associated with a particular scenario4.
Ultimately, this principle is more about the context and proper use of stress tests than about the mechanics of their design and implementation. It is not easy to assess to what extent actual stress testing practice complies with this principle. Instead, this principle should serve as a reminder that stress tests should not be undertaken as an isolated analysis and their results should not be taken as the only measure. After a certain period of time, having compared test results with actual performance, we will see if we were too optimistic or too pessimistic when creating scenarios. Despite greatest caution, there will always be model risk, insufficient data access, or underestimation of the severity of the shock. Therefore, stress test results should be set in a broader context.
1 Macrofinancial Stress Testing—Principles and Practices, IMF, 2012.
2 Report on Guidance to Assess the Systemic Importance of Financial Institutions, Markets, and Instruments: initial considerations IMF/BIS/FSB (2009), Global systemically important banks: Assessment methodology and the additional loss absorbency requirement BCBS (2011)
3 Dodd-Frank Act provides that FED must publish stress test results for large banks.
4 Barnhill, Theodore, Panagiotis Papapanagiotou, Liliana Schumacher, 2002, ― Measuring Integrated Credit and Market Risks in Bank Portfolios: An Application to a Set of Hypothetical Banks Operating in South Africa, Journal of Financial Markets, Institutions and Instruments, 11(5) pp. 401–443
