Previously Published by Forbes

Opinion

September 22, 2014

What Stress Testing Is Not​

Karamjeet Paul
Managing Principal of Strategic Exposure Group and author of "Managing Extreme Financial Risk: Strategies & Tactics for Going Concerns"

With banks getting ready for the year-end regulatory stress testing, it is worth looking at what stress testing is and, more importantly, what it is not.

Typically, stress-testing models simulate adverse conditions to evaluate if a system/structure can survive abnormal circumstances. For banks, the objective of regulatory stress testing, also known as Comprehensive Capital Analysis and Review (CCAR), is “to assess whether the largest bank holding companies operating in the United States have sufficient capital to continue operations throughout times of economic and financial stress …”

​Passing CCAR is important as low grades create a negative perception and accompany regulatory restrictions. However, passing stress tests can also create a false sense of security because of what it is not.

There are infinite combinations of factors that could strain a bank’s capital. Humans are only capable of imagining a small fraction of these. Therefore, no simulation can envision all possible scenarios, and thus prepare for “unexpected and highly consequential” events, also known as black-swan events that stress testing doesn’t address.

Even though black-swans can’t be predicted, Nassim Nicholas Taleb, an authority on black swans and professor of risk engineering at NYU says that “through some mental bias, people think in hindsight that they ‘sort of’ considered the possibility of such event; this gives them confidence in continuing to formulate predictions. But our tools for forecasting and risk measurement cannot begin to capture black swans. Indeed, our faith in these tools makes it more likely that we will continue to take dangerous, uninformed risks.”

The combination of this mental bias and passing stress tests can lull organizations into believing they can survive extreme crises or black swans. A similar complacent mindset existed before 2008 as managers and regulators monitored banks via VaR, a then-fashionable statistical measure of maximum possible losses, without appreciating its severe limitations. This proved not only useless in dealing with the crash, but also may have created a false sense of security pre-2008.

Relying only on stress testing, a critical component of uncertainty management, leaves most institutions unprepared for extreme adversity.

Uncertainty is defined by possible outcomes and probabilities of these outcomes. So the spectrum of uncertainty can be divided into four parts, each needing a different solution.

First, known outcomes with known certainty, or “Known-Knowns,” are easy to deal with. Second, situations where specific outcomes are unknown, but their probabilities are known, or “Unknown-Knowns,” can be handled by using known probabilities to calculate expected losses, which can be covered by pricing premiums. This is traditional risk management. Third, situations where outcomes can be defined and are thus known, but their probabilities are unknown, or “Known-Unknowns,” can be addressed by preparing to deal with the impact of known scenarios. This is stress testing. Fourth, situations where both outcomes and probabilities are unknown, as they can’t be envisioned, or “Unknown-Unknowns,” pose a problem. Black swans arise from Unknown-Unknowns.

To manage uncertainty effectively, all four parts need addressing simultaneously. However, Unknown-Unknowns require a very different approach.

Dealing with the first three parts of the uncertainty spectrum leverages the ability to define events and thus requires an “event-centric” approach. If an event can be defined, its damage can be contained via either controls to reduce probabilities, transaction pricing to cover expected adversities, or capital cushion to absorb losses. Relying solely on this approach, which is what risk management and stress testing do, leaves out Unknown-Unknowns, as they can’t be defined.

The crash of 2008 was a black-swan event where even on the morning of September 15, 2008 – the day Lehman filed for bankruptcy – it was impossible to envision what laid ahead in the next few hours, days or months.

So what should be done for black swans that can’t be defined?

Professor Taleb’s solution: “The answer is simple: We should try to create institutions that won’t fall apart when we encounter black swans.”

To create strong institutions, the focus needs to shift from event definition to damage definition, or to a “damage-centric” approach. By addressing maximum potential damage, or extreme-tail risk, regardless of the event that causes damage, banks can ensure they won’t fall apart in crises. This is not easy today, as there is no damage-centric metric for extreme risk. Bear Stearns and Lehman didn’t just have too much extreme-tail risk; they also didn’t know how close to the precipice they were operating. Such a metric is needed to address extreme-tail risk from Unknown-Unknowns.

There is a concept of “Probable Maximum Loss” in the insurance industry. It measures the maximum damage if the worst happens and all the mitigations work to reduce the impact of the damage. This can be adapted to measure and manage a bank’s strength/fragility in relation to extreme risk.

Stress testing focuses primarily on capital adequacy for “known” variables, but offers no assurance of a bank’s sustainability in times of stress. Even regulatory leaders are beginning to recognize that a different approach is needed. According to William Coen, Secretary General of the Basel Committee on Banking Supervision, “The answer isn’t simply to say you need another X basis points in capital. Maybe the better response is to require the bank to present a detailed plan on how it’s going to better manage these risks and to make sure the bank sticks to that plan.”

To be effective, these plans must address the Unknown-Unknowns or extreme risk to ensure that banks can “continue operations throughout times of economic and financial stress.”