Evaluating Financial Risk Management Systems and Practices: A Conversation with Andy Shaw

Market disruptions – particularly events such as market crashes, liquidity crises, interest rate shocks, and major bankruptcies – often lead to significant investor or institutional losses.

Such events are often followed by litigation and government investigations related to financial firms’ risk management. In such instances, experts are frequently retained to assess whether the firms’ risk management systems and practices were reasonable and consistent with industry standards.

To learn more about the evaluation of risk management systems and practices, Managing Principal D. Lee Heavner sat down with Andy Shaw. Mr. Shaw is the founder and managing director of financial risk consultancy Links Risk, and he specializes in risk management for derivative trading, including the development and validation of risk models and governance frameworks. The two discussed the evolution of risk and risk management standards and Mr. Shaw’s approach to evaluating risk management at financial firms.

What is the goal of risk management, and when do you tend to see disputes over these practices?

For financial institutions, the goal of risk management is to identify and understand risks to the firm’s business objectives, to monitor those risks, and to mitigate them where necessary. It’s important to note that risk management practices are not designed to eliminate risk. Rather, risk is integral to the finance industry, and incurring risk is necessary to generate returns above those you could expect from investing in T-bills.

Litigation around risk management tends to spike after major market declines like those that followed the COVID-19 pandemic or the 2008 global financial crisis. In these situations, it’s typical for investors to argue that the investment manager’s risk management did not comply with prevailing industry standards and practices. For example, risk management claims were central to shareholder derivative litigation against former Credit Suisse executives following multibillion-dollar losses resulting from the collapse of Archegos Capital and defaults from other counterparties.

How do you approach an evaluation of a firm’s risk management systems and practices?

There is no one-size-fits-all risk management formula that applies to all financial institutions. Among other things, risk management standards vary with firm size, business models, regulation, and the extent to which firms are interconnected with other financial institutions and the broader economy. For example, industry standards for risk management are more conservative at SIFIs [systemically important financial institutions] and particularly among G-SIBs [Global Systemically Important Banks] than at buy-side firms that invest client capital.

Similarly, even among firms in the same line of business, there’s no universal set of analytical tools that is appropriate for evaluating risk in all circumstances.  Financial firms may employ a variety of analytical tools, including VaR [Value at Risk], simulations, scenario analysis, and stress testing. Rather than focusing on whether a specific tool was utilized in a specific way, my evaluations assess whether a firm’s risk management system, including its analytical tools, is appropriate for assessing and managing risk in accordance with policy and/or regulatory requirements.

For these reasons, when assessing a risk management function, I consider the firm’s business objectives, its size and resources, applicable regulations, standard practices at comparable institutions, among other factors. I also assess whether the firm has developed a deliberate and structured approach to risk governance. A firm may reasonably choose to accept certain risks; the question is whether those risks were identified, monitored, and mitigated in a manner consistent with the firm’s stated risk appetite and applicable standards. 

 

Can you elaborate on the more conservative standards for SIFIs?

SIFIs are larger, regulated firms like global banks, the failure of which could jeopardize the stability of the broader financial system. For this reason, SIFIs are often referred to as “too big to fail.”  Given their size, complexity, and interconnectedness, SIFIs are subject to greater regulatory expectations, including from the Financial Stability Board [FSB]. Regulation that followed the global financial crisis requires SIFIs to employ risk measures like scenario stress testing and risk management protections around capital, liquidity, counterparty credit risk, risk data aggregation, and corporate governance. Consequently, SIFIs have highly structured and complex risk management systems.

In contrast, while the failure of a typical hedge fund could impose substantial losses on its clients, the fund’s collapse would not jeopardize the economy as a whole. As such, the regulations, objectives, and practices for hedge funds differ markedly from those of SIFIs.

The term “Three Lines Model” is often referenced in risk management discussions. How does this framework apply to investment management firms?

The Three Lines Model, also known as the Three Lines of Defense, is a classification often used to refer to different parts of an organization’s risk management and the different roles these parts undertake. In an investment management firm, the first line of defense would include portfolio managers and other staff who are responsible for managing the primary risks in a firm’s investments on a day-to-day basis. The second line often monitors risk at a more aggregated level than is seen by individual members of the first line – for example across all the managed portfolios as opposed to at the portfolio level. The second line is typically required to establish and monitor compliance with the risk management policies that apply to the first line of defense. In many investment firms, the second line includes risk management and compliance committees that act as the cornerstone of a firm’s risk operations. Finally, the third line performs internal audits of whether the first and second lines are operating effectively. Firms may also engage an external validator; for larger firms, the third line would oversee the external audit. For buy-side firms with limited third-line resources, the second line might conduct the validation or engage the external validator.

What principles related to the evaluation of risk management practices do you emphasize when educating courts or other factfinders?

I already mentioned two: First, risk management should be evaluated in the context of a firm’s business objectives, its size and resources, applicable regulations, and standard practices at comparable institutions. Second, the objective of risk management is not to eliminate risk. Rather, the objective is to identify and optimally manage a firm’s risk exposure, taking into account both economic costs and distribution of potential outcomes.  

I also emphasize two additional points. One, risk management is continually evolving, and an evaluation of a risk management system must be based on contemporaneous practices and regulations. Two, risk management is not intended to prevent the adverse effects associated with all conceivable events. Rather, risk management focuses on managing a firm’s exposure to plausible – including extreme but plausible – events.

 

Could you say more about the evolution of risk management?

Financial risk and risk management systems are always evolving – from regulatory changes, crisis events, and financial innovation. For example, regulatory changes following the global financial crisis increased the complexity of risk management techniques and models, for SIFIs in particular. For this reason, it would be inappropriate to evaluate a firm’s 2006 risk management practices against the standards that were established in 2009 or later.     

In addition, AI is changing risk modeling practices. Typical risk models relied upon by financial firms are explicitly codified and comprehensively understood, allowing for easy validation of their inputs and output. In contrast, output from large language models (LLMs) such as ChatGPT and Claude is the product of a complex framework of interacting optimization routines, trained on wide-ranging data, and it’s not always easy to understand or validate the inputs or processes that influence that output. As with many applications of AI, it’s important to understand the design and constraints of LLM-based risk models and apply these models with care and only when deemed appropriate by the risk management team. 

You mentioned that the goal of risk management systems and practices is not to model every conceivable risk. What did you mean by this?

Reasonable and appropriate financial risk management considers not just the severity of risks but also their plausibility. When evaluating an institution’s stress testing for extreme events, I think about the use of accepted modeling conventions and the application of historical context to assess whether risks are plausible.

Regardless of whether it is possible to identify and hedge against all conceivable events, including, say, a meteor hitting the earth and causing widespread destruction, hedging against all such risks would be too costly to be feasible. In addition, over-hedging for implausible events will often result in a misallocation of capital away from investment opportunities or the mitigation of more plausible risks.

To emphasize this point; novel scenarios occur all the time in financial markets. It’s when novel scenarios become extreme as compared to expectations that larger losses can occur.  Risk managers understand this – it’s an unavoidable consequence of taking investment risk.  A risk manager’s job is to prepare the firm to weather potential storms as best they can while not nullifying returns. ■