Home > Risk > An ERM horror story

An ERM horror story

This week, I was working with the SOX team of a large US-based financial institution. At one point, the senior executive and leader of the team asked me something I had never heard before.

“Our ERM team wants me to provide them a number they can include in their calculation of the company’s residual risk. This is something, they say, is required by the regulators. What do you think of that?”

I have to admit to being stunned. Silent.

Then I couldn’t hold it in any more.

“It’s stupid!” I blurted out.

ERM at this organization sounds like something from a 1920’s horror movie.

How could anybody believe there is value in a single number ‘residual risk’ for a large organization?

Does it make sense to aggregate risk levels for a variety of risk sources, including cyber, compliance, credit, liquidity, competitor, and internal control over financial reporting?

Does that help management make any decision? How is it actionable?

Does it help the regulator understand whether management is putting the interests of stakeholders in jeopardy?

What I will bet is happening is this:

  • Each type of risk at the organization (including but perhaps not limited to those I listed above) are individually assessed. They use a single number for the potential impact (in other words, they don’t consider a range) and then calculate a ‘risk level’ by multiplying that by the likelihood of an event or situation occurring that might have that effect.
  • They then add the risk levels of individual types of risks together.
  • They then, perhaps, compare that number to a pre-determined ‘risk appetite’.

This is wrong on so many levels. I have discussed why many times in this blog and in my books, but:

  • There is a range of potential effects, not a single point
  • Multiplying one point on that range by its likelihood has minimal limited meaning
  • Adding these risk levels together is mathematically unsound
  • The whole process ignores the fact that any event, situation, or decision gives rise to many potential effects – some of which are positive
  • The context for risk-taking is ignored: objectives and strategies, what the organization is trying to achieve. How does this help you assess whether the organization is likely to achieve its objectives?
  • The calculation does not provide the regulators with information that will help them assess whether the organization is unacceptably likely to become illiquid, etc.
  • This is not how people make (or should make) decisions
  • This exercise is likely to mislead rather than provide meaningful and valuable information

I would appreciate your comments.

  1. msfedorov
    August 30, 2019 at 12:08 PM

    Sad but true. Such practice is still ubiquitous even at the regulatory level.

  2. Jorge Jaramillo
    August 30, 2019 at 12:24 PM

    Excelent article, I have faced this same situation before but did not know how to react to it.

  3. Roger Estall
    August 30, 2019 at 1:50 PM

    The only thing that I didn’t understand in your post Norman were the words ‘is unlikely to’ in your last bullet point. This suggests some possibility that good might come of it. Surely it is axiomatic that this approach will not only mislead but actually degrade decision making.

  4. Bill Storage
    August 30, 2019 at 8:42 PM

    Sadly, a large part of the ERM community has perpetuated simplistic early 20th century beliefs like Knightian uncertainty/risk and the notion that risk is a scalar value.

  5. Chamunorwa
    August 30, 2019 at 10:07 PM

    Each and every organisation has different objectives and strategies. The strategies are crafted taking into consideration the culture, risk appetite and tolerance peculiar to the organisation. Risks are assessed using robust methodologies not one size fits all approach. Simply using a number to determine the risk tantamounts to a recipe for disaster.

  6. August 30, 2019 at 10:47 PM

    Sadly this is what happens when regulators, or even commercial or public entities, put ‘the wrong sort of accountant’ in charge of risk and decision systems. Everything has to become a number.
    Here in the UK, it is even common for government entities to require an accountant to take on risk roles, especially board positions. This totally misses the point, implying risk is somehow tied up with numbers and with audit, rather than with decision science, strategies, heuristics and culture. Whilst there are many fine accountants in this world, accountancy is not risk and decision leadership. The two are utterly different disciplines.

  7. August 31, 2019 at 3:07 AM

    Norman, have been saying the same thing for more than twenty years – and more vociferously since the ERM cottage industry sprouted early in the last decade.
    May I suggest taking the notion of “range” a little further to address a particularly problematic aspect of our professional lives – the label “risk management” !
    People, projects, business divisions, teams are managed – “risk” is not managed.
    Further, separating risk from reward or return makes no sense in practice.
    Planting the notion of range into an operationally-useful definition of risk makes this self-evident.
    My experience suggests that risk is most usefully defined as a measure of deviation from a range of expected outcomes over defined time periods.
    Risk is, effectively, a measure of distance – that can be plugged into all kinds of decision-making contexts.

  8. David C Spinks
    August 31, 2019 at 10:02 AM

    Executive directors in regulated firms should have an understanding of the Residual Risks value normally measured in £s this is the amount of exposure remaining after expenditure on risk mitigation has been executed. Risk categories (Operational, Credit and Market) should be calculated as per Basel III recommendations but whilst there might be a relationship across the risk categories each is generally kept separate.

    Risk calculations should be formally tested … again the tools and methods for doing this have been well documented over a number of years.

    • Norman Marks
      August 31, 2019 at 10:23 AM

      Where is the value, other than satisfying the regulators? Does this help ensure the right risks are taken?

      • August 31, 2019 at 10:49 AM

        Nothing to do with regulators. Unless the decision makers understand the risks inherent in their business then how can they possibly take the right decisions. Unless you quantify risk (threat – likelihood – impact – loss – cost to mitigate etc …) how can you manage risk? The finger in the air or Low Medium High today worries me …. we are talking about regulated industry (medical, nuclear, utility, aviation etc….)

        David Spinks – dspinks41@gmail.com

        • Norman Marks
          August 31, 2019 at 10:52 AM

          How can you know which risks to take if you ignore the upside and the need to achieve objectives? Don’t manage risk, manage success. One residual risk number does nothing

          • August 31, 2019 at 11:33 AM

            O I get the upside as well. BUT RBS and Barings are great examples of focus on the upside ignoring the results of audits and concentration on making more money …. look at where that ended!

            Banks have surely got to take a balanced view to risk quantification of “the good”, “the bad” and “the ugly” ….

    • Gregory Sosbee
      August 31, 2019 at 11:08 AM

      “Risk categories (Operational, Credit and Market) should be calculated as per Basel III recommendations but whilst there might be a relationship across the risk categories each is generally kept separate.”

      This is adverse to the SEC’s successful Wells Fargo case that cost a number of jobs plus over $3B in fines and penalties. The SEC made themselves clear that what financial services organizations call “Operational Risk” is just as important as “Credit and Market Risks”. In other words risk = risk which means one all-encompassing review with an enterprise solution. If a regulatory body wants to see only a portion of the calculation, fine, but the organization has to show they are working off the enterprise solution.

      • August 31, 2019 at 11:27 AM

        I have worked inside many banks over a number of years and never seen a case where Operational Risks were mixed with Credit and Market risks. Different metrics. Sure the Enterprise view does need to be considered as per COSO and aligned to the business objectives. OP risk managers and Credit Risk managers come from very different backgrounds, qualifications and experience.

  9. Sid Gale
    August 31, 2019 at 4:00 PM

    The algorithm rules…
    …until it doesn’t.

    ‘Keep it simple, Stupid’
    …and stupid, simple is.

  10. Anonymous
    September 1, 2019 at 3:55 AM

    Agree with Norman. True, ERM outcome is not designed to be number driven. Range and numbers should be justifiably applied for measuring any risk attributes. Overwhelming use of numbers would otherwise numb the ERM effectiveness.

  11. Gary Lim
    September 1, 2019 at 3:14 PM

    Agree with Norman and if it is converted to a number it becomes a risk which is prescriptive a number means this and another number means that, does all the employees of the company has the same imagination for each and every type of risk of the same number. The likelihood scale is a very subjective issue, people who has high tolerance of risk will put it on the scale hence the residual risk computed is on the low side, so no action required.
    However it should be descriptive and when it is read, there is a better feel of the risk.

  12. Rick C.
    September 2, 2019 at 1:09 AM

    Not forgetting Residual Risk’s equally unhelpful older sibling, Inherent Risk. Like trying to argue when buying a house that the price should bear some relation to what it sold for thirty years ago.

  13. September 2, 2019 at 1:25 AM

    Hi Norman,

    Sad but true, that executives (who do not care about risk) or regulators (who do not care about business) ask for something that tells them something they can act on – valid or not.

    HOWEVER; it can actually be done validly – but has to be done for each risk/objective metric separately. Let us take liquidity as an example:
    – The risk exposure in term son an outcome range and probability is analysed and described for each risk which may affect the company’s liquidity. This includes the liquidity effect of uncertainty of sales as well as a bankrupts customer etc.
    – These evaluation are added to your liquidity forecast model
    – The whole thing is Monte Carlo simulated
    and you look at the outcome of this simulation.

    Now imagine the CFO states, that the risk tolerance (/appetite) insist that there must be a 99% certainty of not running completely dry of liquidity, and a 90% certainty that liquidity status will not call for further bank loans/credit facilities to be invoked.

    The liquidity model can now show both to which extent you are complying with your risk tolerance, the likelihood of your needing to invoke further credit facilities/loans, the likelihood of running out of liquidity AND which uncertainties are the most important to address to rectify a possible non-compliance with the defined risk tolerance.

    So … it is only a matter of using/leveraging statistics and analytics in your risk management.

    Now why every risk manager on Earth is not pushing, screaming or just doing this is beyond me, as this is the only valid justification of their position – but that’s another story.

    • Norman Marks
      September 2, 2019 at 6:53 AM

      Sounds like what I say in my books, Hans

  14. Japoy
    September 24, 2019 at 1:38 AM

    Couldn’t agree more with you Norman.

  1. August 31, 2019 at 4:54 AM

Leave a Reply to Anonymous Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: