Home > Risk > What do you think of heat maps?

What do you think of heat maps?

September 8, 2020 Leave a comment Go to comments

Heat maps are one of the most popular ways of comparing individual sources of risk.

A heat map is suggested as a way of reporting in the COSO ERM Framework.

But I dislike them, as do many practitioners. My reasons include:

  • There is a range of possible effects from a possible event or situation, not a single point, and each point in the range has its own likelihood.
  • It doesn’t help you to determine whether to take a risk, because it is without any context of potential reward.
  • Decisions should be based on the big picture. An objective may be affected by multiple sources or risk and opportunity (things that can happen with positive and/or negative effects). Making decisions one source of risk at a time is clearly sub-optimal.
  • It focuses on risks while I want to focus on achieving objectives, what I call success management.
  • There are better methods, which I have described in this blog and in my books.

Grant Purdy shared an article with me (he dared me to write about it) that takes a more satirical view.

An exciting new lexicon for the professional risk manager has a different way of describing heat maps.

What do you think?

  1. GSosbee
    September 8, 2020 at 1:06 PM

    Heat maps have the same problem as simulations, data points – not answers, are produced. The other issue is decision-makers want comparisons of alternatives; not pictures or splatter diagrams.

  2. David
    September 8, 2020 at 5:00 PM

    I suggest the ‘heat map’ is a communication vehicle, not an analytical method. The evaluation of likelihood and impact is anticipated to be done elsewhere before being used to create the ‘heat map’.

    • Norman Marks
      September 8, 2020 at 5:15 PM

      But is it an effective communication tool? I don’t think it has much meaning for the reasons stated.

  3. September 9, 2020 at 9:40 AM

    I’ve suggested using heat maps in two circumstances:
    In risk workshops to highlight significant risks which need controls
    In the ‘Objective, risk and audit universe’ to highlight controls which need checking.

    David (www.internalaudit.biz)

    • grantpurdy
      September 11, 2020 at 8:27 PM

      David, A high relative level of risk can only indicate a high priority for attention. In no way can it indicate that risks are ‘significant’ or that ‘controls’ are needed. To decide what you do with a ‘risk’ that goes through a rating process, you have to further consider how it can be treated and if that is worthwhile using benefit/cost analysis. Often, you just have to put up with the ‘risks’ that are rated highly.

      The IIA published guidance on this that explained why a high risk rating does not induce a ‘control’ that needs checking. It says, quite rightly that risks with a high potential exposure (or maximum forceable consequences) and a low, relative rating are those that need checking.

      There is no point checking controls for risks rated ‘high’. You have either to treat them further or put up with them!

      • Norman Marks
        September 12, 2020 at 5:19 AM

        Disagree, Grant. If there is a possibility of serious harm such that objectives will not be achieved, the controls over that source of risk should be assessed. These controls not only ensure that risk responses are carried out, but that the level of risk is not higher than management believes. There is also an opportunity to see if additional responses can and should be made.

        • grantpurdy
          September 13, 2020 at 5:41 PM

          Norman,

          I agree with you. But that is not what I said.

          I said that auditing and other forms of checking should not be focused the controls for risks with the highest relative ratings. These risks are those that should be given priority consideration for treatment.

          Auditing and other forms of checking should be focused on the risks where we ‘believe’ the relative level of risk is low but the potential maximum consciences are very severe. These are where boards and senior management need ‘assurance’ that the controls are really effective.

          Having said all that, of course, I really don’t know what I’m talking about because I don’t know what risk’ or ‘control’ means! And no one else can agree on these either.

  4. grantpurdy
    September 11, 2020 at 8:20 PM

    Norman,

    I first developed a simple index approach for combining levels of different types of consequence and likelihood in the 80’s. I used a simple table then to combine them itop generate a priority for attention. (It arose when I was looking at a development in Norway involving the construction of a subsea tunnel and pipeline to connect a production rig with shore).

    I think people forget that the matrix is only a device to allow the combination of dissimilar measures and factors that are measured on different scales and with different units. If we just measured the level of risk using $ per year, we would not need matrices.

    The many fatal flaw present in the systems of ‘rapid risk ranking’ (to give its it correct name) that are generally used include:
    • The consequence scales bearing no resemblance to the organisation’s purpose (objectives);
    • Using a generic system, normally because it is given in a standard or book or built into software;
    • Big, inconsistencies in terms of effect when comparing similar levels across the consequence scales;
    • Not having enough levels in the consequence scale to adequately discriminate a range of scenarios;
    • Properly labelling the levels in the scales and having helpfully worded descriptors for the levels in the individual scales;
    • Trying to use likelihood of an ‘event’ rather than the likelihood of a chosen level of consequences;
    • Likelihood scales that only deal with either frequency (return period) or probability (often, of the event) that are inconsistent with the decision being made;
    • Thinking that what is only relative, qualitative risk ranking can be, somehow, converted into absolute quantitative analysis;
    • Whacky semi-quantitative methods that involve other factors and indices – that have no scientific basis;
    • Trying to do mirrored matrices for ‘outcomes’ that have both or either beneficial or detrimental outcomes;
    • Assuming the resulting relative rating is, somehow, an absolute measure of the level of risk and can lead to decisions on ‘acceptance’. All you can do with the results is rank them as a priority for attention. You can’t even decide if they must be treated until you’ve gone through a benefit/cost analysis.

    Most of these problems seem to arise from people either being too clever or, in most cases, not clever enough.

    I think simple scales and a look-up table, well constructed, are still an elegant way to prioritise actions when one axis attempts to reflect a range of dissimilar outcomes for an organisation and the other is some measure of likelihood. Simple is definately beautiful. However, the devil is in the detail and you need to know what you are doing when you construct one of these systems and you should never, never just copy someone else’s and use it blindly.

    To take it to one extreme, even a 2 x 2 matrix is a useful to decision making, providing it is carefully constructed.

    Sadly, however, many practitioners are blind to all this and in the same way that RM has largely become a compliance achieving activity in many parts of the world, they just use a dumb system in a dumb way.

  5. Mike B
    September 12, 2020 at 7:58 AM

    It is a tool, like many tools it has it’s benefits and limitations that you need to be aware of. Yes is built on a single data point estimate approach, never ideal. It does also visually quickly highlight the possible top scaller items to focus on for discussion. As professionals we are accountable to ensure when we are using it we are not creating a false picture for the management team making decisions based on the outcomes.

    • grantpurdy
      September 13, 2020 at 5:43 PM

      Mike,

      It’s just like giving someone who is very confused an AK47.

  6. Audit Monkey
    September 15, 2020 at 3:02 PM

    A load of BS.

  1. September 9, 2020 at 6:44 AM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: