Home > Risk > The cyber heat map

The cyber heat map

February 24, 2019 Leave a comment Go to comments

Vince Dasta of Protiviti makes a good point (pun intended – as will be explained shortly) in Cyber Risk Assessment: Moving Past the “Heat Map Trap”.

Here are a few excerpts:

  • Given the limits on time, attention and resources with which every cyber team must contend, risk assessment plays a critical role in helping set priorities and decide between options. Having a rigorous and accurate risk assessment process goes a long way in determining an organization’s cybersecurity performance.
  • Unfortunately, our observation has been that most cybersecurity professionals significantly overestimate the quality of their risk assessment programs. The common weakness? A reliance on what can be called “pseudo-quantitative” methods, in which risks, benefits and other factors are given labels or colors (such as red, orange, yellow and green) or ratings on an ordinal scale that run, say, from 1 to 5. These approaches have the veneer of objectivity but are actually highly subjective. The illusion of objectivity is all the more deceptive because of the frequent use of scientific-looking heat maps.
  • Monte Carlo simulations generate a probability distribution curve plotting the likelihood of a loss exceeding a certain amount.

Vince argues (quite well, IMHO) for a process that considers what might happen, identifies the various potential impacts should that happen, then uses Monte Carlo methods to develop a chart that shows the range of those potential effects.

In Making Business Sense of Technology Risk, I explain why even this would fall short.

For example:

  • Before you can assess whether the level of risk is unacceptable, you need to decide whether you need to take the risk in order to achieve business objectives. Looking only at the threat side of risk and reward will not lead to a quality business decision.
  • Using only monetary loss measures to ‘value’ the level of risk is not always meaningful to executives making business decisions. They need to be able to compare the need to invest in cyber to the need to invest in product development, marketing, the implementation of new technologies, acquisitions, and so on.
  • Boards and executives are (or should be) focused on achieving objectives. They will be able to make more informed and intelligent decisions if all the risks and opportunities are expressed in terms of their potential effect on the likelihood of achieving enterprise objectives.
  • Heat maps are focused on ‘risks’, assuming (incorrectly) that the level of risk is a point when in fact there is a range of potential effects, each with its own likelihood. Decision-makers should not focus on risks to avoid or mitigate but on the success achieved by taking the right risks.

Even so, I commend Vince for his initiative to help organizations get a better handle on cyber and its potential effect on the organization.

I would like to see everybody considering cyber as just another source of business risk that needs to be weighed, with all other risks to objectives, when making strategic and tactical decisions.

I welcome your views (and comments on the book, once you have read it).

Advertisements
  1. February 24, 2019 at 2:39 PM

    Excellent points!

  2. Osama S.
    February 25, 2019 at 7:04 AM

    Looks familiar somehow, Oh yes it’s using images from my youtube presentation
    13 Reasons Why Heatmaps Must die

    A little credit would have been nice Protiviti
    And Norman, the presentation refers to FAIR, which you recently had some issues with.

  3. Osama S.
    February 25, 2019 at 7:14 AM

    Hi Norman,

    “Using only monetary loss measures to ‘value’ the level of risk is not always meaningful to executives making business decisions. They need to be able to compare the need to invest in cyber to the need to invest in product development, marketing, the implementation of new technologies, acquisitions, and so on.”

    We are asking the business to invest in managing a cyber risk, which will take money away from investing in other things. If the cost/benefit of each investment is expressed in monetary values, why wouldn’t management be able to make a meaningful comparison?

    “I would like to see everybody considering cyber as just another source of business risk that needs to be weighed, with all other risks to objectives, when making strategic and tactical decisions.”

    Doesn’t expressing all risks in monetary value support that? What other ways are there to compare cyber, tech, financial, investment, XYZ risk?

    • February 25, 2019 at 7:41 AM

      That’s what I have been trying to tell you online. Quantifying individual risk rather effect of riskS on objective or decision is meaningless. Hence the question you are asking Norman is not the right question.

      • Osama S.
        February 25, 2019 at 9:17 AM

        Alex, I generally agree but some decisions are more tactical rather than strategic.
        Sometimes your objective is very simple “Let’s close this audit action” then the decision is simply “should we do action 1 or 2 or 3.” Not every decision needs to be traced back to the strategic business objectives.

        • Norman Marks
          February 25, 2019 at 9:20 AM

          But, does it make sense to close the audit item, or should we accept the underlying risk because that is right for the business?

          • Osama S.
            February 25, 2019 at 9:22 AM

            That would be shown in analysis “3” for example in that option we would include losses due to fines, litigation etc.

        • February 25, 2019 at 9:25 AM

          Never said it should be traced back to strategic objectives. Tactical decision 1 or 2 or 3 is a perfect example of how we need to recalculate 3 options with risks in mind, not calculate just cyber risk exposure. Huge difference. This is huge and fundamental. It took me 15 years to finally get. Taleb calls it X and F(X) and dedicated books to it. This is so groundbreaking the whole FAIR methodology falls apart (partially at least), as well as ISO31000, coso and the rest of them

          • Osama S.
            February 25, 2019 at 9:29 AM

            FAIR does only Risk Analysis i.e. measuring the risks in doing 1, 2 or 3 (it also help to scope the question you are really trying to answer) How you present it, what you do with that information, how you use it to make decisions is not the purpose of FAIR. FAIR fills the risk analysis gap that standards ISO 31000, COSO etc. give no practical guidance on.

            • February 25, 2019 at 9:30 AM

              FAIR makes the same mistake everyone else makes, trying to measure X instead of F(X). It’s a dead end

              • Osama S.
                February 25, 2019 at 9:39 AM

                FAIR does measure how risks affect us or the objectives i.e. F(x).
                To help me understand better, can you give me an example, preferably from the cyber world, where you believe that isn’t true?

                • February 25, 2019 at 9:41 AM

                  Let’s have another online debate about it. FAIR does not measure f(X)

                  • Osama S.
                    February 25, 2019 at 6:51 PM

                    Why is everything a debate with you ;-)?
                    Don’t want to debate, want to better understand what you really mean. Throwing Taleb’s x , F(x) at me doesn’t help me understand your point of view. Need a little more elaboration.

                    • February 25, 2019 at 9:40 PM

                      Because talking is easier than writing in comments. I tried to make my point very clear in this video https://youtu.be/uyqrk-gQmCs but may have to write an article about it

                    • Osama Salah
                      February 25, 2019 at 10:02 PM

                      Watched the video, don’t see why FAIR as a Cyber/Operational Risk Analysis model doesn’t fit in the story you are telling.

                    • February 25, 2019 at 10:28 PM

                      Why am I not surprised :))) I will write an article how to make FAIR methodology better soon

    • Norman Marks
      February 25, 2019 at 8:48 AM

      Good questions. Not to avoid answering, but I dedicate much of Making Business Sense of Technology Risk to the issue.

      Quick answer: sometimes the monetary effects of a couple of options compare differently to the effects on objectives. For example, a cyber intrusion that disrupts the business may only cost $3 million out of pocket (the average loss according to studies), but could prevent the company from selling its products for several weeks.

      • Osama S.
        February 25, 2019 at 9:11 AM

        All losses are included in the loss magnitude estimation.
        with “$3 million out of pocket” you may be referring to response costs; which is one type of loss.
        “prevent the company from selling its products for several weeks.” would be another type of loss i.e. productivity loss and would also be included in the loss magnitude calculation. If that’s an objective then it can easily be measured in monetary value. There are other costs like litigation, competitive advantage etc. all these losses need to be modeled and included.

        • Norman Marks
          February 25, 2019 at 9:15 AM

          The $3 million is the total cost identified by companies in the survey.

          Measuring in monetary terms ignores the fact that we need to know the likelihood of the larger level of loss, as well as the likelihood of lower levels of loss.

          Everything can be modeled, but should disparate effects be combined? Better to focus on what information decision-makers need, and they should be focused on the likelihood of achieving their objectives.

          If you want to measure in dollars and then move to reporting the effect on objectives, I am fine with that – as long as the range of potential effects is presented, not a single value.

          • Osama S.
            February 25, 2019 at 9:25 AM

            Most likely achieving their objectives somehow translates in requiring some investment.
            I can’t think now of one that couldn’t be somehow measured in monetary value.

            We need an online chat not a comments box 🙂

  4. Gregory Sosbee
    February 25, 2019 at 8:24 AM

    As an overreaching point, in addition to overestimating the value of IT programs, you can show expense and time to mobilize issues that IT practitioners underestimate.

    To your point, “risk heat maps” should be a measure relative to risk v. risk, and not “the risk”. In my experience, Boards ask “what is the risk” instead of “what is the risk of A compared to B”. As ERM advances in senior executive circles and risk is read as opportunity, the issue should take care of itself, and, probably more importantly, Boards will begin to understand that models produce data points; not answers.

    • Norman Marks
      February 25, 2019 at 8:50 AM

      I want boards to ask how this or that will affect the achievement of our objectives. They need to know whether management is making informed and intelligent decisions and taking the right risks.

  1. February 25, 2019 at 11:44 PM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: