Home > Risk > How should you assess the effectiveness of risk management?

How should you assess the effectiveness of risk management?

January 26, 2018 Leave a comment Go to comments

If an organization seeks to perform at world-class levels, it needs to have highly effective processes and practices for managing what might happen – risk.

They should be assessed and the results shared with the board by several:

  • The CEO, perhaps delegated to the COO or CFO
  • The chief risk officer (if there is one)
  • The head of internal audit

My good friend, Alexei Sidorenko of the Risk Academy recently shared a video on the topic.

He makes some good points, suggesting that assessors consider:

  • Organizational performance
  • Evidence that risk was considered in key decisions
  • The culture of the organization

I think there is more that can and should be done.

I also disagree with the idea that organizational success has a clear correlation with the effectiveness of risk management. Poorly run companies can be lucky and well-run ones unlucky.

In addition to addressing the topic in World-Class Risk Management, I covered the topic in a 2017 IIA post: How Should You Audit and Assess Risk Management?

I said:

Risk management is about:

Setting the right strategies and objectives to deliver value, considering what might happen (risk).

Understanding how the achievement of objectives may be affected by events and situations as management and staff execute those strategies.

Acting to modify the likelihood and effect of those events and situations, recognizing that each event or situation can have multiple consequences — some favorable and some adverse.

Ensuring that decisions are informed and intelligent, whether in setting or modifying strategies, or in executing them every day through management decisions across the extended enterprise, such that the right levels of the right risks are taken.

Monitoring and reporting so that board members and senior managers understand not only the levels of individual sources of risk, but whether they are likely (or not) to achieve each of their objectives.

I also said:

You could audit and assess risk management in a number of ways. For example:

  • An audit of compliance with corporate risk policies and procedures.
  • Assessing risk management maturity, using one of the available risk management maturity models (I have a few in World-Class Risk Management).
  • Assessing whether the principles for effective risk management are achieved (drawing on those in ISO31000:2009 or in COSO ERM 2017 — see here for a discussion).

I personally like a risk and objectives-based approach to pretty much any audit. Here the objective is to manage risk at desired levels. There are multiple risks to achieving that objective (again, described in detail in my book), such as failures to:

  • Include the appropriate people in decisions, where risk is taken.
  • Obtain reliable, current, and timely information on which to base decisions.
  • Address cognitive bias, which can affect both an individual and a group’s assessment of risk.
  • Ensure the desired attitude towards risk: behaviors that are influenced by the culture of the organization, a location, function, or business unit.
  • Obtain buy-in from all key individuals at all levels of management.

This is what I recommend for anybody seeking to audit and assess risk management (or the management or risk).

  • Understand risk management and its principles. The ISO31000:2009 and the 2017 COSO ERM Framework are just two possible sources, but I would also recommend my book and that of John Fraser, Implementing Enterprise Risk Management: Case Studies and Best Practices.

  • Understand what the organization needs from risk management. Start with understanding how and where decisions are made and risks taken. In fact, understanding who makes decisions and therefore takes risk is critical to understanding how risk is managed. Is it centralized or decentralized? Do individuals have a lot of autonomy and decision-making or is consensus required? Is risk dynamic, volatile, or relatively stable?

  • What are the risks to effective risk management? What could go wrong and what needs to go right for there to be reasonable assurance that the right levels of the right risks are taken? (“Right” means what is desired and possibly approved by the executive management team and the board.)

  • What controls are in place to address these risks?

  • Is the design adequate? If the controls are operating consistently as designed, is there reasonable assurance that risk will be managed at desired levels?

  • Perform controls testing to obtain assurance that they are operating effectively as designed.

  • Assess the results of your work. Where is risk management on the maturity curve? What can and should be done to improve it at an appropriate cost? Recognize that one of the costs may be slowing down decision-making and losing operational opportunities.

  • Communicate the results and your insights.

Let me add to that now.

Why not have a series of discussions with decision-makers? Include all the top executives, but also include a good number at varying levels of management across the organization.

Consider questions like these that ask the opinions of the executives, the ones running the organization:

  • Do you (the executive) believe that risk management (which could mean a function or a set of policies and procedures) helps you be successful? Does it increase the likelihood of achieving your and the organization’s objectives?
  • Does it (risk management) help you make better decisions?
  • Does it meet the needs of the organization?
  • Does everybody use/practice risk management as well as they should?
  • Where could improvements be made?
  • Do top management and the board receive the information they need, when they need it?
  • Do the filings with the regulators sufficiently explain how the organization addresses risk?
  • Should a greater or lesser investment be made in risk management?
  • Does risk management give you a competitive advantage?
  • What would you change?

 

I welcome your thoughts.

Advertisements
  1. January 26, 2018 at 1:10 PM

    As many of you know I am helping to lead a global network of real-time dynamic Risk Loss Threat (RLT) industry benchmarking data where all our Members and Partners understand the strategic shareholder value of assessing all KPQ’s, KPI’s, KRI’s and KCI’s on an industry-by-industry basis. In essence, the meaningfulness of risk data comes from working with industry peers to benchmark their performance measurements from many perspectives. Then they can determine just how well they are performing vs peers.

    Companies can get started by…

    1.) Grouping their risk data into meaningful categories KPI’s etc.

    2.) Rolling their data categories up with a risk aggregation methodology so that the aggregated info is more meaningful to the Board of Directors and C-suite exec’s..

    3.) Using Peer Average and Best-In-Class measurement information to assess Risk Loss Threat data.

    By comparing your own company’s performance measurements against industry peers (anonymously) our Member firms cut their risk management costs and drive Continuous Process Improvement (CPI).

    Regards,
    Phil Wilson

    GRC Sphere

    • MIchael Corcoran
      January 26, 2018 at 5:53 PM

      Philip, With all due respect, what the heck are you saying? Don’t baffle us with trademarks and jargon. If this is where the financial services community is going to it is same old s…. Can we talk simply, otherwise I for one will not listen and therefore not hear what you are trying to achieve. Does anybody understand this?

  2. GSosbee
    January 26, 2018 at 1:18 PM

    Norman’s questions are fair and would provide any objective auditor with the information they are seeking. However, in all of my years of dealing with both internal and external auditors, only two external and one internal auditors have been anywhere close to objective. Thus, the unfortunate need for the two accepted standards one of which is nothing more than a check the box effort.

    I would like to see Norman’s questions used as the basis of any risk management benefit audit (the real audit – not the internal policies, procedures and controls audit), and the audit conducted by a qualified real world experienced risk management professional. As Norman points out – it is a discussion.

    • January 26, 2018 at 4:34 PM

      Yes, you are right on, G Sosbee. Norms inputs are categorized a Key Performance Questions (KPQ’s) to which we associate the appropriate KPI’s, KRI’s and KCI’s.

      • Norman Marks
        January 27, 2018 at 7:17 AM

        I suggested that the assessor/auditor have a discussion. These questions can frame the discussion, but hearing whether the executives think the management of risk is effective and why is so much better than simply examining documents.

  3. MIchael Corcoran
    January 26, 2018 at 2:26 PM

    Norman, way to many thoughts for any human being to digest lest respond. IMHO, Communications need to be concise and focused. What do you want the reader to feel emotion about?

    Mike

    • Norman Marks
      January 26, 2018 at 2:29 PM

      Mike, I want the reader to think about what constitutes effective risk management and how to know whether it is present or not.

      Assessing risk management is not an easy task, but it is essential.

      • MIchael Corcoran
        January 26, 2018 at 4:40 PM

        Where do stand on assessing value management?

        • Norman Marks
          January 27, 2018 at 7:15 AM

          Excellent question – it should all be the same, effective management. Consider what might happen, evaluate the situation, then act

          • Mike Corcoran
            January 27, 2018 at 2:04 PM

            That focused and concise, thanks!

  4. Wouter Schram
    January 29, 2018 at 1:08 AM

    Thanks to all for a useful discussion. I fully agree with the objective “think about what constitutes effective risk management and how to know whether it is present or not”. However, I am struggeling. Risk management is not a separate activity, it is fully integrated in the management and running of an organization. A complete insight in risk management is for me an very big subject to tackle. I look at effectiveness of risk management (and control) in nearly every audit I perform, but how do you build the big picture from that.

    • January 29, 2018 at 5:08 AM

      Wouter, I agree that we rate risk-related excellence on process performance and practice performance.

  5. Graeme Alexander
    January 29, 2018 at 1:33 PM

    Douglas Hubbard has written on this topic though with a different slant; rather than assess effectiveness he suggests ‘measuring’ the effectiveness. OK may be that’s a bit pedantic of me but often assessment (like risk management) is very subjective and all in the eye of the beholder. And perhaps this (measurement) is ‘the’ question to ask as everyone can believe that it is helping to make better decisions, the information is going to the right people etc and yet there may still be no measurable improvements.

    Some of Hubbard’s suggestions include use of statistics; direct evidence (ie that the technique identified something that would otherwise not have been found); test the individual components of the approach etc.

    • Norman Marks
      January 29, 2018 at 2:48 PM

      Thanks for sharing this. I don’t see any difference in using the word “measuring”. I also believe that looking for direct evidence will probably not work well when we are talking about the quality of decision-making

  1. January 26, 2018 at 12:23 PM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: