Home > Risk > How do you evaluate your risk management program?

How do you evaluate your risk management program?

The IIA and others are developing guidance on how to assess risk management. For me, there is too much emphasis on the risk management framework and process (the policy, the procedures followed to identify, assess, and respond to risks) and too little on the risk culture.

Risk culture refers to whether the organization embraces and uses risk management as part of strategy-setting and decision-making at all levels of the business. We have seen a number of companies fail or have severe difficulties over the last couple of years because they had good processes but risk culture problems – such as risk managers going unheeded and even being fired for their bad news, failures to communicate risk levels to senior management and the board, and a short-term drive for profits and bonuses trumping consideration of risk.

I developed my own list of high level questions that can be used to assess the risk management process:

1)      Does the organization have an effective risk culture? Attributes would include:

  • Support for and use of risk management at board and executive management levels
  • Devolvement of risk management into decision-making at all levels across the organization
  • A clearly stated risk appetite that has been communicated effectively to all involved in risk management
  • A risk policy and stated risk tolerances that enable a balanced approach to risk management (between risk and reward) and are communicated to all involved in risk management
  • The ability of the executive responsible for risk management to access and influence executives and the board, ensuring that they receive complete, accurate, and timely risk information
  • A common set of risk definitions and processes, enabling a view of risk across the enterprise
  • Sufficient resources for the risk function, including numbers of experienced and trained personnel, budget, and other resources necessary to the task
  • Appropriate training for operational and other managers involved in risk management and daily decision-making
  • Those involved in risk management and decision-making have sufficient information to perform their responsibilities
  • Risk is considered in setting organizational strategies and operating plans. The latter are adjusted as risks change
  • When managing performance, projected results are modified as necessary to reflect risks

2)      Are there adequate processes to ensure risks are promptly and appropriately identified and analyzed? Are appropriate individuals involved in the process? Consider the need for continuous monitoring of risks

3)      Are the processes for evaluating and assessing risks, determining whether they are above tolerances, and selecting risk treatments adequate? Are appropriate individuals involved in the process? Are options for treatment considered and appropriately evaluated?

4)      Are the controls relied upon to manage the more significant risks identified, adequately designed, and operating effectively so that risks are managed within tolerances?

5)      Is risk oversight by the executive leadership and board (including assigned board committees) sufficient? Do they receive appropriate timely information?

6)      Are corrective actions (risk treatments) managed to timely completion?

7)      Is there appropriate monitoring of the risk management process to ensure it continues to function as intended. Are risk policy, risk appetite, risk tolerances, and other parameters and standards current? Do they reflect the current state of the business, its internal and external context, and the risk management needs of the organization?

8)      Is the risk management efficient? Are the results of multiple risk assessment processes and systems efficiently combined to provide consistent reporting across the enterprise?

I would appreciate your comments.

  1. Daniella Penner
    May 3, 2010 at 8:19 PM

    What role does indpendence play in ensuring risk management is able to identify critical issues? I don’t believe I see this addressed in your list of questions.

  2. May 4, 2010 at 4:49 AM

    Excellent post Norman.

    The only observation I offer is the segregation of the risk management function. Is the risk manager compromised by his involvement in a business initiative, process, transaction or function?

    Nick Leeson and the Barings Bank scenario comes to mind. Segregation of duties is also a major risk factor in money laundering.

    Thanks for your insights

    jbm

  3. May 4, 2010 at 11:47 AM

    Norman,

    My thoughts, risk culture and governance are huge—at least (or equally) as important as the nuts and bolts of identifying, assessing, designing, monitoring, responding and follow up. Without appropriate and effective risk culture and governance, I don’t see how you can effective nuts and bolts. I recommend the formulation of fairly definite criteria or questions that people are looking for with respect to evaluating risk culture and governance, and that those criteria/questions be made public. What is the harm of giving everyone more detail about how risk culture and governance will be evaluated? The goal is to raise the level, or at least to set a floor. Help people do that by being more specific as to expectations.

    David Tate, Esq.

  4. May 4, 2010 at 3:37 PM

    Norman,
    fully agree with your comments; I have a suggestion to add to your 10 points:
    Executive management and board need to give the risk function “a place at the table”; in many companies revenue generation is still seen as the most relevant factor for the compensation plans. A good risk culture integrates “risk think” into the compensation plan.
    While I believe that businesses make progress in considering the importance of good risk management we can see again the increased emphasis in revenue generation.

  5. Larry Brown
    May 4, 2010 at 6:42 PM

    Norman – Thanks for your continuing thought leadership. In this instance, your argument, while obviously well intended, is a bit disingenuous. The list of items you present reads much like the attributes of a framework and process; an executive summary of the COSO ERM framework executive summary, perhaps.

    We’ve all heard the analogy of good enterprise risk management being good enterprise management. I think the challenge for those that don’t spend most all their time in the risk and control space (you are preaching to the choir on this list) is designing a risk management activity that leverages the talent of the enterprise managers and affirms their collective responsibility for managing the risks of the enterprise.

    I was at an audit committee roundtable a while back, and a CFO on the panel from a Fortune 500 financial services company described the CRO role not as Chief Risk Officer, but as Chief Risk Organizer, his point being that you need all of the senior officers at the table with the collective knowledge of the organization, external environment, etc. to own the risks (no one person should be tasked as Chief Risk Officer given the pervasiveness of the “task”). The Chief Risk Organizer, much like the grounds crew that puts the stripes down on the field, cuts the grass and smooths the infield, is there in support of the game (the business), but is certainly not the game, or only a small part thereof – the game (enterprise (risk) management) is to be played by the enterprise managers.

    The challenge with the risk management “game” as we know it today is that there are way too many ways to keep score – quantitative, qualitative, both – Top Ten risks list, all x thousand risks list, SOx risks in, SOx risks out, use COSO ERM categories to guide the analysis, home grown risk universe categories, Basel criteria, differing regulatory requirements across industries, ISO, ITIL, COBIT, NIST, et al on the IT side – the list goes on and on.

    So is it any wonder that there’s more than a little bit of “churn” when the risk management topic is discussed, let alone the process of assessing risk management, to your original question?

    Keep up the good work.

    Larry

    I believe you’re onto something with the risk culture theme, but I’d simplify it to the corporate culture – the COSO IC Framework “Tone at the Top” – if you will.

  6. David Doney
    May 4, 2010 at 10:16 PM

    Hello Norman: A very good list. Excellent comments from others to help you as well. I think you should include incentives specifically. Incentives should be discounted based on the risk taken. The most sophisticated risk systems in the world were overridden because executives got huge payouts regardless of the risk taken.

    So to your list I would add:

    1) Are incentive payouts for CEO and CEO direct-reports discounted based on the level of risk taken?

    In the case of the investment banks, their leverage increased from around 20:1 to over 30:1 from 2003-2007. This should have resulted in huge discounts to their payouts, due to the massive increase in firm-wide risk.

  7. nmarks
    May 5, 2010 at 6:16 PM

    Thanks all for the comments. I 100% agree that the voice of risk has to be independent, clear, and heard.

    I like the idea of the Chief Risk Organizer. Much of the concept is covered in the first two bullets in the Risk Culture point. But Larry, you state it better.

    Some have commented that my list is redundant, because BSI and ISO’s frameworks already cover the territory. I disagree. I think there are some points there about attributes and values, but not enough to build an assessment around. ISO is quite weak on the risk culture side.

    With respect to more detail, if we can agree on the questions then more detailed guidance may be developed to help the assessor answer each question. But, I would prefer to keep to principles as each organization should be able to develop framework, organization, systems, and processes that work for it. A trained and experienced assessor should be able to work with general questions – and I hate checklists!

  8. Manuel
    May 6, 2010 at 3:50 AM

    Hi Norman,

    Before reading the entire article, I tried to note down my ideas just having read your headline. The first point that came to my mind is partly covered in your second point. What is really missing is reflecting risk management effectiveness. Nobody really tracks the effectiveness of (historical) risk management measures. There is no “official” learning, experience is not professionally reused.

    Furthermore risk culture differs depending on the kind of company you are looking at. Project-focused companies ought to have a completely different culture compare to financial companies.
    Similar to David, I think that risk appetite should be integrated only carefully. Its a company’s competitive advantage to take risks. I think e.g. in a bigger bank, investment banker should have different risk appetite compare to IT staff 😉

    It would be really interesting to break these strategic guidlines down. But to what? I guess neither checklists nor sotware would suit these demands.

  9. Larry Siminski
    May 6, 2010 at 8:57 AM

    You implied that no one knew and understood these derivatives and the financial pitfalls, and there was no regulation.

    One person did, Brooksley Born, and she was the head of a US government agency that could do something about it, and she attempted to control and regulate it. Six powerful males, in the Government hounded her out of office.

    To say it was simply greed, is to simply focus the blame on a few individuals, away from ourselves. The 10 Trillion dollar mistake was 6 billion people not making enough effort to support someone who was doing the right thing. We all are as much to blame for being sheep as the wolves are.

    http://www.pbs.org/wgbh/pages/frontline/warning/view/?utm_campaign=viewpage&utm_medium=grid&utm_source=grid

  1. April 13, 2014 at 10:27 AM
  2. April 13, 2014 at 8:48 PM
  3. August 24, 2014 at 1:17 PM
  4. August 25, 2014 at 12:11 AM
  5. August 25, 2014 at 2:57 AM
  6. September 3, 2014 at 5:59 PM
  7. September 4, 2014 at 8:07 PM
  8. December 28, 2020 at 10:38 AM
  9. December 28, 2020 at 11:18 AM

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.