Home > Risk > Is the COSO ERM Update a success or failure?

Is the COSO ERM Update a success or failure?

September 9, 2017 Leave a comment Go to comments

A few days ago, COSO published an update to their 2004 ERM Framework. The product, retitled Enterprise Risk Management: Integrating with Strategy and Performance, is available from the AICPA or IIA – see here for the links.

This is their news release, dated September 6. It asserts that:

“The updated edition is designed to help organizations create, preserve, and realize value while improving their approach to managing risk.”

Has it achieved that goal? Or has it failed?

Will it advance practices or has it fallen short of leading thinking?

I am in the process of a careful review of the product and will share the results later.

But I encourage all of you to not only review it but answer my question (is it a success or failure) using a set of questions I shared in June 2016 on this site – upgraded with a few clarifications and couple of additions (at the end).

Even if you don’t provide your own assessment (for whatever reason), consider subscribing or returning to see how others have commented on the product.

My ask is that you assess the updated Framework by rating each of these 14 questions on a scale of 1-10 (10 being perfect). When you rate, consider whether the COSO discussion provides practical guidance or just makes a theoretical point. Will the guidance help organizations actually achieve the principle or point being made?

Then provide your overall pass/fail.

Here are the assessment questions.

  1. Does the draft provide useful guidance that will help leaders of the organization define the mission, objectives, strategies, and plans that will deliver optimal value to stakeholders?
    • If the mission is not optimal, it is unlikely that the objectives will be
    • If the objectives are not optimal, it is unlikely that strategies to achieve them will be
    • …and so on
    • In order to set the optimal mission, objectives, strategies, and plans, leaders need to consider all the possibilities. They need to be able to obtain as clear a view as possible of potential opportunities and harms for all potential options. Their assessment of what might lie ahead, and how it might affect their journey, needs to be performed in a structured fashion – both opportunities and harms – and a reasonable judgment made that takes all of the potential effects of uncertainty into account
    • It is not sufficient to say that you have considered all the options (possibilities) for mission, objectives, strategies, and plans. The processes where those are selected have to involve the right people, consider all the available useful information (which is reliable, timely, and up-to-date), and more – in other words, the risk of setting a wrong or sub-optimal mission, objective, or strategy, has to be at acceptable levels.
    • Organizations need to periodically review their mission and change it as conditions change. Think of Intel, Microsoft, HP, Apple and more
  2. Does the draft provide useful guidance when it comes to executing against the defined mission, objectives, strategies, and plans? Is there sufficient guidance on effective decision-making, and will it move the practice of risk management away from only reviewing, periodically, a list of risks? Will it lead to organizations practicing risk management continuously?
    • The Executive Summary makes the points that risk management must be continuous, enable effective decision-making, and be more than the review of a list of risks
    • But, does the detail of the framework deliver on those promises?
    • As COSO says in their Executive Summary, execution and the optimization of performance rely on decisions that are made not only by leaders in establishing the goals and objectives of the organization, but by managers at every level of the organization every day
    • In order to make good decisions, people need to consider all the potential consequences of the choices they make. Those include not only the harms but also the rewards that may occur. The consideration needs to be structured and based on useful, timely, current, and reliable information
    • Also as COSO says, risk management needs to be an essential part of running the organization and delivering performance. It should not be separate. Does the guidance enable organizations to manage risk as part of the rhythm of the business? Does it help management entwine the consideration of risk into every business process?
  3. Will the guidance still lead people to only identify, assess, and address potential harms? Will risk reporting still be focused on the level of risk rather than the likelihood of achieving each objective?
    • COSO says the consideration of both harms and rewards (in their language, ‘risks’ and ‘opportunities’) is essential if risk management is to be effective
    • While that is essentially what the prior version said, its language focused almost entirely on ‘risk’ and arguably this has led to most organizations only managing potential harms
    • Most organizations limit risk reporting to a list of risks and their level. But if it’s really about achieving objectives, shouldn’t reporting be about whether each objective is likely to be achieved, exceeded, or missed? It should not be limited to an assessment of potential harms
  4. Does the guidance explain clearly and help decision-makers understand and then evaluate all the potential effects of uncertainty?
    • Some look at ‘opportunity’ as the positive side and ‘risk’ as the negative. But, most situations and certainly most decisions have multiple potential consequences. It’s not just reward or just harm, usually it’s both. For example, when you decide to overtake another car on the freeway, there is potential to go faster as well as the potential for a crash. Only by understanding and then weighing all the potential consequences can a good decision be made. As another example, when you purchase a hotel while playing Monopoly, you create the opportunity to obtain rent (and this requires considering the size of that gain and its likelihood) as well as increase the potential to go bankrupt if you land on another’s property and have to pay rent
    • Some assess the ‘level’ of risk as a point – a level of impact and the likelihood of that impact. However, there is almost always a range of potential impacts, each with its separate likelihood. For example, if the organization decides to reduce the price of its products, sales could (a) increase by 10%; (b) increase by 20%; (c) remain the same; (d) change by another percentage. All of these possibilities have different likelihoods. If you wanted to plot the ‘level of risk’, it would be a range or a curve on the chart and not a point
    • The actions and decisions of one affect many. Is the guidance sufficient on this point?
    • Many define the level of risk based on the amount of impact multiplied by its likelihood. But then a 5% likelihood of a $200 loss is the same as a 50% likelihood of a $20 loss. One may be acceptable but the other not. Does COSO discourage the assessment of risk based on this simplistic calculation?
  5. Will the update provide decision-makers with the structure/process they need to decide whether to ‘take the risk’ because of the potential for reward?
    • In real life, people have to ‘balance’ risk and reward
    • Will the guidance provide a disciplined process for identifying and evaluating all the potential effects of each option and only then making an informed decision? Or does it only consider and provide guidance on assessing harms?
    • For example, if the potential for loss is assessed as between $50 (20% likelihood) and $100 (5% likelihood), should a manager ‘take the risk’ when the potential for gain is between $50 (20%) and $250 (5%)?
  6. Will the update lead to providing decision-makers with the guidance they need if they are to make the decisions management and the board want them to make?
    • The great majority of organizations who have a ‘risk appetite statement’ at the entity level have not been able to cascade it down in a way that enables those making the decisions in real life to know what is necessary
    • Different conditions (e.g., whether there is huge public scrutiny, whether the organization is likely to exceed or miss its earnings targets) can lead to executives wanting to change the risk decisions that are made
    • It’s one thing to say that you need to avoid exceeding defined risk limits, but when the reward is high it may be appropriate to take that risk. Does the guidance enable agile decision-making that considers changes in the environment?
  7. Does the update provide sufficient guidance on how to assess and then correct, as necessary, the culture of the organization?
    • It is encouraging that this is now included. Is it sufficient?
  8. Does the update provide sufficient guidance on each stage of the risk management process, including identifying, assessing, evaluating, and treating risk and opportunity? Does it provide sufficient guidance on communications and monitoring, including continuous improvement?
    • There is more to assessing risk (good and bad) than impact and likelihood. Other considerations include duration, speed of onset, and more
    • Many use models. Is this covered sufficiently?
  9. Is the updated COSO guidance on risk appetite and risk tolerance useful? Does it mirror and enable effective decision-making in real life? Does the guidance help to establish not only the upper limit of ‘risk’ that should be taken, but the lower level as well?
    • If organizations don’t ‘take risk’ they will not survive. It is dangerous to be too risk averse
    • How does an organization establish the minimum level as well as the maximum?
    • Does COSO provide sufficient guidance on how to assess both the upside and the downside?
    • Does the updated guidance help people ‘balance’ risk and reward, knowing when to ‘take the risk’? Or does it lead people to evaluate whether the level of harm is acceptable without considering the level of benefit? Does COSO guide people to consider the potential effect on strategies and objectives, or only to assess risk based on some out-of-context measure?
    • The COSO definition of risk appetite in the current framework talks about an amount of risk. Sometimes risk appetite is expressed in terms like “we have no tolerance for this risk”
    • However, in real life people make decisions based not only on the ‘amount’ of risk (harm) but the likelihood of that amount of risk. For example, I might accept a 2% possibility of losing $100 but not a 20% possibility
    • A generic statement like “we have no tolerance for this risk” does not help real life decision-making. While no organization will state a level at which loss of life is acceptable, in many industries the only way to get to zero likelihood is to exit the business
    • What is an acceptable level of variation from objectives? If you set an objective of 10% growth but are willing to accept 5% growth, 5% is your true objective. Alternatively, your objective may remain 10% but you will accept a 7% chance that it will be reduced to 5%
    • Is the ISO 31000:2009 term ‘risk criteria’ better, especially as it can be applied to individual decisions?
  10. Will it be possible to assess the effectiveness of risk management in practice using the updated version?
    • Any assessment should be based on whether the management of risk helps people establish the optimal vision, objectives, strategies, and plans, make better decisions and, as a result, increase the likelihood of achieving objectives
    • Any assessment should identify the areas where the risk of failure in identifying, assessing, evaluating, or taking action to address risk is higher than desired
    • If the assessment is against principles, are those in the COSO draft as good or better than those in ISO 31000:2009?
    • Is all the COSO principles are present and functioning, does that mean that risk management is effective? If one or more are not present, does that mean that risk management is without doubt ineffective?
  11. Will the guidance provide sufficient guidance to enable the board and/or a committee of the board to provide effective oversight?
    • Is the guidance as good as that in South Africa’s King IV Exposure Draft?
  12. Is the updated document consumable? Is it too long? Will it be read, understood, and acted on by all levels of the organization?
  13. Will the updated product help the busy executive or board member understand what risk management is all about, that it is not simply a compliance exercise but can improve the likelihood of quality decisions and the achievement of the right objectives?
  14. Is the 2017 product a sharp improvement on the 2004 version?
    • Are the changes and additions an improvement?
    • Does the updated Framework represent leading thinking?
    • Will it help move practices around the world to greater levels of maturity and effectiveness?
    • Is it better than the ISO 31000:2009 global risk management standard and other guidance that has been provided by regulators, national corporate governance codes, and so on?
    • Would you recommend an executive, board member, or practitioner buying the updated Framework? Or, should they buy my book?
Advertisements
  1. September 9, 2017 at 11:55 AM

    Very good idea. I will respond when finished reading

  2. Farquema A. Cokie
    September 10, 2017 at 6:41 AM

    Great, I admired your analysis and/or questions regarding the 2004 and updated products.

  3. September 10, 2017 at 9:04 AM

    Thank you Norman. I am looking forward to your posts on the new ERM framwork. I have obtained the online version. From my viewpoint, I am going to try to work toward using the new framework with a view toward how a board or board committee including an audit committee should or might use the framework to satisfy or help satisfy duties. So I will also be looking at duty legal requirements, and I’m basically taking the framework as it is and trying to work with it. I’m hoping to put together some presentation materials and working and interacting with others on this. I know that you won’t pull any punches in your blog posts. Thank you for your knowledgeable and experiened input on this important topic. David Tate

  4. September 11, 2017 at 1:58 AM

    Hi Norman, thanks for this. I will go through your questions and come back to you. I’ve also written a post on this, essentially linking to the COSO update (in http://riskmanagementguru.com/coso-updated-enterprise-risk-management-framework.html/).
    Thanks again, regards,
    Antonio

  5. September 11, 2017 at 2:08 AM

    Why is the full COSO 2017 so expensive? That may limit the magnitude of followers.

  6. September 11, 2017 at 7:08 AM

    I’ve talked to several ERM experts and seem to be hearing with consistency that it is miuc improved. Of course, I’m a bit biased as I sit on RIMS’ Standards and Practices Council and we spent quite a bit of time reviewing and giving them hundreds of comments and suggested revisions. We meet this week and will no doubt discuss the group view. Anyway, I’m cautiously optimistic especially since COSO remains controlled by accountants and auditors. I suppose RIMS is supposed to be happy that they allowed Carol Fox to be an “observer”. Frankly, as a former RIMS president and long term practitioner, this remains an insult to me.

  7. Norman Marks
    September 11, 2017 at 10:16 AM

    This is how Hans Laessoe assessed the update:

    Overall, the current COSO is far better than the former (auditor focused control) framework. There is, in my view, still room for improvement, and I do not believe they have captured even present day best practices. Hence, issuing this as the “latest and greatest” will hamper rather than help promoting known best practice in intelligent risk taking.

    1 Does the draft provide useful guidance that will help leaders of the organization define the mission, objectives, strategies, and plans that will deliver optimal value to stakeholders?

    Rating: 8 I believe, as well as can be expected of/delivered by a general “tool” or framework. I would never expect the framework should deliver a process to ensure a board defines the optimal mission/vision/strategies

    It does provide guidance as to test the strategy.
    ====================================================================

    2 Does the draft provide useful guidance when it comes to executing against the defined mission, objectives, strategies, and plans? Is there sufficient guidance on effective decision-making, and will it move the practice of risk management away from only reviewing, periodically, a list of risks? Will it lead to organizations practicing risk management continuously?

    Rating: 3 This is a weak spot in my view. The framework focuses too much of “decision have to be good”, but leave it to the individual management to ensure that, and then provide reactive risk responses to safeguard whatever decisions are made.

    It does not adequately drive/encourage that risk considerations are explicitly embedded in the decision process, but rather simply assumes they are
    ====================================================================

    3 Will the guidance still lead people to only identify, assess, and address potential harms? Will risk reporting still be focused on the level of risk rather than the likelihood of achieving each objective?

    Rating: 3 Despite the foreword’s focus on performance vs objectives, the focus tend to be analysing the individual risk and its likelihood.

    ISO addresses “objectives” and is adamant about the multitude of these. The COSO standard should advocate and guide on best practices on multiple impact parameters (e.g. profit, liquidity, safety, reputation, …)

    Furthermore, the framework does not speak of consolidation (as I have seen it), which means that risks are still only considered individually.
    ====================================================================

    4 Does the guidance explain clearly and help decision-makers understand and then evaluate all the potential effects of uncertainty?

    Rating: 5 It is better than the past, but it appears the framework still looks at two issues in sequence
    – Likelihood of risk materializing
    – Impact, if it happens
    To me, this is an ineffective way of assessing risks as any one of them may have a range of consequences. The view of risks is (still) too simple

    Consider the risk of 5-year old Jack falling on a bicycle.
    – Will it happen, yes almost certainly
    – The impact can be anything from a scratch to a fatality – the likelihood depends on the impact
    Instead one should look at this, and start with the impact
    – Can it kill him – by itself, not really, but if he drives on a road, he may get run over, once fallen (so we do not allow him to ride on the road)
    – Can he get a concussion – yes, it’s not likely, but (today) it is unbearable, so we ask him to wear a helmet
    – Can he break a leg/an arm – yes, it is not very likely, and we will deal with that if it happens. (Using ISO 31000 terminology) it is below our risk tolerance
    – Will he get bruised – yes, very likely, and we will deal with that when it happens
    The risk is more complex that “will he fall, and what is the consequence”. So are most other risks
    ====================================================================

    5 Will the update provide decision-makers with the structure/process they need to decide whether to ‘take the risk’ because of the potential for reward?

    Rating: 8 Assuming the framework is followed and the risk appetite/tolerance is defined in relevant measures – yes, I do believe it provide the base for enabling such a base. The framework does not provide this by itself, yet – it needs to be tailored
    ====================================================================

    6 Will the update lead to providing decision-makers with the guidance they need if they are to make the decisions management and the board want them to make?

    Rating: 8 If used correctly, I believe it does work. If superseded by greed, nothing helps
    ====================================================================

    7 Does the update provide sufficient guidance on how to assess and then correct, as necessary, the culture of the organization?

    Rating: 6 It’s a good step forward – but culture is hard to measure/quantify and embed. I still think there is room for improvement
    ====================================================================

    8 Does the update provide sufficient guidance on each stage of the risk management process, including identifying, assessing, evaluating, and treating risk and opportunity? Does it provide sufficient guidance on communications and monitoring, including continuous improvement?

    Rating: 9 As well as can be done, and there are plenty of parallel approaches to apply as well.

    For a perfect 10, I would need to add Monte Carlo simulation based consolidation to avoid focus is one single risks – and enable a portfolio approach
    ====================================================================

    9 Is the updated COSO guidance on risk appetite and risk tolerance useful? Does it mirror and enable effective decision-making in real life? Does the guidance help to establish not only the upper limit of ‘risk’ that should be taken, but the lower level as well?

    Rating: 7 This could be a lot stronger. ISO 31000 has something in the vocabulary (and for some odd reason, not on the standard). My reading of the ISO vocabulary indicates a useful level of risk appetite and another f risk tolerance.
    – Risk appetite is the level of exposure (ISO: risk profile) which you are willing to take, i.e. which you would not bother to mitigate any further. Treating risks below the risk appetite is a decided “waste” of resoueces
    – Risk tolerance is the level of exposure you are prepared (but not necessarily happy) to take in pursuit of your objectives. Exposure beyond the risk tolerance MUST be treated to lower levels.
    Hence, if your current exposure is somewhere between the “appetite” and the “tolerance” you are in your defined balance. You may still chose to mitigate further, but this should be based on cost/benefit considerations to be effective.

    COSO could be a lot clearer on this, also to help themselves (i.e. PwC and the auditor business) on validation
    ====================================================================

    10 Will it be possible to assess the effectiveness of risk management in practice using the updated version?

    Rating: 3 Largely – NO.

    There is no guidance as to deriving consolidation of a risk portfolio (the risk profile) and hence management cannot know whether or not they are (overall) taking on too much risk or not risk enough.

    A consolidate risk profile can give an indication as to the effectiveness by showing that the exposure level is between the appetite and tolerance levels.

    However, remember, there is no such thing as proof in this business of probabilities.
    ====================================================================

    11 Will the guidance provide sufficient guidance to enable the board and/or a committee of the board to provide effective oversight?

    Rating: 9 If applied correctly, I believe it does. I do not know the RSA exposure draft.
    ====================================================================

    12 Is the updated document consumable? Is it too long? Will it be read, understood, and acted on by all levels of the organization?

    Rating: 6 It is VERY long, and I believe it could have been more effectively made as a “risk mangers technical handbook”. It is not for executives to read – nor is it (I hope) intended to be so.
    ====================================================================

    13 Will the updated product help the busy executive or board member understand what risk management is all about, that it is not simply a compliance exercise but can improve the likelihood of quality decisions and the achievement of the right objectives?

    Rating: 3 No – I fail to see even the attempt to deliver on this
    ====================================================================

    14 Is the 2017 product a sharp improvement on the 2004 version?

    Rating: 8 The 2004 edition was focused on what can be audited and is a control (governance) framework rather than a risk management framework. I believe, this 2017 edition is a lot better.

    I also believe it is better than the ISO 31000:2009. We have yet to see the update of that (long overdue).

    I would like to see the two – and others – to merge into a “best practice” framework which can be used coherently amongst regulators and legislators around. Today – a multinational will have some parts audited based on ISO others on COSO. The fact that their vocabularies are not aligned is disturbing.

    I would recommend a risk manager to buy the standard – as background knowledge in discussion with others. I am afraid, I do not know your book.

  1. September 22, 2017 at 7:48 AM
  2. September 29, 2017 at 8:51 AM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: