Should you adopt the updated COSO ERM Framework? My assessment

I have been working on this for a while. I wanted to be fair to COSO and PwC; several of my friends were involved in the update project on the COSO Board and as advisers. I respect them all.

To perform a detailed assessment, I used the 12 questions I developed to assess the exposure draft with two additional questions at the end. Each is scored on a scale of 1-10, where 10 is best.

But first I need to step back and address whether my wishes and expectations for the update were the same as COSO’s. Then I can give my overall recommendation and then the detailed assessment.

I don’t think they were the same.

I started from the point of view that risk management today is far too often ineffective and needs a catalyst to spark a change.

As I said in my comment letter to COSO on the exposure draft, “surveys, notably by Deloitte[1], have found a huge disconnect between those leading risk management and the executives and directors who should be obtaining value from it. Only a small percentage said that risk management had made a significant contribution to their setting and execution of strategies.”

According to the surveys, executives see risk management as a compliance exercise. It is not seen as essential to running the business day-to-day – when in fact it can and should be critical to success, not just avoiding failure. In my comment letter, I set out a number of reasons.

Its 13 years since the original COSO ERM Framework, 8 years since ISO 31000:2009 was published. Who knows when the next COSO update will be, and the news from the teams working on the ISO update is discouraging.

This was an opportunity for COSO to “leap forward” and “transform how organizations are run, from the setting of the mission, objectives, strategies and plans to the daily operation of the business: how it performs in practice through intelligent and informed decision-making at levels of the extended enterprise”.

Enhancements to the COSO framework would be ‘nice’, but when risk management in practice is failing to be seen as vital to success, ‘nice’ is insufficient.

A leap change rather than incremental change is necessary – and something needs to be a catalyst for that change.

I hoped that would be the COSO ERM update.

It is not.

It is my opinion that COSO and PwC did not seek to incorporate leading thinking and practices. I made sure they had a copy of World-Class Risk Management and was assured that they had read it. I also suggested (in the comment letter and in calls with leaders of the update project) that they involve thought leaders.

They appear to be satisfied with modest improvements, incremental changes that, in my opinion, will not change practices to any great extent. This is their news release.

Leading risk management practitioners are already ahead of what COSO ERM 2017 suggests.

Yes, they have made progress:

• Eliminated the cube
• Stressed the need to consider risk (what might happen) when selecting strategies
• Mentioned (without detail) the needs to enable decision-making and to address bias (more normally called cognitive bias) and culture
• Said that risk management is more than the periodic review of a list of risks

They have also introduced diagrams that purport to show the relationships between strategy, performance, risk, risk capacity, and risk tolerance. Sorry, but I don’t think the diagrams are more than sound in theory. I doubt they work in practice and I question whether they are even theoretically sound as they suggest that you can aggregate all forms of risk and risk capacity, which you cannot in the real world.

So, bottom line, where am I?

The update is an interesting contribution to the world of risk management guidance. But…

• It is insufficient to describe or support the effective management of risk.
• It is also insufficient as a basis for the assessment of risk management.

However, it is worth buying, reading, and considering – along with ISO 31000:2009, my book (which is way ahead of COSO I am afraid), and other guidance.

Now for the 14 questions and my detailed assessment:

 

Question	Assessment
1.      Does the update provide useful guidance that will help leaders of the organization define the mission, objectives, strategies, and plans that will deliver optimal value to stakeholders? ·        If the mission is not optimal, it is unlikely that the objectives will be ·        If the objectives are not optimal, it is unlikely that strategies to achieve them will be ·        …and so on ·        In order to set the optimal mission, objectives, strategies, and plans, leaders need to consider all the possibilities. They need to be able to obtain as clear a view as possible of potential opportunities and harms for all potential options. Their assessment of what might lie ahead, and how it might affect their journey, needs to be performed in a structured fashion – both opportunities and harms – and a reasonable judgment made that takes all of the potential effects of uncertainty into account ·        It is not sufficient to say that you have considered all the options (possibilities) for mission, objectives, strategies, and plans. The processes where those are selected have to involve the right people, consider all the available useful information (which is reliable, timely, and up-to-date), and more – in other words, the risk of setting a wrong or sub-optimal mission, objective, or strategy, has to be at acceptable levels. ·        Organizations need to periodically review their mission and change it as conditions change. Think of Intel, Microsoft, HP, Apple and more	Rating: 7/10 COSO has made a significant improvement in its discussion of the need to embed the consideration of risk into both strategy-setting and execution. I particularly like the reference to scenarios and the need to consider what might happen under each strategic option. Principle 8 is The organization evaluates alternative strategies and potential impact on risk profile. It talks about evaluating strategies and making sure that they are aligned with the mission, vision, and values of the organization. However, there is no discussion of the possibility that the mission or vision is sub-optimal, or that (for reasons such as poor information or not involving the appropriate people) the strategy is not the best. Risks to setting strategy are important and this is a gap in the COSO update.  The update mentions assumptions, but not the possibility that the assumptions are incorrect. Mature organizations should understand that and assess the likelihood of an error that would be significant to the achievement of the strategy or objective; actions should be taken where necessary. I find the discussion of risk appetite, profile, and strategy somewhat confusing. Tt recognizes that some will set appetite before selecting strategy and others will do the reverse; this is a reasonable point to make. However, when discussing the setting or risk appetite and defining risk profile, it assumes strategies and objectives are defined. When selecting strategies, it assumes risk profiles and appetite are in place. I think this could have been written better and as a result I am unsure how people will be able to interpret and use the guidance. What I find lacking is any discussion of the need to assess the likelihood and extent of all potential consequences in a disciplined and systematic fashion. In other words, use similar methods when considering the benefits of a strategy as when assessing potential harms. Further, there is no discussion of the need to take all the potential effects, both good and bad, into account when selecting a strategy. On balance, do the potential benefits outweigh the potential harms? Instead, there is a focus on the list of harms (the risk profile) and risk appetite.
2.      Does the update provide useful guidance when it comes to executing against the defined mission, objectives, strategies, and plans? Is there sufficient guidance on effective decision-making, and will it move the practice of risk management away from only reviewing, periodically, a list of risks? Will it lead to organizations practicing risk management continuously? ·        The Executive Summary makes the points that risk management must be continuous, enable effective decision-making, and be more than the review of a list of risks ·        But, does the detail of the framework deliver on those promises? ·        As COSO says in their Executive Summary, execution and the optimization of performance rely on decisions that are made not only by leaders in establishing the goals and objectives of the organization, but by managers at every level of the organization every day ·        In order to make good decisions, people need to consider all the potential consequences of the choices they make. Those include not only the harms but also the rewards that may occur. The consideration needs to be structured and based on useful, timely, current, and reliable information ·        Also as COSO says, risk management needs to be an essential part of running the organization and delivering performance. It should not be separate. Does the guidance enable organizations to manage risk as part of the rhythm of the business? Does it help management entwine the consideration of risk into every business process?	Rating: 2/10 While the Executive Summary talks about decision-making, there is really no guidance on this. There are no principles and no practical guidance on how decisions should be made, considering all potential consequences. This is critical, as this is where risk is taken in the real world. The section on Performance is all about risks – potential harms. In real life, as distinct from the world of standards and frameworks, people at all levels across the extended enterprise are taking or not taking risk every day. They do this through decisions.  Every decision creates or modifies risk.  The key to the effective management of risk is having decision-makers take the desired amount of the right risk. This is simply not covered. It is simplistic to think that you take risk only as the result of a risk assessment activity. As a result, I have great concern as to whether the COSO update will influence risk-taking in practice, in the real world.
3.      Will the guidance still lead people to only identify, assess, and address potential harms? Will risk reporting still be focused on the level of risk rather than the likelihood of achieving each objective? ·        COSO says the consideration of both harms and rewards (in their language, ‘risks’ and ‘opportunities’) is essential if risk management is to be effective ·        While that is essentially what the prior version said, its language focused almost entirely on ‘risk’ and arguably this has led to most organizations only managing potential harms ·        Most organizations limit risk reporting to a list of risks and their level. But if it’s really about achieving objectives, shouldn’t reporting be about whether each objective is likely to be achieved, exceeded, or missed? It should not be limited to an assessment of potential harms	Rating: 2/10 COSO and I are on totally different pages. They see events or situations (or decisions) having either a positive (opportunity) or negative (risk) opportunity. In the real world, events or situations have not only multiple potential effects, but each is a range and not a point. The framework asks that people identify opportunities as well as risks, but not the combination of good and bad that is likely to follow from an event or situation. Even then, COSO insists on a risk profile (a list of potential harms) and assessing whether risks are within risk appetite, without any consideration of the positive that may accompany a negative. Further, there is only a suggestion to include the effect on objectives as one of the bases for prioritizing risk. If it is all about achieving objectives and fulfilling strategies, then the focus needs to be there and not on risk.  The management of risk needs to be far more than maintaining a risk profile. It has to be about taking the right level of the right risks with every business decision.
4.      Does the guidance explain clearly and help decision-makers understand and then evaluate all the potential effects of uncertainty? ·        Some look at ‘opportunity’ as the positive side and ‘risk’ as the negative. But, most situations and certainly most decisions have multiple potential consequences. It’s not just reward or just harm, usually it’s both. For example, when you decide to overtake another car on the freeway, there is potential to go faster as well as the potential for a crash. Only by understanding and then weighing all the potential consequences can a good decision be made. As another example, when you purchase a hotel while playing Monopoly, you create the opportunity to obtain rent (and this requires considering the size of that gain and its likelihood) as well as increase the potential to go bankrupt if you land on another’s property and have to pay rent ·        Some assess the ‘level’ of risk as a point – a level of impact and the likelihood of that impact. However, there is almost always a range of potential impacts, each with its separate likelihood. For example, if the organization decides to reduce the price of its products, sales could (a) increase by 10%; (b) increase by 20%; (c) remain the same; (d) change by another percentage. All of these possibilities have different likelihoods. If you wanted to plot the ‘level of risk’, it would be a range or a curve on the chart and not a point ·        The actions and decisions of one affect many. Is the guidance sufficient on this point? ·        Many define the level of risk based on the amount of impact multiplied by its likelihood. But then a 5% likelihood of a $200 loss is the same as a 50% likelihood of a $20 loss. One may be acceptable but the other not. Does COSO discourage the assessment of risk based on this simplistic calculation?	Rating: 1/10 As discussed above, this is not covered and is a serious problem IMHO.
5.      Will the update provide decision-makers with the structure/process they need to decide whether to ‘take the risk’ because of the potential for reward? ·        In real life, people have to ‘balance’ risk and reward ·        Will the guidance provide a disciplined process for identifying and evaluating all the potential effects of each option and only then making an informed decision? Or does it only consider and provide guidance on assessing harms? ·        For example, if the potential for loss is assessed as between $50 (20% likelihood) and $100 (5% likelihood), should a manager ‘take the risk’ when the potential for gain is between $50 (20%) and $250 (5%)?	Rating: 1/10 As discussed above, the criteria for determining whether to take a risk does not include any reference to the potential for reward, only the appetite for risk.
6.      Will the update lead to providing decision-makers with the guidance they need if they are to make the decisions management and the board want them to make? ·        The great majority of organizations who have a ‘risk appetite statement’ at the entity level have not been able to cascade it down in a way that enables those making the decisions in real life to know what is necessary ·        Different conditions (e.g., whether there is huge public scrutiny, whether the organization is likely to exceed or miss its earnings targets) can lead to executives wanting to change the risk decisions that are made ·        Its one thing to say that you need to avoid exceeding defined risk limits, but when the reward is high it may be appropriate to take that risk. Does the guidance enable agile decision-making that considers changes in the environment?	Rating: 1/10 COSO says that risk appetite is cascaded down to decisions-makers but provides no practical guidance or examples.
7.      Does the update provide sufficient guidance on how to assess and then correct, as necessary, the culture of the organization? ·        It is encouraging that this is now included. Is it sufficient?	Rating: 4/10 While the update mentions risk culture and emphasizes its importance, there is no practical guidance.
8.      Does the update provide sufficient guidance on each stage of the risk management process, including identifying, assessing, evaluating, and treating risk and opportunity? Does it provide sufficient guidance on communications and monitoring, including continuous improvement? ·        There is more to assessing risk (good and bad) than impact and likelihood. Other considerations include duration, speed of onset, and more ·        Many use models. Is this covered sufficiently?	Rating: 5/10 This is thin, but so is ISO 31000:2009.
9.      Is the updated COSO guidance on risk appetite and risk tolerance useful? Does it mirror and enable effective decision-making in real life? Does the guidance help to establish not only the upper limit of ‘risk’ that should be taken, but the lower level as well? ·        If organizations don’t ‘take risk’ they will not survive. It is dangerous to be too risk averse ·        How does an organization establish the minimum level as well as the maximum? ·        Does COSO provide sufficient guidance on how to assess both the upside and the downside? ·        Does the updated guidance help people ‘balance’ risk and reward, knowing when to ‘take the risk’? Or does it lead people to evaluate whether the level of harm is acceptable without considering the level of benefit? Does COSO guide people to consider the potential effect on strategies and objectives, or only to assess risk based on some out-of-context measure? ·        The COSO definition of risk appetite in the current framework talks about an amount of risk. Sometimes risk appetite is expressed in terms like “we have no tolerance for this risk” ·        However, in real life people make decisions based not only on the ‘amount’ of risk (harm) but the likelihood of that amount of risk. For example, I might accept a 2% possibility of losing $100 but not a 20% possibility ·        A generic statement like “we have no tolerance for this risk” does not help real life decision-making. While no organization will state a level at which loss of life is acceptable, in many industries the only way to get to zero likelihood is to exit the business ·        What is an acceptable level of variation from objectives? If you set an objective of 10% growth but are willing to accept 5% growth, 5% is your true objective. Alternatively, your objective may remain 10% but you will accept a 7% chance that it will be reduced to 5% ·        Is the ISO 31000:2009 term ‘risk criteria’ better, especially as it can be applied to individual decisions?	Rating: 2/10 The update fails to address the points made in the question (1st column). COSO has introduced new charts that purport to show the relationships between levels of performance and the level of risk that needs to be taken to achieve each level of performance, the risk capacity of the organization, the risk tolerance, and the level of variability in performance that is acceptable. But are these charts more than simply interesting? Are they reflective or real life? Are they practical guidance? IMHO, they are flawed. 1.      The relationship between risk and objectives/strategy is many to many. 2.      You simply cannot aggregate all risks to a single strategy/objective. 3.      It is possible for all sources of risk to be individually acceptable but when considered together (using judgment rather than trying to convert risks like compliance or safety to numbers) are unacceptable. 4.      If risks are given values, then the aggregate may appear acceptable when an individual source of risk (e.g., compliance) is not. Thought leaders have questioned the concept of risk appetite, and this section from the update is telling. “Organizations may also choose to exceed the risk appetite if the effect of staying within the appetite is perceived to be greater than the potential exposure from exceeding it. For example, management may accept the risk associated with the expedited approval of a new product in favor of the opportunity and competitive advantage of bringing those products to market more quickly. Where an entity repeatedly accepts risks that approach or exceed appetite as part of its usual operations, a review and recalibration of the risk appetite may be warranted.” In other words, use risk appetite except when judgment tells you not to – because the benefits outweigh the harms. It would be a great deal more useful if guidance would recognize from the beginning that assessing and managing risk out of the context of objectives and potential rewards is less than useful.
10.   Will it be possible to assess the effectiveness of risk management in practice using the updated version? ·        Any assessment should be based on whether the management of risk helps people establish the optimal vision, objectives, strategies, and plans, make better decisions and, as a result, increase the likelihood of achieving objectives ·        Any assessment should identify the areas where the risk of failure in identifying, assessing, evaluating, or taking action to address risk is higher than desired ·        If the assessment is against principles, are those in the COSO draft as good as or better than those in ISO 31000:2009? ·        If all the COSO principles are present and functioning, does that mean that risk management is effective? If one or more are not present, does that mean that risk management is without doubt ineffective?	Rating: 1/10 The recommended approach by COSO is significantly flawed. No guidance is provided on how to assess whether the principles are present and functioning. Compare this to the COSO Internal Control – Integrated Framework, where such guidance is provided: (a) internal control can be considered effective if there is reasonable assurance that risks to objectives are at acceptable levels, and (b) the principles are present and functioning if there are no “major” weaknesses – and the latter is where the weakness means that there is a lack of reasonable assurance that risks to objectives are at acceptable levels. Further, there are no principles or practical guidance on decision-making – which is where risk is actually taken day-to-day. Arguably, the principles can be assessed as present and functioning, yet executive management and the board still sees risk management as failing to make a significant contribution to both the setting and the execution of strategy.
11.   Will the guidance provide sufficient guidance to enable the board and/or a committee of the board to provide effective oversight? ·        Is the guidance as good as that in South Africa’s King IV Exposure Draft?	Rating: 1/10 See #10, above.
12.   Is the updated document consumable? Is it too long? Will it be read, understood, and acted on by all levels of the organization?	Rating: 2/10 It appears long but the practical guidance is short. While it may be read and understood, the valuable comments are terse and few. Much is missing from the guidance in terms of what effective risk management really is – from strategy-setting through execution through decision-making.
13.   Will the updated product help the busy executive or board member understand what risk management is all about, that it is not simply a compliance exercise but can improve the likelihood of quality decisions and the achievement of the right objectives?	Rating: 1/10 It is not persuasive that risk management will help an organization succeed. At best, it might avoid a level of failure. I would not provide the busy executive or board member a copy of the Executive Summary.
14.   Is the 2017 product a sharp improvement on the 2004 version? ·        Are the changes and additions an improvement? ·        Does the updated Framework represent leading thinking? ·        Will it help move practices around the world to greater levels of maturity and effectiveness? ·        Is it better than the ISO 31000:2009 global risk management standard and other guidance that has been provided by regulators, national corporate governance codes, and so on? ·        Would you recommend an executive, board member, or practitioner buying the updated Framework? Or, should they buy my book J?	Rating: 3/10 There are improvements, as reflected in my comment letter. For example, there is language (even if the guidance is thin) on culture, decision-making, cognitive bias, and risk capacity. It simply is well behind leading thinking on risk management and I would not recommend that any organization embrace it and believe that is sufficient. ISO 31000:2009 is not perfect either. Is it better? Perhaps. It is also thin in a number of areas. At minimum, everybody interested in COSO ERM should also read and consider ISO 31000:2009. In some respects, they complement each other. But there is more to risk management, and I in all modesty I believe the guidance in my book is superior.

 

[1] Exploring Strategic Risk reported that “Only 13% of [C-level] respondents believe their risk management processes support, at a high level, the ability to develop and execute business strategies”

Advertisement