Home > Risk > How well did COSO address comments on the ERM draft?

How well did COSO address comments on the ERM draft?

September 22, 2017 Leave a comment Go to comments

Last July, I submitted written comments and suggestions to COSO on the draft of the ERM framework update.

In this post, I remind you of those comments and discuss (see Comment) how well they have been addressed in the final edition. (At the time, I discussed them with several people involved in the update, who all agreed they had merit. However, I got the impression they were reluctant to make the sort of major change I was asking for, saying that COSO might follow the updated framework with thought papers.)

The COSO update has an appendix where they talk about their response to comments. Unfortunately, most of my comments are not addressed in that section.

I will share in a later post my assessment of the final product based on a set of questions that I encourage you to consider. Please join the conversation and share your assessment of the value of the ERM framework update here.

===========================================================================

July, 2016

There’s a lot to like in the update, which in many respects I consider an upgrade.

In fact, I would describe this document as having the potential for a ‘leap forward’, not just a step. It’s more than an ‘upgrade’.

However, it is not yet there. I believe another significant leap forward is required, and this can be delivered through careful and thoughtful consideration of the comments COSO receives on the Exposure Draft (ED) – followed by action to address them.

I believe that while PwC and the COSO Board and its advisors have clearly stepped back and taking a big picture look at its ERM guidance, a second step back and another look at the essentials of risk management should be taken to consider whether the guidance is truly achieving its potential.

What is that potential? It is to transform how organizations are run, from the setting of the mission, objectives, strategies and plans to the daily operation of the business: how it performs in practice through intelligent and informed decision-making at levels of the extended enterprise.

As is said in the Introduction:

“The value of an entity is largely determined by the decisions that management makes—from overall strategy decisions through to day-to-day decisions. Those decisions can determine whether value is created, preserved, realized, or eroded.”

In its ideal state, the management of risk is part of the rhythm of the business[1], entwined[2] into every business process and decision at all levels across the extended enterprise. It is no longer a compliance activity, but an essential ingredient in the success of the organization. It is not limited to avoiding harms, but also encompasses determining when the ability to reap a reward justifies taking the risk of harm.

Comment: COSO has gone a long way to see risk management “entwined” into every business process. However, they have done little IMHO to explain how it is part of decision-making and they have not addressed decisions and actions in the extended enterprise.

They say that an ethical person does the right thing when nobody is watching. Effective risk management is present when there is reasonable assurance that every decision-maker, from the board down to the front-lines, will make the ‘right’ decision without a risk officer present.

Comment: This important concept appears to be missing – that we need reasonable assurance that decision-makers are taking the right risk. Risk appetite is a way to identify after the fact whether too much risk has been taken. It only works proactively when each decision-maker knows which risks to take and I don’t believe that is sufficiently covered in their discussions of risk appetite and tolerance.

In fact, in an ideal world, people don’t think about risk management – it’s simply effective management.

Although the Foreword says (more than implies) that the earlier version had been broadly accepted and should be considered a success, that comment is highly questionable.

Surveys have shown that the ISO 31000:2009 global risk management has been adopted more often in recent years than the COSO ERM Integrated Framework. Many have taken the best of both to develop their own framework, and many experienced risk practitioners and thought leaders have dismissed the COSO product entirely.

Other surveys, notably by Deloitte[3], have found a huge disconnect between those leading risk management and the executives and directors who should be obtaining value from it. Only a small percentage said that risk management had made a significant contribution to their setting and execution of strategies.

There are several reasons for this. They include:

  • Creating the perception that the consideration of risk is something separate from the activity of managing the organization; as the ED says, it should be an integral element in decision-making every day at all levels of the organization

Comment: COSO has made efforts to address this. But the lack of discussion on decision-making and the continuing focus on a risk profile (which they admit is simply a list of risks, a.k.a. a risk register) will likely inhibit meaningful progress. The key point here is that organizations have been managing risk for centuries, often with success, without a formal program or office. As Alex Sidorenko says, talking about ‘risk management’ instead of effective management can actually inhibit a constructive discussion, because the ‘r’ word has a negative connotation in the minds of executives and because it appears to be something different from effective management when in fact it is not. Good managers manage risk all the time; they anticipate what might happen and deal with it; effective boards insist on discussions of what might happen and related scenarios as part of their strategy-setting and performance review discussions.

  • A focus that is restricted to the potential negative effects of uncertainty, considered at intervals rather than continuously

Comment: The need for continuous risk discussions is included, but it is still focused on potential negative effects.

  • A disconnect with management who are looking to enhance performance and deliver value, not just avoid failure

Comment: The update talks about performance but not how to assess the likelihood of achieving strategies and objectives and therefore enable actions to increase the likelihood and extent of success.

  • Reporting risks rather than the likelihood and extent that objectives will be achieved

Comment: This is a major issue that is not effectively addressed.

  • Communicating in a language different from that of the business. This inhibits management’s ability to not only understand at an intellectual level that the management of risk can help them be more effective as managers and successful as business leaders, but actually believe it

Comment: See prior comments.

  • An expressed desire, fueled by regulators and the concept of risk appetite, to ‘manage’ or ‘mitigate’ risk when in real life risk needs to be taken

Comment: I do not see how the update will constructively influence regulators.

  • Failing to understand that events and situations (requiring decisions and choices) create the potential for not just one but multiple effects – both negative and positive effects are likely every time a decision is made or an event or situation presents itself. All potential effects of a decision need to be assessed, generally in the same way, to understand the potential rewards and harms, understand and evaluate options, and consider what should be done to improve the likelihood and extent of success

Comment: This is a major gap in the update.

First, I want to congratulate the Board, its advisors, and PwC for progress on a number of fronts. They include (not in any particular order):

  • Emphasizing that risk management is about addressing the uncertainty that lies between where we are and where we want to be (although not in that language)
  • Restating that risk management is about achieving objectives. This was also in the prior version, but is repeated and emphasized for the great majority that did not see it in the 2004 edition
  • Making the point (I see Jim DeLoach’s influence) that risk management is not about the periodic review of a list of risks (i.e., enterprise list management)
  • Talking about the need to consider what might happen in the future when setting strategies and objectives
  • Restating that decisions need to be made based on an evaluation of both the potentially positive and negative effects of uncertainty
  • Introducing a discussion of risk culture
  • Using the word “anticipate”, which I think is a highly descriptive way to explain what risk management is all about

These are points made in the Executive Summary.

Comment: We should not forget that the update is an improvement on the 2004 version.

I have developed a set of 12 questions to assist in the evaluation of the Exposure Draft and whether it will move the practice of effective management as far forward as it can and should.

Comment: I wonder whether PwC used the set of questions.

My comments are at this 50,000 foot level. They affect much of the detail and I hope the COSO Board and advisors, assisted by PwC, will consider them and then apply them to the detailed content.

 

Final thoughts and suggestions

As I said at the beginning of this response, the ED is an upgrade and has some valuable content. The ideas and aspirations laid out in the Executive Summary are, for the most part, excellent.

However, I have problems that I believe are significant.

  1. The ED continues the focus on harms. There is a huge difference between opportunities (such as the opportunity to take advantage of a competitor’s stumble) and recognizing that any situation, event, decision, or choice can have multiple effects on achieving objectives: some positive as well as some adverse. All have to be assessed and evaluated, not just the harms.

Comment: The executive summary may say that there are multiple potential effects, both positive and negative, but the body talks almost exclusively about harms. There is no discussion of the need to identify, assess, and evaluate all potential effects.

  1. The ED continues to focus on a list of risks. While it talks about decision-making and makes the point that risk management informs decision-making, it is more than that. Every decision is a risk decision. Every decision is about understanding the current situation, what is expected to happen, whether that is acceptable, what options are available, and then making informed choices. That is risk management as well as effective management. It is not just risk-informed decision-making. The best way to improve the management of risk is to improve the decision-making process and capability. If the framework could provide a structured process for decision-making, that would make it both practical and of immense value. Instead, it pays scant attention and continues to talk about generating and maintaining lists of risks.

Comment: The framework body focuses on a risk profile (the same thing as a list of risks, just different language), risk appetite, and so on. There is no discussion of how to weigh all the possibilities, the ranges of good and bad potential effects, to come to an intelligent decision. While the update talks about decision-making, this is absent from the principles and I see no related guidance.

  1. The idea that you can aggregate all risks into a risk profile is alarming. You simply cannot do that and expect to be successful. The potential for each objective to be achieved must be managed individually as well as collectively. Compliance risk should not be aggregated with reputation or financial risk. In fact, there is danger in aggregating different forms of compliance risk; compliance risk in aggregate may appear to be at an acceptable level while the company is significantly in breach of specific regulations or laws.

Comment: This misguided guidance remains prominent.

  1. Finally, and most important of all, risk management is really about anticipating what might happen that would affect your journey from where you are to where you want to be. The COSO Board needs to reconsider how it describes terms like uncertainty, risk, and risk management with this in mind. Good decisions come from understanding what might happen, all possible effects, then making informed, intelligent choices.

Comment: Unfortunately, I do not see sufficient progress. While talking about performance is progress, there is insufficient attention to assessing the likelihood of achieving objectives or on decision-making.

I have pointed out other areas for improvement, such as an expanded discussion and guidance on board oversight, and a major overhaul of the thinking around risk appetite and tolerance. But these are the most crucial issues.

A couple of closing suggestions:

  1. Expand the Advisory Board to include practitioners from around the world, especially from nations where the practice of risk management is more advanced than in the US. Grant Purdy, John Fraser, Richard Anderson, and Martin Davies would be excellent additions.

Comment: While some expert advisors were present (notably, Carol Fox), I wish COSO had brought more thought leaders into the process.

  1. Consider, where possible, the use of plain English instead of technical jargon. This would make the guidance clearer to executives and board members. Talk about optimizing outcomes, achieving success, and so on – the language of the business.

Comment: See prior comments.

There is an opportunity to make a huge leap forward, providing a beacon for world-class risk management, or should I say effective management.

That will require a further step back, a deep breath, a willingness to accept the need for change, the courage to make a huge departure from traditional thinking (which has proven to be failing us), and action.

It is better to take longer to think this through, make the changes thoughtfully, than to tinker with the ED. That, I suggest, will not be sufficient.

 

===========================================================================

Final comment: My impression is that COSO only tinkered with the draft. I understand that they are considering further work, thought papers or similar, that will build on the framework and address some of the points above.

But, have they made a “leap forward”? Have they done enough to move practices forward, in the right direction? Did they want to make that leap forward, or were they too risk averse?

Will this update change the percentage of executives answering the piercing question by Deloitte, “Does risk management support, at a high level, the ability to develop and execute business strategies”, up from 13% close to 80%?

What do you think?

 

[1] “Drive business results by harnessing uncertainty”, EY February, 2015

[2] A great word, far better than ‘integrated’ or ‘embedded’, used by PwC in Risk in review: Going the distance, 2016

[3] Exploring Strategic Risk reported that “Only 13% of [C-level] respondents believe their risk management processes support, at a high level, the ability to develop and execute business strategies”

Advertisements
  1. September 22, 2017 at 9:51 AM

    Norman; I like you, submitted detail comments to COSO. ( https://goo.gl/tLdAgG) While COSO chose to ignore some of my suggestions they did seem to respond to my comments and comments from many others that the body of guidance was still risk centric. While it isn’t the guide I would write or you have called for in your comment letter, I like that they have said risk centric/risk list approaches are the least integrated form of ERM. (I think they could have been much stronger) I do like the push for ERM frameworks to focus on strategic objectives and link to performance. In my mind the three biggest positives are support for objective centric ERM, support for focusing on value creation/strategic objectives and the need to link to performance. I think they should have done more to describe what changes internal auditors should make and why they think ERM can’t/should not be used for the objective ENSURE FINANCIAL DISCLOSURES ARE RELIABLE AND IN ACCORDANCE WITH GAAP. Having voiced my half empty, on balance I like this COSO guidance better than any other guidance they have issued to date.

  2. AndreiDoc
    September 22, 2017 at 4:26 PM

    While I find some pieces of the revised framework difficult to process/ connect with practice I don’t necessarily agree with some of your comments:

    1. Why would there be a real need to identify, assess and evaluate ALL potential effects? Good decision making is also a function of time available/allocated to assess those outcomes and make strategic decisions. Looking into ALL potential effects does not sound … efficient. One might consider starting with a very broad range of outcomes and their likelihood of occurrence and focus on addressing those with the highest assessed risk level (this includes good and bad outcomes). If your risk assessment process is sound and dynamic it will properly capture any change in risk level (that is an outcome becomes more likely or with bigger impact). All potential effects (positive and negative), all the time is not feasible/ efficient.

    One good example is when you need to take a risk informed decision in say… 10 minutes. Are you able to assess all potential effects before the decision? Should you do that, or focus on a few which are most relevant given the context? One needs to prioritise, and human bias would first minimise losses/harm.

    2. Not all decisions are risk decisions. Any decision in expectation of an outcome has a risk level attached to it but some low but some high. Only those above a certain level have sufficient uncertainty attached to them to require a structured risk assessment of outcomes and support course of action. Some decisions are BAU, automated decision making on a daily basis, you don’t explicitly think you manage a risk when you decide you need a Cafe Latte and not a Double Espresso at 4 p.m

    3. Sorry but a risk profile/ risk appetite is not just a list of risks. The problem is not with how it is named it is more with its construction. If it is properly built to be very connected to set objectives than it will support decision making (and yes, often it is not). Support means it is one input into the overall decision mix, not the only one. Do you need your car “dashboard” to get to your destination? Maybe not all the time, but you don’t get rid of it. Are all those buttons, icons, signals needed? Yes, they are; you drive a car by holding the wheel and paying attention to the road, but from time to time you need to check your dashboard (speed, consumption, oil level, time to destination? required speed to get there on time? alternative routes to take?…etc.). Indeed, a key point is that having all dashboard lights on green does not guarantee you will reach your destination. Current risk profile/appetite dashboards most frequently lack a forward looking perspective (e.g. are we on the right track to reach an objective/ what should I do to adjust my trajectory to get back on the right track).

    • Norman Marks
      September 23, 2017 at 6:35 AM

      Andrei, let me take each of your comments in turn.

      1. I agree that it is impractical to include every potential effect. All effects of significance should be considered. But that was not my point. I was saying that both potential positive and adverse effects need to considered, with the same discipline, so that you can weigh everything and make an intelligent decision.

      2. Every decision creates or modifies risk, even the decision whether to get an Espresso or Latte, Each will affect your metabolism differently. Depending on the individual, the Double Espresso may prevent you from sleeping later on – but it might also have the positive effect of helping you with a difficult meeting later that evening. Every decision affects what happens afterwards and therefore is changing risk.

      3. A risk profile that is connected to objectives is inferior to a list of objectives that includes all related risks of significance. Monitoring risks that have no significant effect on your objectives is, as you say yourself, less than optimal because you have limited resources.

      • AndreiDoc
        September 24, 2017 at 2:21 AM

        Norman, for each point:

        1. I know that your core point was around positive/negative, and I agree with that. Just saying that given specific contexts and set-ups, negatives will be top of list (mind) and either no time will be left for a proportional attention to positives or no attention at all given known adversity bias.

        2. Agree with that also, didn’t say otherwise. Just outlined that some decisions have risk management embedded in them structurally (e.g. like the human eyelids that close automatically when something is close to your eye -.> “risk reflex”) and as such, no explicit risk assessment approach is undertaken.

        3. Understood your point. But, “a list of objectives that includes all related risks of significance” is inferior to a combination of such a list + a dashboard tracking performance (KPIs) and risk (KRIs) all-together and thus feeding back to the ones in charge with strategic management/ decision making.

        I guess now we can start another thread about how strategic objectives should be set. I see in practice significant room for improvement in that area.

  3. September 22, 2017 at 8:18 PM

    Andrei,

    Of course, all decisions are “risk decisions” – if you believe that ‘risk’ is concerned with the uncertainty we face achieving our highest level objectives that represent our purpose.

    Every time we make a decision, even if it’s not explicit, we consider to some degree whether the outcomes will be those we desire. If there is not sufficient certainty that the decision will lead to those desired outcomes, we either don’t do anything (which is a decision) or we take extra actions to increase our certainty.

    Just think it out for every simple decision we make. For example, when we wish to buy something on the other side of the road and we have to decide if to cross it there or somewhere else. If we decide it’s safe enough to cross where we are (and not put our ultimate purpose in jeopardy – to live) we might also look carefully to ensure there is no traffic coming so that we improve the certainty we get across the road and still live.

    What you have raised is why, fundamentally, the new COSO ERM is fatally flawed: it still treats risk and it’s management as a separate process that should be somehow grafted onto decision making. And we all know that that approach is doomed to failure because it portrays risk management as seperate and different from normal management and decision making.

    Quite frankly adopting this type of thinking means we will never ultimately convince decision makers to adopt a system process for risk management as part of their decision making. The starting point has to be the (existing) process for decision making not the process for risk management. We need to work from the inside out, not the outside in. In situ, not in vitro!

    Dressing up something as simple as dealing with uncertainty when we make decisions with unnatural confections like ‘risk appetite’,’inherent risk’and ‘risk profiles’ just serves to further prove to normal people that risk management is irrelevant to their day to day existence and should either be left to the ‘experts’ or endured once or twice a year because an audit committee wants a report.

    Nevertheless, I am sure lots of consultants and software vendors will make lots of money out of COSO ERM II – and some of us suspect that might be the motivation for the rewrite.

    • AndreiDoc
      September 23, 2017 at 3:36 AM

      Was your decision to post a comment a risk decision? Did you explicitly assess the potential outcomes and decided on what course of action to follow? Or was it simply an immediate reaction to a thread/subjct of interst to you (linked to one of your objectives). The risk assessment in your decision was automatic: if you were to reply on a medical topic you would have thought 10 times if your answer is relevant, on topic, bold, properly formulated etc. My point is that in many instances risk management is embedded in our routine. In other instances, as the level of uncertainty increases it needs a structured approach, similar to other fields. Why is it wrong to use various tools to support assess risks/outcomes?

      In some instances you also need to convince others how you decided on a course of action, and having a step by step structured method/ documentation supports that process (or even the objective itself e.g. to get from A to B u first need to get approval from your Board on the flight plan; as Boards are part of the ecosystem you will not achieve any objective if you are not able to prove thought process behind proposals, including those risk dashboards

      • September 24, 2017 at 7:46 PM

        Andrei,

        Yes – of course – to all your questions.

        All decisions always involve, to some degree, the consideration that desired outcomes are uncertain. The consideration may mostly be implicit, but it still will be there.

        The point I made (and you ignored) is that COSO ERM II treats risk management as an add-on to decision making while, in reality, it is always already part of it. I also made the point that you can’t realistically improve the integrity of decision making by forcing foreign concepts and artefacts in from outside the process. You have to start working from the ‘inside out’. COSO ERM ignores this central fact and that is why adopting it is doomed to fail.

  4. September 23, 2017 at 1:58 AM

    I wonder whether the problems with the COSO documents on ERM and controls arise because COSO have hired consultants to write them. In order to justify their fees, consultants will inevitably produce lengthy documents so as to decrease the $ per page ratio. This results in documents which are not clearly focused and provide too much detail on the ‘how?’ and not enough on the ‘why?’. They consequently read more like manuals and less like a guiding framework.

    I’m not wishing to ‘consultant bash’ here; PWC seem to have done the job they were hired to do. I just wish COSO had used their own staff with a brief of ‘make it clear and concise; leave the textbooks to other writers’.

  1. September 27, 2017 at 12:33 PM
  2. September 29, 2017 at 8:51 AM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: