10 reasons not to like the COSO ERM framework – a discussion with Grant Purdy
Grant Purdy is a highly respected, veteran risk practitioner based in Melbourne, Australia. He chairs the committee that developed the excellent Australia/New Zealand 4360 risk management standard and has been an active and influential member of the global team that gave us the fine ISO 31000:2009 risk management standard.
In this post, I will share the results of talking to Grant about the merits (mostly the failings) of the COSO Enterprise Risk Management Framework.
Grant told me that recently, after many years, he re-read the COSO ERM framework again, from cover to cover, and discovered that some of his thoughts on its weaknesses have become distorted and exaggerated over time by reading what others had said about it and, in particular, how others have misused the materials to support their own paradigms and agendas. For example, he was convinced that the COSO ERM Framework required risk reporting and did not put enough emphasis on treatment because that is the message that had been put out by many of the large consultancies.
Grant believes that the COSO product has a number of good points but that overall he finds it complex and unwieldy, and can clearly see how many companies would just give up and pay someone to tell them how to implement risk management. He also thinks the cube and the need to keep some alignment going with the Internal Control Framework diagram compromises the flow of the processes given there.
Then, Grant notes there are some big technical flaws that will mean the process being followed will always be deficient and inefficient.
1. “When identifying events, the code mentions external factors; but the majority of the discussion is focused on internal factors, systems, culture etc. The COSO process starts with the internal environment, not the external ones and this fails to reflect the influence that the business environment, regulatory conditions, and external stakeholders have on the risks an organisation faces, its organizational culture, and how they influence its risk appetite and risk treatment priorities. This can easily lead to organisations just focussing inwardly and not actively identifying risks that reflect external factors and circumstances.”
2. “Stakeholders, particularly external ones, are not mentioned and stakeholders’ objectives and their influence on decisions about the significance of levels and types of risk are omitted. This is a critical omission and means the organisation effectively insulates itself from external opinion and stakeholder objectives. Most of the risks we face are caused by an incompatibility between stakeholders’ and our own objectives.”
3. “COSO ERM says that risks are described as events, and events are described and illustrated by examples of sudden, acute occurrences. There is no appreciation of the slow changes in circumstance and situation (for example a deterioration in internal culture or market sentiment) that give rise to some of the most critical risks.”
4. “COSO measures risk in terms of the probability of an event and its “typical” consequences. However, we will not always get the “typical” consequences every time an event occurs. For example, not every time my house is hit by lightning will it burn down. If I estimate the level of risk as the product of the likelihood of the event (being hit by lightning) and the worst consequences (losing my house), I will overestimate it. In fact, there are multiple possibilities: my house could be hit and not be damaged; it could be hit with slight damage; and so on all the way up to being burned down. Each potential consequence would have a different likelihood of occurring.
“Of course, this all sounds rather academic until you actually observe how, in workshops and in life, people who follow the COSO code use a rating system to estimate the level of risk; and they always seem to get it wrong and omit the conditional probabilities that should be applied to the event probability. This means that they always overestimate the level of risk, which prevents individual risks being properly distinguished and compromises any realistic modelling of the effectiveness of controls. The COSO approach to estimating the level of risk reduced the credibility and usefulness of the risk management process because significant consequences are predicted to occur much more frequently than is credible based on historical experience.”
5. “Throughout the document, the term ‘risk likelihood’ is used, but risk does not, per se, have a likelihood. Likelihood is one of the attributes used to measure the level of risk. This is a philosophical trap that can lead the unwary to see a risk as an event and then to use language such as “when the risk occurs”. Risks don’t occur when events occur, risks only ‘exist’ whenever we make objectives. If there are no objectives, then there are no risks. The level of risk (not risk) is described in terms of what can occur (consequences) and how likely they are.”
6. “While there are some concessions to what are called ‘opportunities’, in COSO ERM risks are mostly about losses and risk treatment (response) is about reducing the likelihood and severity of losses. The thinking in the COSO document is not mature enough to appreciate and explain that risk is just the effect of uncertainty in what you set out to achieve and that outcomes can be beneficial, detrimental or both. Certainly, the document does not promote taking risks that have beneficial consequences because you are confident you can treat or tolerate any potential downsides which is, after all, the basis of enterprise: the undertaking of risk for return.”
7. “I find the whole thinking about ‘risk responses’, ‘control activities’ and ‘monitoring’ most confusing and confused and I think most people who read and try to use the code do as well. For example, if you institute an audit regime, this is a good form of risk treatment to reduce the likelihood of unfavourable consequences. However, audit could be required as a matter of policy, could be part of a management process, and could also be part of a monitoring strategy. ISO 31000 clears this all up. Risk treatment refers to the actions you take that lead to the creation of and improvement in controls, and controls are what you employ to modify risk. These controls then require monitoring and review by assurance processes. That’s it.”
8. “The problems with the concept of inherent risk are well-known and the COSO document does not explain why you need to use this artificial, theoretical state where no controls exist, to justify tolerating the present level of risk or doing something more to modify it. In risk analysis it is useful to understand what worse-case consequences could occur if existing controls fail so that we can focus our assurance activities on checking those controls, but this is best dealt with by using the Potential Exposure (inherent consequences) value that does not require any consideration of likelihood.”
9. “The whole area of risk appetite and what COSO ERM calls risk tolerance is handled in a mechanistic and naive way. The thought that before you even do a risk assessment, a board can identify the material risks and tell you how much they are prepared to tolerate puts them on a par with the Gods. What this means in practice is that some Boards may have the ability to think about different types of consequences (not risks) and in some cases they can say how much loss they are prepared to sustain over a period of time compared with the balance sheet and cash flow of the company. However, these are not measures of risk, they are only measures of consequence. For non-monetary consequences, the statements that Boards can make start to get very vague. For example, they might say they never want to kill someone, but they will rarely want to agree on what individual risk of fatality they are prepared to expose their employees or neighbours to.
“Following the COSO prescription of taking these measures of risk appetite and applying them to assess at which level of risk you stop risk treatment is idealised, unrealistic and, in some states, may be illegal. Cost benefit, which is mentioned in relation to ‘response’, is the only way to make this determination, even when it comes to emotive areas like public safety.
“The material in the COSO ERM Framework on risk appetite has led to greater confusion and more wasted consultancy dollars than any other part of the framework.”
10. “The greatest sin – and I’ve left this till last – is that the COSO document confuses and mixes up the framework (the organisational structures, policies, and arrangements put in place to promote, integrate and improve the management of risk) with the process used for risk management, particularly that used for risk assessment, risk treatment and monitor and review. They need to be thought of separately where the framework operates at an organisational level while the process is that which the framework seeks to integrate into all critical organisations processes where decisions are made.”
What’s your view? Do you agree with Grant? If you prefer the COSO ERM Framework to the ISO standard, I would love to hear why. Personally, I like some of the COSO materials (especially the discussion of embedding risk management throughout the organization), but find the cube less than useful and the ISO presentation easier to use.
Leave a comment
Norman Marks on Governance, Risk Management, and Internal Audit 
Norman – Interesting post. COSO had what amounted to a partially sanctioned franchise with the 1992 internal control framework, and it was (somewhat) legitimized (mandated) with Section 404 of Sarbanes-Oxley. The 1992 offering was primarily a liability management framework, with the accountants living in the financial reporting controls slice of the cube, the lawyers in the compliance with laws and regulations slice and “everyone else” in the operations section of the cube.
As far as principles-based offerings go, and considering the intended context (primarily management reporting on internal control over financial reporting (ICOFR)), the 1992 internal control framework remains “evergreen” – e.g., the importance of the “Tone at the Top” is not going away anytime soon. And the attempt to provide three-dimensional context to an organization, even though financial reporting centric and through the liability management lens (i.e., the accountants do not want to be responsible for controls outside of financial reporting controls, nor do the auditing firms want to opine on controls outside of ICOFR) remains admirable. So OK, as far as it goes, using 1992 internal control currency as the medium of exchange for reporting on ICOFR.
COSO ERM was a somewhat uninspired attempt to wrap the 1992 framework with a marketing layer for use by the authors. There was no real attempt to engineer the design of the extended framework, just adding a few risk management shims and attempting to bring the strategy layer into the extended cube to improve marketability to the C-suite and Board (to partially paraphrase Steve Jobs on Microsoft’s adding Windows to DOS, COSO added a road apple to whippped cream when it created the ERM framework).
COSO still has an opportunity to salvage the ERM space with its upcoming rewrite of the framework(s). They have huge market share in the US with SOX, but they have lost a lot of mindshare to ISO and other better thought out approaches to risk management.
Keep up the good work!
Larry
Great Post! Obviously, the COSO Framework still needs work to integrate it with ERM and ISO 31000. I too found the COSO Cube confusing and impossible to incorporate in current control practices. But, COSO did provide companies with the appropriate guidelines to implement ICFR for SOX reporting. Now that the frame is up and taking hold, the walls need some nurturing and modeling inside to complete the circle of protection! I find the adverse comments made about COSO’s lack of integration with external risk factors intriguing.
Good post but since mr purdy reviewed
coso from the perspective of iso 31000
(where he himself contributes to it),
it can not be said an ‘independence’ view.
I have been using both coso and iso 31000
in my consulting activities. Both
have their own strengths and
weaknesses. Let us focus on coso’s
strengths and weaknesses.
Many have questioned about the cube
shape of coso framework. To me it is
a very clear message of an integrated
model. It means every part of the
model should not be understood
separately without the other and
the flow starts from the internal
invironment down to the monitoring
and review. It is a framework of ERM.
What is intoduced in ISO 31000 as
a risk management process which is
different and seperated from the
ISO 31000 risk management framework
is a foreign concept to COSO integrated
framework. In ISO, consequently, its
risk management process can be used
separately within a different set
of framework (as long as the famous
typical ISO PDCA formula is there).
While COSO introduced us an integrated
ERM framework.
COSO’s used of risk appetite is a very
important strategic approach to risk
management. First of all it requires
the board to have a proper knowledge
of the company’s capacity to persue
its objectives. It is a scarcity issue
here and any company’s board should
define it effectively. Secondly,
it defines the limit of risks taking.
A corporate board without proper
understanding and clear definition of
company’s capacity and capability
will fail to discourage
risk taking appropriately. Either
they neglect it at all or setting a
too low/too high level of risk taking.
It is not ‘playing God,’ it is an
obligation of the board –
a responsibility.
However, COSO weaknesses
lies on its concept of risk
(merely as negative/adverse impacts);
While the concept of risk is very well
defined by ISO 31000. So I entirely
agree with Mr Purdy’s analysis in this
regard. The consequences are abound.
COSO does not help us to be positive
about risk. The COSO’s concept of risk
fails to encourage a proper formulation
of risk appetite. I do hope that COSO
would consider to make a paradigm
shift in revising the concept of risk
in its framework in the future.
I’ve said it before and I’ll say it again, there are some elements of COSO ERM which are not looking too bad as the document reaches it’s 16th birthday.
In particular, I find the expansion of the COSO cube to include strategic risk as a risk class in it’s own right to be many years ahead of the curve. Indeed if we’d focussed on getting capabilities in place around this we’d be in a better place today.
I direct your attention to a similar comparison written by Legend Felix Kloman and summarized in a post I called “COSO ERM vs ANZ 4360 Deathmatch”
http://riskczar.com/2009/09/09/h-felix-kloman-coso-erm-vs-anz-4360-deathmatch/
COSO and COSO ERM may be hard to use, but the flaws are greatly overstated. It is used by 13,000 SEC registrants, companies, their CRO’s internal and external auditors in the USA and elsewhere. ISO 31000 should be useful as a reference framework/model, but we will have to see as it is more widely adopted outside of Australia/NZ. I am however tired of the concerted effort to “body slam” COSO ERM and the related hyperbole.
Grant Purdy wrote to me recently and said about ISO 31000: “One purpose of the ISO standard is to regulate the risk management industry to ensure consistency.” Regulate? Is ISO the new world order? Is ISO 31000 that grand? If this is the mantra of this group it will probably turn a lot of companies and professionals off.
I am always fascinated by the number of folks that write critiques as relates to a standard on one thing or another but have never actually worked with the document. Take the reference to 13,000 SEC registrants above. Unlikely that anywhere near this total have ever read COSO ERM let alone used it. Add to this the ability to publish gibberish instead of factually addressing each of the points raised by Grant Purdy and what you end up with are critiques with no validity whatsoever.
I have actually expended 4,000 hours or so give or take on the COSO ERM framework and can relate and discuss all of Grant’s points and in fact add others.
It is clearly recognizable that most of the folks that do not understand ISO 31000 hail from the United States and that is because of the condition of risk management in this country- poor quality
And lo, a warrior came from overseas. One steeled in the might of his binders, the armor of his standards notebooks.
This warrior, this “ARNOLD SCHANFIELD” did come to the land and find that nay, the risk managers of that land had neither bowed at the alters of the ISO, nor drank deep from the cup of ISACA.
ARNOLD SCHANFIELD, he who had travled seemingly at the speed of light to quickly, ardently reproach those of lesser intellect did fail. ARNOLD SCHANFIELD simply trolled their thoughts in comments, shook his head, and logged off.
Fare thee well, ARNOLD SCHANFIELD, Fare thee well….
Norman and Grant, excellent post.
Generally I’ve found that GRC people like COSO because in emphasizes the role of control and compliance functions, whereas Quality and Risk Analysis types prefer ISO 31000 because it emphasizes systems and performance optimization.
Todd, I agree that one must assess an organization’s strategic risks as part of an ERM effort, but the cube paradigm fails because it makes Strategy appear to be of equal importance to other elements like financial reporting. Clearly strategic decisions are much more fundamental and can completely shift the context for all the other dimensions. So, including strategy as a the cube is simply an inappropriate conceptual metaphor.
Arnold and Alex, sorry to say, but in my experience doing risk management in a major American corporation, I’ve found that most people, including many professional risk managers, find it quite difficult to fully “get their heads around” a very formal and structured conceptual model for risk management. First, they struggle to get an intuitive grasp of the statistics of risks, as Grant describes when he says that “they always seem to get it wrong … they always overestimate the level of risk.” If you don’t grasp the maths, you’re much less likely to render an accurate risk assessment. Second, people generally struggle to understand the concepts behind risk. This is most apparent in their poor use of terminology, which reflects confusion over things like, as Grant again pointed out, the difference between a risk and an event, or using redundant terms like “risk likelihood” that have no precise meaning. But more fundamentally, people faile to understand things like that Strategy is far more fundamental that Reporting, or that risk does not exist in the absence of an objective. I don’t blame this on COSO as much as I blame it on the inherently non-intuitive (for most people) nature of risk concepts, and America’s general cultural preference, outside of the technical literati, for immersing themselves in environments that are emotion-driven and avoiding environments driven by logic and mathematics. That said, I don’t think Europe or Asia are much better at it, though Australia seems to have a unique aptitude for it.
Reblogged this on Christian Lenehan MSc in Risk Management.