How do you audit enterprise risk management?
The IIA published a Practice Guide, Assessing the Risk Management Process, in 2019. It is recommended guidance and not mandatory. What is mandatory in the IIA Standards is performing an assessment, and this Practice Guide (PG) is intended as helpful advice on how to do it. (While the Standards say that you must perform an assessment, I am assured that you don’t need to do so every year (regardless of the actual words used) when the risk is low – for example, if it was assessed and found effective the prior year.)
The PG starts well:
Around the world, risk management activities and initiatives are required and expected by regulators, rating agencies, and a host of other stakeholders in major industries including financial services, government, manufacturing, energy, health services, and more. However, risk management is driven by more than regulations and external forces. Implementing efficient and effective risk management benefits organizations of any type and size by helping them to achieve operational and strategic objectives and to increase value and sustainability, ultimately better safeguarding their stakeholders.
Internal auditors must evaluate the effectiveness and contribute to the improvement of risk management process (Standard 2120 – Risk Management). Benchmarking the current state of the organization’s risk management against a risk management maturity model is a good place to start this type of assessment. Benchmarking may help the internal audit activity communicate with senior management and the board about the organization’s level of risk management maturity and about aspiring to improve the process and advance in maturity. This information also enables internal auditors to appropriately tailor each engagement, taking into account the maturity of the area or process under review.
I like the fact that the PG promotes the use of a maturity model. I recently shared one that Michael Rasmussen developed and have a more extensive on in Risk Management for Success.
While the PG appears to understand that there is a “positive side to risk” (i.e., good things can happen, usually referred to as opportunities, as well as negative, generally referred to as risks), it falls into the same trap as almost everybody else by focusing on the negative side. For example, it talks about risk registers (lists of risks, also known as risk profiles), heat maps, and risk appetite. It fails to recognize the need to take risk, even taking more risk when the business opportunities and needs require.
The PG contains material that is worth considering (especially if you are only interested in auditing compliance with risk policies and procedures), I think there is a better way.
It starts with the recognition that if risk management is effective, leaders and other decision-makers will say so.
They will acknowledge that risk management is helping them make better informed and intelligent decisions that are contributing to the success of the organization, the achievement of objectives.
By ‘risk management’, I am not talking only about any risk function; I am talking about how the organization as a whole understands the more significant things that might happen, and uses that information in setting objectives, goals, objectives, and strategies, and then executing on them through every-day decisions.
So the audit starts by asking leaders and decision-makers, not only at the top but in other positions:
- Does risk management (broadly defined) help you set your goals and objectives and then execute on them for success? If so, to what extent? Is it sufficient?
- Is it helping you make informed and intelligent decisions? If so, to what extent? Is it sufficient?
- Do you have confidence that others are making the best informed and intelligent decisions?
- What is working well?
- What needs improvement?
- Are risk practitioners (if there are any such specialists) effective? Are they proficient? Are they helping you succeed?
- What should be changed?
While this can be asked in a survey, I strongly encourage the auditor to sit down with each individual and listen carefully. Start here and see what answers you get. Listen carefully.
If there are issues, understand the root causes and go from there.
You may find that everybody is complying with stated policies, risk limits, and even risk appetite statements – but this is not helping the organization succeed!
Seek to assess effectiveness rather than compliance. Help the organization succeed rather than avoid failure.
I welcome your thoughts.
Leave a comment
Norman Marks on Governance, Risk Management, and Internal Audit 
I have and can assess ERM with two questions:
Are you having structured conversations about risks at the board, executive and departmental level (e.g. by means of risk workshops).?
Do you have established risk criteria so that risks can be prioritized on a consistent basis across the organization?
Without these, then IMHO you don’t have ERM.
John, how does this provide assurance that people are taking the right level of the right risks?
How does it provide assurance that people are making good decisions?
For once, we are on different pages.
1. Would take longer to explain.
2. IMHO nothing but history can tell if good or the right decisions are made.
I believe the major stumbling point for this guidance is captured in the title – “Assessing the Risk Management Process.” Too much focus is on seeing risk management as PRIMARILY a process that can be bolted on and not as an integral part of decision making. For example, the text states: “Well-governed and successful organizations use the risk management process to coordinate the direction and control of risk exposure in a way that enables the organization to meet its objectives.” I don’t believe reliance on a “process” to accomplish this is insufficient.
Good point
Norman, please don’t leave me/us in suspense. Please explain how to decide if people are making good decisions.
John, I certainly am not going to try to second-guess decisions that have already been made. (Although I will ask whether they are comfortable they made a reasonable decision, in hindsight.) I am going to ask people whether they believe their and other people’s decisions follow a reasonable process, including the consideration of what might happen. There are quite a few questions that can be asked, starting with “how do you make your decisions?” I would also ask whether they are confident they are using all the relevant information, and who else is involved or consulted.
I believe effective risk management enables informed and intelligent decisions. So I need to ask about decision-making to test whether risk management is effective.
Auditors are in no position to judge risk management decisions as their skill sets are different. Pass the FRM exam and then perhaps you could start appreciating what risk management really means.
I am not sure that passing the FRM exam means you understand effective risk management, helping the organization succeed by making informed and intelligent decisions. Most risk managers in financial institutions only assess downside risk rather than recognizing that there are upsides (opportunities).
You must have had bad experiences with auditors in the past. There are many who know how to take a business perspective and can assess the effectiveness of ERM.
Nice article Norman!
Thinking back over my career, the organizations I’ve been part of that had the most effective risk management processes worked tirelessly to make risk awareness part of the everyday decision making process for all employees.
They trained employees on how to make more effective decisions including considering both the upsides and downsides of decisions, clearly articulating the assumptions embedded in their decisions, for more significant decisions getting independent perspectives to root out bias, etc.
They also provided employees with training on economic thinking and other areas to provide the appropriate tools for effective decision making.
They strove for continuous learning by evaluating decisions in retrospect “Was it an effective decision?” “What important factors did we not consider?” Did we have overweight certain factors, and if so why?” etc.
They coupled that with a culture that encouraged everyone to candidly question and challenge decisions, not by attacking the person or the actual decision, but by asking probing questions about the underlying process.
Creating that type of culture is hard work, but from what I have seen is a very solid investment. We still made bad decisions, but fewer of them, and without too many repeated bad decisions which I see plague other organizations at times.
Excellent! You are fortunate to have that experience. Thanks for sharing.
Not to be totally facetious, but I wonder if ISO31000 could create a reciprocal standard to rate internal audit effectiveness. Or at least the risk of internal audit failure.
Having been the only internal auditor involving the drafting ISO 31000 in working group 262, I can easily state that would be beyond the bandwidth of the 31000 group. They know very little of modern internal auditing. Similarly, most internal auditors know very little of modern risk management. Unfortunately, the two groups typical operate in separate worlds or only with decades old stereotypes of the other group.
Both could learn so much from each other, as long as they work with leading practitioners.
To Bruce’s point, a really big opportunity was missed after the 2000 credit crisis, as no one (to my knowledge) researched what internal auditors had reported re credit risk and how it was handled by audit committees. We have a few anecdotes about this but it could have been a. major worthwhile project.
Norman hear that role of ERM in decision making is one area of focus that can be useful. Organizational culture to ERM I find plays big role, for example when ERM is considered as slowing things down vs contributing to better outcomes. Auditing ERM role in decision outcomes is tricky to avoid audit bias, as might have an outcome that is unfavorable or not as expected yet the decision was properly considered ERM at the point in time it was made.