The Culture of the Audit Department: Cop or Consultant?

September 26, 2022 5 comments

I had the privilege of working with Mike Jacka as a member of an IIA International Committee for many years. He’s not only smart, but funny – a great speaker if you can get him.

He continues to write for the IIA’s magazine and blog.

His latest is Mind of Jacka: Not Quite a Manifesto.

Mike shares an interesting list of what he calls “the more intangible, human elements that allow us to function amid the contradictions that define our profession. To quote a fairly famous document, we hold these truths to be self-evident.”

I agree with the underlying premise, that internal audit exists to help the organization and its leaders succeed. It’s not about finding fault. It’s not about making ourselves look good and boasting about how much value we deliver.

While there are times when we have to be the cop, finding serious issues that threaten the success of the organization, most of the time we should be the consultant, providing valuable assurance, advice, and insight on what matters to the success of the organization.

Some auditors prioritize finding fault. For example, I knew one CAE who told his team that an audit report with no findings was unacceptable.

These auditors will not gain the trust of management. They will not be seen as helpful, only as potential impediments that consume valuable time.

Management will hide things and certainly not invite them in. They will resist and not be interested in working together to identify the best path forward.

I don’t believe in that approach, that mindset.

Do your testing and assessment with the objective of confirming everything is ok, rather than to find fault.

Recognize that success is a shared success – the success of the organization.

Here’s Mike’s list. I have highlighted my favorites.

  • We want to make things better.
  • We want to make our departments better.
  • We want to make our profession better.
  • We want to make our organizations better.
  • We want to make ourselves better, as well as those around us.
  • We are partners with the organization and the people who work within that organization.
  • No matter how much we are battled, beaten, and bruised, we still recognize that we are partners.
  • We have our egos, and we succumb to them, push them aside, or use them to become better.
  • Our clients have their egos, and we bow to them, fight them, or use them to understand how we can better work with those clients.
  • We take an active role in the success of our organizations, not sitting back and letting events happen around and to us.
  • We have a unique blend of skills, access, and opportunities.
  • We grab our opportunities, allowing us to place ourselves into the consciousness of those with whom we work.
  • We don’t have all the answers.
  • Sometimes we don’t have many answers.
  • Sometimes we don’t have any answers. And there is nothing wrong with that.
  • We are salespeople.
  • We are selling ourselves.
  • We are selling assurance.
  • We are selling improvement and betterment.
  • We are not selling a report.
  • We know we can count on each other — as professionals, as businesspeople, and as human beings.
  • And, ultimately, we are nothing more than people working with people to raise the bar in everything that is done.

I expect some will say that some of these are wrong, because we must be both independent and objective.

Yes, we must be independent within the organization, and we must be objective in our assessments and reports.

But that must not interfere with our ability to help the organization succeed.

We need to have that mindset and have it drive our actions as auditors and trusted advisors to the business.

Don’t be the person who is seen as always finding fault.

I welcome your thoughts.

When management fails to implement audit recommendations

September 23, 2022 4 comments

Last year, the Independent Casino Commission (in New South Wales, Australia) appointed Adam Bell, SC (an attorney) to lead an independent inquiry into The Star Pty Ltd. (Star Entertainment Group).

The stated objective was to “assess The Star’s suitability to hold a casino licence and to examine compliance with its legal obligations. In September 2021”.

This month, the results of the review (the “Bell report”) were released by the Commission. This is part of what ABC News reported:

The Star Entertainment Group has been found unsuitable to operate its casino in Sydney after a damning inquiry into the company.

The inquiry, led by Adam Bell SC, was held earlier this year and heard allegations of money laundering, organised crime links and fraud at its casino in Pyrmont.

Philip Crawford, the NSW Independent Casino Commission chief, said the report made for “sad reading” and detailed Star’s “scant regard” for harm minimisation.

“The institutional arrogance of this company has been breathtaking,” he said.

“And their willingness to take risks in pursuit of financial goals has been appalling.

“Our major concern with regard to the Star remains its culture. There doesn’t seem to be any short-term fix.”

Mr Crawford said Star had allowed money laundering and organised crime to infiltrate the casino, and took “deliberate steps” to cover their tracks.

He said some of that conduct continued even after the public inquiry began.

“They tended to ignore the risk inherent in all of their conduct, and then they tried to hide their conduct,” he said.

“Financial goals seemed to have been the main driver of their conduct.”

Key points:

  • The report says the casino’s protections against money laundering were unsatisfactory
  • The inquiry also heard about links to organised crime and fraud
  • Philip Crawford says senior executives “didn’t have a clue” what was going on at the company

The Guardian filled in a few details in their report.

Among the management failures was one that has relevance for all organizations, especially audit and risk practitioners. This is from Inside Asian Gambling (see the highlighted section):

…the Bell Report details a wide range of reasons for finding The Star unsuitable – among them the illegal use of China UnionPay cards to fund gambling at The Star Sydney, Star’s dealing with Asian junket operator Suncity Group and the company’s response to independent audits of its anti-money laundering (AML) and counter terrorism financing (CTF) controls.

The Financial Review picked up on this in their reporting:

The Star Entertainment Group’s “clear failings” in responding to its internal auditors’ concerns are symptomatic of a wider attitude by companies to ignore or water down negative reports by these teams despite the “serious” risks of doing so, their professional body says.

A NSW regulatory inquiry into Star last week declared it unsuitable to hold a casino licence and found serious failures of corporate governance and culture at the company.

Several of these failures – such as rejecting and then trying to hide an explosive report by KPMG into issues with the company’s anti-money laundering measures, which was commissioned by its internal audit team – showed “clear failings in how the internal audit process was handled”, CEO of the Institute of Internal Auditors (IIA) Peter Jones said.

The inquiry heard that Star’s then-CEO, Matt Bekier, was “hostile” and “sulky” about the report, originally claiming it was wrong, and that its internal audit team were “put under a lot of pressure for putting up a report that the directors took such exception to”.

But commissioning the KPMG report “was in line with best practice” for internal auditors, Mr Jones said, and Star’s directors ignored it “at their peril”.

He said this attitude to internal audit was “by no means unique” to Star, however, pointing to similar failings found by financial services companies investigated by the Hayne royal commission and Crown Resorts in separate government inquiries.

A recent IIA survey found that 45 per cent of members believed their recommendations in internal audit reports were not always acted on in a timely way, while one in 10 of the professionals say they have been sanctioned after giving their employers’ management or audit committees an unfavourable report.

“Internal audit is all about contributing to and protecting organisational value … ignored recommendations have serious implications,” Mr Jones warned.

“They often flag major cultural flaws within an organisation. If systemic issues cannot be addressed, it’s a major issue for directors as they have a fiduciary responsibility for the organisation’s welfare.”

Directors ignoring internal audit recommendation may also be “found negligent and legally accountable for issues identified”, he added, suggesting that “regulators will demand retribution to save face and accountability with the public”.

“The bottom line for directors is to ignore the advice of internal auditors at their peril.”

While Star’s board eventually accepted and acted upon the KPMG recommendations, Mr Jones said that “as it was some time before this occurred, much damage had already been done”.

“In any organisation, the first line of assurance is line management; the second compliance and risk; and the third and final is internal audit,” he said.

“A commitment to robust internal audit practices is essential for any organisation that holds a position of responsibility and privilege, such as a casino.”

Accountants Daily carried further comments by Peter Jones, including:

“A commitment to robust internal audit practices is essential for any organisation that holds a position of responsibility and privilege, such as a casino.”

He said the Bell report highlighted a number of clear failings into how the internal audit process was handled within the organisation.

He said: “According to the report, the in-house internal audit team engaged KPMG to carry out an independent review of the Anti-Money Laundering and Counter-Terrorism Financing program, as part of its licence obligations.

“This is in line with our best practice recommendations and we believe was an appropriate step by the internal audit team.”

However, the failures came in senior management’s reaction.

He said: “Specifically (according to the Bell Report):

The report was not given to the Audit Committee until the day before their meeting in late May 2018.

The message from the CEO was that there were a number of problems and inaccuracies within the report.

As the Audit Committee did not have the time to thoroughly review the report at that time, management was given time to address the issues with KPMG.

KPMG was pressured by senior management to change a number of findings within the report. The internal audit team (and other management) was given a clear message “that bad news was unwelcome”.

The report was erroneously treated as legally privileged and was subsequently held back from the regulators (AUSTRAC) for around two years.”

One of the KPMG auditors, quoted in the Bell report, said Star chief executive at the time Matt Bekier was “hostile” and failed to greet the auditors or make eye contact shortly after the Audit Committee was given their findings.

“Mr Bekier was sat down, turning the pages of the report, essentially berating us for the whole entire time of that meeting,” he said.

Mr Jones said it was important to note that KPMG reviewed its report and stood by its original findings and in addition, its recommendations were all subsequently accepted by Star.

“However, as it was some time before this occurred, much damage had already been done,” Mr Jones said.

All the reporting focused on management failures.

But there were clear failures, from my reading, by internal audit.

  1. The head of Star’s internal audit team (the CAE) is the person who should be ensuring the audit committee receives any audit report promptly, whoever performs the work. Instead, the report was not given to the audit committee until the day before their meeting.
  2. It is not clear that the CAE took ownership of the audit and report.
  3. The KPMG report included audit recommendations instead of agreed action items. This leaves the audit committee guessing: do they accept the opinion of the auditors or of management?
  4. The CAE allowed the report to be issued before it was ready, before agreement had been reached with management. The reports say that the audit committee was unable to have a constructive discussion and asked KPMG and management to work it out. If the CAE knew that there was serious disagreement, especially if management tried to interfere with the integrity of the audit, he/she should have alerted the audit committee ahead of the audit committee meeting. I believe the CAE should not have allowed a dispute of this magnitude with management in front of the committee. One option would have been to tell them that KPMG had identified serious issues, but he/she was still reviewing them with the management team and the report would be issued shortly. If the report has to be issued without agreement, that should be stated front and center in the report – reluctantly.

When you issue a report with recommendations, requesting a management response, you are (IMHO) asking for trouble. It is infinitely better to sit down and talk to management, agreeing on the facts, their implications, whether anything should be done, and what actions should be taken by whom and when.

In this case, it is clear that management did not agree, only accepting the recommendations later.

I am not persuaded that the CAE made sure the disagreements were fully aired. I suspect that KPMG did their audit, wrote a report with recommendations, shared it with management, and left the scene – job anything but done.

Internal audit fails if they are unable to work with management to drive action when it is needed. Such discussions, especially listening to management, are hard and take time. They can delay the report significantly. But a report that doesn’t lead to action when it is needed has little value!

The IIA Australia executive who referred to the high percentage of recommendations not being accepted by management as a management failure is, IMHO, mistaken. It’s an internal audit failure.

We need to know how to communicate and, especially, listen.

If management doesn’t see the need to act, to accept a recommendation, it won’t get done.

We also need to be sufficiently humble and open to being shown that we are wrong. There may be mitigating factors and the risk may not be as high as we think.

Perhaps the risk is not sufficiently high that it merits the use of scarce management time and money to fix.

Perhaps there is a better solution than we were suggesting.

On the other hand, I have seen more cases than I care to mention where management did something because “the auditor told me to do it” – and what they did was not in the best interests of the organization.

Let’s discard the idea that audit reports should include recommendations.

Let’s replace it with the notion that we should add value by providing assurance and influencing appropriate change. Reports should include agreed action items instead of recommendations (even if there is also a management response).

I welcome your thoughts.

POSTSCRIPT:

This was just reported, today:

On September 13, 2022, the Central Bank of Ireland fined Danske Bank €1.82m for transaction monitoring failures in its anti-money laundering (AML) and terrorist financing systems. Pursuant to the Central Bank’s administrative sanctions procedure, Danske Bank was reprimanded by the Central Bank for multiple breaches of the Criminal Justice (Money Laundering & Terrorist Financing) Act 2010 (CJA) between 2010 and 2019.

During this time, Denmark’s Danske Bank failed to ensure its automated transaction monitoring system monitored the transactions of certain customer groups in its Dublin-based branch. This led to the exclusion of specific customer categories from the transaction monitoring process, including some customers rated by the bank as medium and high risk.

According to the enforcement action, the root cause of these failures was found in the out-of-date data filters applied within Danske’s automated transaction monitoring system, which had not been updated since being applied to the Irish branch in 2006. In failing to examine whether the data filters were appropriate within the system, Danske Bank did not consider the specific requirements of the CJA when it was brought into force in Ireland in 2010.

As a result of an internal audit in May 2015, Danske Bank became aware of the inadequacies in its transaction monitoring system and the nature of the risks it posed. However, the bank failed to notify the Irish branch of these issues and did not take appropriate action for nearly four years. Between August 31, 2015, and March 31, 2019, it is estimated that 348,321 transactions processed through the Irish branch were not monitored for money laundering and terrorist financing risk.

Share this video interview with your board and top management

September 19, 2022 9 comments

Today, I will review two very different sources for perspectives on risk management. Then I will provide a link to a paper by a law firm of relevance for board members and CROs.

The first is an interview with Robert Finocchio on ‘Risk Oversight and Assessment’. If you look at his background, you will see that he is deserving of respect. After a career with technology companies, including ten years at 3Com Corp. where he was the President of 3Com Systems, and three years as the CEO of Informix Corp., he has been a board member and chair of the audit committee for multiple companies.

The interview is from 2011, but what he has to say resonates strongly with me.

He is asked, “Managing risk, how is that best done by a board member?”

Robert says:

Whenever I think about risk or talk about risk from the context of being in the boardroom, an important first principle is that a director’s job, the board’s job, is NOT to minimize risk. The director’s job is to make sure the company takes the right risks [and] knows what risks they are taking.

I couldn’t agree more.

I talk (incessantly, perhaps) about taking the right level of the right risks. That requires knowing what they are, as well as the reasons for taking them.

Please consider sharing the video with your board and top management.

The second is a marketing piece from Wolters Kluwer, a software company. A better approach to risk management is clearly intended to lead people to consider the risk management solution[1], but they have some wisdom to share in the process.

Here are some highlights:

  • A great deal of the difficulty in managing risk has been imposed on them, but bankers have brought some of it on themselves, too. The focus at many institutions continues to be on individual sources of risk in isolation from others. Each source tends to be examined only from a narrow point of view within each department, with little regard for other risks or other functions at the bank. The result is that risk is carved into ever finer pieces. This segmented way of doing things is time consuming and unproductive, and it can generate inaccurate, inconsistent results, especially when the calculations used to arrive at them are performed on separate systems using diverse, discrepant analytical models.
  • A better approach is to conceive of risk holistically, in four dimensions. Instead of isolated islands of risk – credit risk, market risk, operational risk and so on – risk should be understood as a single phenomenon in which all types influence one another in ways that change continuously over time.
  • Assessing risk this way produces more accurate results more efficiently, and lets you derive more benefit from them because they provide a truer depiction of the real world, where relationships among critical elements are complex and ever changing and need to be considered at multiple levels of granularity, from the minute to the very large.
  • A holistic approach to risk management gives you a fuller, more meaningful understanding of your activities and your operating environment and its risks. It allows you to respond to all your priorities, from compliance and reporting to business projections, such as for capital and liquidity planning, under multiple scenarios, to making short- and long-term decisions, when your need to act quickly, decisively and correctly is greatest.
  • The reason that this may not be clear at some institutions is the continued partition of key functions into silos whose occupants focus on what is in front of them to the exclusion, sometimes, of what is all around them. Finance officers fixate on the reward part of the balance between risk and reward, risk officers on the risk part. As for compliance officers, whatever the big picture may be, they tend to be concerned mainly that their colleagues draw within the lines. These concentrations of interest are understandable; these specialists are focusing on the jobs they were hired to do. But their bosses in the C-suite, of course, are interested in achieving the right reward for the overall risk the company is taking, and how they can manage different stakeholders.
  • All in all, maintaining functional silos and their accompanying legacy systems is a waste of resources – time, money and your employees’ effort – and the results it produces, even after the checks and reconciliations, may be inaccurate and of limited value in meeting your compliance and business objectives. You may end up with little more than a collection of isolated facts and figures about various risks, with no deeper understanding of how they interact with one another – the interdependencies that supervisory authorities have asked banks to factor more into their thinking about risk – or insight into what matters most: how to optimize the balance of risk and reward, and therefore return on equity.

I can see how technology might help leaders see the (holistic) big picture. However, we must be careful not to reduce it to a single number that we compare to ‘risk appetite’.

As CEO or board member, I would like to understand all the more significant sources of risk and reward, both individually and together, to make an informed and intelligent decision.

Wolters Kluwer think they have the solution. I am sure others think they do, perhaps better.

Either way, practitioners need to stop assessing and acting on risk in a silo. They also need to make sure decision-makers have all the information they need.

The law firm of Wachtell, Lipton, Rosen & Katz recently  shared a long paper (as you would expect from a law firm) on Risk Management and the Board of Directors. While it is focused on making sure you don’t take too much risk, rather than taking the right level of the right risks to optimize performance, it has some valuable links and discussion on related legal issues.

I welcome your thoughts on any of the above.

[1] While I work from time to time with various software companies, mostly presenting on one of their webinars, I am independent and do not endorse any product from any vendor. I do not have a relationship at this time with Wolters Kluwer.

Updated Internal Audit Core Principles

September 14, 2022 26 comments

The IIA is in the process of revamping their International Professional Practices Framework (IPPF), including the Mission, Core Principles, and Standards.

I think that is an excellent move and am encouraged by what I have heard and seen of the Evolution update in progress.

There is one area where I think that we (collectively, as a form of crowdsourcing) can help. That is around the updated Core Principles (“the principles”).

I would like to share with you my thoughts to get your related comments and upgrades.

One of the criticisms of the COSO frameworks is that there are too many principles – a criticism I agree with. For example, they have many more than in the ISO 31000 risk management standard.

We should have a few principles for the IPPF’s principles.

  1. Effective internal audit in conformance with the Standards requires that all the principles are present and functioning.
  2. Present and functioning means that there are no major deficiencies in the achievement of the principle.
  3. Therefore, the only principles that should be included in the IPPF are those necessary for an effective internal audit function. A proposed principle is not relevant if it is not necessary, if internal audit can be effective in its absence.
  4. Achievement of the principles should not only be necessary for effective internal auditing, but also for the internal audit function to be a trusted partner of both management and the board.

An example of #3 is in the COSO Internal Audit Framework. One of its principles is that the board is independent of management. However, that is generally not the case for family and similar organizations. Internal control in family businesses can be effective even if the board is composed of family members.

Being a trusted partner is not absolutely necessary for an internal audit function to be effective, notably when there are problems with the culture of the organization and the leadership of the management team. But is very much a desirable attribute.

Turning our attention to the principles that should be included in the IPPF, I think more attention should have been given to updating the last three of the current Core Principles.

These are around the product of our services, that Internal Audit:

  • Provides risk-based assurance.
  • Is insightful, proactive, and future-focused.
  • Promotes organizational improvement.

I was privileged to be on the ReLook Task Force that developed them only a few years ago. We wanted them to be short and to the point, but the updated principles are more expressive. That’s probably a good move.

I would like your thoughts on these as a replacement and expansion of the principles around the valuable products of the internal audit function.

  • Provides constructive assurance, advice, and insight on what matters to the success of the organization, including the achievement of its enterprise objectives, when it is needed by management and the board.
  • Is forward-looking, focused on the effectiveness of the organization’s governance, management of risk and opportunity, and related systems of internal control in providing reasonable assurance of the organization’s current and future success.
  • Focuses on what matters to the success of the organization, the achievement of enterprise objectives, addressing both current and future risks and opportunities that might have a significant effect on its success.
  • Works with management, listening in a collaborative manner and exercising its independent, professional judgment, to promote improvement in the organization’s systems of governance, management of risk, and internal control.
  • Shares the results of its work through a combination of timely written and oral communications that are fair, balanced, concise, clear, and actionable.

What do you think?

What have I missed and how would you upgrade my ideas?

Please share here (not only on LinkedIn) so comments are in one place and can be reviewed by IIA staff.

Thank you in advance.

Life and risk management are both complicated.

September 12, 2022 3 comments

The world in which we live is turbulent, with so much happening all the time.

That applies to our business as well as our personal lives.

Naturally, we try to simplify the complex. Hard to survive otherwise!

Our brains are (at least for most of us) unable to fully grasp everything that is happening, so when we make decisions we often set aside some things and focus more on others. Maybe we then cross our fingers and hope those things we set aside don’t come back to hurt us.

Understanding risk, with all the interconnectivity and complexity it demands, so that you can make an intelligent and informed decision, is not easy.

Risk practitioners can help us see the big picture, all the things that might happen with a significant effect, both positive and negative.

But even risk practitioners simplify – whether deliberately or not. They may:

  • Forget that there is a range of potential effects of both a risk and an opportunity, each with its own likelihood. Instead, they represent the level of risk as a point.
  • Assess risks singly, ignoring the fact that multiple things can and do happen.
  • Think it’s about minimizing risk instead of taking the right level of the right risks.
  • Leave the assessment of upsides to others (undefined and often non-existent).
  • Ignore the fact that some risks can happen multiple times each year, not just once, and with different effects.
  • Ignore the risk that risk data and/or assessments may be incorrect.
  • Fail to understand and take bias into account.
  • Provide one report to management, even though different decision-makers require different data.
  • Ignore the fact that risks and their levels change frequently, yet they assess them monthly or less frequently.
  • Only consider a tiny percentage of all the risks that might have a significant effect on objectives. This is because management says they only want to review the top ten or twenty risks, or because they simply don’t have the bandwidth to do more.
  • Don’t consider whether information needed to assess and respond to new risks will be sufficiently timely (the “risk clockspeed” issue, as explained by Keith Smith).
  • Don’t give sufficient consideration to issues like the duration of any effects of an incident, or how extensive reputation damage may be.

A recent publication by software vendor Origami Risk, 2022 Mid-Year State of Risk Report, talked about both risk complexity and risk velocity. (Risk velocity is the speed of onset of a risk event, and risk clockspeed is the time that it will take to get the information you need.)

So, risk is complicated. But the human brain doesn’t always work well with complexity.

We don’t want to overcomplicate things, because:

  • The extra analysis takes time, and sometimes the information is needed at speed.
  • It may actually make the information harder to digest and apply to the situation, for example if an aggregation of multiple risks comes up with a single number or assessment.
  • Sometimes, simpler information is enough.

You also don’t want to oversimplify things, because:

  • You might miss some important information.
  • The information might be misunderstood.
  • People can get into the habit of seeking easy answers to complex situations.

Where is the balance?

I suggest that we always ask whether the decision-makers have sufficient and reliable information to make a quality and timely decision, given time, cost, and other constraints.

I also suggest that practitioners don’t fall in love with their own tools and black magic, making a simple situation complex.

I welcome your thoughts.

More talk about cybersecurity risk

September 9, 2022 1 comment

People continue to talk about cybersecurity and risk, but not always in a way that I think makes a lot of sense. Here’s a sample.

The IIA

The IIA finds a disparity between the level of risk internal auditors assign to cyber and the percentage of their audit plan allocated to addressing it. Just this week, their The Standard online newsletter advertised an upcoming conference:

Strengthen Your Cyber Risk Plan

Cybersecurity continues to be a pervasive challenge, with 85% of audit leaders in the recent 2022 Pulse of Internal Audit survey ranking it high or very high risk in their organizations. Yet it only covers 11% of audit plans. How are you managing cyber risks in your plan? We have practical implementation tools for you at our Cybersecurity Virtual Conference on October 27.

Register today.

This is nonsense. Dedicating 11% of all internal audit resources to one source of business risk (especially as so much is allocated to SOX) means that CAEs are taking it very seriously indeed! In fact, it may well have more resources allocated to it than any other source of business risk.

I’m not saying that the conference won’t be of value. I don’t know. I am saying that the conclusion drawn in the marketing and the Pulse report is misleading.

PCAOB

The PCAOB Staff recently issued an edition of Spotlight, Audit Committee Resource. It contains some useful points about the external auditor’s assessment of fraud risk (as it relates to the possibility of material misstatements of the financials). But it also suggests that the Audit Committee ask these three questions of the external auditor:

  • What is the auditor’s view on management’s cybersecurity risk assessment approach, overall cyber assessment, and conclusions?
  • Did the auditor identify and assess cybersecurity risks and evaluate potential cyber breaches within the company’s operations, which may have an effect on financial reporting? If so, what were the results of the auditor’s procedures?
  • Has the auditor changed its overall approach to addressing cybersecurity risks as a result of increased cyber threats to corporations and government agencies from external sources?

The likelihood that a breach would result in a material error in the financial statements filed with the SEC is (in almost every case) slight. Hackers don’t break in to manipulate the financials. So why should the external auditor be concerned?

By all means they should perform a risk assessment for SOX (I like using the IIA’s GAIT Methodology), but the real risk from a breach is operational, not financial reporting.

If I was on the Audit Committee, I would want the external auditor to focus on the real sources of risk to the financial statements rather than waste their time and my money.

There are better ways to spend money, such as on cyber defenses, than on encouraging the external auditor to believe that cyber is an area of risk to the financial statements – or pretending that they have the competence to assess how management assesses the business risk from cyber breaches.

Deloitte

Writing last month in the Wall Street Journal, Deloitte had better advice on cyber for boards. They had a good summary of the SEC’s cybersecurity proposal:

cybersecurity proposal by the Securities and Exchange Commission (SEC) in March has sparked increased discussions about cyber risk in corporate boardrooms. Boards at many companies are asking what measures they should consider taking to help improve governance and risk management ahead of the new SEC rules.

The proposed rules aim to enhance and standardize disclosures regarding cybersecurity risk management, strategy, governance, and incident reporting. The SEC received nearly 150 comment letters on the proposal and is expected to issue final requirements later this year. If adopted as proposed, the new rules would require prompt reporting of material cybersecurity incidents and disclosures in periodic filings focused on:

  • Policies and procedures to identify and manage cybersecurity risks
  • Management’s role in implementing cybersecurity policies and procedures
  • Corporate directors’ cybersecurity expertise, if any, and the board’s oversight of cybersecurity risk
  • Updates about previously reported material cybersecurity incidents

Even before the proposal was issued, oversight of cybersecurity risk had become an increasing area of focus for boards. A survey by Deloitte and the Center for Audit Quality of 246 audit committee members published in January found that two-thirds of participants with oversight responsibility for cybersecurity expected to spend more time on the topic in the coming year. In addition, 62% identified cybersecurity as one of the company’s top risks to focus on in 2022.

Intelligently, they did not mention financial reporting in their list of risks and threats:

The list of threats includes theft of information, disruption of functions, ransomware demands, destruction of hardware and software, and corruption of data.

The financial risks that can stem from loss of confidentiality, integrity, critical business processes, and information assets can be substantial. In addition to direct costs, operational impacts such as an inability to produce goods and services, system downtime, missed opportunities, and an outsize focus on incident or breach management impacts can be significant. A company’s brand, one of its greatest assets, can be damaged significantly from the loss of customer trust that can occur with cyber incidents.

They make sense with:

Boards can consider several measures to promote an increased focus, beginning with a cyber risk assessment by business area that includes the company’s readiness for a cyber incident, the response plan, and the recovery plan. Evaluation of the organization’s cyber incident response plan is also critical at the board level, with a focus on the controls surrounding business functions and what steps will be taken in the event of an incident. The board can also set an expectation that the incident response plan has been practiced through scenario planning or wargaming exercises to improve the company’s ability to respond and recover in the event of an attack. The teams for such a review should include senior management from each line of business and corporate function.

McKinsey & Company

Also in August, the consulting firm McKinsey shared Creating a technology risk and cyber risk appetite framework.

They start with:

When it comes to technology risk and cyber risk, financial institutions are increasingly shifting toward a risk-based approach to determine their priorities for controls. Those controls should be based on their current security capabilities, the likelihood of threats, and the impact of any potential cyber breach. However, the question remains: can organizations really make strategic, objective decisions about which controls they should and should not implement, given their appetite for technology risk and cyber risk?

Their reference to a “risk-based approach” takes you to their 2019 publication, The risk-based approach to cybersecurity.

The 2022 piece asserts (my emphasis):

Risk-based management measures risk against an organization’s risk appetite to determine where further technology and cyber controls are needed. The goal is to reduce the remaining technology and cyber risks to a point the business can tolerate. To succeed, it must have clear, measurable statements on its technology risk and cyber risk appetite, defined in business terms, with clear ownership.

However much I dislike the idea of an enterprise having a single risk appetite (amount of risk), I agree that risk limits (or criteria) are useful when it comes to specific sources of business risk.

The key part of the McKinsey quote is that any criteria are “defined in business terms, with clear ownership”. They explain (my emphasis):

Many organizations find that they already have components of an optimal risk appetite framework (such as thresholds for key risk indicators) or overarching, enterprise-wide statements that present the overall appetite for risk as high, medium, or low. These organizations, however, struggle to measure their risk appetite against real-world business events and to agree on risk appetite–based thresholds for metrics.

For example, it is easy for organizations to say that they have a low appetite for cyber risk. But debate begins when they ask what constitutes such a low appetite in terms of control implementation and when the first and second lines of defense ask whether residual risk falls within or outside of that overall appetite. To manage technology risk and cyber risk effectively, organizations must lay out an objective risk appetite framework that supports business decisions on risk and uses objective metrics and reporting to achieve alignment with the risk appetite.

In other words, they point out that calling the risk appetite as “low” means nothing when it comes to decision-making.

McKinsey clarifies with (my emphasis):

An organization’s risk appetite should be measurable and aligned with business objectives. The business should set the risk appetite together with the technology teams, basing it on how much technology and data impact they would accept to achieve business objectives. Those technology teams should ask the business questions, such as how many minutes of unplanned downtime it is willing to accept for a specific business service, how much sensitive data it would accept losing to achieve its objectives, and what combination of cyber investment, cyber control, and business enablement it needs to manage cyber risk during day-to-day operations. These insights should determine the organization’s risk appetite and the associated control objectives.

Interpreting again, the level of potential service interruption that would be considered acceptable (remembering that there is a range or potential levels, each with its own likelihood) is determined based on how it might affect the achievement of business objectives.

The 2019 piece has some important statements, including (my highlights):

  • First, our perspective is that cyberrisk is “only” another kind of operational risk. That is, cyberrisk refers to the potential for business losses of all kinds—financial, reputational, operational, productivity related, and regulatory related—in the digital domain. Cyberrisk can also cause losses in the physical domain, such as damage to operational equipment. But it is important to stress that cyberrisk is a form of business risk.
  • Decisions about how best to reduce cyberrisk can be contentious. Taking into account the overall context in which the enterprise operates, leaders must decide which efforts to prioritize: Which projects could most reduce enterprise risk? What methodology should be used that will make clear to enterprise stakeholders (especially in IT) that those priorities will have the greatest risk reducing impact for the enterprise? That clarity is crucial in organizing and executing those cyber projects in a focused way.

Yes. Cyber should not be risk-assessed based on the threat to information assets, but on threats to the achievement of enterprise objectives!

Organizations succeed by achieving their objectives, not by simply avoiding harms – even harms to information assets!

Consider this statement by McKinsey:

If the objective is to reduce enterprise risk, then the efforts with the best return on investment in risk reduction should draw the most resources. This approach holds true across the full control landscape, not only for monitoring but also for privileged-access management, data-loss prevention, and so forth. All of these capabilities reduce risk somewhat and somehow, but most companies are unable to determine exactly how and by how much.

I don’t think McKinsey goes nearly far enough.

Let’s upgrade that last statement in two steps. First (with changes highlighted):

If the objective is to reduce enterprise risk, then the efforts with the best return on investment in risk reduction should draw the most resources. This approach holds true across the full control landscape, not only for monitoring but also for risks related to privileged-access management, data-loss prevention, safety, compliance, change control, supply chain, government actions, competitors, customer satisfaction, reputation, credit, cash flow, exchange rates, and so forth. All of these capabilities reduce risk somewhat and somehow, but most companies are unable to determine exactly how and by how much.

In other words, how should management and the board allocate scarce resources between all the various sources of risk to enterprise objectives?

Far too few assess cybersecurity risk and investment decisions in this way.

Let’s take it to the next level by modifying the objective as well.

If the objective is to achieve enterprise objectives, taking the right level of the right risks and opportunities, then the efforts with the best return on investment in risk reduction should draw the most resources. This approach holds true across the full control landscape, not only for monitoring but also for risks and opportunities related to the timely introduction of new products and services, the completion of major systems projects and upgrades, the hiring of new personnel, the initiation of marketing initiatives, the acquisition of other organizations, obtaining new customers, privileged-access management, data-loss prevention, safety, compliance, change control, supply chain, government actions, competitors, customer satisfaction, reputation, credit, cash flow, exchange rates, and so forth. All of these capabilities can increase the likelihood of achieving objectives somewhat and somehow, but most companies are unable to determine exactly how and by how much.

Boards and executives are in the business of running the entire business, not just technology and not just protecting the organization from the consequences of a cybersecurity breach.

The sooner everybody remembers that, including InfoSec practitioners, the sooner those organizations will start taking the right level of the right cybersecurity (and other) risks.

I welcome your thoughts.

Risk report vs. risk information

September 5, 2022 5 comments

Alexei Sidorenko has a great blog that we should all subscribe to, the Risk-Academy Blog. He describes it as “Controversial thoughts about modern day risk management in non-financial companies”.

He recently wrote “What should an awesome risk report look like?”, in which he said:

If we wanted to really make a difference to decision makers we would switch from risk reporting to risk-adjusted performance reporting instead. Risk managers always have a choice: generate own risk reports or use the outputs of risk analysis to improve existing performance and management reports instead. To me the choice is clear. Integrating risk information into existing management reporting is the future.

The first suggestion he makes is:

1. Probability of achieving a target or an objective / likelihood of success

A useful metric that risk managers should communicate to decision makers is the probability of meeting/achieving an objective or target. Think of it as achievability given the risks. If your performance report has targets or objectives, then risk managers can measure and report how achievable they are and whether they are more achievable today than last month. Norman Marks calls this likelihood of success and Tim Leech calls objective centric. I provide a step by the step guide how to do it here.  This can be represented as a single number (70% probability of achieving business plan objective) or as bands (forecasted performs falls within acceptable range). Separate likelihood of success needs to be reported for each significant objective. Archer Insight, for example, does a good job presenting risk information as probability distributions around the objective.

As you might imagine, I am pleased that Alexei has this as the first of the five items he would include in a risk-adjusted performance report.

This is what I said in Risk Management for Success:

Reporting to management and the board

In Risk Management in Plain English, I suggested a format for performance reporting (performance integrated with risk reporting). I have since reviewed this with multiple executives and boards and they liked the actionable information it provides.

Objective YTD Status Fall short Achieve target Exceed target
Revenue growth of 10%

9.85%

15% 80%

5%

EPS improvement of 5%

8.00%

10% 80%

10%

Maintain customer satisfaction levels

98.00% 8% 90%

2%

Improve market share by 5%

5.00%

20% 70%

10%

Introduce new product on time and budget

72.00% 30% 65%

5%

An executive or board discussion around a report like this will focus on the areas where the current status and/or likelihood of achieving an objective by the end of the year are unacceptable. In the example above, these are highlighted in red. There will also be discussion of those pinkish areas, where achievement is marginal.

By drilling down into those cells, management and the board can identify which risks and opportunities are drivers of the assessment[1]. They can then determine the appropriate actions to improve the likelihood of success.

For example, I can imagine a report being discussed at a weekly meeting of the CEO and his or her direct reports. Jane sees that the likelihood of achieving the revenue target is only 80%. She asks what would happen if she joined the team in a meeting with a major customer, increasing the likelihood of that deal closing. That underlying factor is adjusted and she can then see that the likelihood of hitting the 10% revenue growth number increases to 85%.

The report has not only provided actionable information but led directly to a CEO decision and action.

Note that the report also identifies where there is a possibility of exceeding targets. I would expect those to be discussed with a view to improving those possibilities as well.

One of the values of a report like this is that an executive can consider where to allocate additional resources. It not only highlights all the areas that merit attention, but also enables a comparison of their severity. Then options can be considered, including letting one objective remain at a questionable level while attention is given to another.

The smart organization will prioritize its objectives.

For example, the year before I joined one company, it was very close to bankruptcy. The CFO held cash meetings twice each day, just to make sure they could make it to the next meeting. While the ability to make their revenue and profit targets was very important, it was even more important to generate cash. They granted significant discounts and sacrificed profits to close a sale that would bring them fast funds.

Another organization may find itself in trouble with the regulators for non-compliance with, say, anti-bribery laws. It might have to sacrifice profits and market share objectives, redirecting funds planned for a marketing initiative to upgrading its ethics staffing, processes, and systems.

Alex has four other items he would include in periodic reports to management:

  • Risk-adjusted performance metrics
  • VaRs, EaRs, cVaRs
  • Limit breaches and activated stop losses
  • Transparent methodology with a back test

OK.

I suggest a principle we should follow:

Help leaders and decision-makers get the information they need.

While Alexei’s suggestions are excellent, these are from the perspective of the risk practitioner.

I am suggesting we need to look at this from the perspective of the leader and decision-maker.

Find out what they need and only then figure out what to give them!

A second principle is:

The success of any organization depends on the quality of their decisions.

Decisions should be informed and intelligent. They should be made by the right people, at the right time, with an understanding of what might happen (i.e., risk and opportunities).

Then:

Different people need different information to inform their decisions.

While I am a strong believer in managing the organization so that there is at least an acceptable likelihood of achieving enterprise objectives, there is more.

Consider the needs for risk-related information of these individuals:

  • The CEO
  • The CFO
  • The Treasurer
  • The head of Sales
  • The head of Marketing
  • The CIO
  • The COO
  • The CISO
  • The Chief Compliance Officer
  • The head of Procurement
  • The Safety Officer
  • The head of Human Resources
  • The head of Manufacturing
  • The head of Engineering
  • The head of Product Development
  • The manager of Physical Security
  • The head of Investor Relations
  • and the list goes on

Each has different decisions to make and needs different information. We can’t expect them to find all the information they need in the same report.

Yet, a poor decision by any one of them might have serious ramifications on the ability of the organization to achieve its objectives.

The risk practitioner should work to ensure each has the information they need.

There’s a difference between providing a report and providing information. For example, a CISO needs to be alerted every time there is a serious attack on the cyber defenses. A CFO needs to know as soon as there is a significant movement in exchange or interest rates. A Manufacturing executive needs to information about manufacturing or supply chain issues as soon as they occur.

Reports are, by their nature, periodic. But risk management should be a continuous activity.

In other words, we need to tie in KPI and KRI into the discussion.

In his recent posts and videos, Alexei has made the point that the most important part of risk management is the risk assessment. While that is important, and the risk practitioner can bring excellent tools and techniques to develop valuable insights, it is of little use if it is not used by the right people in their decision-making.

Each decision should have reasonable information about what might happen (i.e., risk analysis).

A final premise:

More decisions are being made every day that require an understanding of risk than the risk practitioner has resources to provide.

Where does that leave us?

We have to rely on management to collect and analyze risk information in the absence of the risk practitioner.

My advice:

  1. Work with those responsible for periodic (and hopefully continuous) performance reporting to make it risk informed. Make sure leaders understand the likelihood they will achieve their and the enterprise’s objectives. Feel free to adapt and use my suggested report format, above.
  2. Work with them and those who own each enterprise objective to develop the next level down of reporting. Take each objective and identify the related risks and opportunities, highlighting which are at acceptable levels and which are not.
  3. Talk to management to understand which of their decisions are most critical, and help them obtain the information they need.
  4. Help train management to make quality, risk-informed decisions.
  5. Allocate your time to where it will be of most value.
  6. Constantly ask if you are doing what you should be doing to help the organization succeed, which is far more than avoiding failure. Adapt.

I welcome your thoughts.

P.S. If you liked World Class Risk Management, I suggest you read the book that continues the discussion, Risk Management for Success,

[1] The left side of a bowtie or a tornado analysis may help.

Balanced and fair audit reports

September 2, 2022 2 comments

I think it’s fair to say that operating management doesn’t look forward to an internal audit report.

However they may feel about the competence and professionalism of the auditors, they know from experience that the formal reports at the end of an audit won’t make them look good.

The best they can hope for is an absence of significant ‘findings’ and an opinion that says their work is ‘adequate’.

Is that fair and balanced?

Is that an accurate representation of the quality of work that management and their team are producing?

Is that what you would like to hear from your manager in a performance review: an absence of significant issues?

I think we can and should do better.

If a department or business unit is doing well, we should say so.

If they have adopted what might be considered a best practice that could be adopted elsewhere, we should say so.

If they have made significant progress since the last audit? We should say so.

If individuals should be commended, we should do so.

Frankly, this is one of the problems with a formal, written audit report. If we were to either augment or (perhaps better) replace a written report with an in-person briefing of management, we would be far more likely to say what we are reluctant to write.

The converse is also true.

We are reluctant to include in the report (but might say in a briefing) that individuals lack experience or competence. But we should find a way to say it.

We are in love with traffic light audit reports and opinions, where the highest grade is a B+. Yet sometimes management deserves an A+!

Sarah Bareilles wrote a song, Brave, which captures what I think about this.

I welcome your comments.

Perhaps the greatest and least practiced skill for internal auditors

August 29, 2022 7 comments

Whenever I see papers or presentations by consultants on the evolution of internal auditing, usually by adopting new technologies, I am at once amused and frustrated.

What these papers ignore is that so much more can be achieved by ensuring internal auditors perform the basics well.

For example, in my experience few internal audit departments focus on the more significant risks and stop auditing issues that would never have a serious effect on enterprise objectives.

You see this when audit thought leaders and practitioners talk about agile internal auditing, where they break down a lengthy audit (perhaps multiple man-months in length) into sprints. They prioritize the sprints, auditing the more significant areas first and the less significant ones in later sprints.

The problem is that those later sprints involve auditing issues that wouldn’t rise to the high-risk level on their own.

The lengthy audit should be cut down dramatically so that it only includes in its scope those issues of significance to the success of the enterprise as a whole.

Another example is the need to sit down and have a constructive discussion with operating management when potential issues surface. The first priority should be to agree on the facts and whether there is a problem. Once that is achieved and we can agree on the significance of the problem, then we need to focus on what needs to be done (if anything). Sometimes, the risk is one that should be taken!

Auditors should stop prioritizing reporting issues and instead prioritize helping the organization succeed!

Before adopting new tools, lets optimize the ones we already have.

Let’s optimize out ability to LISTEN!

I have admired Tom Peters for decades. He challenges us with provocative statements and ideas, most of which are fundamentally accurate. I gave a copy of his book, The Pursuit of WoW!, to each of my direct reports at Tosco.

As an aside, I adopted his principles in designing a WoW! Audit Department:

WoW audit department

But let’s get back to listening, perhaps the greatest and least practiced skill.

We need to listen to each other. The CAE needs to listen to management and the board, but especially to his or her team! I don’t see that as a strength of many CAEs.

Individual internal auditors need to listen to everybody as well, especially to those they are auditing – from the department head to the most junior employee or contractor. In fact, find a way to listen to suppliers, customers, and others in the extended enterprise.

I advise everybody not to “go and talk to people”. Instead, “go and listen”. If you are talking more than 40% of the time, you are not listening enough.

Active listening, paying attention, is a rare and very hard skill to learn.

I have exchanged messages on Twitter with Tom (we follow each other). When he wrote about Managing by Walking Around, I persuaded him that we should be Managing by Listening Around.

Here is a recent short (3:37 minutes) video you should listen to, The Little Big Things: One More Way to Pursue Excellence.

I welcome your thoughts.

Testing data vs. testing controls

August 24, 2022 19 comments

In a recent post of his on LinkedIn, Joseph Kassapis wrote:

I was reading a typically excellent blog/post of Norman Marks on Control Testing (in the context of commenting on 2 reports on SOX Controls Testing), and was struck and intrigued by his insistence/emphasis on testing “Data” in the mistaken impression that this amounts to testing the Control(‘s effectiveness). He named this twice in his post as a fallacy/defect in the reports, and it instantly caught my attention, being something I always found extremely interesting and important: to what extent correct output can be taken to mean/evidence correct mechanism.

External Audit standards, as I fairly confidently recall/understand, expressly preclude this position, i.e. state that the correctness of the recorded transactions, as regards their aspects controlled by the control, can in no way and under no circumstances be taken as evidence of soundness/effectiveness of the control; and I sort of ‘resented’ this, regretted it, wished it was not there; without actually being able to really/genuinely fault it, logically; rather minding its being inconvenient, making things harder, depriving us of easy tests and forcing us to conceive harder ones, (towards the already very hard task/goal of attaining satisifaction of effective functioning of Control), easier said than done !

Nobody else seems to, elaborate either, on this very important principle. Nobody seems to take it up. Except, it seems, Norman Marks. In the sense that at least he does consider it is there, it is important, and it is grossly abused. I was badly hoping he would go on to elaborate, in this blog pot, but he didn’t.

I don’t know if he elaborated elsewhere. He can inform/refer us. Whether or not he did, in the past, I would dare invite/provoke/challenge him to do so now. With another, dedicated post. Enlightening us. As he always does.

OK, Joseph. Here we go.

I start with a premise: our objective is to obtain reasonable assurance that the controls relied upon to manage the risk (whether SOX and ICFR, or some other business risk) are (a) adequately designed and (b) operating effectively as designed.

In other words, we are performing an audit of the system of internal controls for that risk.

The situation is different if we are trying to validate that the data (or information, such as in a report) is complete and accurate.

The value of an opinion on the system of internal control is that it provides continuing assurance, while validating the data provides point in time assurance. Validating the data or the information in a report may confirm that that instance of the report is complete and accurate, but it doesn’t tell you that the next instances will be. For that, you either have to continue testing and validating each instance or rely on the system of internal controls.

The quality of assurance is different. An opinion on the system of internal controls only provides reasonable assurance that each instance is complete and accurate, whilst validating data provides more absolute assurance that the data is correct.

Now, let’s return to the challenge.

I have been leading a SOX Masters class for many years, usually multiple times each year. In that class, I ask participants:

“Has your home been burglarized in the last five years or so?

In all that time, only one person raised their hand. (Good news.)

I then ask:

“Does that prove you always closed and locked your doors and windows every time you left home?”

(I don’t even go so far as to ask whether they set the alarm.)

They smile ruefully, very much aware that they have failed to do so: their controls were not operating effectively, yet they did not have an incident (or data exception, if you like).

Consultants are pushing the notion that you can use analytics and other methods like AI and RPA to test controls.

There are very few opportunities to do so, as these techniques may provide some level of assurance that the data is free of error (if not always omissions). But they rarely provide acceptable evidence that the controls management have in place even exist, let alone are adequately designed and operating effectively.

Taking another example.

The city of San Jose, my hometown, has implemented a number of controls to limit accidents at busy intersections. They include:

  • Traffic lights
  • Lane and other street markings
  • Periodic police visits
  • Reliance on controls performed by others, such as DMV’s driver licensing controls

If you ran analytics and found that there were no accidents reported at the intersection of Stevens Creek Boulevard and Winchester Boulevard in 2022, does that prove that any of the controls were working?

No. I can tell you that there were times when the lights did not work but drivers exercised appropriate caution.

While detecting that there were incidents may indicate that controls were not working (more work needs to be done to confirm that), the lack of exceptions does not provide assurance that controls were in place, adequately designed, or operating effectively.

I hope that helps.

By the way, the intersection example illustrates another issue that many don’t understand.

The system of internal control only provides reasonable assurance. It does not provide absolute or perfect assurance.

COSO’s Internal Control Framework provides some examples of the limitations, but there is more.

When you test internal controls, you may find exceptions.

For example, you inspect the traffic lights and find that they were inoperative for a few hours on one day.

If that only happened once over a period of a year, I would call that an “isolated incident”. It is reasonable to accept the occasional breakdown.

But if it happened several times in a month, I would call it a “control breakdown”.

You can have effective internal controls despite isolated incidents, but not when there have been control breakdowns.

That is why when we find exceptions we need to expand the sample size to determine whether we have an isolated incident, which would acceptable, or a control breakdown – when we would assess that the control has failed to operate effectively as designed.

I welcome your comments.

Where do our SOX programs stand today? Two reports

August 22, 2022 1 comment

Two firms recently released reports on SOX Compliance trends: Protiviti and Deloitte.

I need to make one important point.

When I was responsible for SOX at my company, I wanted to find out what our internal SOX compliance costs were. To my surprise, more than 50% of the costs were incurred by management: supporting testing by both internal and external audit teams, maintaining the documentation, answering questions, and helping with the scoping.

The surveys on cost performed by firms like these two tend to ignore the management-related costs. Keep that in the back of your mind as we review the two reports.

Protiviti shared the results of their annual SOX surveys in Assessing SOX internal costs, hours, controls and other trends in the results of Protiviti’s 2022 Sarbanes-Oxley Compliance Survey. It has a great deal of information and is worth downloading and reading.

Protiviti’s Executive Summary includes this (with my highlights):

Escalating compliance costs, time and efforts have a silver lining: They are driving more investments in automation and technology tools that generate greater efficiencies — and potentially cost savings as well as effectiveness and coverage benefits — into the SOX compliance process. Our data indicates that technology tools currently support an average of one-fourth of SOX compliance work across all companies, and a majority of programs deploy audit management and/or GRC platforms. These results are promising: Greater use of enabling technologies can, over time, help moderate jumps in internal SOX compliance costs. That said, more progress is needed. Many programs have yet to begin using an audit management platform while most have yet to leverage more advanced technology tools in their SOX programs.

There also are opportunities to pursue procedural and structural changes in SOX compliance programs. Shared services or “centers of excellence” approaches — managed internally or by an external outsourcing partner — offer substantial opportunities for efficiency improvements, especially when it comes to the highly defined and repeatable tasks, such as gathering and organizing evidence, and control testing, that dominate SOX compliance efforts. Many of the forces driving internal SOX compliance costs and hours higher are, for the most part, beyond the control of companies. This is not the case with investments in compliance automation and broader technology enablement as well as alternative delivery models that generate greater efficiency over the long term. Internal audit and finance leaders, together with their C-suite colleagues, should avoid delaying their evaluation and pursuit of opportunities in these areas.

I have highlighted two sections:

  1. While technology can provide useful functionalities in managing a SOX compliance program, the ROI for what can be expensive software is not always clear for companies without hundreds of key controls. In addition, my experience with some of the software is that it doesn’t always support the top-down and risk-based approach explained in PCAOB and SEC guidance; it doesn’t identify significant accounts and then the key controls relied upon to prevent or detect potential material errors of omissions in those accounts.

The consulting firms preach that you can use technology for testing. However, the potential is not nearly as great as they indicate. We need to perform testing that provides reasonable assurance of the existence, design, and operation of the key controls we rely on. Most of the software tests the data, not the controls – and just because the data is clear you cannot assume that the controls are in place, adequately designed, and consistently operating as they should.

Protiviti says this later on, which is highly questionable:

Automation platforms and applications bring greater efficiency to SOX compliance activities. The deployment of process mining, advanced analytics, robotic process automation (RPA) and continuous monitoring, along with other advanced technological tools, can significantly reduce the volume of manual compliance tasks as well as retention risks associated with subjecting internal full-time staff to heavy loads of repetitive, task-driven work.

  1. These “shared service centers” for SOX testing, if outsourced, are a return to the use of expensive consulting firms for testing – not something I recommend. If they are run in-house, staffed by people who do nothing else, then they may not be in tune with the business. I would think twice (or more) before doing this. There is huge value in a SOX team that suggests better controls and process improvements in addition to testing key controls.

Protiviti tells us in the report that, on average, 41% of SOX internal costs is for outsourced resources.

On the other hand, this is correct:

A combination of internal and external factors creating volatility — technology-driven transformation and innovation, talent shortages, strategic pivots and more — is contributing to rising SOX compliance costs. More companies spend $2 million or more on compliance while fewer spend $500,000 or less. A surge in the number of smaller companies spending $2 million or more in SOX compliance costs likely reflects last year’s significant increase in initial public offerings (IPOs), driven by special purpose acquisition companies (SPACs).

The chart on page 12 of the report is very useful information. It shows the typical time taken for various activities, such as testing for operational effectiveness or adequate design of a key control. Unfortunately, Protiviti did not distinguish between manual and automated controls.

The results in one chart disappointed me: the percentage of controls where the external auditors relied on management testing. The average was just 26% and only 10% of respondents said external auditors’ reliance exceeded 50%.

Protiviti tells us:

In assessing year-over-year trends in external auditor reliance on management controls testing, percentages show a year-over-year decline — i.e., external auditors appear to be relying less on this testing.

Two points:

  1. At my company, EY told the audit committee they relied on my team for 80%. At the SOX Masters training I lead, a number of attendees have reported similar levels of reliance.
  2. It is important to recognize that the external auditors can rely entirely (with review) on management’s testing of key controls that are not high risk, but they can also reduce their work by placing partial reliance with limited reperformance.

I found it interesting that according to the survey, in the average company 50” of the key controls are automated, up from 33%.

I also found it interesting that the average company has 52 significant applications, and more than half of them are cloud applications. That seems too high.

I wonder whether they have done a good job in using the top-down and risk-based approach to identify significant applications, or whether they have included applications that are involved in financial reporting but don’t contain any automated controls or other IT-dependent controls.

I am also surprised that many companies either test key reports (IPE) on a rotational basis (which should not be allowed) or only once and then not until the report is changed – 21% rotational and 36% just once. That conflicts with my empirical experience with the number of companies who have employed a baselining or benchmarking approach.

As a reminder, except when benchmarking is used for IT-dependent controls, every SOX year has to stand on its own.

Let me make one important statement:

The best path to reducing SOX compliance costs and improving effectiveness is through application (and re-application every year) of the top-down and risk-based approach. Right-size your controls!

The Deloitte report is SOX modernization: Optimizing compliance while extracting value.

They seem to agree with my important statement, above, when they say:

A SOX program that has not been challenged in years may be stale, which could be a drain on resources and impede performance, particularly if this compliance program is treated more like a “check-the-box” activity.

Deloitte also comments, with my highlights:

Management’s responsibilities related to internal control over financial reporting is to obtain reasonable assurance over the reliability of financial reporting, not absolute assurance, and the concept of “reasonableness” is objective with a range of judgments and methodologies that could be considered appropriate. Performing an effective risk assessment can help management identify areas with risks of material misstatement within the company and determine which of those areas it should focus its efforts.

Many factors could contribute to a lagging SOX program. Over time, risks evolve, or new risks are identified, and the response may have been to design new controls without always taking into consideration if any existing controls should be modified or removed. Additionally, once risks are identified, the level of risk may not be considered, such as if it’s a lower risk or a significant risk, which could result in not spending enough time in areas of significant risk or spending too much time in areas of lower risk. Controls could also have been added to manage an issue or deficiency identified without actually addressing the root cause.

Deloitte goes on to provide good advice on the risk assessment process.

But they fail miserably by recommending testing data instead of controls:

Automated testing consists of profiling certain populations and transactions with real-time results, allowing a company to be able to test up to 100 percent of the population and potentially achieve more assurance for less time and cost.

As a reminder: the data can be 100% clean even though nobody is performing the controls. Just think about how many times you left your windows open and/or doors unlocked when you left home, and even though those controls were not operating you were not burglarized.

Deloitte makes one good point, but they don’t go far enough.

They talk about automating a current manual process. That can certainly provide both efficiency and effectiveness.

But why not go further and consider whether the process should be changed – with or without modernization. There’s little point in automating an inefficient process!

If you are responsible for your company’s SOX program, I urge you to consider my SOX Masters class (one is planned for September). You can also purchase the IIA’s Management Guide to Sarbanes-Oxley Section 404.

I welcome your comments and experiences.

If you are involved in SOX compliance, you should know about the IIA’s GAIT Methodology

August 17, 2022 1 comment

A fact: most companies have included far too many IT General Controls (ITGC) in their scope for SOX.

Why: because they have taken an approach to scoping ITGC that is disconnected from the top-down and risk-based approach used to identify key controls within business processes. The scoping of ITGC has resulted in including ITGC controls in scope where a failure would not present a reasonable possibility of a material error omission in the financial statements.

“The identification of risks and controls within IT should not be a separate evaluation. Instead, it should be an integral part of management’s top-down, risk-based approach to identifying risks and controls and in determining evidential matter necessary to support the assessment.” – SEC Interpretive Guidance

The IIA recognized that there was a need to help practitioners define the right scope of ITGC for SOX, and a team of experts (including a representative from the PCAOB) developed the GAIT Methodology.

GAIT continues the top-down and risk-based approach recommended for companies by the SEC and mandated for their auditors in the PCAOB’s Auditing Standard 2201 (formerly AS5).

“The auditor should use a top-down approach to the audit of internal control over financial reporting to select the controls to test.” – PCAOB Auditing Standard 2201

“Management should identify those risks of misstatement that could, individually or in combination with others, result in a material misstatement of the financial statements (financial reporting risks).” – SEC Interpretive guidance

“In an audit of internal control, if the auditor selects an IT-dependent control for testing, the auditor should test the IT-dependent controls and the IT controls on which the selected control relies to support a conclusion about whether those controls address the risks of material misstatement.” – PCAOB Staff Alert No. 11

“For purposes of the evaluation of ICFR, management only needs to evaluate those IT general controls that are necessary for the proper and consistent operation of other controls designed to adequately address financial reporting risks.” – SEC Interpretive Guidance

Since its publication in 2007, GAIT has been adopted with great success by hundreds of companies and accepted (even recommended) by their CPA firms.

It has helped those organizations right-size their ITGC scope for SOX. Although it is focused on getting the scope right, rather than on cutting unnecessary ITGC out of their SOX scope, companies have been able to reduce the number of ITGC key controls significantly.

15 years have passed since GAIT was published. During that time, technology has advanced and practitioners have gained far more experience in SOX compliance.

It was time to update GAIT.

That update has now been completed (with the help of an eminent review panel of practitioners and partners from independent audit and consulting firms) and the product is available for free download by visiting a dedicated page on this website.

GAIT has stood the test of time very well! This is not surprising as it continues to be used extensively.

Its principles and methods continue to apply, even as technology and its use have changed.

The updated version of GAIT, developed independently from the IIA but with their full knowledge, simplifies the text, adds real-life examples, and references relevant regulatory guidance. The IIA is focused on an update to their International Professional Practices Framework and was not able to lead or participate in the update, but it is expected they will turn to their own update in 2023.

The dedicated web page includes links to the original GAIT Methodology, as well as to the two GAIT products that followed: for general technology-related business risk (GAIT-R), and for the assessment of ITGC deficiencies for SOX.

Comments and feedback are welcome.

What if we just abandon “risk management”?

August 15, 2022 11 comments

Earlier this year, Marco Nutini asked this challenging question in a newsletter he shared on LinkedIn.

He starts with:

Calm down, I don’t want to ruin my source of daily bread, let alone create a fuss.

Several internationally recognized authors have already addressed a recurring theme in the Risk literature: if a company does not manage risks, but manages decisions, why use the term “Risk Management”?

For example, Grant Purdy and Roger Estall devoted an entire section of their book, Deciding (2020), to propose the temporary eradication of the term. Grant was a nominated expert to the working group that wrote ISO 31000 and ISO Guide 73. Both standards were inspired by AS/NZ 4360:2004, to which Grant was a key contributor. So, I guess he is in a privileged position to give his opinion.

Marco quotes Grant and Roger’s argument that the terms “risk” and therefore “risk management” have multiple meanings and that means they really have no meaning. Therefore, we should stop using the terns.

This is not a view I ascribe to, although I do dislike the four-letter word “risk” because it sparks a negative reaction from most business executives.

Instead, Marco suggests:

“…what we now call ERM (Enterprise Risk Management) is a tangle of three distinct, yet interconnected fields of knowledge, something like modes of Risk Management:

  • Strategic Assumptions Assurance: A set of tools developed to assess an organization’s chance of achieving its goals and honoring its performance forecasts. It is supposed to support the strategy execution and monitoring processes.
  • Risk-Informed Decision Making: This mode has a diffuse, broad scope. As the name implies, it aims to ensure that the organization’s decision-making processes gather and use intelligently the necessary information for decision making under uncertainty. This mode is called Sufficient Certainty by Grant Purdy and Roger Estall, also the name of their consultancy from Australia.
  • Risk Control: A mode that has a transactional and compliance scope. It seeks to design and maintain a control environment that keeps residual risks at the planned levels. It is analogous to the “routine management” of Quality. Many people think that this is what Risk Management is all about.

This resonates more with me (see my last blog post).

The first of the three seems very similar to my idea of top-down risk management, which focuses on whether there is an acceptable likelihood of achieving each of the enterprise’s objectives.

The second is what I referred to decision-based risk management.

But I see the third as a subset of the first two. Some might say that this is how an organization responds to, manages, or mitigates risk.

The problem is that it overlooks the positive aspect of risk: opportunities. We need controls to ensure that they are taken as and when appropriate.

Marco’s newsletter/LI post is quite long, and I will let you read the rest. The only comment I will make is that he makes everything seem complicated, whereas I always seek (but don’t always find) simplicity.

Please share your comments here as well as against his post.

P.S. Happy belated birthday, Marco!

Decision-based Risk Management

August 12, 2022 9 comments

WARNING: This is likely to be a controversial post!

I have been talking (OK, preaching) about the need to manage the likelihood of achieving objectives (i.e., success) rather than limiting yourself and the organization by managing or mitigating risks. You need to take risks if you ever want to achieve objectives; the key is taking the right level of the right risks. I especially dislike managing individual risks, or a silo of risks, absent the context of what we are trying to achieve as an organization.

To repeat: we need to take the right level of the right risks for success.

That’s a top-down approach to risk management.

But there is another dimension to risk management.

Both ISO 31000 and COSO ERM talk about the need for intelligent decision-making, where leaders understand:

  • Where they stand
  • Whether that is a problem
  • What might happen going forward, both risks and opportunities
  • The best path to follow, balancing or weighing risks and potential reward

I recently did a video presentation on this topic that will be shown as part of the RAW 2022 conference in a couple of months.

The idea is that if risk practitioners want to help people make informed and intelligent decisions, they must:

  • Understand what decisions (especially crucial decisions for success) are to be made, both strategic and tactical
  • Make it easy for decision-makers to find and then use the information they need about what might happen
  • Help them have all the important information they need for their decision, not just threat assessments or information from a silo perspective (like cyber, supply-chain, compliance, etc.)
  • Help them see the big picture and weigh the pros and cons of each option

Decision-makers won’t find the actionable information they need if all they have is the same huge list of risks everybody has. They need something designed to help them make the smart decisions they need to make at the speed of the business; something tailored to them and their needs.

The information must be:

  • Relevant
  • Reliable
  • Complete
  • Current
  • Timely
  • Easy to find and use

Those risk functions that have changed the name to “decision support” or similar are going to be ahead of the game in this respect.

But practitioners have to satisfy the need for both dimensions: decision-based and top-down (also known as success management or objective-centric risk management – see the work of Tim Leech).

Some might add a third dimension: bottom-up.

This is where somebody identifies a risk (or opportunity) by reading a paper, hearing from a board member of a concern, or as the result of a silo risk management function’s work.

Silos

In order to properly assess the bottom-up risk, it needs to be added to the big picture. Given all other sources of risk, how would it, affect the achievement of enterprise objectives?

For example, a board member reads an article that talks about risks to the supply chain if you are importing goods from Taiwan. (A purely hypothetical situation.) In order to assess the risk, you need to know what you might be importing from Taiwan and how any disruption might affect your revenue or other aspect of the business. It has to be put into context and considered alongside other related sources of risk.

You add it to the top-down dimension to see a revised big picture.

Big picture

In my books, I mentioned the concept of a tipping point[1]. While from a siloed perspective (in this case, supply-chain risk management) the risk may seem low and acceptable, when added to the big picture it may take the whole past the tipping point. While it was previously seen as acceptable, adding one more source of risk makes it unacceptable.

But there’s another dimension. That supply-chain risk might also potentially affect decisions, so it should be added to those pictures as well.

Yes, risk criteria (my preferred language, from ISO 31000) may exist and be used to evaluate risk. That’s OK if the criteria or risk limits are derived based on the achievement of objectives and updated as conditions change. But its not OK if they are based on risks to the silo instead of to the whole business and its success.

One word of caution.

Risk practitioners don’t have to provide all the information themselves. It’s perfectly fine, even desirable, if management is able to find and use the information they need to achieve success through informed and intelligent decision-making by themselves.

The risk practitioner, in my opinion, should be an enabler and an aide. If management doesn’t need your help, step aside – your job is done, at least for now.

But often, the information needs to be gathered from sources across the extended enterprise. It needs to be brought together to see the big picture. That can be hard when different methods are used (such as when the CISO insists on reporting risk to information assets in his silo rather than to the business objectives).

The risk officer can be the linguist and translator, the big picture painter. (They should fight for risk assessments that are apples to apples, even from diverse sources.)

Sometimes, the information may appear to be in conflict, requiring facilitation by the risk practitioner. Bring people together to resolve these conflicts, and help everybody involved.

The risk practitioner should collaborate with performance management and the finance team for management and board reporting, so they can see the big picture likelihood of achieving objectives.

In other words, there remains a role for the risk officer, but the primary role is to help management see the big pictures and make informed and intelligent decisions on the path to success.

The risk team needs to talk to and (especially) listen to leaders and decision-makers.

  • Understand their needs (and that may mean changing their perception of what they need if they are not managing the likelihood of success, or are satisfied with making decisions based on the rumbling of their gut)
  • Make sure their needs are met
  • Stay alert to changes in those needs
  • Help them (individually and together) be successful

What do you think?

[1] Made famous by Malcom Gladwell in The Tipping Point, How Little Things Can Make A Big Difference (2006)

You can’t audit this!

August 8, 2022 19 comments

I have heard that in one form or another over my career.

The first came when I was an internal audit manager for a financial institution. The senior vice president for Human Resources said she was a big supporter of internal audit, but my team and I couldn’t audit her area.

I asked why and she explained that since none of us had any experience working in HR, we didn’t have the competence (my word) to perform an audit of HR.

I was able to get her to give us a chance. We might not be experts in running HR, but we were experts in processes, risks, and controls. When I asked where she had a problem, she pointed me to one that had been troubling her for months. I had one of my team (who had recently completed a class in operational auditing) perform the audit. He soon identified the process problem to her great surprise. She was so impressed she wrote both of us a letter of commendation and took me to lunch, letting me drive her Cadillac!

Years later, when I was leading the internal audit at Tosco, one of the IT managers told me I couldn’t audit their very old financial system. It was too complicated. I had fun with that, as I was able to read the COBOL code and identify a number of their coding errors. Internal auditors can easily be underestimated.

A more serious situation arose when Tosco started trading in derivatives to hedge its commodity purchases and sales, with an occasional speculative position taken under the close supervision of the CEO.

This was a significant source of risk to the company, and I knew that none of the current staff had the necessary experience or training to audit the related processes. We could audit for compliance with policies and procedures, but we wouldn’t know whether they were the right ones for the business.

I hired an expert to lead the first audit with me as his assistant and pupil. He was a former manager of trading operations and now specialized in consulting and performing such audits or reviews. I added my audit expertise, and we got the job done. Our main issue was the need for upgraded policies and procedures, both to provide discipline over the trading and to ensure appropriate accounting. Over time, I got specialized training myself, weaned the consultant off the payroll, hired people with experience auditing trading operations, and built a strong competency within the team.

I have taken this approach many times, hiring an individual with experience in the business operation to supplement the team. For example, I did it with audits of sales contract management, procurement, the tax department, and white hat hacking. One technique used by many CAEs, including me, is to borrow  subject matter experts from the business (in a different area to ensure there are no conflicts) and use them as guest auditors, adding experience and insight to the audit team.

The most recent challenge came in the last week, when my good friend Alexei told me that internal auditors didn’t have the competency to perform an effective audit of risk management.

I disagree, but the cynical Norman wants to ask him a question first:

“Alex, how many organizations have effective risk management, what you would call RM2, leaders agree it is helping them make quality decisions and take the right level of the right risks for success?”

I think he will reply that it’s a small number.

Most organizations are managing a list of risks instead of managing the business, They fail to recognize in their program that sometimes you need to take more risk to achieve success. Instead they believe that every risk needs to be managed or mitigated.

So cynical Norman thinks that auditing risk management and reaching an opinion on its effectiveness at the great majority of organizations is very easy! It is quickly evident that risk management is a compliance activity at that organization; most if not all executives fail to see much value in it to them or the business.

The auditor should conclude that risk management is not effective in helping leaders run the business. The far more difficult question to answer is why. The auditor adds value when he or she can point to the changes necessary to bring it to an acceptable level of maturity.

In other words, it is insufficient to audit for compliance with risk management policies and procedures when those procedures are not helping the organization succeed in doing anything other than manage a list of risks.

I and many others hold the Certification in Risk Management Assurance (CRMA) from the IIA. Does that certification automatically mean that we have the experience and competence to audit risk management?

No. (I have the ability based on my experience, not because I have a CRMA). I know of several auditors (whom I will not name) who hold the CRMA but have never audited risk management and I doubt they have a sufficient understanding of effective risk management to do it well.

But that doesn’t mean it can’t be done and done well. It just takes people who appreciate what effective risk management looks like, understand the business, and can use their common sense.

If the internal audit team doesn’t have individuals with the required experience and understanding, they can bring on a consultant to help them. For example, a company could hire Alexei or one of my other friends around the world! (Although I helped one audit team with high-level advice – including to use the maturity model in Risk Management for Success – I am trying to be retired so won’t take on any projects of length).

There are other areas where an internal audit may be a challenge, even for the largest internal audit department.

Last week, I met an old friend in San Francisco. She is a CAE for who I have great respect.

I mentioned that I thought auditing ‘talent management’ (how you ensure you have the right employees to run the organization for success) is hard. She thought it was easy, as her company has many processes to address the risk/need. Her team can audit those processes.

I see it differently. When I lead my SOX Masters training, we talk about the fact that the attendees’ companies all have processes for hiring, training, performance reviews, and so on – yet none of them would want to rely on them to ensure that every control is performed by competent individuals. Rather than test controls in those processes, we rely on walkthroughs and tests of specific controls where we assess the experience and knowledge of the individuals performing the key controls.

The difficult question to answer in an audit is whether the processes implemented by the business provide reasonable assurance that its objective(s) will be achieved.

While hiring programs may provide reasonable assurance that individuals with the potential to excel are hired, when they turn out to be less than stars it is difficult to change them out. It’s a sad reality.

Talent management is also inextricably linked to the ability of management to lead and inspire excellence.

Can it be audited? I believe it can, but it’s not always that easy.

You can audit for compliance with policies and procedures. But auditing for effectiveness requires more judgment and experience.

You have to be able to assess whether those policies and procedures are the right ones, providing reasonable assurance that the related risks will be managed at an acceptable level.

This is where specialized expertise and experience comes in handy.

A similar situation arises with cybersecurity. My friend and I disagree on this as well. She is correct that there are processes and policies that we can audit against. But how can you reach an opinion as to whether the right level of security is in place for the business and its risks – especially when threats and hacker techniques are changing all the time?

With the right people and the right approach, I think you can audit pretty much everything. I was able to audit creativity in the Marketing function at one company, believe it or not.

What do you think?

More Risk Assessment Danger

August 4, 2022 15 comments

When I was setting up ERM for Business Objects S.A., I was surprised by the reaction of the General Counsel, David.

I had already met with the CEO and his other direct reports. Now David and I were meeting so I could get his insights on the more significant sources of risk to the company and its objectives.

“I’m not going to answer your questions about risk.”

I was shocked and asked him why, since both the board and his boss, the CEO, wanted this done.

Even though I told him that his insights were critical, he politely but firmly told me he would not share what he thought the likelihoods were of each of the events and situations most likely to cause a significant problem for Business Objects.

He went further, saying he would not provide any assessment of risk relating to legal actions by or against the company that would be documented by me.

David believed, with some justification, that documenting his (and the company’s) assessment of risks could itself create an unacceptable level of risk.

Why is there danger in risk assessment? (Beyond the risk of getting the risk assessment wrong, leading to bad business decisions, as discussed in my last post.)

Consider safety risk: the possibility that an individual might sustain serious harm while on our premises or when using our products. The company may publish a risk appetite statement that declares it has zero appetite or tolerance for safety risk. Yet, it continues to operate – meaning it is actually accepting some level of risk.

Now consider that management performs a risk assessment and (correctly) assesses that there is a low level of safety risk. For the sake of argument, let’s say it determines that the likelihood of loss of life is 0.5%, of serious injury 2.5%, and of minor injuries 3.75%. Relying on that, management decides not to upgrade some of their equipment using the argument that the cost would be prohibitive and the benefit (including the reduction in safety risk) minimal.

Then there is an incident with loss of life and other serious injuries to personnel, including both employees and contractors.

A lawsuit surfaces the risk assessment and management’s decision to accept the risk.

The union and the press blame the company for accepting the likelihood of death and injury for the sake of profit.

A similar situation can arise with compliance risk.

In theory, and probably in public, no company will accept any level of compliance risk.

In practice, they must if they are to be in business.

So when they decide not to hire additional compliance personnel because the cost exceeds the benefit, and they then violate data privacy laws or anti-money-laundering regulations, significant penalties and business disruption may ensue.

Taking this to a practical example, I have been working with a nonprofit that helps refugees in the Ukraine and many other nations around the world.

The chair of the audit committee would like to know what its risk appetite is, meaning the total amount of risk the organization is willing to take in pursuit of its objectives.

But how do you set an acceptable level of risk when people’s lives are at risk? It can’t be zero, because taking risk is necessary if you are going to send employees and others into a dangerous area to rescue people.

My point is this. The risk practitioner should understand where and when a formal, documented risk assessment or statement of risk appetite might be a source of risk should it become public.

I am certainly not saying that there is no need for or value in a risk assessment for compliance and safety risk.  There is value, especially when allocating resources to areas of greatest compliance risk.

What I am saying is that we have to be careful how we quantify, document, and report it. At Business Objects, I found a way to perform the analysis “at direction of counsel” to provide some level of safety.

What do you think?

Risk Assessment Danger

July 31, 2022 26 comments

Every so often, we hear about a military mission where something went wrong. The intelligence might have said, for example, that a targeted individual was thought to be in a certain location – so the military attacked that location but did not find the sought-after person.

In the same way, business leaders make decisions based (at least in part) on information about risks and opportunities.

If a risk assessment is unreliable, wrong decisions may be made with serious effects.

For example, if the risk is seen as ‘high’ that a competitor will shortly release an advanced version of a competitive product, the management team may decide to accelerate the launch of its own product even though its development team say they are not quite ready.

On the other hand, if the competitive product release risk is assessed as ‘low’, then management may wait and spend more time on product quality.

If the risk assessment is faulty and leads management to make the wrong decision, there may be severe damage.

Going to market too early with a less than perfect product can lead to customer dissatisfaction and longer-term revenue losses.

Going to market too late allows competitors to steal market share and for people to question the ability of the company to be a market-leader.

Are risk officers (CROs and their teams) confident in the risk assessments they make or facilitate?

If a risk (of any type) is assessed as, let’s say, ‘high’ (whatever that means), how confident is the CRO and/or the management team in that assessment?

Are they 100% confident? I doubt it.

How about 90% or 80%?

In fact, I doubt that many CRO’s think about the likelihood that any of the risk assessments they make or facilitate are reliable.

I believe that CROs need to understand the likelihood that each risk assessment is or is not reliable.

Related risk factors may include:

  • Cognitive bias. See previous posts: Understand your own bias as a practitioner and Are your business decisions failing because they are biased?
  • Incomplete information, including not involving all the people who have relevant information and insights
  • Information that is out of date
  • Inaccurate information, for example portraying risk as a point instead of a range
  • Hidden or difficult to find and use information. For example, I understand some organizations have a risk matrix with more than 50 columns let alone the number of rows. How can decision-makers be expected to find the nuggets of actionable information they need in such a mess of data.

Of course, many factors may lead to risk assessments that need to be taken with a grain, a pinch, or a bucket of salt.

The issue is whether the CRO understands the level of salt required. Should management make business decisions based on the available risk assessments.

If the likelihood of error in a risk assessment is unacceptable, should the decision be delayed until improvements are made – if that is even possible?

What do you think?

There are other dangers in risk assessment, which I will discuss in a later post.

Talking about Risk Governance

July 25, 2022 12 comments

My thanks to Alex Sidorenko, who recently wrote about The Directors and Chief Risk Officers Group (DCRO) on his blog in Companies need intelligent risk-taking to survive according to DCRO Institute.

I really like the shift from talking about risk management to risk-taking.

Alex says:

Avoiding risk altogether is the single surest way to fail over time, as innovation, competition, and customer lethargy will slowly eat away at the advantages you currently enjoy. Because there is plenty of evidence that organizations don’t take risk well – or at least well enough for long-run interests – we need to adopt practices that ensure our future.

The DCRO Institute [is] a collaboration among practicing board members and C-suite executives has developed an extensive program to help current and aspiring board members become comfortable with the positive governance of risk-taking. In just its first year, registrants for its programs come from more than 65 countries, and graduates of its flagship Board Members’ Course on Risk, an intensive study program, are found serving in boardrooms and C-suites on five continents.

He goes on to assert:

Boards and senior executives who embrace risk in this framework foster an environment of innovation, allowing organizations to grow at rates that allow them to escape the well-documented corporate fade in performance.

When a board changes its view of how risk is governed and taken, the transition to embracing risk carries throughout the organization to every employee, especially those that face customers. Today when most talk about risk, they still think of the fear of loss or uncertainty, especially given our current health, social, economic, and political climate. Loss and uncertainty are partially correct conceptualizations of risk, but both fall short of the approach we need to take to be our best fiduciaries.

The staged transition from the board’s embrace of risk-taking, to the C-suite’s implementation of that guidance, to the frontline employees’ management of essential risk-taking, leads us to the most crucial conceptual change of risk-taking: its impact on the trust that all capital providers and external influencers have in us. Organizations have an expressed purpose and stakeholders trust us to pursue that purpose in value-enhancing ways. That trust, in turn, makes all transactions more effortless and less expensive.

DCRO’s Guiding Principles for Board Risk Committees (published in 2018) lists seven principles:

  1. At any organization, the full board has the overall responsibility for risk governance. In many cases, the full board will benefit from the focused and specialized support of a well-structured and competent board risk committee.
  2. The focus of a board risk committee is to link the risk-taking activities of an organization with its strategic objectives. It provides the full board with the capacity to evaluate the risk management infrastructure and capabilities of the organization and to challenge the effectiveness of management’s pursuit of strategic objectives from a return-on-risk perspective.
  3. Board risk committee meeting agendas should be guided by best practices, stakeholder expectations, and regulatory requirements. Agendas should cover topics that include a review of risk culture, strategy, tolerance for loss, and both internal and external communications.
  4. Regular meetings with key executives and independent information gathering from stakeholders are both essential for the board risk committee to develop a full narrative of a company’s risk-taking activities.
  5. The board risk committee must interact with other board committees to ensure full coverage of the organization’s risk profile and the interdependencies across its risk and performance drivers.
  6. Board risk committees should be populated with Qualified Risk Directors who are competent to govern the risks to which the organization is exposed.
  7. The board risk committee should provide sufficient guidance and information to allow the full board to issue a simple-language disclosure about the organization’s risk culture and control processes. Further, and only if warranted, the full board should issue a statement that the organization’s risk philosophy, infrastructure, processes, and capital base are “fit for purpose.”

Frankly, the only one that resonates with me is the second. The rest are ho-hum. The first sentence in #2 is the key:

The focus of a board risk committee is to link the risk-taking activities of an organization with its strategic objectives.

I will come back to that, but first want to share some interesting excerpts, with my highlights.

  • Formal and effective implementation of a board risk committee fosters a corporate environment in which the most value can be created from an organization’s limited risk-taking capacity. Garnering the most benefit from risk-taking requires both an understanding of downside risks, from either action or inaction, as well as an understanding of the drivers of success.
  • The full board’s responsibility for risk oversight and governance mirrors its responsibility for oversight of strategy and the evaluation of results.
  • A board risk committee helps the full board to evaluate if the organization is taking risks that will truly generate value after accounting for their costs, both actual and prospective. It further helps to focus the full board’s attention on the organization’s most critical risks and risk management capabilities.
  • Board risk committees should meet quarterly or monthly, depending on the complexity of the organization and overall cadence of full board meetings. The focus of the conversations should be on linking the organization’s risk-taking activities with its strategic objectives and evaluating whether the return on risk-being-taken is sufficient to support strategic goals.
  • At least annually, the committee should independently gather information from key stakeholders in their supply chain, from customers, line employees, securities analysts, investment bankers, and regulators. The committee may go even further and create a stakeholders committee to advise it on external perceptions of the organization for alignment with the representations made by internal sources. To be clear, this is not intended to be a two-way flow of information, but rather a way for the board risk committee to receive additional perspectives on the work of the organization.
  • The committee should always consider ways to avoid barriers that prevent risk information from reaching the highest levels of an organization. Regular meetings with randomly selected line employees from key business and operational units may provide additional perspective on emerging risk or cultural issues that have not yet garnered the attention of senior management or that may contradict the representations they are making to the committee. These types of conversations can also help to identify obstacles to the free flow of critical information to the board.

The last two bullet points are controversial, at least in my opinion.

The idea that the members of the board committee should meet with “randomly selected employees” and other stakeholders is a strange one. I am not persuaded that directors should do that, especially as I am not sure they will receive sufficient information from a small sample to challenge management’s position. I would prefer that management justify how they arrived at their assessments.

Another controversial suggestion relates to where there is a combined Audit and Risk Committee.

DCRO points out that there is a lot of work for such a committee. It has a full slate just on the Audit Committee side. DCRO also asserts that understanding financial reporting doesn’t mean that you understand risk and risk-taking.

So they suggest that there might be dual chairs, one for each responsibility of the committee.

I am not in favor of that, although I do agree that combining Audit and Risk may give short shrift to the oversight of risk-taking.

The same criticism applies when the Audit Committee is expected to address risk, even though it is not part of their name. In those cases, DCRO points out that attention to risk-taking is often one of the last items listed in the committee’s charter.

My personal belief is that there should be a Risk and Strategy committee.

When you have a Risk committee, it may devolve into a focus on managing and mitigating risk (a list of risks, more often than not). This is especially true when there is a separate Strategy committee.

Going back to the second DCRO principle:

The focus of a board risk committee is to link the risk-taking activities of an organization with its strategic objectives.

Isn’t this best achieved by a Risk and Strategy committee?

Whatever you believe, I think the DCRO guidance is useful and should be considered by every Risk, Audit, Audit and Risk, and Risk and Strategy committee.

What do you think?

A brave root cause analysis and how COSO might help

July 22, 2022 7 comments

I have been a big fan of the IIA’s magazine for a long time, having been both a contributor and a member of its editorial board.

A recent piece tackled a topic that I believe is important, not only for internal auditors but also for risk practitioners in an article titled, Digging Deep (available to IIA members).

The lead-in paragraph says:

Using COSO-based root cause analysis to connect reasons for control failures with internal control principles can help identify weaknesses across the organization.

Now I’m not sure the author understands that root cause analysis has nothing whatsoever to do with the COSO Internal Control Framework.

However, that COSO framework’s principles can point to some areas, such as competency and information, that can help understand the true root cause of an internal control failure – so the author just got the wording wrong.

She says this well:

Conducting a root cause analysis is a way internal audit can add value to the organization by looking beyond identified symptoms of internal control weaknesses to the underlying reasons for why they exist. Without an RCA, recommended corrective actions often fail to address the actual cause of a problem, and the issue may persist or evolve.

In fact, if the auditor doesn’t perform a root cause analysis it is highly likely that only the symptom is identified and addressed, rather than the underlying disease.

RCA should not be considered an additional step. It should be mandatory for every identified control weakness.

The author has a useful section on the different ways a root cause analysis can be performed.

  • Five Whys: Asking “why” five times to drill down to the true cause of a finding.
  • Pareto Chart: Presenting potential causes for the identified problems on a chart from the highest to the lowest frequency to focus on areas of improvement with the greatest impact.
  • Fishbone Diagram: Assessing potential causes grouped into categories (people, process/methods, equipment, materials, measurement, environment) to establish a relationship with the identified problem.
  • Scatter Plot Diagram: Testing correlation between variables by plotting potential root cause (an independent variable) against the effect (dependent variable).

I would add a caveat: whichever method you choose (I prefer the first), you have to keep inquiring until the true root cause is identified.

In other words, you may have to ask “why” six, seven, or more times until you are satisfied that the root cause has been identified, and only then can corrective actions be considered.

Consider this. An audit or review has identified that reconciliations are not being completed on time.

  1. Why? Because people are too busy.
  2. Why are they busy? They have too much work to do in other areas and the reconciliations are lower priority tasks.
  3. Why do they have too much work? People have left and not been replaced.
  4. Why have they not been replaced? The manager has not been able to fill the positions.
  5. Why hasn’t he been able to fill the positions? Candidates are asking for too much money, more than the company can offer.
  6. Why is the company not able to offer sufficient compensation? Because the Human Resources department mandates a salary and bonus range for these positions that is lower than candidates with the required experience and ability demand.
  7. Why…..?

And on it goes until the true root cause, which in this case is in a different department than the symptom, is identified.

The other three methods (Pareto chart, Fishbone diagram, and Scatter plot diagram) may not be sufficient. For example, you may identify a common point of failure for multiple control issues. But then you have to ask “why” several times to get to why that cause existed.

Where the article goes astray is in its attempt to list ‘common root causes’ for deficiencies in particular areas. If you have been able to access and read the article, you will see what I mean. We can set aside the rest of that article.

So are there common root causes?

I would start with the principle that holds true in 99.99% of cases: the root cause is people related. It may be:

  • Controls are performed by people with insufficient training, experience, or competency (addressed by a COSO principle). The author has identified competency weaknesses and lack of training as common root causes, but they are not root causes. The auditor needs to ask why these conditions exist. Why didn’t competent people get hired? Why wasn’t adequate training provided? Several more whys may be needed before the true root cause is identified.
  • Controls are performed by people who have not received the information they need to do their job well (another COSO principle). Again, the article just says the common root cause is insufficient internal communication. But why did that happen? And why, and why, and why.
  • Management is lacking in some way, whether it is in how people are directed, how they are motivated, or some other issue.

Take one example from Auditing that Matters. Loretta Forti is our heroine, conducting an audit that focused on the timeliness of approval for capital expenditures (Authorizations for Expenditures, or AFEs).

I had asked her to perform an audit of the AFE process after I discovered that expenditures with a very high ROI were taking so long to be approved that the opportunity passed!

It was relatively easy to find out how the process worked. Once a month, the division CFO gathered all the Vice Presidents and they collectively reviewed all the AFEs and the analysis prepared by Mike Passaretti and his team [the Capital Expenditure department]. They would take about half a day to discuss them and decide which they would propose should move forward and what the priority was for each.

The next meeting, typically the following day, was with the division CEO, Bob. The CFO and all the Vice Presidents would review with Bob the AFEs they believed should go forward. When he felt that the total was too high or disagreed with the VPs’ recommendations, the executives had to debate which would be approved, which might be deferred, and which would be declined. This meeting also took a half-day on average.

Because of the intense review and approval process, each executive was careful to ensure all the AFEs they proposed had complete and accurate analyses included in the package. Mike and his team were equally careful with their review and analysis. This all took time.

It was clear to Loretta, as it was to all the Vice Presidents and the CFO, that the process was too long, consumed far too much executive time, and often cost more than the spending itself (if you count the cost of the VPs’ time)!

The question was why the process was this way.

The CFO and VPs all agreed, usually with language they wouldn’t use with children around, that they hated both the all-VP meeting and the meeting with Bob. They said they didn’t have the time to spare and asked for our help to get the process – both time and cost – under control.

Loretta and I met to talk about what we were to do. Rather than share my opinion, for once I did the smart thing and asked Loretta for her opinion.

At first, she didn’t know what to say. But as she realized she could say what was on her mind, and with some gentle guidance from me, she said it: the CEO was the problem. He was the only one who wanted these long and expensive meetings. Only when he was persuaded to change his mind could it be changed.

I knew Bob quite well, having worked with him before he moved into his current position with the company. He was one of the executives with whom I met frequently to discuss the business and he had shared a number of confidences with me.

I was sure that he would listen to Loretta and had a suspicion he would find it easier to understand himself if he met one-on-one with her. Both a formal meeting with the CFO present and a larger meeting with the three of us (Bob, Loretta, and I) might make it harder for him to look in the mirror.

And so it was. I persuaded him to meet with Loretta and she, in turn, trusted me when I told her she would not only be safe but would enjoy herself.

I admit that I was a little nervous as I waited in my office for Loretta. Then she appeared in the doorway, all smiles!

She told me that the meeting went brilliantly. Bob was charming, as usual, and showed great respect for her – even though she was ‘only’ a manager. He let her explain what she had found and that the long process was preventing timely investment to seize market opportunities. In addition, not only was it consuming a lot of expensive executive time, but it was taking them away from running the business.

This was critical, explaining the issue in terms of how it affected the business and its success. Auditors who talk in their language (what I call “technobabble”), rather than the language of the executives they are attempting to inform or persuade (which is the objective of an audit report) are unlikely to succeed.

Loretta said that Bob responded with silence, clearly thinking about what she had said.

Then he shocked her by telling her that he was the problem. He recognized that his insistence on discussing and approving every AFE could not continue. Bob told Loretta she had done an excellent job and that he would like to talk to me.

When I met Bob later that week, he repeated his praise for Loretta. Then he asked for my opinion. Again I was smart and didn’t give him my opinion straight away. Instead, I asked him why he wanted to approve every AFE.

After a short hesitation, he said that perhaps he should only approve major capital expenditures instead of every one. I concurred, saying that was what I was used to and would advise.

But I kept at it. Why had he insisted on approving every AFE? This was not what he had done in his previous positions with the company, nor was it what he was used to working directly for Tom O’Malley – a consistent and effective delegator.

Then he looked again in the mirror and saw his true self.

“Norman, I can see now that I didn’t trust my direct reports enough to make these decisions!”

We talked about this for a while. Either he had the wrong people in these key positions, in which case he needed to replace them, or he needed to trust the people he had and delegate more effectively. He didn’t hesitate before saying he had excellent people; he just had to let go, take a little more risk, and trust and delegate.

For the next couple of weeks, Loretta and I had a trail of VPs visiting us to express their thanks for Loretta’s great work. Bob had changed the entire process, with new delegations of authority such that the VPs could approve most AFEs, the CFO would have to approve all over a certain value, and Bob was only involved in truly major capital expenditures.

Going back to the statement I made earlier, that PEOPLE are almost always the root cause, in one way or another, root cause analysis may surface some ugly truths.

It can take a lot of interpersonal and even political skills for the auditor (with the CAE’s active assistance) to discuss the issue and root cause with management, obtain their agreement on the facts, and work with them on the appropriate corrective action.

They are often unable or unwilling to face those facts.

Consider situations where:elephant in the room

  • A manager is a poor leader, failing to delegate, motivate, inspire, etc.
  • The employee charged with performing the control has too much work and management is unwilling to hire additional staff.
  • A manager is unable (might be incapable) to persuade more senior management that there is a need to address a risk, to hire more people, to change direction, etc.
  • People are talking in different languages, such as senior management and the cybersecurity staff.
  • The company’s systems are old and need to be replaced at a cost of tens of millions, which is not in the budget.
  • The CEO is a bully and gets his direct reports to compete instead of working together.
  • The Marketing team distrusts the people in the front lines, and therefore loses touch with the needs and wants of the customer base.
  • The manager is biased against individuals who don’t look like him or her, creating a hostile environment and failing to get the best out of employees.
  • The culture established and reinforced by management’s actions discourages creativity and risk-taking, and stifles performance.
  • Management is not trusted or respected.
  • People are motivated to achieve their personal performance goals rather than what is best for the organization.

A root cause analysis that is not afraid of identifying and reporting people failures is essential.

The COSO principles are useful, but they are insufficient. Only some of the bulleted situations above are covered by them.

I am reminded that the former CEO of GE, Jack Welch, was once asked what problems he faced every day. His answer was:

  1. People
  2. People
  3. People

They are the root of (almost every) control failure.

We need to be brave to see and help others see the true situation.

I welcome your thoughts.

The agile risk appetite

July 18, 2022 4 comments

If you have been reading this blog or my books, you know I have significant reservations about the concept of “an amount of risk” that would be acceptable in pursuit of objectives.

However, I recognize the need for limits and policies when it comes to risk-taking. They help guide decision-makers on what risks and outcomes are desirable to leaders of the organization. We could call them ‘risk criteria’ (ISO), while some refer to them as ‘risk appetites’ or ‘risk tolerances’ (COSO). I prefer to avoid those terms as they focus on ‘risk’ with the inevitable negative connotation (i.e., we must manage or mitigate risk) instead of guiding people to take the right level of the right risks in the circumstances (such as the potential for reward). Let’s use ordinary business language instead of risk technobabble.

For example, these are useful:

  • Spending approval authorities
  • Credit limits
  • Policies on the level of credit that can be given to customers, with escalation to more senior individuals or even the board as needed
  • Approval levels for capital expenditures, including reserving certain expenditures to the CEO or the board
  • Policies of who can approve journal entries, purchase orders, inventory write-offs, etc.
  • Policies with limits on the use of derivative instruments
  • Policies on commodity or currency hedging
  • …and so on

My point today is that all of these, whatever you call them, need to be “agile”.

The environment within which organizations function is volatile – as or more volatile than any prior period.

There is uncertainty about:

  • Local and global economies
  • The supply of raw materials and components
  • The speed of the supply chain
  • The availability of personnel, both in specialist positions and minimum wage jobs
  • Disruption caused by sanctions
  • Consumer confidence
  • …and more

In these times, organizations need to be agile. They need to be able to adapt intelligently and at speed, without sacrificing the long term at the altar of the short.

If policies and limits, etc. don’t change as business needs change, you are highly unlikely to be taking the right level of the right risks.

I am reminded of a real-life situation that I wrote about in World-Class Internal Auditing.

The Treasurer at Tosco was a senior member of the Finance team, highly respected by company leadership. He had been a key member of the management team during the lean years at Tosco; shortly before I joined when the company was “leaking cash”, he had led twice-daily meetings of the financial team to ensure there was sufficient cash to make it to the next day!

So it was important that we make a good impression when we performed our first audit of his area.

At the same time, he was a gruff curmudgeon (he reminded me of the late, great Alastair Sim as Scrooge in “A Christmas Carol”) that scowled every time I saw him – and other executives told me that he shared that disposition with everybody except the CFO.

So, I set the auditor, Laura Morton (now Nathlich), two tasks: the first was to perform an audit and provide an objective assessment of whether the Treasury function was meeting the needs of the corporation; the second was to get the Treasurer (Craig Deasy) to smile!

Laura exceeded my expectations (something she went on to do regularly).

As I had expected, Craig’s area was in very good shape. It reflected his personality as a disciplined, careful individual that had a deep understanding of the business and its needs.

But, Laura identified one issue that only deepened Craig’s frown.

She pointed out that the company’s investment policy limited overnight investment of cash to the safest of all investments, which had the lowest of all rates of return. While this was the policy that had been approved by the board, the level of risk being taken (clearly a very conservative one) was inconsistent with the general attitude of the company to taking risk!

The company was a significant “player” in the commodity derivatives market, not only to hedge the price it would pay for its raw materials (crude oil) and the price it would obtain for its refined products (gasoline, diesel, jet fuel, and so on), but it also had a truly speculative position. (The manager in charge of our derivatives trading desk was permitted to make speculative trades of several million dollars, subject to supervision by Pete Sutton, a Vice President. Over the years, he was consistently profitable.)

So it was taking millions of dollars of risk in the commodities market but unwilling to take any risk in its overnight investments?

Laura recommended that the investment policy be reconsidered. That was a wise move. Only management can decide how much risk it is willing to take, but we (as the independent and objective internal audit team) can challenge them when appropriate.

Craig reluctantly agreed that Laura had a point – not on technical controls philosophy but on business grounds. He discussed it with the CFO and they agreed to change the policy.

I met with Craig and Laura to review the final report before it went to the audit committee. He gave Laura a reluctant smile and acknowledged that it was a professional audit.

Since then, when I talk to groups of internal auditors about ‘world-class internal auditing’ and ‘how internal audit can add value’, I ask “Do your audit customers smile?”

But the other lesson for me was that internal auditors should not try to eliminate every risk they see.

In my early years, we would identify “findings” and assess the level of risk they presented. The level of risk (high, medium, or low was the typical scale) would drive the sense of urgency when we reported the issues and recommended corrective action by management.

This audit was one of the first where I applied the lessons I had learned in line management, that it is not about eliminating risk – it is about taking the right risk, based on understanding the potential downside, the potential upside, and the cost of any actions.

When the policy was developed, it was the right policy for those times. But times had changed, without the policy being updated.

Some will tell you that policies and other guidance should be reviewed on a regular basis. They will suggest an annual review.

That’s fine, but is it fast enough in these turbulent times?

Are we being agile if we only update policies and practices annually (if that)?

Let’s recognize that agility requires being flexible, with appropriate reviews and approvals, with our risk criteria and other guidance.

Let’s encourage everybody to challenge existing policies and procedures, drawing the attention of management to guidance that used to be but is no longer best for our business.

Don’t accept “we can’t do that because of our firm’s policy” if that is holding us back from success.

I welcome your thoughts.