Get on board and help drive transforming change!

This week, I was privileged to attend and contribute to a great conference chaired by my good friend, Gene Kim. His organization is IT Revolution.

The DevOps Enterprise Summit in Las Vegas finished today, but in the video library you can see some of the great presentations. I was only there for half a day, but heard inspiring stories from American Airlines, Mattel, and the US Navy (the videos are in the library – watch them! Mine is there too.) I see there will be a virtual conference in December.

What I witnessed was how companies were able to transform their technology and even product (toys, in the case of Mattel) development processes.

Instead of change taking months or years, new code, apps, or products were being developed and deployed in days or less.

Change like this can transform an entire company, helping it respond with agility to market changes and opportunities.

My role in the conference was to talk about how internal audit can be a partner and help drive the change – but it requires the auditor to jump on board and embrace the revolution, and management to open their eyes, ears, and arms to them.

What is DevOps? This is how Splunk describes it:

DevOps is an approach to IT delivery that combines people, practices and tools to break down silos between development and operations teams. DevOps teams accelerate the development of applications and services and, with a more responsive approach to management of the IT infrastructure, can deploy and update IT products at the speed of the modern marketplace.

DevOps bridges the gap between “dev” and “ops” — in other words, software development, where the code behind applications is created, and IT operations, where those applications are put into production, available to end users, and maintained. DevOps emerged from two earlier trends: The agile development movement and lean manufacturing principles. The former emphasizes short sprints of work and rapid iteration to create a more responsive IT development organization, and the latter minimizes waste and maximizes productivity in factories.

DevOps solves a bottleneck problem associated with agile development. If agile developers are producing new software or code updates at a higher frequency, then traditional operations teams will struggle to get the software tested and live in a timely manner, and the actual value of rapid development is consequently lost. Ultimately, while the agile movement made the design and building of software more iterative and flexible, that approach did not extend through the full software development lifecycle (SDLC) into deployment.

As a culture or philosophical approach, DevOps is dedicated to continuous improvement, collaboration and transparency. DevOps sees IT operations holistically in terms of value. Its goal is not to focus on individual work silos, but on the entire flow from initial idea to available product or functionality — optimizing everything in between, with an eye toward achieving greater business value at a faster pace. High-performing DevOps teams see not only faster code iterations and deployments, but overall shorter time to market for new ideas, fewer bugs and a more stable infrastructure.

I was introduced to DevOps ten years ago when Gene was writing The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win. One of the characters was the head of internal audit, and another was the chief information security officer. I was able to help him with some advice on the two individuals and how they would work with the others to “help the business win”.

It was amazing to hear so many technology and product executives talk about how this book, now in its third edition, had inspired them to what is a true revolution in technology development and deployment.

I told the (my guess) 1,400 participants that I almost wished I hadn’t retired. It would have been exciting to be part of this breaking all the rules, whether I was in IT, audit, or management.

So what should the role be of internal audit?

Is it to stand and watch as the transformation happens?

Is it to write a report that says IT is breaking all the rules?

Or is it to jump on board and help them break the old rules – replacing them with processes, controls, and rules more fitting for the disruptive VUCA world of today and tomorrow?

Forget about writing audit reports when there is so much change.

Be the trusted advisor, the independent friend, rather than the naysayer.

Understand the rationale for change and make sure it is done well.

Help the team with advice and insight on risk, controls, and security.

Help break down any barriers between the developers, InfoSec, and the business.

All on board!

P.S., if you IT team has not embraced DevOps, find out why not. Is it still taking months or longer to provide the business with the technologies they need?

I welcome your sharing.

Good practice guidelines for the Enterprise Risk Management function

I like this review of the guidelines published a couple of years ago by internal auditors in the Nordics.

Guest blogger Marinus de Pooter highlights some good points and areas of weakness.

====================================================================

A steering group drawn from the Institutes of Internal Auditors for the Nordic and Baltic countries issued the ‘Good practice guidelines for the Enterprise Risk Management function’ in 2020. The target group is organizations that would like to establish an ERM function or develop their existing risk management function further.

The aim of this document is to set a common benchmark and to facilitate the Internal Audit function when evaluating the effectiveness of risk management processes. When reading the Guidelines I asked myself this question: To which extent do they help a business manager to run his or her organization(al unit) better?

What I like is that they authors talk about the management of positive and negative uncertainty [p. 1]. Contrary to many approaches, according to them risk management is not only about mitigating events with undesirable consequences.

I agree that the emphasis should be on assisting decision-makers with dealing with meaningful uncertainty. Risk management’s field of expertise is in evaluating and communicating the uncertain elements so that there is a fully informed basis for taking a decision. [p. 9]

The focus on value for the stakeholders is promising, too. Through the identification and proactive evaluation of threats and opportunities an organisation can protect as well as create value for its stakeholders. [p. 18] However, the reader gets the impression that ‘value’ mainly refers to money than to the many other things in life that stakeholders attach value to, such as safety, environmental protection, social contribution, beauty, customer friendliness and so on.

The authors use the undisputed assumption that risk management is indispensable. The same goes for an independent ERM function. To ensure the operation and implementation of sound risk management in a holistic fashion it has been found necessary to have a person or function dedicated to this activity. [Executive Summary]

The Guidelines state: The organisation should appoint one person with the overall responsibility for the Enterprise Risk Management function. [p. 14] Why do organisations  need such a function to start with? Many family-owned businesses for example are pretty successful without having a risk management function. Apparently they are capable of benefiting from their opportunities and facing their threats.

The Guidelines are mainly about how to run an ERM function. Appendix 1 contains a 17-point plan for the establishment of a risk management function. It lists the typical paraphernalia such as: separate policy, risk appetite statements, implementation plan, job descriptions, risk owners, IT application, risk reports et cetera. Conventional risk management thrives in a compliance-driven context. If not mandated by regulators, would entrepreneurs, directors and managers still create all these risk management phenomena?

The focus of the Guidelines is on dealing with risk. It is not primarily focused on helping management to increase the likelihood of their success through the reconciliation of strategic and operational dilemmas. It states: Executive Management regularly reviews reports showing the development of significant risks as well as the status of actions taken to treat the risks. [p. 18] As a business manager I would rather receive reports expressing the estimated likelihoods of my team underachieving, meeting and overachieving our key performance indicators in the coming period.

The Guidelines state: The objective of ERM is to ensure the correct amount of risk exposure. [p. 2] However, there is no unit of measure to determine the ‘amount of risk’. If you try to express it in financial terms you will soon find out that what you value the most in life cannot be monetized.

ISO 31000 defines ‘risk’ as the ‘effect of uncertainty on objectives’; ‘effect’ being ‘deviation from the expected’. The Guidelines do not address the essential notion that it is all about managing the expectations of your core stakeholders. As a decision-maker you should focus on creating and protecting value for them. Life is not primarily about identifying, assessing, treating and monitoring risks. The future-proofness of your organisation is dependent on whether your core stakeholders remain satisfied with your performance.

Different stakeholders have diverging interests, needs and expectations. Hence, as a decision-maker you always have to reconcile dilemmas. The Guidelines do not address balancing the pros and cons when analysing your options and making your decisions.

According to the Guidelines: Executives should ensure that the risk management process is fully integrated across all levels of the organization and is strongly aligned with objectives, strategy and culture. [p. 3] The typical ERM pitfall is first creating a separate risk management system and then trying to squeeze all these concepts and tools into your regular business management. I don’t know any success story of this myself.

Maintaining risk lists mainly serves compliance purposes. Risk registers aren’t consulted when people have to make important decisions. Approaches for dealing with the uncertain future should start from the perspective of the decision-makers and help them to face their challenges. How can they best be supported to make balanced choices?

The Guidelines promise that ERM becomes a tool for the balanced prioritisation of resource utilization. [p. 4] Do you need separate risk management for the allocation of your scarce resources in order to able to deliver products and services that meet requirements and expectations? Looking ahead and asking questions like ‘what-if?’ and ‘what-can-happen?’ are part and parcel of just (capacity) management.

The ‘three lines of defence’ model is embraced [p. 11] The Guidelines address the common issues of the delineation of the responsibilities of ERM versus other support functions such as Compliance and Internal Audit. [p. 1] The document also talks about the application of a holistic perspective and about avoiding ‘silo’ thinking’. [p. 3] The reality is that the ‘three lines’ model causes lots of fuss about who is part of which line. And particularly about what these colleagues are supposed to do and to refrain from doing.

Does the 2^nd line comprise of all business enabling functions or only of those that control and monitor risks (risk oversight)? Are the support functions primarily advisors, policy makers and challengers? Or are they internal inspectors, too? Do they even have the right of veto? These questions warrant a separate discussion.

The regulators in the Financial Services industry require an independent (sheriff-type) Risk Management function aimed at holding down their colleagues in commercial functions. An inspectorate rather than a decision support function. This background presumably has led to the guideline that it is a prerequisite that the function does not perform or have responsibility for operations or make decisions which affect the business operations. [p. 14] Instead of creating another Compliance or Internal Audit type function I would rather emphasize the benefits of the role of the ‘critical friend’ for decision-makers.

The Guidelines state that employees in the ERM function shall respect and contribute to the organisation’s legitimacy and ethical objectives. [p. 15] However, the ethical dimension in decision-making is not emphasized in the document. Take for example dilemmas associated with the cost implications of employee safety, environmental protection and animal welfare. In addition, the document does not underscore the importance of biases. The same goes for our serious limitations to comprehend the complexity of the future caused by too many actors and factors.

The Guidelines use a deviant meaning for risk tolerance. They state that it is more of a given based on the organisation’s financial robustness, the enforcement by authorities, or other external factors determining the impact when a risk materialises. They refer to it as the level of risk an organisation is able to absorb without significantly impacting the achievement of its strategic objectives. [p. 31] The latter resembles COSO ERM’s definition of risk capacity: The maximum amount of risk that an entity is able to absorb in the pursuit of strategy and business objectives.

The risk profile is featured in the document, too. The conventional risk diagram is presented stating: The green area defines the desired performance and given risk appetite. [p. 32] It has already been discussed in detail elsewhere that the ‘heatmap’ is a misleading tool.

The document mentions a couple of creative additions to the ever expanding risk vocabulary such as risk gaps, risk picture and risk landscape. My recommendation is to try to avoid risk jargon at all costs. The more words you use starting with ‘risk’ the more people are inclined to think that it is all about something different than ordinary management.

Appendix 1 contains an impressive list of 26 reasons for failure in the establishment of ERM. [p. 22-25] It recommends curative actions for each of these items. However, in my view the solution is not to try to fix ERM. Considered closely, it is not about managing risk, but about managing expectations.

• Risks (opportunities and threats) are not an end in themselves; they help arrive at appropriate (hard and soft) controls.
• Controls are also not an end in themselves; they help create more robust business processes.
• Processes are not an end in themselves; they help achieve objectives in a structured way.
• Objectives are not an end in themselves; they help clarify which value you need to create and protect for your core stakeholders in order to keep them satisfied.

Regardless of the sector in which your organization operates the lasting satisfaction of your core stakeholders is the pre-eminent condition for your future-proofness.

Is it worth following the Guidelines presented in the document? I welcome your thoughts.

Marinus de Pooter is owner of MdP | Management, Consulting and Training. Previously he worked as Director of Finance with Ernst & Young Global Client Consulting, as European Director Internal Audit with Office Depot and as ERM Solution Leader with EY Advisory.

Are you trusted and valued?

All of us want to be both trusted and valued by our peers and customers in operating and senior management.

How do we know whether we are? Can we trust the feedback, if any, that we get?

I have developed a tool that you can adapt and use.

It asks each individual to rate whether you exhibit the traits described in words or phrases on a scale of 1 (hardly ever) to 10 (most definitely).

I suggest getting the list to each person in a way that allows them to return their assessment anonymously. One way is to enlist a trusted partner in the Legal or HR team, one that will not share the results with anybody else. Then either you or they send the list to each person you identify and ask for replies to be sent to your partner. Your partner strips the assessment from any email it is attached to and sends you just the assessment.

Here it is.

Dear Executive

(John or Jane Doe) is asking for your help in understanding how they are perceived by their (peers/customers/clients/etc.)

Please rate whether they hardly ever (score of 1) or most definitely (score of 10) demonstrate each of these attributes.

To keep your assessment confidential, please return your assessment to (Barbara Jones) in the (Legal/HR) department. She will forward only your anonymous assessment to (John/Jane).

We would appreciate receiving your completed assessment by (before H freezes over).

Attribute	Rating
Trusted (by you)
Intelligent
Constructive
Listens
Wastes your time
Stubborn
Imaginative
Understands the business
Understands YOUR business
Practical
Flexible
Emotional
Timely
Theoretical
Professional
Delivers on commitments
Fair
A partner
Helps you be successful
Gets in the way

Feel free to tailor it and let me know whether you use it and the results you obtain.

Comments welcome.

Maximize internal audit’s impact

There’s an interesting article published by the International Federation of Accountants (IFAC), We need to Maximize Internal Audit’s Impact.

It is based on the results of a survey by the UK’s Chartered Institute of Public Finance and Accountancy (CIPFA) of internal auditors in the public sector. While it might appear that the results would only apply to public sector functions, I believe they have wider relevance.

The report said (with my emphasis and comments):

• Increasing the impact of internal audit on an organisation is essential if it is allowed to become more effective. But what do we mean by ‘impact’? We define it as ‘internal audit’s ability to support the organisation in achieving its strategic objectives and goals.’ Some of the main areas to assess whether internal audit is having an impact include good engagement with senior managers, whether internal audit’s priorities are clearly aligned with organisational ones, providing timely and meaningful assurance, the ability to challenge constructively and the freedom to be dynamic and change focus.
  • Comment: I like the emphasis on helping the organization be successful – which is far more than managing risks that have a negative effect.
• 93% of internal auditors thought that they contributed to the effective management of the organisation, while 88% of clients agreed.
• Almost three-quarters (73%) of heads of internal audit who responded believed that they acted as an independent critical friend, but only 43% of management agreed with this. Perhaps of more concern, our research identified only 35% of audit committee members thought that internal audit provided this role. Ninety one per cent of heads of internal audit said they provide advice on new systems and developments, but only 62% of managers agreed. This disparity in how internal audit’s impact is currently viewed is common across a range of different services and roles provided by internal audit – with clients consistently believing internal audit’s input is significantly less than what the heads of internal audit believe. Perhaps one of the best ways to significantly increase the impact that internal audit makes is to look to the future. We asked respondents to identify three key areas that internal audit should focus on in the future which will have the greatest impact on the organisation. Cybersecurity was the top priority, with just under 60% of respondents wanting internal audit to focus on this key strategic area in the next three years. Just over 50% identified digitisation and data use within organisations as the next most important area while 47% thought that climate change and sustainability would be important areas of focus for internal audit professionals.
  • Comment: the phrase “independent friend” is a curious one that I have not seen before. Trusted advisor is perhaps the more common version. Even so, CAEs have to do more than talk about value, they have to deliver it. Only when management sees that internal auditors are actually helping them be successful will they believe it.
• The area of internal financial risk, which internal audit has traditionally provided assurance in, such as payroll and income, are generally already well managed with little exposure to risk. So, does internal audit still have a role to play in mitigating financial risk? Thirty-five per cent of respondents said they thought financial viability was a key area for the future. This includes more strategic areas such as financial resilience and medium- and long-term financial strategies – both of which carry considerable risk to the organisation. Without seeking to influence the financial policies themselves, internal audit can provide vital independent assurance to decision makers to allow them to take on more risk and be more ambitious.
  • Comment: I would change this to include auditing cash management, including cash flow. Remember what is King!
• …internal audit should take a more strategic role in areas of future importance, which may be outside of its traditional activity, if it is to have a real impact on the organisation. Of course, audit professionals cannot be expected to become experts in areas such as cybersecurity and climate change, but they can provide independent assurance, critical analysis, strategic advice, promote transparent decision making, put in place good governance arrangements and help to mitigate risk in all of these areas.
  • Comment: you can’t be effective in auditing the business if you don’t understand it.
• Organisations must invest in continually upskilling their internal audit teams with life-long learning, continuing professional development and expose them to developing areas of strategic interest. If internal audit is to keep up with the pace of change in an organisation, then it starts with giving those teams the skills they need.
  • Comment: Conferences and seminars (in person, preferably) are valuable. May I plug my books as well?
• Internal audit managers must become better advocates for their teams, and promote the contribution of internal audit throughout the organisation. This way, managers and clients will better understand its impact.
  • Comment: the best way to sell is through performance.

There’s a great deal of value in these words. But it all starts with the internal auditors’ mindset.

Are they there to find fault, or to help the organization succeed?

If it’s the latter, then prove it!

I welcome your thoughts.

Pick audits to perform that can make a real difference

I continue to see internal audit functions performing audits that are highly unlikely to make a difference to the success of the organization.

One article I saw that was (sadly) on the IIA’s blog site asserted that risk-based auditing ensured that every entity in the audit universe is audited at least once every few years. The article thought that the level of risk determined how often the entity might be audited.

If an entity is low risk, there are always other areas that should be audited first. While an argument might be made that the longer it is before an area is audited the higher the risk, that is usually not true. If the entity represents 0.5% of the corporate revenues, it will never be a high risk.

In a presentation on agile auditing, the entirety of an entity (in this case, a process) was audited pretty much every year. Risk was used to determine which aspect of the process would be audited first. I read the same in an article by a different internal audit leader.

May I suggest two principles:

1. Only perform audits where the risk to the enterprise (not just an entity within the enterprise) is significant.
2. Focus audits on areas where, if controls are not adequately designed or operating, there would be a significant risk to the enterprise.

For example, if an audit of the management and accounting for inventory in France is identified as a high priority, focus on the product lines that make up the bulk of the inventory. If some lines only make up 2-5% of the inventory, the time it would take to audit related controls is probably not justified.

Here are some questions you can ask to test the audit plan:

1. If controls in the targeted area were to fail (recognizing that it is highly unlikely that every single control would fail), how serious would it be to the enterprise as a whole? Who would need to be involved, paying personal attention to remediation. Note that I said involved, not just informed. Receiving communications of the results is routine. Having to take action is not.
2. What could you cut out of the scope so that you are focusing only on the areas and potential issues that would matter to the enterprise?
3. Is the audit less important than other potential projects that are not in the audit plan? Note: the audit plan should be based on enterprise risk, not on the availability of competent auditors. If there’s a high-risk area and you don’t have internal staff who can perform an audit, get them.
4. Does the plan represent today’s areas of risk, and the right audits to perform?

For more on this, I refer you to my two books:

Auditing that Matters: a seminal book that CAEs have purchased for their entire team

Auditing at the Speed of Risk with an Agile, Continuous Audit Plan: this year’s follow-up book

Before I leave the topic, one word of caution for those relying on management’s risk assessment.

The ERM process probably assesses the current level of risk and assumes that controls are functioning properly.

But we know that is not always the case, and we should consider the likelihood of control failures in our own risk assessment process.

I welcome your thoughts.

CyberSecurity Survey

This is a guest post by my good friend and fellow OCEG Fellow, Brian Barnier.

====================================================================

Help us to help you. Think.Design.Cyber’s research study on the human experience and innovation in cybersecurity is waiting for you. Your 5 mins can help take cybersecurity to the next level!

Far too many brilliant, committed cyber pros are feeling burnout, stress and “hamster wheel” syndrome. Achieving cybersecurity objectives and innovating in cybersecurity are becoming increasingly complex. It doesn’t have to be this way.

We have heard you in our talks and workshops. There are specific difficulties preventing cyber pros from more easily protecting people and companies from danger, and have a better work life balance. Board members tell us they are perplexed about why cybersecurity metrics and investment are different from other business functions. CISOs and senior leaders are under pressure to meet expectations and make sure nothing happens on their watch.

To gain clarity in action, we are conducting a first of its kind survey to understand the experiences of mission-driven cybersecurity professionals across the world and to explore how cybersecurity innovation compares to other business functions.

From whatever your vantage point, you are in the powerful position to share your perspective to help others around the world. Our aspiration is to discover how to better innovate in methods to setup cyber pros for success and empower them to more easily protect people and companies. Your participation makes the findings more robust.

The survey takes only 5 mins, is anonymous (no need to sign-in to Google forms), including demographic questions needed to qualify for research/academic publishing.

Here’s the link… https://forms.gle/PP834unZnKoh6fcG8

Thank you for taking 5 minutes to help in this landmark research initiative!

The latest insights on fraud from the ACFE

The Association of Certified Fraud Examiners (ACFE) has been sharing the results of its annual survey of fraud practitioners (its members) for many years.

It has now released the 12^th edition, Occupational Fraud and Abuse 2022: A Report to the Nations.

As usual, it contains a wealth of useful information.

But it has to be put into context.

How significant is fraud risk?

The ACFE says that CFEs estimate that “organizations lose 5% of revenue to fraud every year”.

Is that credible?

Actually, it is not totally beyond the realms of the possible – although it seems high to me! In the convenience store business, we were willing to accept up to 3% of revenue in store thefts because the cost of bringing it down would be greater than the reduction in losses; but for the business as a whole, including its gas station and oil refining sides, the total losses from fraud were far less. In the technology companies where I was CAE, fraud existed (mostly revenue fraud) but was, again, far less than 3%.

Testing this further…

If we take the ACFE survey results for the Energy Industry, 97 cases were reported with a median loss of $100,000. The median across all industries was $117,000 and the average loss was $1,783,000. If we assume the same ratio between the median and the average applies to Energy, that means that the average loss per case was $147,821,368.

I am going to make the reasonable assumption that each ACFE member reported the more significant case at their company. Another assumption: total losses for each company were about $300,000.

If that represents 5% of revenue, then the average company’s revenue was about $6 billion.

But the energy sector has 105 companies (globally) with more revenue than that, including 3 with revenues greater than $400 billion, and another 12 with revenue greater than $100 billion.

I haven’t been able to find a source for the average company revenue in the sector, but I think it has to be much much more than $6 billion.

I believe 5% is high and the real number (based on my experience) is less than half of that.

What does this mean for the practitioner?

5% (or 2.5%) of revenue is a lot.

But does it represent a critical source of risk?

How worried should we be if the potential for loss from a fraud is $150,000?

How much would it cost to prevent such a loss?

I am pleased that fraud doesn’t often appear in lists of top risks. It shouldn’t.

That’s not to say it should be ignored.

But at the same time, practitioners should make sure that the level of resources they allocate to addressing fraud risk is appropriate to the level of risk.

Does your company have a stated level of tolerance for fraud, as we did on the convenience store business?

What do you think losses from fraud at your company were as a percentage of revenue?

Are practitioners doing too much or too little?

I will let you read and absorb the report as it contains a lot of information, very well presented.

I welcome your thoughts.

The Culture of the Audit Department: Cop or Consultant?

I had the privilege of working with Mike Jacka as a member of an IIA International Committee for many years. He’s not only smart, but funny – a great speaker if you can get him.

He continues to write for the IIA’s magazine and blog.

His latest is Mind of Jacka: Not Quite a Manifesto.

Mike shares an interesting list of what he calls “the more intangible, human elements that allow us to function amid the contradictions that define our profession. To quote a fairly famous document, we hold these truths to be self-evident.”

I agree with the underlying premise, that internal audit exists to help the organization and its leaders succeed. It’s not about finding fault. It’s not about making ourselves look good and boasting about how much value we deliver.

While there are times when we have to be the cop, finding serious issues that threaten the success of the organization, most of the time we should be the consultant, providing valuable assurance, advice, and insight on what matters to the success of the organization.

Some auditors prioritize finding fault. For example, I knew one CAE who told his team that an audit report with no findings was unacceptable.

These auditors will not gain the trust of management. They will not be seen as helpful, only as potential impediments that consume valuable time.

Management will hide things and certainly not invite them in. They will resist and not be interested in working together to identify the best path forward.

I don’t believe in that approach, that mindset.

Do your testing and assessment with the objective of confirming everything is ok, rather than to find fault.

Recognize that success is a shared success – the success of the organization.

Here’s Mike’s list. I have highlighted my favorites.

• We want to make things better.
• We want to make our departments better.
• We want to make our profession better.
• We want to make our organizations better.
• We want to make ourselves better, as well as those around us.
• We are partners with the organization and the people who work within that organization.
• No matter how much we are battled, beaten, and bruised, we still recognize that we are partners.
• We have our egos, and we succumb to them, push them aside, or use them to become better.
• Our clients have their egos, and we bow to them, fight them, or use them to understand how we can better work with those clients.
• We take an active role in the success of our organizations, not sitting back and letting events happen around and to us.
• We have a unique blend of skills, access, and opportunities.
• We grab our opportunities, allowing us to place ourselves into the consciousness of those with whom we work.
• We don’t have all the answers.
• Sometimes we don’t have many answers.
• Sometimes we don’t have any answers. And there is nothing wrong with that.
• We are salespeople.
• We are selling ourselves.
• We are selling assurance.
• We are selling improvement and betterment.
• We are not selling a report.
• We know we can count on each other — as professionals, as businesspeople, and as human beings.
• And, ultimately, we are nothing more than people working with people to raise the bar in everything that is done.

I expect some will say that some of these are wrong, because we must be both independent and objective.

Yes, we must be independent within the organization, and we must be objective in our assessments and reports.

But that must not interfere with our ability to help the organization succeed.

We need to have that mindset and have it drive our actions as auditors and trusted advisors to the business.

Don’t be the person who is seen as always finding fault.

I welcome your thoughts.

When management fails to implement audit recommendations

Last year, the Independent Casino Commission (in New South Wales, Australia) appointed Adam Bell, SC (an attorney) to lead an independent inquiry into The Star Pty Ltd. (Star Entertainment Group).

The stated objective was to “assess The Star’s suitability to hold a casino licence and to examine compliance with its legal obligations. In September 2021”.

This month, the results of the review (the “Bell report”) were released by the Commission. This is part of what ABC News reported:

The Star Entertainment Group has been found unsuitable to operate its casino in Sydney after a damning inquiry into the company.

The inquiry, led by Adam Bell SC, was held earlier this year and heard allegations of money laundering, organised crime links and fraud at its casino in Pyrmont.

Philip Crawford, the NSW Independent Casino Commission chief, said the report made for “sad reading” and detailed Star’s “scant regard” for harm minimisation.

“The institutional arrogance of this company has been breathtaking,” he said.

“And their willingness to take risks in pursuit of financial goals has been appalling.

“Our major concern with regard to the Star remains its culture. There doesn’t seem to be any short-term fix.”

Mr Crawford said Star had allowed money laundering and organised crime to infiltrate the casino, and took “deliberate steps” to cover their tracks.

He said some of that conduct continued even after the public inquiry began.

“They tended to ignore the risk inherent in all of their conduct, and then they tried to hide their conduct,” he said.

“Financial goals seemed to have been the main driver of their conduct.”

Key points:

• The report says the casino’s protections against money laundering were unsatisfactory
• The inquiry also heard about links to organised crime and fraud
• Philip Crawford says senior executives “didn’t have a clue” what was going on at the company

The Guardian filled in a few details in their report.

Among the management failures was one that has relevance for all organizations, especially audit and risk practitioners. This is from Inside Asian Gambling (see the highlighted section):

…the Bell Report details a wide range of reasons for finding The Star unsuitable – among them the illegal use of China UnionPay cards to fund gambling at The Star Sydney, Star’s dealing with Asian junket operator Suncity Group and the company’s response to independent audits of its anti-money laundering (AML) and counter terrorism financing (CTF) controls.

The Financial Review picked up on this in their reporting:

The Star Entertainment Group’s “clear failings” in responding to its internal auditors’ concerns are symptomatic of a wider attitude by companies to ignore or water down negative reports by these teams despite the “serious” risks of doing so, their professional body says.

A NSW regulatory inquiry into Star last week declared it unsuitable to hold a casino licence and found serious failures of corporate governance and culture at the company.

Several of these failures – such as rejecting and then trying to hide an explosive report by KPMG into issues with the company’s anti-money laundering measures, which was commissioned by its internal audit team – showed “clear failings in how the internal audit process was handled”, CEO of the Institute of Internal Auditors (IIA) Peter Jones said.

The inquiry heard that Star’s then-CEO, Matt Bekier, was “hostile” and “sulky” about the report, originally claiming it was wrong, and that its internal audit team were “put under a lot of pressure for putting up a report that the directors took such exception to”.

But commissioning the KPMG report “was in line with best practice” for internal auditors, Mr Jones said, and Star’s directors ignored it “at their peril”.

He said this attitude to internal audit was “by no means unique” to Star, however, pointing to similar failings found by financial services companies investigated by the Hayne royal commission and Crown Resorts in separate government inquiries.

A recent IIA survey found that 45 per cent of members believed their recommendations in internal audit reports were not always acted on in a timely way, while one in 10 of the professionals say they have been sanctioned after giving their employers’ management or audit committees an unfavourable report.

“Internal audit is all about contributing to and protecting organisational value … ignored recommendations have serious implications,” Mr Jones warned.

“They often flag major cultural flaws within an organisation. If systemic issues cannot be addressed, it’s a major issue for directors as they have a fiduciary responsibility for the organisation’s welfare.”

Directors ignoring internal audit recommendation may also be “found negligent and legally accountable for issues identified”, he added, suggesting that “regulators will demand retribution to save face and accountability with the public”.

“The bottom line for directors is to ignore the advice of internal auditors at their peril.”

While Star’s board eventually accepted and acted upon the KPMG recommendations, Mr Jones said that “as it was some time before this occurred, much damage had already been done”.

“In any organisation, the first line of assurance is line management; the second compliance and risk; and the third and final is internal audit,” he said.

“A commitment to robust internal audit practices is essential for any organisation that holds a position of responsibility and privilege, such as a casino.”

Accountants Daily carried further comments by Peter Jones, including:

“A commitment to robust internal audit practices is essential for any organisation that holds a position of responsibility and privilege, such as a casino.”

He said the Bell report highlighted a number of clear failings into how the internal audit process was handled within the organisation.

He said: “According to the report, the in-house internal audit team engaged KPMG to carry out an independent review of the Anti-Money Laundering and Counter-Terrorism Financing program, as part of its licence obligations.

“This is in line with our best practice recommendations and we believe was an appropriate step by the internal audit team.”

However, the failures came in senior management’s reaction.

He said: “Specifically (according to the Bell Report):

The report was not given to the Audit Committee until the day before their meeting in late May 2018.

The message from the CEO was that there were a number of problems and inaccuracies within the report.

As the Audit Committee did not have the time to thoroughly review the report at that time, management was given time to address the issues with KPMG.

KPMG was pressured by senior management to change a number of findings within the report. The internal audit team (and other management) was given a clear message “that bad news was unwelcome”.

The report was erroneously treated as legally privileged and was subsequently held back from the regulators (AUSTRAC) for around two years.”

One of the KPMG auditors, quoted in the Bell report, said Star chief executive at the time Matt Bekier was “hostile” and failed to greet the auditors or make eye contact shortly after the Audit Committee was given their findings.

“Mr Bekier was sat down, turning the pages of the report, essentially berating us for the whole entire time of that meeting,” he said.

Mr Jones said it was important to note that KPMG reviewed its report and stood by its original findings and in addition, its recommendations were all subsequently accepted by Star.

“However, as it was some time before this occurred, much damage had already been done,” Mr Jones said.

All the reporting focused on management failures.

But there were clear failures, from my reading, by internal audit.

1. The head of Star’s internal audit team (the CAE) is the person who should be ensuring the audit committee receives any audit report promptly, whoever performs the work. Instead, the report was not given to the audit committee until the day before their meeting.
2. It is not clear that the CAE took ownership of the audit and report.
3. The KPMG report included audit recommendations instead of agreed action items. This leaves the audit committee guessing: do they accept the opinion of the auditors or of management?
4. The CAE allowed the report to be issued before it was ready, before agreement had been reached with management. The reports say that the audit committee was unable to have a constructive discussion and asked KPMG and management to work it out. If the CAE knew that there was serious disagreement, especially if management tried to interfere with the integrity of the audit, he/she should have alerted the audit committee ahead of the audit committee meeting. I believe the CAE should not have allowed a dispute of this magnitude with management in front of the committee. One option would have been to tell them that KPMG had identified serious issues, but he/she was still reviewing them with the management team and the report would be issued shortly. If the report has to be issued without agreement, that should be stated front and center in the report – reluctantly.

When you issue a report with recommendations, requesting a management response, you are (IMHO) asking for trouble. It is infinitely better to sit down and talk to management, agreeing on the facts, their implications, whether anything should be done, and what actions should be taken by whom and when.

In this case, it is clear that management did not agree, only accepting the recommendations later.

I am not persuaded that the CAE made sure the disagreements were fully aired. I suspect that KPMG did their audit, wrote a report with recommendations, shared it with management, and left the scene – job anything but done.

Internal audit fails if they are unable to work with management to drive action when it is needed. Such discussions, especially listening to management, are hard and take time. They can delay the report significantly. But a report that doesn’t lead to action when it is needed has little value!

The IIA Australia executive who referred to the high percentage of recommendations not being accepted by management as a management failure is, IMHO, mistaken. It’s an internal audit failure.

We need to know how to communicate and, especially, listen.

If management doesn’t see the need to act, to accept a recommendation, it won’t get done.

We also need to be sufficiently humble and open to being shown that we are wrong. There may be mitigating factors and the risk may not be as high as we think.

Perhaps the risk is not sufficiently high that it merits the use of scarce management time and money to fix.

Perhaps there is a better solution than we were suggesting.

On the other hand, I have seen more cases than I care to mention where management did something because “the auditor told me to do it” – and what they did was not in the best interests of the organization.

Let’s discard the idea that audit reports should include recommendations.

Let’s replace it with the notion that we should add value by providing assurance and influencing appropriate change. Reports should include agreed action items instead of recommendations (even if there is also a management response).

I welcome your thoughts.

POSTSCRIPT:

This was just reported, today:

On September 13, 2022, the Central Bank of Ireland fined Danske Bank €1.82m for transaction monitoring failures in its anti-money laundering (AML) and terrorist financing systems. Pursuant to the Central Bank’s administrative sanctions procedure, Danske Bank was reprimanded by the Central Bank for multiple breaches of the Criminal Justice (Money Laundering & Terrorist Financing) Act 2010 (CJA) between 2010 and 2019.

During this time, Denmark’s Danske Bank failed to ensure its automated transaction monitoring system monitored the transactions of certain customer groups in its Dublin-based branch. This led to the exclusion of specific customer categories from the transaction monitoring process, including some customers rated by the bank as medium and high risk.

According to the enforcement action, the root cause of these failures was found in the out-of-date data filters applied within Danske’s automated transaction monitoring system, which had not been updated since being applied to the Irish branch in 2006. In failing to examine whether the data filters were appropriate within the system, Danske Bank did not consider the specific requirements of the CJA when it was brought into force in Ireland in 2010.

As a result of an internal audit in May 2015, Danske Bank became aware of the inadequacies in its transaction monitoring system and the nature of the risks it posed. However, the bank failed to notify the Irish branch of these issues and did not take appropriate action for nearly four years. Between August 31, 2015, and March 31, 2019, it is estimated that 348,321 transactions processed through the Irish branch were not monitored for money laundering and terrorist financing risk.

Share this video interview with your board and top management

Today, I will review two very different sources for perspectives on risk management. Then I will provide a link to a paper by a law firm of relevance for board members and CROs.

The first is an interview with Robert Finocchio on ‘Risk Oversight and Assessment’. If you look at his background, you will see that he is deserving of respect. After a career with technology companies, including ten years at 3Com Corp. where he was the President of 3Com Systems, and three years as the CEO of Informix Corp., he has been a board member and chair of the audit committee for multiple companies.

The interview is from 2011, but what he has to say resonates strongly with me.

He is asked, “Managing risk, how is that best done by a board member?”

Robert says:

Whenever I think about risk or talk about risk from the context of being in the boardroom, an important first principle is that a director’s job, the board’s job, is NOT to minimize risk. The director’s job is to make sure the company takes the right risks [and] knows what risks they are taking.

I couldn’t agree more.

I talk (incessantly, perhaps) about taking the right level of the right risks. That requires knowing what they are, as well as the reasons for taking them.

Please consider sharing the video with your board and top management.

The second is a marketing piece from Wolters Kluwer, a software company. A better approach to risk management is clearly intended to lead people to consider the risk management solution[1], but they have some wisdom to share in the process.

Here are some highlights:

• A great deal of the difficulty in managing risk has been imposed on them, but bankers have brought some of it on themselves, too. The focus at many institutions continues to be on individual sources of risk in isolation from others. Each source tends to be examined only from a narrow point of view within each department, with little regard for other risks or other functions at the bank. The result is that risk is carved into ever finer pieces. This segmented way of doing things is time consuming and unproductive, and it can generate inaccurate, inconsistent results, especially when the calculations used to arrive at them are performed on separate systems using diverse, discrepant analytical models.
• A better approach is to conceive of risk holistically, in four dimensions. Instead of isolated islands of risk – credit risk, market risk, operational risk and so on – risk should be understood as a single phenomenon in which all types influence one another in ways that change continuously over time.
• Assessing risk this way produces more accurate results more efficiently, and lets you derive more benefit from them because they provide a truer depiction of the real world, where relationships among critical elements are complex and ever changing and need to be considered at multiple levels of granularity, from the minute to the very large.
• A holistic approach to risk management gives you a fuller, more meaningful understanding of your activities and your operating environment and its risks. It allows you to respond to all your priorities, from compliance and reporting to business projections, such as for capital and liquidity planning, under multiple scenarios, to making short- and long-term decisions, when your need to act quickly, decisively and correctly is greatest.
• The reason that this may not be clear at some institutions is the continued partition of key functions into silos whose occupants focus on what is in front of them to the exclusion, sometimes, of what is all around them. Finance officers fixate on the reward part of the balance between risk and reward, risk officers on the risk part. As for compliance officers, whatever the big picture may be, they tend to be concerned mainly that their colleagues draw within the lines. These concentrations of interest are understandable; these specialists are focusing on the jobs they were hired to do. But their bosses in the C-suite, of course, are interested in achieving the right reward for the overall risk the company is taking, and how they can manage different stakeholders.
• All in all, maintaining functional silos and their accompanying legacy systems is a waste of resources – time, money and your employees’ effort – and the results it produces, even after the checks and reconciliations, may be inaccurate and of limited value in meeting your compliance and business objectives. You may end up with little more than a collection of isolated facts and figures about various risks, with no deeper understanding of how they interact with one another – the interdependencies that supervisory authorities have asked banks to factor more into their thinking about risk – or insight into what matters most: how to optimize the balance of risk and reward, and therefore return on equity.

I can see how technology might help leaders see the (holistic) big picture. However, we must be careful not to reduce it to a single number that we compare to ‘risk appetite’.

As CEO or board member, I would like to understand all the more significant sources of risk and reward, both individually and together, to make an informed and intelligent decision.

Wolters Kluwer think they have the solution. I am sure others think they do, perhaps better.

Either way, practitioners need to stop assessing and acting on risk in a silo. They also need to make sure decision-makers have all the information they need.

The law firm of Wachtell, Lipton, Rosen & Katz recently  shared a long paper (as you would expect from a law firm) on Risk Management and the Board of Directors. While it is focused on making sure you don’t take too much risk, rather than taking the right level of the right risks to optimize performance, it has some valuable links and discussion on related legal issues.

I welcome your thoughts on any of the above.

[1] While I work from time to time with various software companies, mostly presenting on one of their webinars, I am independent and do not endorse any product from any vendor. I do not have a relationship at this time with Wolters Kluwer.

Updated Internal Audit Core Principles

The IIA is in the process of revamping their International Professional Practices Framework (IPPF), including the Mission, Core Principles, and Standards.

I think that is an excellent move and am encouraged by what I have heard and seen of the Evolution update in progress.

There is one area where I think that we (collectively, as a form of crowdsourcing) can help. That is around the updated Core Principles (“the principles”).

I would like to share with you my thoughts to get your related comments and upgrades.

One of the criticisms of the COSO frameworks is that there are too many principles – a criticism I agree with. For example, they have many more than in the ISO 31000 risk management standard.

We should have a few principles for the IPPF’s principles.

1. Effective internal audit in conformance with the Standards requires that all the principles are present and functioning.
2. Present and functioning means that there are no major deficiencies in the achievement of the principle.
3. Therefore, the only principles that should be included in the IPPF are those necessary for an effective internal audit function. A proposed principle is not relevant if it is not necessary, if internal audit can be effective in its absence.
4. Achievement of the principles should not only be necessary for effective internal auditing, but also for the internal audit function to be a trusted partner of both management and the board.

An example of #3 is in the COSO Internal Audit Framework. One of its principles is that the board is independent of management. However, that is generally not the case for family and similar organizations. Internal control in family businesses can be effective even if the board is composed of family members.

Being a trusted partner is not absolutely necessary for an internal audit function to be effective, notably when there are problems with the culture of the organization and the leadership of the management team. But is very much a desirable attribute.

Turning our attention to the principles that should be included in the IPPF, I think more attention should have been given to updating the last three of the current Core Principles.

These are around the product of our services, that Internal Audit:

• Provides risk-based assurance.
• Is insightful, proactive, and future-focused.
• Promotes organizational improvement.

I was privileged to be on the ReLook Task Force that developed them only a few years ago. We wanted them to be short and to the point, but the updated principles are more expressive. That’s probably a good move.

I would like your thoughts on these as a replacement and expansion of the principles around the valuable products of the internal audit function.

• Provides constructive assurance, advice, and insight on what matters to the success of the organization, including the achievement of its enterprise objectives, when it is needed by management and the board.
• Is forward-looking, focused on the effectiveness of the organization’s governance, management of risk and opportunity, and related systems of internal control in providing reasonable assurance of the organization’s current and future success.
• Focuses on what matters to the success of the organization, the achievement of enterprise objectives, addressing both current and future risks and opportunities that might have a significant effect on its success.
• Works with management, listening in a collaborative manner and exercising its independent, professional judgment, to promote improvement in the organization’s systems of governance, management of risk, and internal control.
• Shares the results of its work through a combination of timely written and oral communications that are fair, balanced, concise, clear, and actionable.

What do you think?

What have I missed and how would you upgrade my ideas?

Please share here (not only on LinkedIn) so comments are in one place and can be reviewed by IIA staff.

Thank you in advance.

Life and risk management are both complicated.

The world in which we live is turbulent, with so much happening all the time.

That applies to our business as well as our personal lives.

Naturally, we try to simplify the complex. Hard to survive otherwise!

Our brains are (at least for most of us) unable to fully grasp everything that is happening, so when we make decisions we often set aside some things and focus more on others. Maybe we then cross our fingers and hope those things we set aside don’t come back to hurt us.

Understanding risk, with all the interconnectivity and complexity it demands, so that you can make an intelligent and informed decision, is not easy.

Risk practitioners can help us see the big picture, all the things that might happen with a significant effect, both positive and negative.

But even risk practitioners simplify – whether deliberately or not. They may:

• Forget that there is a range of potential effects of both a risk and an opportunity, each with its own likelihood. Instead, they represent the level of risk as a point.
• Assess risks singly, ignoring the fact that multiple things can and do happen.
• Think it’s about minimizing risk instead of taking the right level of the right risks.
• Leave the assessment of upsides to others (undefined and often non-existent).
• Ignore the fact that some risks can happen multiple times each year, not just once, and with different effects.
• Ignore the risk that risk data and/or assessments may be incorrect.
• Fail to understand and take bias into account.
• Provide one report to management, even though different decision-makers require different data.
• Ignore the fact that risks and their levels change frequently, yet they assess them monthly or less frequently.
• Only consider a tiny percentage of all the risks that might have a significant effect on objectives. This is because management says they only want to review the top ten or twenty risks, or because they simply don’t have the bandwidth to do more.
• Don’t consider whether information needed to assess and respond to new risks will be sufficiently timely (the “risk clockspeed” issue, as explained by Keith Smith).
• Don’t give sufficient consideration to issues like the duration of any effects of an incident, or how extensive reputation damage may be.

A recent publication by software vendor Origami Risk, 2022 Mid-Year State of Risk Report, talked about both risk complexity and risk velocity. (Risk velocity is the speed of onset of a risk event, and risk clockspeed is the time that it will take to get the information you need.)

So, risk is complicated. But the human brain doesn’t always work well with complexity.

We don’t want to overcomplicate things, because:

• The extra analysis takes time, and sometimes the information is needed at speed.
• It may actually make the information harder to digest and apply to the situation, for example if an aggregation of multiple risks comes up with a single number or assessment.
• Sometimes, simpler information is enough.

You also don’t want to oversimplify things, because:

• You might miss some important information.
• The information might be misunderstood.
• People can get into the habit of seeking easy answers to complex situations.

Where is the balance?

I suggest that we always ask whether the decision-makers have sufficient and reliable information to make a quality and timely decision, given time, cost, and other constraints.

I also suggest that practitioners don’t fall in love with their own tools and black magic, making a simple situation complex.

I welcome your thoughts.

More talk about cybersecurity risk

People continue to talk about cybersecurity and risk, but not always in a way that I think makes a lot of sense. Here’s a sample.

The IIA

The IIA finds a disparity between the level of risk internal auditors assign to cyber and the percentage of their audit plan allocated to addressing it. Just this week, their The Standard online newsletter advertised an upcoming conference:

Strengthen Your Cyber Risk Plan

Cybersecurity continues to be a pervasive challenge, with 85% of audit leaders in the recent 2022 Pulse of Internal Audit survey ranking it high or very high risk in their organizations. Yet it only covers 11% of audit plans. How are you managing cyber risks in your plan? We have practical implementation tools for you at our Cybersecurity Virtual Conference on October 27.

Register today.

This is nonsense. Dedicating 11% of all internal audit resources to one source of business risk (especially as so much is allocated to SOX) means that CAEs are taking it very seriously indeed! In fact, it may well have more resources allocated to it than any other source of business risk.

I’m not saying that the conference won’t be of value. I don’t know. I am saying that the conclusion drawn in the marketing and the Pulse report is misleading.

PCAOB

The PCAOB Staff recently issued an edition of Spotlight, Audit Committee Resource. It contains some useful points about the external auditor’s assessment of fraud risk (as it relates to the possibility of material misstatements of the financials). But it also suggests that the Audit Committee ask these three questions of the external auditor:

• What is the auditor’s view on management’s cybersecurity risk assessment approach, overall cyber assessment, and conclusions?
• Did the auditor identify and assess cybersecurity risks and evaluate potential cyber breaches within the company’s operations, which may have an effect on financial reporting? If so, what were the results of the auditor’s procedures?
• Has the auditor changed its overall approach to addressing cybersecurity risks as a result of increased cyber threats to corporations and government agencies from external sources?

The likelihood that a breach would result in a material error in the financial statements filed with the SEC is (in almost every case) slight. Hackers don’t break in to manipulate the financials. So why should the external auditor be concerned?

By all means they should perform a risk assessment for SOX (I like using the IIA’s GAIT Methodology), but the real risk from a breach is operational, not financial reporting.

If I was on the Audit Committee, I would want the external auditor to focus on the real sources of risk to the financial statements rather than waste their time and my money.

There are better ways to spend money, such as on cyber defenses, than on encouraging the external auditor to believe that cyber is an area of risk to the financial statements – or pretending that they have the competence to assess how management assesses the business risk from cyber breaches.

Deloitte

Writing last month in the Wall Street Journal, Deloitte had better advice on cyber for boards. They had a good summary of the SEC’s cybersecurity proposal:

A cybersecurity proposal by the Securities and Exchange Commission (SEC) in March has sparked increased discussions about cyber risk in corporate boardrooms. Boards at many companies are asking what measures they should consider taking to help improve governance and risk management ahead of the new SEC rules.

The proposed rules aim to enhance and standardize disclosures regarding cybersecurity risk management, strategy, governance, and incident reporting. The SEC received nearly 150 comment letters on the proposal and is expected to issue final requirements later this year. If adopted as proposed, the new rules would require prompt reporting of material cybersecurity incidents and disclosures in periodic filings focused on:

• Policies and procedures to identify and manage cybersecurity risks
• Management’s role in implementing cybersecurity policies and procedures
• Corporate directors’ cybersecurity expertise, if any, and the board’s oversight of cybersecurity risk
• Updates about previously reported material cybersecurity incidents

Even before the proposal was issued, oversight of cybersecurity risk had become an increasing area of focus for boards. A survey by Deloitte and the Center for Audit Quality of 246 audit committee members published in January found that two-thirds of participants with oversight responsibility for cybersecurity expected to spend more time on the topic in the coming year. In addition, 62% identified cybersecurity as one of the company’s top risks to focus on in 2022.

Intelligently, they did not mention financial reporting in their list of risks and threats:

The list of threats includes theft of information, disruption of functions, ransomware demands, destruction of hardware and software, and corruption of data.

The financial risks that can stem from loss of confidentiality, integrity, critical business processes, and information assets can be substantial. In addition to direct costs, operational impacts such as an inability to produce goods and services, system downtime, missed opportunities, and an outsize focus on incident or breach management impacts can be significant. A company’s brand, one of its greatest assets, can be damaged significantly from the loss of customer trust that can occur with cyber incidents.

They make sense with:

Boards can consider several measures to promote an increased focus, beginning with a cyber risk assessment by business area that includes the company’s readiness for a cyber incident, the response plan, and the recovery plan. Evaluation of the organization’s cyber incident response plan is also critical at the board level, with a focus on the controls surrounding business functions and what steps will be taken in the event of an incident. The board can also set an expectation that the incident response plan has been practiced through scenario planning or wargaming exercises to improve the company’s ability to respond and recover in the event of an attack. The teams for such a review should include senior management from each line of business and corporate function.

McKinsey & Company

Also in August, the consulting firm McKinsey shared Creating a technology risk and cyber risk appetite framework.

They start with:

When it comes to technology risk and cyber risk, financial institutions are increasingly shifting toward a risk-based approach to determine their priorities for controls. Those controls should be based on their current security capabilities, the likelihood of threats, and the impact of any potential cyber breach. However, the question remains: can organizations really make strategic, objective decisions about which controls they should and should not implement, given their appetite for technology risk and cyber risk?

Their reference to a “risk-based approach” takes you to their 2019 publication, The risk-based approach to cybersecurity.

The 2022 piece asserts (my emphasis):

Risk-based management measures risk against an organization’s risk appetite to determine where further technology and cyber controls are needed. The goal is to reduce the remaining technology and cyber risks to a point the business can tolerate. To succeed, it must have clear, measurable statements on its technology risk and cyber risk appetite, defined in business terms, with clear ownership.

However much I dislike the idea of an enterprise having a single risk appetite (amount of risk), I agree that risk limits (or criteria) are useful when it comes to specific sources of business risk.

The key part of the McKinsey quote is that any criteria are “defined in business terms, with clear ownership”. They explain (my emphasis):

Many organizations find that they already have components of an optimal risk appetite framework (such as thresholds for key risk indicators) or overarching, enterprise-wide statements that present the overall appetite for risk as high, medium, or low. These organizations, however, struggle to measure their risk appetite against real-world business events and to agree on risk appetite–based thresholds for metrics.

For example, it is easy for organizations to say that they have a low appetite for cyber risk. But debate begins when they ask what constitutes such a low appetite in terms of control implementation and when the first and second lines of defense ask whether residual risk falls within or outside of that overall appetite. To manage technology risk and cyber risk effectively, organizations must lay out an objective risk appetite framework that supports business decisions on risk and uses objective metrics and reporting to achieve alignment with the risk appetite.

In other words, they point out that calling the risk appetite as “low” means nothing when it comes to decision-making.

McKinsey clarifies with (my emphasis):

An organization’s risk appetite should be measurable and aligned with business objectives. The business should set the risk appetite together with the technology teams, basing it on how much technology and data impact they would accept to achieve business objectives. Those technology teams should ask the business questions, such as how many minutes of unplanned downtime it is willing to accept for a specific business service, how much sensitive data it would accept losing to achieve its objectives, and what combination of cyber investment, cyber control, and business enablement it needs to manage cyber risk during day-to-day operations. These insights should determine the organization’s risk appetite and the associated control objectives.

Interpreting again, the level of potential service interruption that would be considered acceptable (remembering that there is a range or potential levels, each with its own likelihood) is determined based on how it might affect the achievement of business objectives.

The 2019 piece has some important statements, including (my highlights):

• First, our perspective is that cyberrisk is “only” another kind of operational risk. That is, cyberrisk refers to the potential for business losses of all kinds—financial, reputational, operational, productivity related, and regulatory related—in the digital domain. Cyberrisk can also cause losses in the physical domain, such as damage to operational equipment. But it is important to stress that cyberrisk is a form of business risk.
• Decisions about how best to reduce cyberrisk can be contentious. Taking into account the overall context in which the enterprise operates, leaders must decide which efforts to prioritize: Which projects could most reduce enterprise risk? What methodology should be used that will make clear to enterprise stakeholders (especially in IT) that those priorities will have the greatest risk reducing impact for the enterprise? That clarity is crucial in organizing and executing those cyber projects in a focused way.

Yes. Cyber should not be risk-assessed based on the threat to information assets, but on threats to the achievement of enterprise objectives!

Organizations succeed by achieving their objectives, not by simply avoiding harms – even harms to information assets!

Consider this statement by McKinsey:

If the objective is to reduce enterprise risk, then the efforts with the best return on investment in risk reduction should draw the most resources. This approach holds true across the full control landscape, not only for monitoring but also for privileged-access management, data-loss prevention, and so forth. All of these capabilities reduce risk somewhat and somehow, but most companies are unable to determine exactly how and by how much.

I don’t think McKinsey goes nearly far enough.

Let’s upgrade that last statement in two steps. First (with changes highlighted):

If the objective is to reduce enterprise risk, then the efforts with the best return on investment in risk reduction should draw the most resources. This approach holds true across the full control landscape, not only for monitoring but also for risks related to privileged-access management, data-loss prevention, safety, compliance, change control, supply chain, government actions, competitors, customer satisfaction, reputation, credit, cash flow, exchange rates, and so forth. All of these capabilities reduce risk somewhat and somehow, but most companies are unable to determine exactly how and by how much.

In other words, how should management and the board allocate scarce resources between all the various sources of risk to enterprise objectives?

Far too few assess cybersecurity risk and investment decisions in this way.

Let’s take it to the next level by modifying the objective as well.

If the objective is to achieve enterprise objectives, taking the right level of the right risks and opportunities, then the efforts with the best return on investment in risk reduction should draw the most resources. This approach holds true across the full control landscape, not only for monitoring but also for risks and opportunities related to the timely introduction of new products and services, the completion of major systems projects and upgrades, the hiring of new personnel, the initiation of marketing initiatives, the acquisition of other organizations, obtaining new customers, privileged-access management, data-loss prevention, safety, compliance, change control, supply chain, government actions, competitors, customer satisfaction, reputation, credit, cash flow, exchange rates, and so forth. All of these capabilities can increase the likelihood of achieving objectives somewhat and somehow, but most companies are unable to determine exactly how and by how much.

Boards and executives are in the business of running the entire business, not just technology and not just protecting the organization from the consequences of a cybersecurity breach.

The sooner everybody remembers that, including InfoSec practitioners, the sooner those organizations will start taking the right level of the right cybersecurity (and other) risks.

I welcome your thoughts.

Risk report vs. risk information

Alexei Sidorenko has a great blog that we should all subscribe to, the Risk-Academy Blog. He describes it as “Controversial thoughts about modern day risk management in non-financial companies”.

He recently wrote “What should an awesome risk report look like?”, in which he said:

If we wanted to really make a difference to decision makers we would switch from risk reporting to risk-adjusted performance reporting instead. Risk managers always have a choice: generate own risk reports or use the outputs of risk analysis to improve existing performance and management reports instead. To me the choice is clear. Integrating risk information into existing management reporting is the future.

The first suggestion he makes is:

1. Probability of achieving a target or an objective / likelihood of success

A useful metric that risk managers should communicate to decision makers is the probability of meeting/achieving an objective or target. Think of it as achievability given the risks. If your performance report has targets or objectives, then risk managers can measure and report how achievable they are and whether they are more achievable today than last month. Norman Marks calls this likelihood of success and Tim Leech calls objective centric. I provide a step by the step guide how to do it here.  This can be represented as a single number (70% probability of achieving business plan objective) or as bands (forecasted performs falls within acceptable range). Separate likelihood of success needs to be reported for each significant objective. Archer Insight, for example, does a good job presenting risk information as probability distributions around the objective.

As you might imagine, I am pleased that Alexei has this as the first of the five items he would include in a risk-adjusted performance report.

This is what I said in Risk Management for Success:

Reporting to management and the board

In Risk Management in Plain English, I suggested a format for performance reporting (performance integrated with risk reporting). I have since reviewed this with multiple executives and boards and they liked the actionable information it provides.

Objective	YTD Status	Fall short	Achieve target	Exceed target
Revenue growth of 10%	9.85%	15%	80%	5%
EPS improvement of 5%	8.00%	10%	80%	10%
Maintain customer satisfaction levels	98.00%	8%	90%	2%
Improve market share by 5%	5.00%	20%	70%	10%
Introduce new product on time and budget	72.00%	30%	65%	5%

An executive or board discussion around a report like this will focus on the areas where the current status and/or likelihood of achieving an objective by the end of the year are unacceptable. In the example above, these are highlighted in red. There will also be discussion of those pinkish areas, where achievement is marginal.

By drilling down into those cells, management and the board can identify which risks and opportunities are drivers of the assessment[1]. They can then determine the appropriate actions to improve the likelihood of success.

For example, I can imagine a report being discussed at a weekly meeting of the CEO and his or her direct reports. Jane sees that the likelihood of achieving the revenue target is only 80%. She asks what would happen if she joined the team in a meeting with a major customer, increasing the likelihood of that deal closing. That underlying factor is adjusted and she can then see that the likelihood of hitting the 10% revenue growth number increases to 85%.

The report has not only provided actionable information but led directly to a CEO decision and action.

Note that the report also identifies where there is a possibility of exceeding targets. I would expect those to be discussed with a view to improving those possibilities as well.

One of the values of a report like this is that an executive can consider where to allocate additional resources. It not only highlights all the areas that merit attention, but also enables a comparison of their severity. Then options can be considered, including letting one objective remain at a questionable level while attention is given to another.

The smart organization will prioritize its objectives.

For example, the year before I joined one company, it was very close to bankruptcy. The CFO held cash meetings twice each day, just to make sure they could make it to the next meeting. While the ability to make their revenue and profit targets was very important, it was even more important to generate cash. They granted significant discounts and sacrificed profits to close a sale that would bring them fast funds.

Another organization may find itself in trouble with the regulators for non-compliance with, say, anti-bribery laws. It might have to sacrifice profits and market share objectives, redirecting funds planned for a marketing initiative to upgrading its ethics staffing, processes, and systems.

Alex has four other items he would include in periodic reports to management:

• Risk-adjusted performance metrics
• VaRs, EaRs, cVaRs
• Limit breaches and activated stop losses
• Transparent methodology with a back test

OK.

I suggest a principle we should follow:

Help leaders and decision-makers get the information they need.

While Alexei’s suggestions are excellent, these are from the perspective of the risk practitioner.

I am suggesting we need to look at this from the perspective of the leader and decision-maker.

Find out what they need and only then figure out what to give them!

A second principle is:

The success of any organization depends on the quality of their decisions.

Decisions should be informed and intelligent. They should be made by the right people, at the right time, with an understanding of what might happen (i.e., risk and opportunities).

Then:

Different people need different information to inform their decisions.

While I am a strong believer in managing the organization so that there is at least an acceptable likelihood of achieving enterprise objectives, there is more.

Consider the needs for risk-related information of these individuals:

• The CEO
• The CFO
• The Treasurer
• The head of Sales
• The head of Marketing
• The CIO
• The COO
• The CISO
• The Chief Compliance Officer
• The head of Procurement
• The Safety Officer
• The head of Human Resources
• The head of Manufacturing
• The head of Engineering
• The head of Product Development
• The manager of Physical Security
• The head of Investor Relations
• and the list goes on

Each has different decisions to make and needs different information. We can’t expect them to find all the information they need in the same report.

Yet, a poor decision by any one of them might have serious ramifications on the ability of the organization to achieve its objectives.

The risk practitioner should work to ensure each has the information they need.

There’s a difference between providing a report and providing information. For example, a CISO needs to be alerted every time there is a serious attack on the cyber defenses. A CFO needs to know as soon as there is a significant movement in exchange or interest rates. A Manufacturing executive needs to information about manufacturing or supply chain issues as soon as they occur.

Reports are, by their nature, periodic. But risk management should be a continuous activity.

In other words, we need to tie in KPI and KRI into the discussion.

In his recent posts and videos, Alexei has made the point that the most important part of risk management is the risk assessment. While that is important, and the risk practitioner can bring excellent tools and techniques to develop valuable insights, it is of little use if it is not used by the right people in their decision-making.

Each decision should have reasonable information about what might happen (i.e., risk analysis).

A final premise:

More decisions are being made every day that require an understanding of risk than the risk practitioner has resources to provide.

Where does that leave us?

We have to rely on management to collect and analyze risk information in the absence of the risk practitioner.

My advice:

1. Work with those responsible for periodic (and hopefully continuous) performance reporting to make it risk informed. Make sure leaders understand the likelihood they will achieve their and the enterprise’s objectives. Feel free to adapt and use my suggested report format, above.
2. Work with them and those who own each enterprise objective to develop the next level down of reporting. Take each objective and identify the related risks and opportunities, highlighting which are at acceptable levels and which are not.
3. Talk to management to understand which of their decisions are most critical, and help them obtain the information they need.
4. Help train management to make quality, risk-informed decisions.
5. Allocate your time to where it will be of most value.
6. Constantly ask if you are doing what you should be doing to help the organization succeed, which is far more than avoiding failure. Adapt.

I welcome your thoughts.

P.S. If you liked World Class Risk Management, I suggest you read the book that continues the discussion, Risk Management for Success,

[1] The left side of a bowtie or a tornado analysis may help.

Balanced and fair audit reports

I think it’s fair to say that operating management doesn’t look forward to an internal audit report.

However they may feel about the competence and professionalism of the auditors, they know from experience that the formal reports at the end of an audit won’t make them look good.

The best they can hope for is an absence of significant ‘findings’ and an opinion that says their work is ‘adequate’.

Is that fair and balanced?

Is that an accurate representation of the quality of work that management and their team are producing?

Is that what you would like to hear from your manager in a performance review: an absence of significant issues?

I think we can and should do better.

If a department or business unit is doing well, we should say so.

If they have adopted what might be considered a best practice that could be adopted elsewhere, we should say so.

If they have made significant progress since the last audit? We should say so.

If individuals should be commended, we should do so.

Frankly, this is one of the problems with a formal, written audit report. If we were to either augment or (perhaps better) replace a written report with an in-person briefing of management, we would be far more likely to say what we are reluctant to write.

The converse is also true.

We are reluctant to include in the report (but might say in a briefing) that individuals lack experience or competence. But we should find a way to say it.

We are in love with traffic light audit reports and opinions, where the highest grade is a B+. Yet sometimes management deserves an A+!

Sarah Bareilles wrote a song, Brave, which captures what I think about this.

I welcome your comments.

Perhaps the greatest and least practiced skill for internal auditors

Whenever I see papers or presentations by consultants on the evolution of internal auditing, usually by adopting new technologies, I am at once amused and frustrated.

What these papers ignore is that so much more can be achieved by ensuring internal auditors perform the basics well.

For example, in my experience few internal audit departments focus on the more significant risks and stop auditing issues that would never have a serious effect on enterprise objectives.

You see this when audit thought leaders and practitioners talk about agile internal auditing, where they break down a lengthy audit (perhaps multiple man-months in length) into sprints. They prioritize the sprints, auditing the more significant areas first and the less significant ones in later sprints.

The problem is that those later sprints involve auditing issues that wouldn’t rise to the high-risk level on their own.

The lengthy audit should be cut down dramatically so that it only includes in its scope those issues of significance to the success of the enterprise as a whole.

Another example is the need to sit down and have a constructive discussion with operating management when potential issues surface. The first priority should be to agree on the facts and whether there is a problem. Once that is achieved and we can agree on the significance of the problem, then we need to focus on what needs to be done (if anything). Sometimes, the risk is one that should be taken!

Auditors should stop prioritizing reporting issues and instead prioritize helping the organization succeed!

Before adopting new tools, lets optimize the ones we already have.

Let’s optimize out ability to LISTEN!

I have admired Tom Peters for decades. He challenges us with provocative statements and ideas, most of which are fundamentally accurate. I gave a copy of his book, The Pursuit of WoW!, to each of my direct reports at Tosco.

As an aside, I adopted his principles in designing a WoW! Audit Department:

But let’s get back to listening, perhaps the greatest and least practiced skill.

We need to listen to each other. The CAE needs to listen to management and the board, but especially to his or her team! I don’t see that as a strength of many CAEs.

Individual internal auditors need to listen to everybody as well, especially to those they are auditing – from the department head to the most junior employee or contractor. In fact, find a way to listen to suppliers, customers, and others in the extended enterprise.

I advise everybody not to “go and talk to people”. Instead, “go and listen”. If you are talking more than 40% of the time, you are not listening enough.

Active listening, paying attention, is a rare and very hard skill to learn.

I have exchanged messages on Twitter with Tom (we follow each other). When he wrote about Managing by Walking Around, I persuaded him that we should be Managing by Listening Around.

Here is a recent short (3:37 minutes) video you should listen to, The Little Big Things: One More Way to Pursue Excellence.

I welcome your thoughts.

Testing data vs. testing controls

In a recent post of his on LinkedIn, Joseph Kassapis wrote:

I was reading a typically excellent blog/post of Norman Marks on Control Testing (in the context of commenting on 2 reports on SOX Controls Testing), and was struck and intrigued by his insistence/emphasis on testing “Data” in the mistaken impression that this amounts to testing the Control(‘s effectiveness). He named this twice in his post as a fallacy/defect in the reports, and it instantly caught my attention, being something I always found extremely interesting and important: to what extent correct output can be taken to mean/evidence correct mechanism.

…

External Audit standards, as I fairly confidently recall/understand, expressly preclude this position, i.e. state that the correctness of the recorded transactions, as regards their aspects controlled by the control, can in no way and under no circumstances be taken as evidence of soundness/effectiveness of the control; and I sort of ‘resented’ this, regretted it, wished it was not there; without actually being able to really/genuinely fault it, logically; rather minding its being inconvenient, making things harder, depriving us of easy tests and forcing us to conceive harder ones, (towards the already very hard task/goal of attaining satisifaction of effective functioning of Control), easier said than done !

…

Nobody else seems to, elaborate either, on this very important principle. Nobody seems to take it up. Except, it seems, Norman Marks. In the sense that at least he does consider it is there, it is important, and it is grossly abused. I was badly hoping he would go on to elaborate, in this blog pot, but he didn’t.

I don’t know if he elaborated elsewhere. He can inform/refer us. Whether or not he did, in the past, I would dare invite/provoke/challenge him to do so now. With another, dedicated post. Enlightening us. As he always does.

OK, Joseph. Here we go.

I start with a premise: our objective is to obtain reasonable assurance that the controls relied upon to manage the risk (whether SOX and ICFR, or some other business risk) are (a) adequately designed and (b) operating effectively as designed.

In other words, we are performing an audit of the system of internal controls for that risk.

The situation is different if we are trying to validate that the data (or information, such as in a report) is complete and accurate.

The value of an opinion on the system of internal control is that it provides continuing assurance, while validating the data provides point in time assurance. Validating the data or the information in a report may confirm that that instance of the report is complete and accurate, but it doesn’t tell you that the next instances will be. For that, you either have to continue testing and validating each instance or rely on the system of internal controls.

The quality of assurance is different. An opinion on the system of internal controls only provides reasonable assurance that each instance is complete and accurate, whilst validating data provides more absolute assurance that the data is correct.

Now, let’s return to the challenge.

I have been leading a SOX Masters class for many years, usually multiple times each year. In that class, I ask participants:

“Has your home been burglarized in the last five years or so?

In all that time, only one person raised their hand. (Good news.)

I then ask:

“Does that prove you always closed and locked your doors and windows every time you left home?”

(I don’t even go so far as to ask whether they set the alarm.)

They smile ruefully, very much aware that they have failed to do so: their controls were not operating effectively, yet they did not have an incident (or data exception, if you like).

Consultants are pushing the notion that you can use analytics and other methods like AI and RPA to test controls.

There are very few opportunities to do so, as these techniques may provide some level of assurance that the data is free of error (if not always omissions). But they rarely provide acceptable evidence that the controls management have in place even exist, let alone are adequately designed and operating effectively.

Taking another example.

The city of San Jose, my hometown, has implemented a number of controls to limit accidents at busy intersections. They include:

• Traffic lights
• Lane and other street markings
• Periodic police visits
• Reliance on controls performed by others, such as DMV’s driver licensing controls

If you ran analytics and found that there were no accidents reported at the intersection of Stevens Creek Boulevard and Winchester Boulevard in 2022, does that prove that any of the controls were working?

No. I can tell you that there were times when the lights did not work but drivers exercised appropriate caution.

While detecting that there were incidents may indicate that controls were not working (more work needs to be done to confirm that), the lack of exceptions does not provide assurance that controls were in place, adequately designed, or operating effectively.

I hope that helps.

By the way, the intersection example illustrates another issue that many don’t understand.

The system of internal control only provides reasonable assurance. It does not provide absolute or perfect assurance.

COSO’s Internal Control Framework provides some examples of the limitations, but there is more.

When you test internal controls, you may find exceptions.

For example, you inspect the traffic lights and find that they were inoperative for a few hours on one day.

If that only happened once over a period of a year, I would call that an “isolated incident”. It is reasonable to accept the occasional breakdown.

But if it happened several times in a month, I would call it a “control breakdown”.

You can have effective internal controls despite isolated incidents, but not when there have been control breakdowns.

That is why when we find exceptions we need to expand the sample size to determine whether we have an isolated incident, which would acceptable, or a control breakdown – when we would assess that the control has failed to operate effectively as designed.

I welcome your comments.

Where do our SOX programs stand today? Two reports

Two firms recently released reports on SOX Compliance trends: Protiviti and Deloitte.

I need to make one important point.

When I was responsible for SOX at my company, I wanted to find out what our internal SOX compliance costs were. To my surprise, more than 50% of the costs were incurred by management: supporting testing by both internal and external audit teams, maintaining the documentation, answering questions, and helping with the scoping.

The surveys on cost performed by firms like these two tend to ignore the management-related costs. Keep that in the back of your mind as we review the two reports.

Protiviti shared the results of their annual SOX surveys in Assessing SOX internal costs, hours, controls and other trends in the results of Protiviti’s 2022 Sarbanes-Oxley Compliance Survey. It has a great deal of information and is worth downloading and reading.

Protiviti’s Executive Summary includes this (with my highlights):

Escalating compliance costs, time and efforts have a silver lining: They are driving more investments in automation and technology tools that generate greater efficiencies — and potentially cost savings as well as effectiveness and coverage benefits — into the SOX compliance process. Our data indicates that technology tools currently support an average of one-fourth of SOX compliance work across all companies, and a majority of programs deploy audit management and/or GRC platforms. These results are promising: Greater use of enabling technologies can, over time, help moderate jumps in internal SOX compliance costs. That said, more progress is needed. Many programs have yet to begin using an audit management platform while most have yet to leverage more advanced technology tools in their SOX programs.

There also are opportunities to pursue procedural and structural changes in SOX compliance programs. Shared services or “centers of excellence” approaches — managed internally or by an external outsourcing partner — offer substantial opportunities for efficiency improvements, especially when it comes to the highly defined and repeatable tasks, such as gathering and organizing evidence, and control testing, that dominate SOX compliance efforts. Many of the forces driving internal SOX compliance costs and hours higher are, for the most part, beyond the control of companies. This is not the case with investments in compliance automation and broader technology enablement as well as alternative delivery models that generate greater efficiency over the long term. Internal audit and finance leaders, together with their C-suite colleagues, should avoid delaying their evaluation and pursuit of opportunities in these areas.

I have highlighted two sections:

1. While technology can provide useful functionalities in managing a SOX compliance program, the ROI for what can be expensive software is not always clear for companies without hundreds of key controls. In addition, my experience with some of the software is that it doesn’t always support the top-down and risk-based approach explained in PCAOB and SEC guidance; it doesn’t identify significant accounts and then the key controls relied upon to prevent or detect potential material errors of omissions in those accounts.

The consulting firms preach that you can use technology for testing. However, the potential is not nearly as great as they indicate. We need to perform testing that provides reasonable assurance of the existence, design, and operation of the key controls we rely on. Most of the software tests the data, not the controls – and just because the data is clear you cannot assume that the controls are in place, adequately designed, and consistently operating as they should.

Protiviti says this later on, which is highly questionable:

Automation platforms and applications bring greater efficiency to SOX compliance activities. The deployment of process mining, advanced analytics, robotic process automation (RPA) and continuous monitoring, along with other advanced technological tools, can significantly reduce the volume of manual compliance tasks as well as retention risks associated with subjecting internal full-time staff to heavy loads of repetitive, task-driven work.

2. These “shared service centers” for SOX testing, if outsourced, are a return to the use of expensive consulting firms for testing – not something I recommend. If they are run in-house, staffed by people who do nothing else, then they may not be in tune with the business. I would think twice (or more) before doing this. There is huge value in a SOX team that suggests better controls and process improvements in addition to testing key controls.

Protiviti tells us in the report that, on average, 41% of SOX internal costs is for outsourced resources.

On the other hand, this is correct:

A combination of internal and external factors creating volatility — technology-driven transformation and innovation, talent shortages, strategic pivots and more — is contributing to rising SOX compliance costs. More companies spend $2 million or more on compliance while fewer spend $500,000 or less. A surge in the number of smaller companies spending $2 million or more in SOX compliance costs likely reflects last year’s significant increase in initial public offerings (IPOs), driven by special purpose acquisition companies (SPACs).

The chart on page 12 of the report is very useful information. It shows the typical time taken for various activities, such as testing for operational effectiveness or adequate design of a key control. Unfortunately, Protiviti did not distinguish between manual and automated controls.

The results in one chart disappointed me: the percentage of controls where the external auditors relied on management testing. The average was just 26% and only 10% of respondents said external auditors’ reliance exceeded 50%.

Protiviti tells us:

In assessing year-over-year trends in external auditor reliance on management controls testing, percentages show a year-over-year decline — i.e., external auditors appear to be relying less on this testing.

Two points:

1. At my company, EY told the audit committee they relied on my team for 80%. At the SOX Masters training I lead, a number of attendees have reported similar levels of reliance.
2. It is important to recognize that the external auditors can rely entirely (with review) on management’s testing of key controls that are not high risk, but they can also reduce their work by placing partial reliance with limited reperformance.

I found it interesting that according to the survey, in the average company 50” of the key controls are automated, up from 33%.

I also found it interesting that the average company has 52 significant applications, and more than half of them are cloud applications. That seems too high.

I wonder whether they have done a good job in using the top-down and risk-based approach to identify significant applications, or whether they have included applications that are involved in financial reporting but don’t contain any automated controls or other IT-dependent controls.

I am also surprised that many companies either test key reports (IPE) on a rotational basis (which should not be allowed) or only once and then not until the report is changed – 21% rotational and 36% just once. That conflicts with my empirical experience with the number of companies who have employed a baselining or benchmarking approach.

As a reminder, except when benchmarking is used for IT-dependent controls, every SOX year has to stand on its own.

Let me make one important statement:

The best path to reducing SOX compliance costs and improving effectiveness is through application (and re-application every year) of the top-down and risk-based approach. Right-size your controls!

The Deloitte report is SOX modernization: Optimizing compliance while extracting value.

They seem to agree with my important statement, above, when they say:

A SOX program that has not been challenged in years may be stale, which could be a drain on resources and impede performance, particularly if this compliance program is treated more like a “check-the-box” activity.

Deloitte also comments, with my highlights:

Management’s responsibilities related to internal control over financial reporting is to obtain reasonable assurance over the reliability of financial reporting, not absolute assurance, and the concept of “reasonableness” is objective with a range of judgments and methodologies that could be considered appropriate. Performing an effective risk assessment can help management identify areas with risks of material misstatement within the company and determine which of those areas it should focus its efforts.

Many factors could contribute to a lagging SOX program. Over time, risks evolve, or new risks are identified, and the response may have been to design new controls without always taking into consideration if any existing controls should be modified or removed. Additionally, once risks are identified, the level of risk may not be considered, such as if it’s a lower risk or a significant risk, which could result in not spending enough time in areas of significant risk or spending too much time in areas of lower risk. Controls could also have been added to manage an issue or deficiency identified without actually addressing the root cause.

Deloitte goes on to provide good advice on the risk assessment process.

But they fail miserably by recommending testing data instead of controls:

Automated testing consists of profiling certain populations and transactions with real-time results, allowing a company to be able to test up to 100 percent of the population and potentially achieve more assurance for less time and cost.

As a reminder: the data can be 100% clean even though nobody is performing the controls. Just think about how many times you left your windows open and/or doors unlocked when you left home, and even though those controls were not operating you were not burglarized.

Deloitte makes one good point, but they don’t go far enough.

They talk about automating a current manual process. That can certainly provide both efficiency and effectiveness.

But why not go further and consider whether the process should be changed – with or without modernization. There’s little point in automating an inefficient process!

If you are responsible for your company’s SOX program, I urge you to consider my SOX Masters class (one is planned for September). You can also purchase the IIA’s Management Guide to Sarbanes-Oxley Section 404.

I welcome your comments and experiences.

If you are involved in SOX compliance, you should know about the IIA’s GAIT Methodology

A fact: most companies have included far too many IT General Controls (ITGC) in their scope for SOX.

Why: because they have taken an approach to scoping ITGC that is disconnected from the top-down and risk-based approach used to identify key controls within business processes. The scoping of ITGC has resulted in including ITGC controls in scope where a failure would not present a reasonable possibility of a material error omission in the financial statements.

“The identification of risks and controls within IT should not be a separate evaluation. Instead, it should be an integral part of management’s top-down, risk-based approach to identifying risks and controls and in determining evidential matter necessary to support the assessment.” – SEC Interpretive Guidance

The IIA recognized that there was a need to help practitioners define the right scope of ITGC for SOX, and a team of experts (including a representative from the PCAOB) developed the GAIT Methodology.

GAIT continues the top-down and risk-based approach recommended for companies by the SEC and mandated for their auditors in the PCAOB’s Auditing Standard 2201 (formerly AS5).

“The auditor should use a top-down approach to the audit of internal control over financial reporting to select the controls to test.” – PCAOB Auditing Standard 2201

“Management should identify those risks of misstatement that could, individually or in combination with others, result in a material misstatement of the financial statements (financial reporting risks).” – SEC Interpretive guidance

“In an audit of internal control, if the auditor selects an IT-dependent control for testing, the auditor should test the IT-dependent controls and the IT controls on which the selected control relies to support a conclusion about whether those controls address the risks of material misstatement.” – PCAOB Staff Alert No. 11

“For purposes of the evaluation of ICFR, management only needs to evaluate those IT general controls that are necessary for the proper and consistent operation of other controls designed to adequately address financial reporting risks.” – SEC Interpretive Guidance

Since its publication in 2007, GAIT has been adopted with great success by hundreds of companies and accepted (even recommended) by their CPA firms.

It has helped those organizations right-size their ITGC scope for SOX. Although it is focused on getting the scope right, rather than on cutting unnecessary ITGC out of their SOX scope, companies have been able to reduce the number of ITGC key controls significantly.

15 years have passed since GAIT was published. During that time, technology has advanced and practitioners have gained far more experience in SOX compliance.

It was time to update GAIT.

That update has now been completed (with the help of an eminent review panel of practitioners and partners from independent audit and consulting firms) and the product is available for free download by visiting a dedicated page on this website.

GAIT has stood the test of time very well! This is not surprising as it continues to be used extensively.

Its principles and methods continue to apply, even as technology and its use have changed.

The updated version of GAIT, developed independently from the IIA but with their full knowledge, simplifies the text, adds real-life examples, and references relevant regulatory guidance. The IIA is focused on an update to their International Professional Practices Framework and was not able to lead or participate in the update, but it is expected they will turn to their own update in 2023.

The dedicated web page includes links to the original GAIT Methodology, as well as to the two GAIT products that followed: for general technology-related business risk (GAIT-R), and for the assessment of ITGC deficiencies for SOX.

Comments and feedback are welcome.

What if we just abandon “risk management”?

Earlier this year, Marco Nutini asked this challenging question in a newsletter he shared on LinkedIn.

He starts with:

Calm down, I don’t want to ruin my source of daily bread, let alone create a fuss.

Several internationally recognized authors have already addressed a recurring theme in the Risk literature: if a company does not manage risks, but manages decisions, why use the term “Risk Management”?

For example, Grant Purdy and Roger Estall devoted an entire section of their book, Deciding (2020), to propose the temporary eradication of the term. Grant was a nominated expert to the working group that wrote ISO 31000 and ISO Guide 73. Both standards were inspired by AS/NZ 4360:2004, to which Grant was a key contributor. So, I guess he is in a privileged position to give his opinion.

Marco quotes Grant and Roger’s argument that the terms “risk” and therefore “risk management” have multiple meanings and that means they really have no meaning. Therefore, we should stop using the terns.

This is not a view I ascribe to, although I do dislike the four-letter word “risk” because it sparks a negative reaction from most business executives.

Instead, Marco suggests:

“…what we now call ERM (Enterprise Risk Management) is a tangle of three distinct, yet interconnected fields of knowledge, something like modes of Risk Management:

• Strategic Assumptions Assurance: A set of tools developed to assess an organization’s chance of achieving its goals and honoring its performance forecasts. It is supposed to support the strategy execution and monitoring processes.
• Risk-Informed Decision Making: This mode has a diffuse, broad scope. As the name implies, it aims to ensure that the organization’s decision-making processes gather and use intelligently the necessary information for decision making under uncertainty. This mode is called Sufficient Certainty by Grant Purdy and Roger Estall, also the name of their consultancy from Australia.
• Risk Control: A mode that has a transactional and compliance scope. It seeks to design and maintain a control environment that keeps residual risks at the planned levels. It is analogous to the “routine management” of Quality. Many people think that this is what Risk Management is all about.

This resonates more with me (see my last blog post).

The first of the three seems very similar to my idea of top-down risk management, which focuses on whether there is an acceptable likelihood of achieving each of the enterprise’s objectives.

The second is what I referred to decision-based risk management.

But I see the third as a subset of the first two. Some might say that this is how an organization responds to, manages, or mitigates risk.

The problem is that it overlooks the positive aspect of risk: opportunities. We need controls to ensure that they are taken as and when appropriate.

Marco’s newsletter/LI post is quite long, and I will let you read the rest. The only comment I will make is that he makes everything seem complicated, whereas I always seek (but don’t always find) simplicity.

Please share your comments here as well as against his post.

P.S. Happy belated birthday, Marco!