Home > Risk > How do you measure the effectiveness of internal audit?

How do you measure the effectiveness of internal audit?

I want to thank Dr. Rainer Lenz for telling me about the new paper he and Dr. Marc Eulerich have written for the IIA’s Internal Audit Foundation. (I also want to commend the Foundation and the IIA Dallas Chapter, the sponsor, for their innovative crowd-funding of the paper.) Rainer and I have exchanged thoughts and ideas about internal audit for years, and I respect him and his contributions to the profession.

The products of the Foundation are intended as leading research. They do not represent guidance.

Defining, Measuring, and Communicating the Value of Internal Audit: Best Practices for the Profession has some excellent content, especially the quotes from CAEs. I will focus on those before explaining why I think it falls short.

  • Internal auditors and internal audit functions have been struggling — some more than others — to find convincing answers addressing one fundamental question: What is the added value of internal auditing in the specific organizational context?
  • Internal audit’s perceived value and its standing in the profession itself and among its stakeholders is still often described as hazy and enigmatic.
  • Deloitte (2018) finds that only about 40 percent of CAEs believe that their function has strong impact and influence within the organization and only 46 percent think that stakeholders are aware of internal audit’s services. In other words, more than 50% of internal audit’s key stakeholders do not see the added value of their audit functions.
  • …there is a difference between the value internal auditors think they rendered and what their stakeholders perceive.
  • “[What we] try to do is help the company identify the top risks, determine whether or not the management and risk management practices are adequate to deal with those risks or whether or not additional work needs to be done […]. Then I’m providing the assurance that it’s in place and operating the way it should be […]. I’m like your doctor or your dentist, I can’t brush your teeth for you, but I can tell you here are the steps you need to do to be healthy and I don’t want to be a police officer. I want to be that person that helps you get healthy, but I can’t do it for you.” —CAE of a large multinational technology company
  • “The audit committee and the management board know that we are going after the right topics and provide advice about these hot topics and we have a lot of them in our company. That is, I would say, our number one value. Number two, obviously, is that the audit committee and our board of management not only know that we go after the right topics, but that we have the competencies to tackle those topics.” —CAE of a large, listed infrastructure company
  • …the survey responses unambiguously suggest assurance services as internal audit’s core value.
  • …in some organizations, stakeholders actually completely deny internal audit’s value.
  • “Our value proposition cuts across all of the types of risks that the company sees, going from operational through financial and regulatory. We have to offer assurance for the audit committee and the C-level.” —CAE of a large multinational company from the financial industry
  • “We are providing the assurance: is everything (e.g., controls) in place and operating the way it should be?” —CAE of a large listed multinational company
  • “We are seen as the trusted advisor at least for management, we give them advice and also give the audit client advice, how they can do better. We are not only the bad ones, telling them what they are doing wrong. We also tell them how they can do better. Thus, it is important to be ready to switch your roles.” —CAE of a large listed multinational company
  • “How would I define the strategic value of internal auditing? From the perspective of the person receiving the value, they (the stakeholders) are able to say, I can use this information from internal auditing. I needed this information and I can actually make things better.” —CAE of a large national governmental organization
  • [Only] about half of the participants stated that they deliver significant value.
  • …company size does not prevent internal auditors from adding value to the organization. The overall picture regarding total assets and revenues suggests that there is no association between company size and value creation.
  • …the self-perception of audit leaders surveyed is that the audit committee and senior management (the two central stakeholders of internal audit) are very satisfied with the work.
  • …the added value of internal audit can be made clear through direct communication between the CAE and key stakeholder groups. The direct contact with both senior management and the audit committee provides the internal audit function with the opportunity to demonstrate and discuss its value performance and establish a relationship built on trust.
  • Truly audit what matters to the success of the organization. Become a respected value driver of the organization.

I find it refreshing and exciting to see the ideas and even the language I have been promoting for many years repeated by the authors and the CAEs they talked to. Just look at that last sentence I quoted. It’s something I might have said.

Now for the criticism. (Sorry, Rainer.)

  • The paper talks about ‘assurance’ as being limited to financial reporting and compliance. This is a major misunderstanding. As referenced by the CAEs that are quoted, assurance relates to all sources of ‘risk’ and opportunity. “Is everything in place and operating the way it should be?”
  • No reference is made to the Core Principles for the Professional Practice of Internal Auditing. (I thank Paul Hicks for pointing this out.)
  • The value of anything is what people are willing to pay for it – as a general rule.
  • To know whether the stakeholders on the board and in top management believe they are receiving full value is to ask them. To quote from the paper, “there is a difference between the value internal auditors think they rendered and what their stakeholders perceive.” If they say, for example, “I can use this information from internal auditing. I needed this information,” then you are adding value. I cover this extensively in Auditing that Matters, with examples of positive responses to the question of “How are we doing” of:
    • “Keep it up or your fired”, a joke by the CFO before awarding me a huge bonus
    • “You help us sleep through the night” from an audit committee chair
    • “You have yet to perform an audit I wouldn’t gladly pay for” from a divisional CEO
    • “You help us stay efficient” from another divisional CEO
    • “I want you to attend the IT committee meetings” from a board member who chaired that committee
    • I (an executive) don’t want to cut internal audit budgets when we are having layoffs.
  • The metrics discussed in the paper are, by and large, measures of ineffectiveness. For example, completion of the audit plan measures whether you have continued to audit what used to matter, rather than what matters today and tomorrow. If you have an audit plan that is continuously updated, by definition you are completing it.
  • There are other metrics which are more useful, such as the number of requests from management for assistance. A soft one, which defies measurement, is the speed with which executives respond to e-mails or requests for a meeting.
  • While I agree with the use of a maturity model in assessing internal audit performance, the one in the publication is poor. Providing effective assurance on what matters, when it matters, satisfies all three levels of the paper’s model, and there is insufficient attention to providing insight as well as advice. I have a much more sophisticated model. Unfortunately, it is not free: Is your Internal Audit World-Class? A Maturity Model for Internal Audit.
  • The publication has contradictory information without explanation:
    • …only about 40 percent of CAEs believe that their function has strong impact and influence within the organization and only 46 percent think that stakeholders are aware of internal audit’s services. In other words, more than 50% of internal audit’s key stakeholders do not see the added value of their audit functions.
    • [Only] about half of the participants stated that they deliver significant value.
    • Survey participants indicate that more than 80% of the stakeholders are either “very satisfied” or “satisfied.”

Finally, if you have to tell the audit committee and CEO how valuable you are, you are lost. If they don’t already believe you are valuable, then you are doing something wrong.

If I was on the audit committee of an organization, I would assess internal audit based on:

  1. Are they helping me be effective as a board member, providing the assurance, advice, and insight on what matters, when it matters, in an actionable form that I need?
  2. Does the management team believe and trust internal audit’s assurance, advice, and insight? Do they agree that internal audit provides the information they need on what matters, when it matters, and in an actionable form?

When I started out as a CAE many, many years ago, I started to fall into the trap of trying to put a value on our assurance, advice, and insight. The number is meaningless.

I turned instead to asking my stakeholders some simple open-ended questions, such as “are we helping you as much as we should” or “are we doing something that is not valuable to you?”

The only thing that matters is the assessment of the customer. Having said that, there is good advice available elsewhere (hint) on how to build and then measure a world-class internal audit function.

I welcome your thoughts.

  1. June 25, 2020 at 4:25 PM

    If an IAU does not matter to the Board and Executive Management, simply, it is ineffective. If any of the members of these organisations keep on calling the CAE, day and night, for advice/opinion, that’s a good indication of being effective.

  2. Michael Corcoran
    June 25, 2020 at 6:14 PM

    A dump too much. What is your point?

    • June 26, 2020 at 12:49 PM

      Thanks Michael. I appreciate any feedback. Hope there is something of value in this co-authored paper for the interested reader.

  3. June 25, 2020 at 8:49 PM

    First, thanks for bringing the latest research to our attention. I think the point that value is determined by the customer is crucial. Our product is information useful to our clients, not audits. That means we should have at least two general engagement types to add the sort of value needed by management: 1) Assurance that compares reality against a standard for mature processes; and 2) Consulting to help develop immature processes as a partner.

    When the CEO asked IA to jump in and help improve how we were delivering services for a major client, I was both thrilled by the request (definitely working on something that mattered) and concerned about how to deliver the value. There was no simple recipe. They asked for help, not an audit. I used all the tools I could think of; risk assessment, process mapping, project management, and facilitated solution brainstorming. What are the known issues and who’s fixing them? What are the barriers for our client-facing personnel? Where does IA have something to add and where should we stay out of the way?

    P.S. Most IA metrics are designed to measure the internal workings of our audit processes, for IA management looking downward into the IA organization, and are of little concern to clients. One exception is if there is a long lag in providing audit reports. That is one metric to fix right away, live editing sessions with the key parties the main solution there. Another is progress on resolving open issues in follow-up, a key reflection on the “tone at the top” of the organization and the relevance of findings.

    • Norman Marks
      June 26, 2020 at 5:33 AM

      Excellent

  4. Bertrand
    June 26, 2020 at 4:30 AM

    I do not agree with all these critics about IA. I believe our function is structurally prone to critics. The same could be said for many professions, with an emphasis of course for the control functions (police, justice…). I agree that all our audit reports are not great (“grands crus”), since we are often general practitioners auditing specialists, which is quite challenging for us. However, our main stakeholders have also often conflicting interests (“to be the eyes and ears of the audit committee or of the CEO”). Moreover, we uncover sometimes issues directly related to these important stakeholders. We are kind of agents of transparency, sometimes alone with our moral compass. Transparency enforce good governance but it does not help to make friends. In such circumstances, there is not always (if ever) a correlation between the quality of our work and the satisfaction of our stakeholders. We also need a CEO and an audit committee who believe in internal audit, who are ready to invest in IA and who stand for the subsequent transparency. Therefore yes our stakeholders opinion is important, but even more so if they act first in the best interest of the company. If you want a proof that our profession is not all that bad, have a look at the great papers issued by our prominent members: Norman, Rainer…!

    • June 26, 2020 at 12:46 PM

      Thanks Bertrand. Let‘s keep building on each others‘ work 🙂

    • Norman Marks
      June 26, 2020 at 12:51 PM

      Bertrand, I don’t know who you are saying is criticizing IA. I think everybody who has commented is passionate about its value and only wants to see that upgraded, if possible.

  5. DAVID PETRISKY
    June 26, 2020 at 4:45 AM

    Let’s be honest about the paper’s methodology: interviewing 11 CAEs and then sending surveys to IIA members is not an acceptable way to measure the stakeholders’ valuation of our services. I would prefer to see an analysis of IA budgets (e.g. as % of Revenue, trended over 5 years, say) as evidence of how organizations value IA.

    • Norman Marks
      June 27, 2020 at 12:52 PM

      That is a tempting metric, but what about those organizations that have a department only because it is required and view it as a necessary expense? My department was extremely cost-efficient. Should we be penalized for that?

      • David L Petrisky
        June 27, 2020 at 1:42 PM

        It may not be the best metric theoretically, but it would have the advantages of being objective, verifiable, comparable across entities, and already available to the CAE. Now, it’s true that not every IA shop has the same expectations placed on it – e.g. how involved IA is with SOX or Fraud investigations, if at all – but eventually, with enough data, patterns and benchmarks would emerge. Thanks for listening!

  6. Mark Williams
    June 27, 2020 at 12:39 PM

    Thank you, this is a fantastic conversation. Please can I offer my simple answer to the question:

    Q. How do you measure the effectiveness of internal audit?
    A. Time to Value (time it takes to deliver value to our Stakeholders)

    Perhaps I am being over simplistic, but I’ve yet to find a better metric as a measure of success for any IA improvement initiative (e.g. Agile in IA in my case)?

    • Norman Marks
      June 27, 2020 at 12:50 PM

      Good idea. My question back to you is how do you know whether you are delivering the most value?

      • Mark Williams
        June 27, 2020 at 1:33 PM

        Ok, how about this? This is my thinking and I’m keen to share for feedback:
        – Value (includes cost)
        – Quality (does not)
        What I think you’re asking about is not Value, but Quality…
        I think the Quality of IA is measured by three metrics (in no particular order:
        1. Raising new issues – a simple count?
        2. Reiterating known issues – also a simple count?
        3. Assurance provided – binary yes / no?

        • Norman Marks
          June 27, 2020 at 1:37 PM

          Interesting ideas. There is huge value, value that people would pay for, in assuring leaders that they can rely on their systems and controls.

          Raising new issues is less valuable than effecting change that contributes significantly to success.

          I stand by the premise that the only value is in the eyes of the customer and can only be measured by them.

      • Mark Williams
        June 27, 2020 at 1:50 PM

        Just to conclude. I think Value in IA is not about getting more done, but about getting the right things done. Therefore, perhaps the four “mega metrics” are?:
        – Time to Value
        – Value (this is the cost / efficiency metric)
        – Quality (the three I stated above)
        – Stakeholder feedback
        Strangely I see so many IA departments optimised not for these things, but throughput (i.e. a count of how many audits done in a given period against a predicted count (which is error prone as our environment too complex and unpredictable to do accurately – see Cynefin model). Very odd.

  7. Paul
    July 7, 2020 at 10:58 PM

    Honestly, Sometimes I wonder if I am in the right profession. Maybe because all I do is SOx for now and I can’t see the big picture but so far all I seem to be doing is ticking the box. When I work I keep asking my self “if I were a business owner who is not required to comply with SOx, would I honestly pay for this service ?” More often than not, the answer I come up with is no. I can’t justify how some of the testing we do actually helps the organisation becomes successful other than fulfilling compliance with a regulation. Perhaps I would get to appreciate internal auditing more as I progress in my career. For now however, I just don’t see the value. Great article Norman.

  1. June 25, 2020 at 1:21 PM
  2. June 29, 2020 at 6:01 AM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: