Where do our SOX programs stand today? Two reports

Two firms recently released reports on SOX Compliance trends: Protiviti and Deloitte.

I need to make one important point.

When I was responsible for SOX at my company, I wanted to find out what our internal SOX compliance costs were. To my surprise, more than 50% of the costs were incurred by management: supporting testing by both internal and external audit teams, maintaining the documentation, answering questions, and helping with the scoping.

The surveys on cost performed by firms like these two tend to ignore the management-related costs. Keep that in the back of your mind as we review the two reports.

Protiviti shared the results of their annual SOX surveys in Assessing SOX internal costs, hours, controls and other trends in the results of Protiviti’s 2022 Sarbanes-Oxley Compliance Survey. It has a great deal of information and is worth downloading and reading.

Protiviti’s Executive Summary includes this (with my highlights):

Escalating compliance costs, time and efforts have a silver lining: They are driving more investments in automation and technology tools that generate greater efficiencies — and potentially cost savings as well as effectiveness and coverage benefits — into the SOX compliance process. Our data indicates that technology tools currently support an average of one-fourth of SOX compliance work across all companies, and a majority of programs deploy audit management and/or GRC platforms. These results are promising: Greater use of enabling technologies can, over time, help moderate jumps in internal SOX compliance costs. That said, more progress is needed. Many programs have yet to begin using an audit management platform while most have yet to leverage more advanced technology tools in their SOX programs.

There also are opportunities to pursue procedural and structural changes in SOX compliance programs. Shared services or “centers of excellence” approaches — managed internally or by an external outsourcing partner — offer substantial opportunities for efficiency improvements, especially when it comes to the highly defined and repeatable tasks, such as gathering and organizing evidence, and control testing, that dominate SOX compliance efforts. Many of the forces driving internal SOX compliance costs and hours higher are, for the most part, beyond the control of companies. This is not the case with investments in compliance automation and broader technology enablement as well as alternative delivery models that generate greater efficiency over the long term. Internal audit and finance leaders, together with their C-suite colleagues, should avoid delaying their evaluation and pursuit of opportunities in these areas.

I have highlighted two sections:

1. While technology can provide useful functionalities in managing a SOX compliance program, the ROI for what can be expensive software is not always clear for companies without hundreds of key controls. In addition, my experience with some of the software is that it doesn’t always support the top-down and risk-based approach explained in PCAOB and SEC guidance; it doesn’t identify significant accounts and then the key controls relied upon to prevent or detect potential material errors of omissions in those accounts.

The consulting firms preach that you can use technology for testing. However, the potential is not nearly as great as they indicate. We need to perform testing that provides reasonable assurance of the existence, design, and operation of the key controls we rely on. Most of the software tests the data, not the controls – and just because the data is clear you cannot assume that the controls are in place, adequately designed, and consistently operating as they should.

Protiviti says this later on, which is highly questionable:

Automation platforms and applications bring greater efficiency to SOX compliance activities. The deployment of process mining, advanced analytics, robotic process automation (RPA) and continuous monitoring, along with other advanced technological tools, can significantly reduce the volume of manual compliance tasks as well as retention risks associated with subjecting internal full-time staff to heavy loads of repetitive, task-driven work.

2. These “shared service centers” for SOX testing, if outsourced, are a return to the use of expensive consulting firms for testing – not something I recommend. If they are run in-house, staffed by people who do nothing else, then they may not be in tune with the business. I would think twice (or more) before doing this. There is huge value in a SOX team that suggests better controls and process improvements in addition to testing key controls.

Protiviti tells us in the report that, on average, 41% of SOX internal costs is for outsourced resources.

On the other hand, this is correct:

A combination of internal and external factors creating volatility — technology-driven transformation and innovation, talent shortages, strategic pivots and more — is contributing to rising SOX compliance costs. More companies spend $2 million or more on compliance while fewer spend $500,000 or less. A surge in the number of smaller companies spending $2 million or more in SOX compliance costs likely reflects last year’s significant increase in initial public offerings (IPOs), driven by special purpose acquisition companies (SPACs).

The chart on page 12 of the report is very useful information. It shows the typical time taken for various activities, such as testing for operational effectiveness or adequate design of a key control. Unfortunately, Protiviti did not distinguish between manual and automated controls.

The results in one chart disappointed me: the percentage of controls where the external auditors relied on management testing. The average was just 26% and only 10% of respondents said external auditors’ reliance exceeded 50%.

Protiviti tells us:

In assessing year-over-year trends in external auditor reliance on management controls testing, percentages show a year-over-year decline — i.e., external auditors appear to be relying less on this testing.

Two points:

1. At my company, EY told the audit committee they relied on my team for 80%. At the SOX Masters training I lead, a number of attendees have reported similar levels of reliance.
2. It is important to recognize that the external auditors can rely entirely (with review) on management’s testing of key controls that are not high risk, but they can also reduce their work by placing partial reliance with limited reperformance.

I found it interesting that according to the survey, in the average company 50” of the key controls are automated, up from 33%.

I also found it interesting that the average company has 52 significant applications, and more than half of them are cloud applications. That seems too high.

I wonder whether they have done a good job in using the top-down and risk-based approach to identify significant applications, or whether they have included applications that are involved in financial reporting but don’t contain any automated controls or other IT-dependent controls.

I am also surprised that many companies either test key reports (IPE) on a rotational basis (which should not be allowed) or only once and then not until the report is changed – 21% rotational and 36% just once. That conflicts with my empirical experience with the number of companies who have employed a baselining or benchmarking approach.

As a reminder, except when benchmarking is used for IT-dependent controls, every SOX year has to stand on its own.

Let me make one important statement:

The best path to reducing SOX compliance costs and improving effectiveness is through application (and re-application every year) of the top-down and risk-based approach. Right-size your controls!

The Deloitte report is SOX modernization: Optimizing compliance while extracting value.

They seem to agree with my important statement, above, when they say:

A SOX program that has not been challenged in years may be stale, which could be a drain on resources and impede performance, particularly if this compliance program is treated more like a “check-the-box” activity.

Deloitte also comments, with my highlights:

Management’s responsibilities related to internal control over financial reporting is to obtain reasonable assurance over the reliability of financial reporting, not absolute assurance, and the concept of “reasonableness” is objective with a range of judgments and methodologies that could be considered appropriate. Performing an effective risk assessment can help management identify areas with risks of material misstatement within the company and determine which of those areas it should focus its efforts.

Many factors could contribute to a lagging SOX program. Over time, risks evolve, or new risks are identified, and the response may have been to design new controls without always taking into consideration if any existing controls should be modified or removed. Additionally, once risks are identified, the level of risk may not be considered, such as if it’s a lower risk or a significant risk, which could result in not spending enough time in areas of significant risk or spending too much time in areas of lower risk. Controls could also have been added to manage an issue or deficiency identified without actually addressing the root cause.

Deloitte goes on to provide good advice on the risk assessment process.

But they fail miserably by recommending testing data instead of controls:

Automated testing consists of profiling certain populations and transactions with real-time results, allowing a company to be able to test up to 100 percent of the population and potentially achieve more assurance for less time and cost.

As a reminder: the data can be 100% clean even though nobody is performing the controls. Just think about how many times you left your windows open and/or doors unlocked when you left home, and even though those controls were not operating you were not burglarized.

Deloitte makes one good point, but they don’t go far enough.

They talk about automating a current manual process. That can certainly provide both efficiency and effectiveness.

But why not go further and consider whether the process should be changed – with or without modernization. There’s little point in automating an inefficient process!

If you are responsible for your company’s SOX program, I urge you to consider my SOX Masters class (one is planned for September). You can also purchase the IIA’s Management Guide to Sarbanes-Oxley Section 404.

I welcome your comments and experiences.

Advertisement

Is your SOX program both effective and efficient?

Protiviti’s surveys and reports are always worth reading. One I look forward to is their annual survey on SOX compliance.

Those of you who are responsible for the SOX program or SOX testing at their organization are likely to find the benchmarking info in the 2019 survey, Benchmarking SOX Costs, Hours and Controls of interest.

However, I want to share (again) a note of caution.

Protiviti and others are talking about the use of analytics and other tools, such as RPA, for SOX testing.

But, the purpose of the SOX testing is to:

• Confirm that the design of the controls relied upon to prevent or detect a material error or omission in the financial statements filed with the SEC are sufficient, if they are operated as designed, to address such a possibility. The likelihood of a material error or omission is less than reasonably possible.
• Confirm, with a reasonable level of assurance, that those controls are being performed consistently as designed.

The end product is an assessment as to whether the system of internal control over financial reporting is effective; that means that the controls are sufficient to provide reasonable assurance that a material error or omission would be prevented or detected.

What do these newer technology tools do for us?

For the most part, they provide some level of assurance that the data, and possibly the transactions, are free from error.

But do they provide any assurance that the system of internal control is effective?

While the presence of errors is a strong indicator that the controls are not sufficient, the absence of errors is not a strong indicator that the controls are effective!

The data may be free from error even though the controls are not being performed at all!

In my SOX training classes (the next one is in October), I ask the attendees how many of them have had their homes burglarized in the last year. Only on the rare occasion has anybody raised their hand.

I then ask whether the fact that they have not been burglarized is proof that they locked all the doors and windows before they left the house.

I remember one time in England when, as an IT auditor, I was flowcharting and identifying controls in a very complex integrated system. One of the controls that management had identified was a comparison between data at one point in the system to the data at a much later point (a “run to run” control). When I examined the logic of the program that did the comparison, I found that it was coded incorrectly. At each point, early (file E) and late (file L), a file was created that could be compared. But the comparison program was comparing data in file E to data on file E – instead of file L.

The control was doing nothing. But the data happened to be clean anyway (we checked).

So, when it comes to the use of technology tools, will they provide the evidence you need that the controls relied on are both adequately designed and operated? Do they test the controls or only the data?

My second note of caution is to remain focused on whether the system of internal control over financial reporting provides reasonable assurance that material errors will either be prevented or detected. That refers to the possibility of errors in the consolidated financial statements filed with the SEC.

Too many, typically under pressure from the external auditors, are adding controls without asking whether they are needed to prevent or detect a material error.

                WHERE’S THE RISK?

The scope does not, and typically should not, include controls that would never result in material weaknesses should they fail. It’s not a matter of whether they are important controls, or required to address the risk-du-jour. It’s a matter of whether they are being relied upon to prevent or detect a material error in the filed financials.

One final point: I don’t care how many ‘entity-level’ controls you have. I only care whether you have selected the right controls to include in scope.  By ‘right’ I mean the combination of controls that can be relied on to function consistently and address the risk of a material error, and are efficient to operate and test.

I welcome your thoughts.

A new front opens in the SOX battle

One of the issues that I address in my SOX Master Classes (the next one is in February) has come of age.

I am talking about the certification signed by the CEO and CFO and included in the quarterly filing with the SEC – the one required by Section 302 of the Sarbanes-Oxley Act.

The issue is this:

• The CEO and CFO are required by law to assess the state of internal control over financial reporting (and disclosure control) every quarter and report whether or not it is effective as of the date of the quarterly filing.
• For their own as well as the company’s protection, they need to have a reasonable basis for that assessment.
• Tests of internal control over financial reporting are typically spread over the year. Some perform tests in every quarter; some during at least a couple of quarters; and few limit their testing to the fourth quarter.
• Deficiencies in the controls are identified during that testing.
• Those deficiencies may be assessed as potential material weaknesses if not corrected and retested prior to the end of the year.
• As a result, potential material weakness frequently not only exist but are known to exist at the time that the CEO and CFO are required to assess and certify internal control over financial reporting.
• But, for whatever reason, these potential material weaknesses either are not reported to the CEO and CFO (which fails one of the Section 302 requirements: they have to certify that they know about control issues) or are ignored.
• The CEO and CFO may certify that the systems of internal control and disclosure controls are adequate when they are not.

This is what I have to say in Management’s Guide to Sarbanes-Oxley Section 404: Maximize Value Within Your Organization:

In the past, most CEOs and chief financial officers (CFOs) have signed their annual and quarterly certifications—which are included in the financial statements filed with the SEC on Form 10-Q and required by Section 302 of Sarbanes-Oxley—without a rigorous examination of internal controls. Ideally, management has integrated the quarterly and annual assessment processes. Although management is not required to test all its key controls every quarter, it should perform some degree of testing each quarter to support the quarterly Section 302 certification. At a minimum, the Section 302 certification process should include a consideration of the status of the Sarbanes-Oxley project, the results of testing, the severity of any identified control deficiencies, and management’s corrective action plans.

When I was writing the book, I talked to the SEC about this issue. They said that they understood it but it was not a priority at that time.

Well “the times, they are a-changing”.

This recently appeared on the CFO magazine web site in an article on SEC Focuses on Internal Control by a former chief accountant of the SEC’s Division of Enforcement. In the middle of the article is this section:

Specific issues that investigators have been addressing include whether a material weakness: (1) existed in a reporting period before a restatement; (2) was adequately described as to scope; (3) existed, even if there was no material error; and (4) existed in connection with controls and procedures for disclosure, or in connection with 302 certification processes.

In the book and in the class, I recommend that management and the SOX PMO consider how the results of testing during earlier quarters are incorporated into the Section 302 certification process.

For example, is the SOX PMO (or equivalent) included in the disclosure review process?

When potential material weaknesses are discovered during SOX or internal audit testing, my suggestion is to review the issue with the legal function. They can advise the CEO and CFO whether this should be disclosed as part of the Section 302 certification.

This new front is clearly starting to open.

Don’t let it pull you under.

I welcome your comments.

Going crazy with COSO 2013 for SOX

For some reason, I only just saw a new PwC publication, Present and functioning: Fine-tuning your ICFR using the COSO update, dated November 2014.

PwC provided the project team for the COSO 2013 update of the Internal Controls – Integrated Framework, so their advice and insight should merit our attention.

The trouble is that it very easy to go overboard and do much more work than is necessary to update your SOX program for COSO 2013.

I fear that PwC may help people go crazy, rather than perform the few additional procedures necessary. I respect those who have said, rightly in my view, that if you were able to comply with the requirements of COSO 1992 (the original version) and either the SEC guidance (in their Interpretive Guidance) or PCAOB Standard Number 5, you should already be in compliance with COSO 2013.

The key is to be able to demonstrate that.

We need to remember these facts:

1. Neither the SEC nor the PCAOB has updated regulatory guidance for management or the external auditor since the release of COSO 2013. That guidance, reinforced by the PCAOB October 2013 Staff Practice Report) mandates a top-down and risk-based approach. It requires a focus on the potential for a material error or omission in the financial statements filed with the SEC.
2. COSO 2013 says that internal control is effective when it reduces the risk to the achievement of objectives to acceptable levels. For SOX, that means that there are no material weaknesses.
3. COSO 2013 also says that a principle can be deemed present and functioning if there are no “major deficiencies” that represent a significant level of risk to the achievement of the objective – in other words, there are no material weaknesses due to a failure of elements relating to a principle.

Now let’s have a look at what PwC has to say.

“With the COSO’s 1992 Control Framework being superseded by the 2013 updated edition on December 15, 2014, now is the time for companies to use the updated framework to evaluate the effectiveness of their systems of internal control over financial reporting.”

I agree with this statement. This is a great opportunity to ensure an effective and efficient program is in place.

“The updated framework formalizes 17 principles that stipulate more granular evaluative criteria to help a company’s management assess the design and operating effectiveness of its ICFR.”

They forget to say that COSO informs us that internal control is effective if it reduces risk to the achievement of objectives to acceptable levels. They also forget to remind us that the SOX assessment must be top-down, risk-based, and focused on the potential for a material error or omission.

“We don’t believe that implementation of the 2013 framework affects management’s existing control activities…. assuming that a company’s control activities have been assessed as effective, reevaluating them according to the 2013 framework is not necessary.”

While there is an element of truth to this, organizations should not be assessing control activities in isolation – they should be assessing whether the combination of controls provides reasonable assurance that there are no material errors or omissions. Focusing on one component by itself is insufficient and, I believe, incorrect.

In addition, the selection of controls for reliance should always be re-evaluated as the business is likely to have changed, including materiality, significant accounts and locations, and so on.

“We believe the most immediate value of applying the 2013 framework lies in the opportunity it provides for taking a fresh look at indirect entity-level controls.”

Again, the SOX scoping should be focused on the combination of controls that provides reasonable assurance. In addition, some principles (such as the hiring and training of employees, or the provision of training and obtaining certification of employees in the code of conduct) are performed at the activity level. COSO tells us that activities in each of the COSO components may exist at any level of the organization. So, we need to recognize that indirect controls may operate at the entity (corporate) level, activity level, or any level in between (such as at the business unit or regional level).

Having said which, the principles do offer us a new opportunity to determine which of these indirect controls need to be included in scope because a failure would represent an unacceptable level of risk – because they raise to an unacceptable level the likelihood that one or more key direct control relied on to prevent or detect a material error or omission might fail.

But, it all has to be within the context that we are focusing the scope, and the SOX program as a whole, on the risk of a material error or omission!

“…fine-tune the design and related documentation of indirect ELCs [entity-level controls] through mapping them to principles.”

Many have misguided organizations, telling them to “map their controls to the principles”. The proper guidance is to “identify the controls you are relying on to provide reasonable assurance that the principles are present and functioning”. Again, we need to remember that the principles can be deemed present and functioning if a failure would not represent a material weakness.

It is correct to say that if you have indirect controls (at entity or another level) that are not required to provide that reasonable assurance, they do not need to be included in scope for SOX.

“…we have noted the following areas in which management’s assessment has indicated room for optimization or improvement in control documentation.”

I suspect that the issue is not limited to control documentation! There is always room for improvement and it is useful to see what PwC has identified.

“Leading companies are formalizing or clarifying and incorporating into their evaluations of ICFR certain indirect ELCs that support existing human resources policies. Such controls usually consist of approvals of new hires and employee transfers (including background checks and assessments of requisite skills and experience when appropriate), requirements for professional certifications and training (e.g., in new and complex accounting standards), succession planning and retention of competent employees, and periodic reviews of employee performance to assess requisite skill levels and conduct. Compensation programs aligned with expected performance, competencies, and behaviors are also important to support ICFR objectives.”

If you believe that any organization’s HR policies and practices provide the assurance you need that every single key control is performed by individuals with the appropriate experience, knowledge, training, and so on, I have a bridge to sell you!

While it is very important to have excellence in hiring, training, supervision, career development, promotion and so on, I do not believe that for SOX it is productive to spend much time on controls in this area.

I very much prefer to assess the capabilities and competence of each control owner as part of the evaluation of the design and operation of each individual key control.

“In many organizations, the evaluation of fraud risks related to financial reporting is integrated into the overall assessment of financial-reporting risks……… In identifying and evaluating those risks, management investigates incentives, pressures, opportunities, attitudes, and rationalizations that might exist throughout the company in different departments and among various personnel.”

The first statement is (I hope) true, although I personally perform a separate assessment of fraud risk (focused on the risk of a material error or omission due to fraud) and generally find that they are addressed by the controls already identified for mistakes.

PwC talks about ‘scenarios’, while I talk about ‘fraud schemes’. In each case, we are talking about ‘how’ the fraud would be committed – an essential step in understanding the true nature of the risk and the controls that would prevent or detect it, if material.

However, going crazy about the fraud triangle is not recommended. We should focus on how we can provide reasonable assurance that a material error or omission due to fraud might be prevented or detected, and remember that the number of people with the ability to commit such a fraud is limited. More than 80% of reported material frauds have been perpetrated by the CEO and CFO acting together, not individuals “throughout the company in different departments and among various personnel.” Rationalization, for example, is an intensely personal action and not something that can be detected by looking broadly at even a segment of the workforce.

“Companies taking a thoughtful approach in transitioning to the 2013 framework—rather than viewing it as a mere compliance exercise—are finding value in the identification of opportunities to strengthen their ICFR.”

We are back on solid ground.

The focus has to remain solidly grounded on identifying and then testing the design and operation of the controls relied upon to prevent or detect a material error or omission. A top-down and risk-based approach is mandated.

Going beyond this may have value in improving operations and the achievement of other (than SOX) business objectives.

But let’s not go crazy!

I welcome your comments and, especially, your experiences with COSO 2013 and your external auditors.

By the way, I think it is well past time for COSO to issue a statement or other guidance to set people straight on the COSO 2013 principles when it comes to SOX. They need to explain that the primary evaluation criterion for effective internal control is whether there is reasonable assurance that risk to the achievement of principles is at an acceptable level. Then they need to explain that the principles offer more granulated guidance that can be used in assessing that risk and whether it is acceptable, but assessing the principles without the context of risk is misunderstanding COSO 2013.

Do you agree?

 

Let’s get practical with COSO 2013

The highly respected periodical Compliance Week just published an article that appears to reflect misunderstanding if not panic in some quarters over the updated COSO Internal Control – Integrated Framework.

Written by Tammy Whitehouse with a title of COSO Framework Update Introduces New Measure of Deficiency , the piece includes quotes from interviews with a number of consultants and other experts, including the new chairman of COSO and me.

I was struck by a number of apparent inferences:
1. There is confusion over the use by COSO of the term “major deficiency” when for SOX purposes we have been using the terms deficiency, significant deficiency, and material weakness.
2. People are worried because they are unsure how the updated framework will affect their SOX program.
3. Some seem to believe that if they have a deficiency related to one risk area that will affect their assessment for SOX.

Let’s take each of these in turn. By the way, Tammy was unable to include in the article my comments on these points, presumably for lack of space.

1. When it comes to SOX, we continue to use the same terms as before. As Bob Hirth pointed out, COSO recognizes that when there are regulations in an area, as there are for SOX with SEC and PCAOB guidance, that takes precedence.

We should also recognize that the term in COSO is not new. Organizations and their internal auditors have been using it for decades.

2. If an organization was truly basing their SOX assessment on the prior version of the COSO framework, they are already “compliant” with the updated version. The only issue is that they have to be able to show how they achieve the principles. This can be done with minimal effort through a management self-assessment. Where the level of risk justifies, consistent with current practice, related key controls are identified and tested. (I expect the IIA will publish an update to my SOX book in a few months that guides this process.)

3. If you have a deficiency in a different risk area, such as in compliance with safety regulations or the delivery of revenue growth, that will not prevent you from assessing your internal control over financial reporting as effective.

I believe the only implementation issue will be on how much evidence you need to support management’s assertion that all the principles are present and functioning. I believe, and have many experts supporting me on this, that you need to consider the risk to the financial statements if there is a defect in a principle. That risk is indirect; please refer to the discussion in AS5 on entity-level controls that have an indirect effect. The greater the risk, the greater the need for key controls to support and augment management’s self-assessment.

As for those who are concerned because they haven’t read this long document. Let me reassure you. If you read and understood COSO 1992 you will not find anything different in 2013, at least anything substantial. The 17 principles were there before; they now have been emphasized. Some discussions, such as on monitoring, are improved.

This is NOT a radical new document that should cause concern.

But that doesn’t mean you should leave implementation for SOX to 2014! Do your self-assessment now and start any remediation now, because it will take time to upgrade issues like the composition and practices of the audit committee, or the training of staff.

I welcome your comments.