Continuous controls monitoring – grossly misunderstood!
One of my regular complaints is about people who assert continuous controls monitoring is an automated technique. Sorry, but while automation can monitor transactions and changes to master data for integrity, it is not a complete solution to the monitoring of controls.
Continuous monitoring of transactions to inspect their integrity can be 100% automated, with just the review of exceptions manual.
But, the continuous monitoring of controls can only be partially achieved through automation. Consider:
- Testing transactions does not provide positive assurance that controls are present and operating effectively. They only tell you that the transactions are clean. (If the transactions are clean, you have a strong indication that controls are not present or ineffective. But monitoring is about confirming controls are present)
- Some controls (such as the review by a manager of a reconciliation, the performance of a physical inventory count, or employee understanding of the code of conduct and other key policies) do not lend themselves to automated testing
To perform continuous monitoring of controls, you need a combination of techniques: automated monitoring, automated control testing, and other tests such as surveys and manual test procedures.
Some talk about the acronym CCM/T (continuous control monitoring/transactions). This is (IMHO) wrong. You can have CM/T (continuous monitoring/transactions) and you can have CCM (continuous control monitoring) – a partly automated and partly manual process. But you can’t have CCM/T.
Leave a reply to Jay R. Taylor Cancel reply
Norman Marks on Governance, Risk Management, and Internal Audit 
Norman, your thoughts regarding these definitional distinctions are on target as usual. I appreciate your thought leadership in this and other areas.
Your comments are very helpful to the IA profession as we try to move our data analytics / continuous monitoring initiatives forward. I believe everyone is struggling with this. The insight you’ve provided here helps us better design our CCM, by ensuring we take the holistic approach to looking at all the key controls (manual and automated)within a well-designed control framework – – if we really intend to do either CCM or CCM/T.
the way you define CCM, CCM/T or CM/T is correct and helps to clearify the differences between them.
It is really necessary to have a clear understanding about controls monitoring. Generally, it is obviously possible to use automated monitoring as long as you ensure that irregularities are recognized and managed – which is usually a manual activity.
Norman – You make some interesting points. The most important one, I think, is that if you are not monitoring a control then it can’t be called continuous controls monitoring. With that in mind, it’s important to remember that the generally accepted definition of continuous controls monitoring includes four different types of controls (transactions (CCM-T), user access (CCM-SOD), master data (CCM-MD) and application configuration (CCM-AC)). When we talk about monitoring controls we’re almost always – as you suggest – looking not just at one of these controls (e.g. CCM-T). The real value is in monitoring and correlating exceptions across multiple types of controls (e.g. did a user change the payee address on a vendor and then approve a PO for that vendor). And, of course, you have to be able to do it across multiple applications.
To your point that monitoring can be done manually…I’d agree. In fact, traditional auditing is basically manual monitoring right? But it’s not continuous. The value of CCM is that it’s analyzing 100% of transactions 100% of the time. Of course, there are controls that aren’t suited (and never will be) to automation. In my experience working with customers to deploy CCM roughly 60-75% of controls can be automated. The rest will always be things that require other tests, as you say (e.g. we’ll never be able to automat a test for measuring “tone at the top”). Where I would disagree with your post is that it suggests that every control will have a manual component and a component that can be automated. I’m not sure that’s true. Yes…in a portfolio of controls there will be those that will never be automated. But in my experience there’s a good 60%+ that can be automated with CCM.
I would like to share my research into effective continuous auditing. You can download it (CRCA) from my LinkedIn Profile: http://www.linkedin.com/profile?viewProfile=&key=1228225&trk=tab_pro
To the point in M. Evan’s comment.
A control is something that you do to make sure something is being done right.
You can have:
– controls over transactions, to ensure they are valid, complete, accurate, etc.
– controls over changes to master files, to ensure they are valid, accurate, etc.
– controls over access, to ensure that is is approved, necessary, not a fraud risk, not misused, etc.
– controls over configurations, not just of applications, but of any automated control or activity, to ensure they are correct, and not changed without approvals, testing, etc.
– and so on.
Monitoring controls means that you are performing activities that provide ongoing assurance that the controls (over whatever) are in place, adequate to the task, and operating effectively.
I accept CM/T, CM/MD, CM/UA (far more than SOD), CM/Config (more than applications), etc. I also accept CCM.
I don’t accept CCM/T, etc., because those techniques are monitoring the activity not the control over the activity.
The CCM/T/MD/SOD/AC nomenclature is unfortunate as it does not accurately describe the domain. The current CCM naming convention has more to do with the order vendors developed solutions and the efforts by industry analysts to create a category. To the best of my knowledge ACL pioneered the name CCM to describe their solution which evaluates data. The initial products from firms such as Virsa/SAP, Approva and Logical Apps monitored SOD controls and thus they were included as CCM solutions.
Oversight is a vendor in the “CCM” domain as it is currently defined and based on experiences implementing solutions we’d advocate an evolution in the naming convention that reflects the real world usage.
In broad terms organizations are continuously monitoring controls and/or continuously monitoring data. Monitoring controls serves compliance/attestation requirements. Monitoring data can also support compliance requirements as well as serving operational needs. It is valuable to find a duplicate vendor or customer record whether or not you’re concerned about compliance.
A naming hierarchy based on value and usage would be Continuous Monitoring (CM)with subcategories of Continuous Controls Monitoring (CCM) and Continuous Transaction Monitoring (CTM). CCM should have the categories outlined by Norman. CTM should include master data since an update to a vendor file is a form of a transaction in our experience. (If the market is insistent on separting master data from transaction data then you would have Continous Data Montiring with sub-subcategories of transactions and master data).
As popular as the space is becoming it’s far from too late to correct the naming conventions. We’re just at the beginning of this market.
Patrick, why would you say “it’s far from too late to correct the naming conventions”? The trains may have left the stations but it is not out of line to identify which one is the Amtrack passenger train, the bullet train, or the coal-fired Disney version.
In fact, it would be very helpful to practitioners such as myself to have a better definition of available processes and products to help me find what my organization needs to fill a particular set of needs.
Disagree? Not that it will be easy to modify naming conventions … but a better taxonomy here would be helpful to all of us.
I agree with Patrick. I support the concept of changing the terms to CCM (as defined in the IIA GTAG on continuous auditing) and CTM (which would include all forms of transaction, such as master file changes, intrusion attempts, etc).
Just one point of clarification, the current set of SAP products addresses the monitoring of both automated and manual controls (for all risks, not just compliance or financial risks), the monitoring of business application transactions (including master file changes), and the monitoring of systems access – all across multiple technologies (i.e., not just SAP, but Oracle and other).
Please see my second post, today, extending the discussion.
Hi, Norman – You have to consider that CCM, while technically detective, has two preventive qualities:
1 – Quite often it is monitoring the preventive controls within a business application – it ensures the preventive controls are preventing
2 – Since CCM operates daily or weekly, in most cases, and it looks at a lot more transactions than a quarterly manual sample, it obtains a preventive nature. In other words, the chances are much better I may detect something in time to prevent it from getting worse.
So, CCM is technically detective – but when compared to a quarterly manual sample, it has preventive qualities.
Cordially –
French
First, thanks to all for the comments. Together with comments I have received off-line, these have been helpful in refining my thinking.
Let me make a suggestion: CCM/T might be a valid and useful term if it was defined as including not only the monitoring of transations but also the ability to perform manual tests of controls over transactions, support a management self-assessment process, and use surveys. Do you agree?
The same would have to be applied to CCM/MD, since it is essential not only to ensure that all master file changes are valid and accurate, but that all necessary changes were made. Consider the need to process changes to credit limits, customer tax status (exempt/non-exempt), product prices, etc.
CCM/SoD (which should be changed to reflect not just SoD but user access in general) may be sufficient with just monitoring actual access. That is where the risk lies, not in failing to grant access.
CCM/AC may also be sufficient with only monitoring of changes. Controls are assured by monitoring if it is supplemented by periodic examination of the configurations for validity.
Do you agree?
Apologies for a belated addition to this thread….
I agree with both Norman and Patrick on the distinction between CCM and CTM. I suspect that Patrick is also correct that ACL pioneered the term CCM. I came up with ACL Continuous Controls Monitoring, abbreviated to ACL CCM, as the name for the product that we launched in 2003. A few months after launch I presented the product to the Continuous Auditing Symposium at Rutgers University. Professor Efrim Boritz from the University of Waterloo interrupted me to point out that ACL’s product was not monitoring controls but transactions. I replied that it was monitoring controls effectiveness – but he was, at least in literal terms, quite right. Examination of financial or operational transactional data is not examination of the control itself – even though it can be argued to be a more useful activity.
I have usually argued that analysis of transactional data does 2 things. It can provide an indication of whether a specific control is operating effectively. It can also provide an indication of a risk for which no control was implemented. I think this can apply to CTM in cases where certain risks and anomalies can be detected without testing for compliance with a specific control rule. This can provide an additional aspect of risk monitoring to the CTM process.
I do not quite agree with Patrick’s distinction between monitoring controls and monitoring data. Either way, when using computer technology, it is data that is being examined. I see the distinction as being between monitoring data in a control setting (such as one that prevents the same invoice number from a specific vendor being processed twice – or one that prevents an ERP user from performing a specific action) and monitoring data that represents a specific financial or business activity (a transaction) to which a control should be applied. Using this type of distinction, I would agree with Patrick that monitoring of changes to master data is really another type of transaction monitoring, since it is not a control setting which is being changed.
I certainly agree with Norman’s opening statements about controls monitoring being more than automated techniques. I had begun to question recently whether, in a technology context, control monitoring was really a useful term, since a control is more than just a computer setting. It is a combination of processes that need to be effectively in place to mitigate the risk of something occurring.
Norman’s last post on an expanded view of CCM/X is interesting. I don’t think it helps to clarify the basic difference between monitoring control settings and monitoring transactions. But perhaps what we are saying is these are just 2 specific automated techniques that support the 4 categories of overall CCM.
I have enjoyed reading the many comments on this topic, and I feel like it has sharpened my understanding of CCM. I am responsible for facilitating the implementation of a continuous controls monitoring process at the company I work for. I am a huge fan of ACL; however, ACL is very expensive. I am curious to know if anyone can recommend a less expensive CAAT used in CCM. As of now, I am expected to use MS Excel and Access. Does anyone use Access for CCM? Any advice, forewarnings, success stories are welcomed and appreciated. I already understand the many limitations that exist in using MS Excel for this. I am mainly interested in the use of MS Access for CCM or another CAAT similar to ACL (but less expensive).