Auditing Section Research Summaries Space

A Database of Auditing Research - Building Bridges with Practice

This is a public Custom Hive  public

research summary

    Design and Evaluation of a Continuous Data Level Auditing...
    research summary posted February 15, 2015 by Jennifer M Mueller-Phillips, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.01 Substantive Analytical Review – Effectiveness, 08.08 Projecting Interim Testing Conclusions Year End, 08.09 Impact of Technology on Audit Procedures 
    508 Views
    Title:
    Design and Evaluation of a Continuous Data Level Auditing System
    Practical Implications:

    This paper is intended to prompt auditors to take advantage of easier access to population data in today’s digital business environment. By abandoning sampling auditors can develop much more sophisticated models of behavior that can identify anomalies in ways that were not possible before. Auditors can also be more creative in how they treat data, be it in aggregating it across organizational subunits or in larger and smaller time units. Most innovative of all, auditors and/or managers have the ability to continually update their expectation models by investigating errors and anomalies in real time and correcting them, so that the model is not based on flawed data. We find that such error correction greatly improves the accuracy of analytical procedures. Perhaps the most important finding, however, is that almost all the various expectation models we used gave similarly strong results which implies that what really matters is the size of the data set. Once auditors move away from sampling they will find that population data provides great statistical power when developing analytical procedures that reduces the reliance on finding just the right such procedure.

    For more information on this study, please contact Alexander Kogan.

    Citation:

    Kogan, A., M. Alles, M. Vasarhelyi and J. Wu. 2014. Design and Evaluation of a Continuous Data Level Auditing System. Auditing: A Journal of Practice and Theory. 33 (4): 221-245.

    Keywords:
    continuous auditing (CA), analytical procedures (AP), population data, auditing practice.
    Purpose of the Study:

    The purpose of this paper is to demonstrate how audit practice may change when auditors have access to real time population data, how to use real world data to develop APs for CA, and compare different analytical procedures in a CA context. In the paper we develop a framework for a continuous data level auditing system and uses a large sample of procurement data from a major health care provider to simulate an implementation of this framework. The first layer of the framework monitors compliance with deterministic business process rules and the second layer consists of analytical monitoring of business processes. A distinction is made between exceptions identified by the first layer and anomalies identified by the second one. The unique capability of continuous auditing to investigate (and possibly remediate) the identified anomalies in ‘‘pseudo-real time’’ (e.g., on a daily basis) is simulated and evaluated.

    Design/Method/ Approach:

    Our simulated implementation of the data-oriented CA system focuses on the procurement-related BPs and utilizes the data sets extracted from the data warehouse of a healthcare management business with many billions of dollars in assets and close to two hundred thousand employees. The data sets include all procurement cycle daily transactions from October 1st, 2003 through June 30th, 2004.  The number of transaction records for each activity ranges from approximately 330,000 to 550,000. Since we have access to population data, the first step is to undertake tests of details to detect violations of key controls. Once that is done we turn to determining whether there are anomalies that do not violate any established controls but which may be nonetheless indicative of potential problems.

    The implementation of the analytical procedure component of the CA system requires creation of the models of expected behavior to enable anomaly detection which we label “continuity equations” (CE). We use advanced statistical models to extract CE from the data, and then by seeding errors we determine how effectively the CE model identifies anomalies. We also investigate the effect of conducting AP on data aggregated in either time or geographically and also the implication of error correction.

    Findings:

    Our research shows that when auditors have access to population data there can be significant changes in the role and sequence of audit procedures. Since data access is not a constraint, tests of detail can be carried out first on the complete population data to find exceptions to controls and for transaction verification. Then APs can be used, again, on the complete population data, to find anomalies. This paper shows that while there are differences in the predictive ability and detection performance of various CE models, all models perform reasonably well and no single model performs better on all aspects. From this two important conclusions can be drawn: First, the choice of a particular model across the candidate CE models is less important than the fact that all models yield fairly effective AP tests.  Our second conclusion from the fact that all the CE models yield reasonably effective analytical procedures is that when auditors have access to complete transaction data, the richness of that disaggregate data combined with the reorganization of auditing workflow to implement pseudo-real time error correction makes BP problem detection robust across a variety of expectation models. In other words, it is the nature of the data that serves as audit evidence that is the primary driver of audit effectiveness, with the selection of the specific AP a second order concern—not because the audit benchmark is not important, but because auditing at the process level makes anomalies stand out much more obviously in the data.

    Category:
    Auditing Procedures - Nature - Timing and Extent, Auditor Judgment
    Sub-category:
    Evaluation of Errors - Statistical and Non-statistical, Impact of Technology on Audit Procedures Confirmation – Process and Evaluation of Responses, Substantive Analytical Review – Effectiveness