JIS Senior Editors' Blog

Journal of Information Systems

This is a public blog  publicRSS

blog entry

    Recent publication - Exception Prioritization in the...
    blog entry posted August 26, 2016 by Roger Debreceny, last edited August 26, 2016 
    Recent publication - Exception Prioritization in the Continuous Auditing Environment: A Framework and Experimental Evaluation

    Exception Prioritization in the Continuous Auditing Environment: A Framework and Experimental Evaluation

    A paper that is coming out in a  future issue isPei Li, David Y. Chan and Alexander Kogan  Exception Prioritization in the Continuous Auditing Environment: A Framework and Experimental Evaluation. Here is a blog post by the authors.

    An important potential benefit of a continuous auditing system is the improvement of the efficiency of auditing through the automation of audit procedures.  However, the inherent nature of a continuous auditing system may in fact diminish any economic benefits from automation.  Researchers have found that the large volume of exceptions generated by a continuous auditing system can be overwhelming for an internal audit department.  Exceptions are irregular or suspicious transactions, or internal controls violations identified by the continuous auditing system that need to be manually investigated by the auditor.  As a result, a large number of exceptions diminishes the economic efficiency gained through automation. 

    In this study, we propose a framework that systematically prioritizes exceptions based on the likelihood of an exception being erroneous or fraudulent.  The framework is based on an initial set of rules that are generated by internal auditors to detect irregular transactions.  These rules are assigned a confidence level depending on their effectiveness in detecting errors or fraud.  The continuous auditing system identifies transactions that violate a single rule or multiple rules and labels those transactions as exceptions.  The suspicion score of each of these exceptions is generated using the Dempster-Shafer theory of belief functions.  Then, the auditors are guided to investigate those exceptions that have the highest suspicion scores.

    The framework incorporates an advanced feature that learns from identified errors and fraud after each iterative process through employing the following two methods.  First, the confidence level of a rule that contributed to finding erroneous or fraudulent transactions is revised on the basis of investigative results of auditors.  Second, a rule learner algorithm is implemented to add new rules to the original set of rules that were developed by the auditors.  Although the confidence levels of the auditor-based rules have been refined, the newly identified erroneous and fraudulent transactions might have new attributes, which are not represented in the existing rules.  This method captures the attributes of errors and fraud to create new rules that will attempt to find similar instances subsequently.

    The framework consists of six stages: 1) generation of exceptions using defined rules, 2) assignment of suspicion scores to exceptions using belief functions, 3) exception prioritization, 4) exception investigation, 5) rule confidence level update utilizing back propagation, and 6) rule(s) addition utilizing a rule learner algorithm.  

    We validated the proposed framework using a simulated experiment. The experiment used accounts payable transactional data from a technology company and simulated irregular transactions.  The results from the experiment provide evidence that the proposed framework has the ability to effectively prioritize erroneous or fraudulent transactions.  Furthermore, the results indicates that using back propagation to refine the confidence levels of rules and using a rule learner algorithm to generate additional rules helped improve the effectiveness of the prioritization of exceptions in subsequent iterations of the process.