This article is important to practitioners as well as academics because they will be using data analytics in accounting and auditing tasks and will need to specify system design characteristics needed to effectively accomplish these tasks. The authors identify several research questions for further study.

Schneider, G. P., J. Dai, D. J. Janvrin, K. Ajayi, and R. L. Raschke. 2015. Infer, Predict, and Assure: Accounting Opportunities in Data Analytics. Accounting Horizons 29 (3): 719-742.

The objective of this paper is to examine how data analytics will impact the accounting and auditing environment, identify emerging management and regulatory challenges, and outline new research opportunities. To incorporate and process both structured and unstructured data to support decisions, accountants are working with a new set of sophisticated tools known as data analytics. Data analytics is the process of using structured and unstructured data through the applications of various analytic techniques such as statistical and quantitative analysis and explanatory and predictive models to provide useful information to decision-makers. Data analytics involves complex procedures that extract useful knowledge from large data repositories. Compared with conventional approaches, data analytics offer advantages in terms of cost-effectiveness), scalability, and capability to identify new patterns in real time.

Several challenges and risks may arise with data analytics. First, how can voluminous data stored in heterogeneous and differently organized data sources be converted into structured, hence well interpretable, format? In doing so, uncorrelated data needs to be filtered out. The challenge is to identify what data needs to be filtered out. Further, how can structured data repositories be managed, processed, and transformed in order to derive needed information for decision-making purposes? Finally, data analytics applications often are highly scalable.

This article is a commentary. 

The authors expand upon the challenges and risks via adopting the organizing principles of the metatheory of AIS and apply it to data analytics. The first principle states that data analytics research should be task-focused. Their analysis concentrated on three tasks to which accountants often apply data analytics: infer, predict, and assure. The second organizing principle notes that task requirements are the start of the process that establishes the set of system design characteristics needed. They note that the lack of accepted models of data analytics and related perceptions is a significant challenge that should be considered. The third principle suggests that the impact of data analytics on task performance should be examined within the context of cognitive, technological, and organizational contingency factors. They identify several research questions related to each of these contingency factors. Finally, the fourth principle states that the outcome of data analytics is task performance. The authors discuss how evaluating the infer, predict, and assure tasks completed with data analytics may occur at either the individual or organizational level. In addition, often the outcome of data analytics contains private and/or confidential information and more research is needed to examine how organizations can address their responsibilities to maintain privacy and confidentiality.

http://www.auditingresearchsummaries.org

The Big Data qualities of Volume, Velocity, Variety, and Veracity contribute to the creation of the following Big Data Gaps: Data Consistency, Data Integrity, Data Identification, Data Aggregation, and Data Confidentiality. These Big Data Gaps create challenges for current CA systems. The paper outlines possible solutions to these gaps along with needed research topics with the aim of increasing the applicability of continuous auditing systems to Big Data. Big Data is a business phenomenon that is here to stay, and CA systems need to adapt to its challenges.

Zhang, J., X. Yang, and D. Appelbaum. 2015. Toward Effective Big Data Analysis in Continuous Auditing. Accounting Horizons 29 (2): 469-476.

Big Data originates from traditional transactions systems, as well as new sources such as emails, phone calls, Internet activities, social media, news media, sensor recordings and videos, and RFID tags. Since much of this Big Data informs and affects corporate decisions that are important to both internal and external corporate stakeholders, auditors will need to expand their current scope of data analysis.

Certain qualities, known as the four Vs, define the term Big Data: namely, massive Volume or size of the database, high Velocity of data added on a continuous basis, large Variety of types of data, and uncertain Veracity. Due to volume and velocity, the application of continuous auditing (CA) has become increasingly relevant for the automation and real-time analysis of Big Data. However, massive volume and high velocity also introduce gaps between the present state of audit analytics and the requirements of Big Data analytics in a continuous audit context. Moreover, variety and uncertain veracity present challenges beyond the capability of current CA methods. The purpose of this paper is to identify these gaps and challenges and to point out the need for updating the CA system to accommodate Big Data analysis.

This article is a commentary.

The authors identify and discuss potential remediation for the five Big Data Gaps:

• Data Consistency: Big Data systems supporting key business processes usually consist of a patchwork of different systems, where data may be fully or partially replicated, the informational content may be overlapped, and more derived data may be stored. This situation gives rise to the serious gap in data consistency.
• Data Integrity: the volume and types of data are so expansive that it becomes more difficult to identify individual data as well as data sets that have been modified/ deleted/ hidden/ destroyed because of operating error, procedural error, illegal access, and/or network transmission failures. This difficulty in identifying integrity issues can create a domino effect that causes other reliable data to lose their value for audit analysis purposes, thus increasing audit risk in a Big Data, continuous audit environment.
• Data Identification: refers to records that link two or more separately recorded pieces of information about the same individual or entity.
• Data Aggregation: necessary for the normal operation of continuous auditing using Big Data and to meaningfully summarize and simplify the Big Data that is most likely coming from different sources.
• Data Confidentiality: certain data, or more often the associations among data points, are sensitive and cannot be released to others.

The authors identify the nine CA Challenges:

• Audit on data with different formats
• Audit on asynchronous data
• Audit on conflicting data
• Audit on illegally tampered data
• Audit on incomplete data
• Audit on data with various identifiers
• Audit on aggregated data
• Search encrypted data
• Audit on encrypted data

http://www.auditingresearchsummaries.org

The authors outline potential pitfalls in the use of big data noted in auditing and psychology literature. Specifically, they focus on areas in which auditor decision making may be negatively influenced. They note that big data may lead to inefficient and/or incorrect decisions resulting from having too much information, not being able to determine what is relevant to the decision, not finding correct patterns in the data/finding incorrect patterns, and having data which may be too ambiguous to be used effectively. They then outline potential solutions to these problems, such as having decision aids, fitting the task to the system to auditor experience level, and providing contextual (“big picture”) training.

Brown-Liburd, H., H. Issa, and D. Lombardi. 2015. Behavioral implications of big data’s impact on audit judgment and decision making and future research directions. Accounting Horizons 29(2): 451-468.

Big data—high-volume, high-velocity, and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making—has been the new trendy topic in the future of auditing. This paper outlines potential implications on the actions of the auditor resulting from the use of big data, specifically drawing from literature on psychology and auditing.

This paper summarizes the current literature in psychology and auditing specifically to bring to light potential issues resulting from incorporation of big data in auditing. No data is used and no analyses are performed.

The authors find four primary areas where research in auditing and psychology indicate potential negative effects of big data on audit performance, specifically in auditor judgment and decision making:

• Auditors may make inefficient decisions, struggle to differentiate relevant information, struggle to identify relationships between detail and overall trends, and disregard large portions of information (information overload).
• Auditors may make poorer decisions due to the excess of irrelevant information innate in big data. They may be unable to filter relevant information from the noise, thus being less efficient in analyzing data and potentially using irrelevant information in decision making (information relevance).
• The sheer magnitude of data to analyze may force auditors to become worse at recognizing patterns, applying knowledge to the audit task, weighting evidence, or extrapolating patterns into longer times series (pattern recognition).
• Auditors—if they have a low tolerance for ambiguity—may neglect information once a simple solution or response is identified, potentially ignoring relevant but more complex information (ambiguity).

The authors then provide examples of solutions to these problems, such as 1) providing decision models 2) having inexperienced auditors use expert systems (requires minimal auditor interpretation) 3) provide auditors with more contextual experience and training so they can interpret patterns in big data 4) leverage predictive models that can indicate areas of increased risk.

www.auditingresearchsummaries.org

This study makes three primary contributions to the accounting and auditing communities of researchers, practitioners, and regulators. First, it is the first study to document economically significant benefits consistent with COSO’s assertions that formal ICM activities enhance the strength of internal control systems and the efficiency of external examinations of such internal control systems. Second, this study contributes to the literature in accounting information systems (AIS) by documenting specific benefits associated with strategically focused information technology. Finally, it enhances the IT-related auditing literature.

However, there needs to be further research to examine the link between other types of ICM technology and specific outcomes. Similarly, it remains an open question as to how technology impacts other areas of internal control monitoring. COSO asserts the role of monitoring not only aids the financial reporting process, but also ultimately the organization’s overall system of governance, including operational decision-making. Though this study documents benefit-oriented assurance outcomes, the research area remains fruitful with respect to the impact of ICM technology on other audit quality measures.

For more information on this study, please contact Adi Masli.

Masli, A., G. F. Peters, V. J. Richardson, and J. M. Sanchez. 2010. Examining the Potential Benefits of Internal Control Monitoring Technology. The Accounting Review 85 (3): 1001-1034. 

The Committee of Sponsoring Organizations of the Treadway Commission (COSO) asserts that improved ICM practices should enhance the efficiency, effectiveness, and assurance of internal control processes. In January 2009, COSO issued guidance on ICM and observed that firms often struggle with realizing the benefits of ICM-related activities. The purpose of the current study is to examine the potential benefits that firms can realize from implementing ICM technology specifically aimed at monitoring and supporting the effectiveness of their internal control systems.

The authors develop and test hypotheses for three explicit potential benefits associated with internal and external assurance outcomes: (1) more effective internal control systems, (2) enhanced audit efficiency, and (3) timely audit reporting. They identify 139 announcements of ICM technology purchases across the time period 2003–2006. The control group consists of all available observations listed in Audit Analytics SOX 404 Internal Controls database during the same periods.

Consistent with the hypotheses, the study documents positive associations between ICM technology initiatives and subsequently stronger internal controls, enhanced audit efficiency, and timely audit reports. Collectively, the main results suggest that ICM technology yields important benefits in both internal and external assurance outcomes. In some of this research’s tests, the findings suggest transformative-oriented ICM technology initiatives yield greater assurance benefits compared to compliance-oriented ICM initiatives.

http://www.auditingresearchsummaries.org

This paper is intended to prompt auditors to take advantage of easier access to population data in today’s digital business environment. By abandoning sampling auditors can develop much more sophisticated models of behavior that can identify anomalies in ways that were not possible before. Auditors can also be more creative in how they treat data, be it in aggregating it across organizational subunits or in larger and smaller time units. Most innovative of all, auditors and/or managers have the ability to continually update their expectation models by investigating errors and anomalies in real time and correcting them, so that the model is not based on flawed data. We find that such error correction greatly improves the accuracy of analytical procedures. Perhaps the most important finding, however, is that almost all the various expectation models we used gave similarly strong results which implies that what really matters is the size of the data set. Once auditors move away from sampling they will find that population data provides great statistical power when developing analytical procedures that reduces the reliance on finding just the right such procedure.

For more information on this study, please contact Alexander Kogan.

Kogan, A., M. Alles, M. Vasarhelyi and J. Wu. 2014. Design and Evaluation of a Continuous Data Level Auditing System. Auditing: A Journal of Practice and Theory. 33 (4): 221-245.

The purpose of this paper is to demonstrate how audit practice may change when auditors have access to real time population data, how to use real world data to develop APs for CA, and compare different analytical procedures in a CA context. In the paper we develop a framework for a continuous data level auditing system and uses a large sample of procurement data from a major health care provider to simulate an implementation of this framework. The first layer of the framework monitors compliance with deterministic business process rules and the second layer consists of analytical monitoring of business processes. A distinction is made between exceptions identified by the first layer and anomalies identified by the second one. The unique capability of continuous auditing to investigate (and possibly remediate) the identified anomalies in ‘‘pseudo-real time’’ (e.g., on a daily basis) is simulated and evaluated.

Our simulated implementation of the data-oriented CA system focuses on the procurement-related BPs and utilizes the data sets extracted from the data warehouse of a healthcare management business with many billions of dollars in assets and close to two hundred thousand employees. The data sets include all procurement cycle daily transactions from October 1st, 2003 through June 30th, 2004.  The number of transaction records for each activity ranges from approximately 330,000 to 550,000. Since we have access to population data, the first step is to undertake tests of details to detect violations of key controls. Once that is done we turn to determining whether there are anomalies that do not violate any established controls but which may be nonetheless indicative of potential problems.

The implementation of the analytical procedure component of the CA system requires creation of the models of expected behavior to enable anomaly detection which we label “continuity equations” (CE). We use advanced statistical models to extract CE from the data, and then by seeding errors we determine how effectively the CE model identifies anomalies. We also investigate the effect of conducting AP on data aggregated in either time or geographically and also the implication of error correction.

Our research shows that when auditors have access to population data there can be significant changes in the role and sequence of audit procedures. Since data access is not a constraint, tests of detail can be carried out first on the complete population data to find exceptions to controls and for transaction verification. Then APs can be used, again, on the complete population data, to find anomalies. This paper shows that while there are differences in the predictive ability and detection performance of various CE models, all models perform reasonably well and no single model performs better on all aspects. From this two important conclusions can be drawn: First, the choice of a particular model across the candidate CE models is less important than the fact that all models yield fairly effective AP tests.  Our second conclusion from the fact that all the CE models yield reasonably effective analytical procedures is that when auditors have access to complete transaction data, the richness of that disaggregate data combined with the reorganization of auditing workflow to implement pseudo-real time error correction makes BP problem detection robust across a variety of expectation models. In other words, it is the nature of the data that serves as audit evidence that is the primary driver of audit effectiveness, with the selection of the specific AP a second order concern—not because the audit benchmark is not important, but because auditing at the process level makes anomalies stand out much more obviously in the data.

http://www.auditingresearchsummaries.org