Although little research evidence exists on the effectiveness of audit sampling, auditors should consider the effectiveness of audit sampling compared to other sources of evidence, and the use of statistical compared to nonstatistical sampling for both tests of controls and tests of details to develop the most effective and efficient sampling plans. Auditors that use nonstatistical sampling techniques should evaluate procedures to determine whether sample sizes and evaluation of results are comparable to sample sizes and conclusions reached using statistical methods. Auditors also often fail to project sample misstatements and explicitly consider sampling risk; auditor performance in the evaluation of samples is enhanced with the use of standardized sampling templates.

For more information on this study, please contact Randy Elder, rjelder@syr.edu.

Elder, R. J., A. D. Akresh, S. M. Glover, J. L. Higgs, and J. Liljegren.  2013. Audit sampling research: A synthesis and implications for future research. Auditing: A Journal of Practice & Theory 32 (Supplement 1): 99-129

While research has influenced auditing standards for audit sampling, academic research provides limited insights into the current use of audit sampling. We synthesize relevant research based on a sampling decision framework and suggest areas for additional research. Important judgments include determining:

• Does sampling apply
• What type of sampling to apply (e.g., attribute or monetary sampling)
• Whether to use statistical or nonstatistical techniques, including appropriate inputs to determine sample size and evaluate results
• Consideration of environmental factors such as regulation, litigation, competition, culture, and technology 

We first design a framework of the audit sampling process based on existing auditing standards and guidance. We then review relevant literature for each step in the audit process. A fairly extensive literature exists on some sampling issues, such as determination of sample size and projection of misstatements found in the sample. An extensive, but generally dated literature also exists on various statistical sampling techniques. However, limited evidence exists for many issues related to audit sampling.

Auditing standards and guidance on audit sampling have not changed significantly since SAS No. 39 (1981) and the first Audit Sampling Accounting and Auditing Guide (AICPA 1983). However, a review of the literature suggests there have been major changes in sampling practices over the last three decades. Key findings from previous research include:

• Limited research evidence exists on the extent of the use of statistical and nonstatistical sampling for tests of controls and tests of details, and how use of these methods has changed over time or across client characteristics or other environmental factors.
• Little research evidence also exists as to the effectiveness of audit sampling relative to other audit procedures or the effectiveness of nonstatistical audit sampling relative to statistical audit sampling in providing sufficient audit evidence.
• When auditors select samples statistically (e.g., randomly) and evaluate the results nonstatistically, research suggests they may be prone to decision biases.
• Auditors often underestimate risks in order to minimize the extent of testing in tests of details, which could potentially compromise audit effectiveness.
• Several studies find that auditors may not consistently project sample misstatements as required by auditing standards, which could lead to incorrect acceptance of accounting populations. However, more recent research suggests that when decision aids such as templates are used, auditors do usually project misstatements observed in the sample to the population.  

http://www.auditingresearchsummaries.org

The study has implications for audit planning and practice that are not specifically discussed in the paper.  First, the study provides some support for a seemingly obvious position that seniors/managers are more efficient in identifying financial statement errors than audit staff.  Second, the study shows that when auditors are given time to freely sort errors (e.g., when the errors have already been identified in the audit), they prefer to classify the errors by audit objective.  This finding is somewhat counterintuitive, given that auditors’ knowledge is primarily structured around transaction cycles.  The authors suggest that this finding may be attributed to auditors thinking about the causes of errors when sorting through errors, which would correspond more directly to audit objectives than to transaction cycles.    

Coyne, M. P., S. F. Biggs, and J. S. Rich. 2010. Priming/Reaction-Time Evidence of the Structure of Auditors’ Knowledge of Financial Statement Errors. Auditing: A Journal of Practice and Theory 29(1):99-123.

Research into how knowledge of financial statement errors is organized in auditors’ memory has an important practical goal to improve decision making in auditing contexts.  Prior research has concluded that auditors structure their knowledge of financial statement errors primarily around audit objectives.  This stream of research has typically used cue sorting method, and it has been believed that other research methods would reach the same conclusions regarding how auditors structure their knowledge of financial statement errors.  

However, the cue sorting method has one important limitation for studying memory structure. The act of sorting cues requires a participant to develop sorting strategies and make conscious choices about sorting cues into categories. These sorting strategies and choices may or may not be representative of the structure or organization of the knowledge in participants’ memory.  For example, an auditor may decide to sort financial statement errors alphabetically, which is an unlikely organization of the auditor’s knowledge.  The purpose of this study is to determine if the finding that auditor knowledge of financial statement errors is organized primarily around audit objectives can be replicated with the priming/reaction time method.  This method is often preferred in psychological research for studying the structure of knowledge in memory.

Coyne et al. collect their evidence via an experiment with audit staff, seniors, and managers as participants.  In this experiment auditors complete a task designed to assess how financial statement errors are organized in auditors’ memory.  The task is completed by each auditor initially using the method used in prior research (the free sort method), and then using the method preferred in psychological research for studying the structure of knowledge in memory (the priming/reaction time method).  The free sort method requires the auditors to sort 35 financial statement errors into groups of “audit differences that go together.”  In the priming/reaction time method, Coyne et al. measure participants’ speed of recognition of several financial statement error descriptions as audit objectives or transaction cycle primes. The experiments were conducted in the early 2000’s time period.

• Coyne et al. find that when auditors use the free sort method, the results of prior research are generally replicated.  That is, auditors sort errors in a way that indicates that they structure their knowledge of financial statement errors primarily around audit objectives.
• Coyne et al. find that seniors’/managers’ average financial error recognition speed in the priming/reaction time method are significantly faster than the audit staff’s average recognition speed. 
• Coyne et al. find that all auditors’ speed of financial statement error recognition is faster when they are primed to think of transaction cycles, than when they are primed to think of audit objectives.  This effect is especially pronounced for seniors/managers.  Thus, Coyne et al. find that seniors’/managers’ speed of financial error recognition is significantly faster when they are primed to think of transaction cycles than audit objectives.    
• In a supplemental analysis, Coyne et al. find that seniors/managers are marginally more accurate than staff, and are significantly more accurate than staff with transaction cycle primes.  This finding further supports the study’s conclusion that seniors/managers organize their knowledge about financial statement errors around transaction cycles. 
• In a supplemental analysis, Coyne et al. find that staff are more accurate with transaction cycle primes than audit objective primes.  This finding provides some evidence that staff may organize errors around transaction cycles, and no evidence that staff organize errors around audit objectives.