Auditing Section Research Summaries Space

A Database of Auditing Research - Building Bridges with Practice

This is a public Custom Hive  public

Posts

  • 1-5 of 5
  • Jennifer M Mueller-Phillips
    Behind the Numbers: Insights into Large Audit Firm Sampling...
    research summary posted July 27, 2015 by Jennifer M Mueller-Phillips, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.02 Sample Selection – use of statistical sampling 
    Title:
    Behind the Numbers: Insights into Large Audit Firm Sampling Policies.
    Practical Implications:

    Given the limited evidence on firms’ sampling policies after Sarbanes-Oxley, these findings contribute to the current literature on audit sampling and provide insights into sampling policies and procedures that are important for researchers, educators, regulators, and practitioners to better understand the application of audit sampling in the current audit environment. This study provides evidence on current sampling practices and identifies important differences in sampling policy among the largest audit firms. Responses represent firm policy, and although the sampling experts indicate that they believe that firm guidance is followed in the field, actual sampling practices may vary.

    Citation:

    Christensen, B. E., Elder, R. J., & Glover, S. M. 2015. Behind the Numbers: Insights into Large Audit Firm Sampling Policies. Accounting Horizons 29 (1): 61-81.

    Keywords:
    audit misstatements, audit sampling, materiality, statistical sampling
    Purpose of the Study:

    This study addresses a number of research questions regarding the current state of audit sampling. Audit sampling is a fundamental audit testing procedure. Over the last two decades there have been significant changes in audit approaches, including strategic systems auditing in the 1990s and federally mandated audits of internal control over financial reporting for large public companies as a result of the Sarbanes-Oxley Act of 2002 (SOX). Revisions to audit approaches have the potential to change the nature and extent of audit sampling techniques used by accounting firms. For instance, the requirement to audit internal control over financial reporting has necessarily increased the extent and importance of tests of controls, many of which are performed using sampling. Additionally, the Public Company Accounting Oversight Board (PCAOB) has identified sampling as an area needing more emphasis, and inspection reports have identified multiple issues regarding audit sampling, including small and non-representative samples and incorrect or lack of error projection, among others.

    The study focuses on the policies in place at the firms and not necessarily how these policies are implemented in the field. However, due in part to internal firm and federal oversight, discussions with firm experts indicate that audit teams are expected to comply with firm sampling guidance. 

    Design/Method/ Approach:

    The survey asked respondents a number of open-ended questions regarding sampling policies and practices currently in place at the Big 4 and two other international accounting firms. The authors worked in coordination with the participating firms. A version was sent by email in Spring 2013 to one of the Big 4 firms for completion and feedback, after which some additional clarifications were made before distributing electronically to the other firms.

    Findings:
    • The sampling methods differ significantly among the largest auditing firms; while some emphasize statistical methods, others use nonstatistical methods.
    • Firms frequently use different inputs to these sampling models, thus resulting in relatively different sample sizes.
    • The authors find variation in the planned level of expected error, and they also find differences in error projection methods used and how firms respond to identified errors and misstatements.
    • Sampling approaches and parameters within most firms are identical for large public and smaller private companies despite the likely differences in business and engagement risk.
    • Some firms have significantly changed their approach to revenue testing due to PCAOB inspections, relying more heavily on substantive testing using sampling than other substantive testing such as analytical procedures.
    • Some firms have significantly changed their approach to revenue testing due to PCAOB inspections, relying more heavily on substantive testing using sampling than other substantive testing such as analytical procedures.
       
    Category:
    Auditing Procedures - Nature - Timing and Extent
    Sub-category:
    Sample Selection – use of statistical sampling
  • Jennifer M Mueller-Phillips
    Field Data on Accounting Error Rates and Audit Sampling
    research summary posted February 19, 2015 by Jennifer M Mueller-Phillips, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.02 Sample Selection – use of statistical sampling, 08.04 Auditors’ Professional Skepticism, 09.0 Auditor Judgment, 09.08 Evaluation of Errors – Statistical and Non-statistical 
    Title:
    Field Data on Accounting Error Rates and Audit Sampling
    Practical Implications:

    This study provides several important practice implications. First, prior research on audit sampling that relied on the assumption of relatively large error rates may not provide useful guidance for post-SOX audit sampling populations. Second, auditing educators, regulators, and standard setters benefit from an updated understanding of how auditors apply audit sampling guidance in auditing standards when using audit sampling in the field. For example, knowing the relatively high compliance (compared with prior periods) with requirements in auditing standards should impact the way audit sampling is taught in universities and firm trainings, how peer and federal inspectors address audit sampling issues, as well as the need for further clarity of auditing standards. Auditors also benefit as they consider the sampling techniques and input assumptions that will produce the most effective and efficient sampling plans. Specifically, an important implication of our study is related to the impact of standardized sampling templates. The firm in this study mandated the use of such templates, which contributed to levels of explicit consideration of error projection, sufficiency of sample sizes, and of sampling risk in planning and evaluating sample testing.

    For more information on this study, please contact Steve Glover.

    Citation:

    Durney, M., R. J. Elder, and S. M. Glover. 2014. Field data on accounting error rates and audit sampling. Auditing: A Journal of Practice and Theory 33 (2): 79-110

    Keywords:
    Sampling, error rates, error projection, sampling risk
    Purpose of the Study:

    Prior research has examined error characteristics of accounting populations. Many studies investigating audit sampling techniques rely on assumptions concerning the error characteristics of accounting populations. Prior research studies examining auditor performance when using audit sampling have reported:

    • Decreasing sample sizes for tests of details.
    • Auditors frequently fail to project sample errors.
    • Auditors do not consider sampling risk when projecting sample errors.

    These studies involve data from periods preceding the events resulting in the Sarbanes–Oxley Act (hereafter, SOX). Much has happened since SOX, including a renewed focus on audit quality, new auditing and accounting standards, and the creation of the PCAOB. Using proprietary post-SOX data from a large accounting firm, the authors report on:

    • Error rates in populations subject to audit sampling.
    • Auditor compliance with auditing standards with regards to error projection, sample size, and consideration of sampling risk.
    Design/Method/ Approach:

    Data for the study is comprised of the results of 160 different sampling applications from a large auditing firm in 2005 and 2006. The sampling applications were applied across a range of financial statement accounts including accounts receivable, inventory, loans, expenses, plant additions, and revenues. All the tests were substantive tests of details and meant to be representative of the population.

    Findings:

    The authors find the following:

    • Error rates in populations subject to audit sampling are significantly lower in magnitude and frequency than researchers have traditionally assumed.
    • Significantly larger sample sizes and higher error projection rates than reported in previous studies using pre-SOX data.
    • Explicit consideration of sampling risk by auditors.
    Category:
    Auditing Procedures - Nature - Timing and Extent, Auditor Judgment
    Sub-category:
    Conclusions Based on Samples, Evaluation of Errors - Statistical and Non-statistical, Sample Selection – use of statistical sampling
  • Jennifer M Mueller-Phillips
    Audit Sampling Research: A Synthesis and Implications for...
    research summary posted December 1, 2014 by Jennifer M Mueller-Phillips, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.02 Sample Selection – use of statistical sampling, 08.03 Conclusions Based on Samples, 09.0 Auditor Judgment, 09.08 Evaluation of Errors – Statistical and Non-statistical 
    Title:
    Audit Sampling Research: A Synthesis and Implications for Future Research
    Practical Implications:

    Although little research evidence exists on the effectiveness of audit sampling, auditors should consider the effectiveness of audit sampling compared to other sources of evidence, and the use of statistical compared to nonstatistical sampling for both tests of controls and tests of details to develop the most effective and efficient sampling plans. Auditors that use nonstatistical sampling techniques should evaluate procedures to determine whether sample sizes and evaluation of results are comparable to sample sizes and conclusions reached using statistical methods. Auditors also often fail to project sample misstatements and explicitly consider sampling risk; auditor performance in the evaluation of samples is enhanced with the use of standardized sampling templates.

    For more information on this study, please contact Randy Elder, rjelder@syr.edu.

    Citation:

    Elder, R. J., A. D. Akresh, S. M. Glover, J. L. Higgs, and J. Liljegren.  2013. Audit sampling research: A synthesis and implications for future research. Auditing: A Journal of Practice & Theory 32 (Supplement 1): 99-129

    Keywords:
    Audit sampling, audit evidence, post-SOX environment
    Purpose of the Study:

    While research has influenced auditing standards for audit sampling, academic research provides limited insights into the current use of audit sampling. We synthesize relevant research based on a sampling decision framework and suggest areas for additional research. Important judgments include determining:

    • Does sampling apply
    • What type of sampling to apply (e.g., attribute or monetary sampling)
    • Whether to use statistical or nonstatistical techniques, including appropriate inputs to determine sample size and evaluate results
    • Consideration of environmental factors such as regulation, litigation, competition, culture, and technology 
    Design/Method/ Approach:

    We first design a framework of the audit sampling process based on existing auditing standards and guidance. We then review relevant literature for each step in the audit process. A fairly extensive literature exists on some sampling issues, such as determination of sample size and projection of misstatements found in the sample. An extensive, but generally dated literature also exists on various statistical sampling techniques. However, limited evidence exists for many issues related to audit sampling.

    Findings:

    Auditing standards and guidance on audit sampling have not changed significantly since SAS No. 39 (1981) and the first Audit Sampling Accounting and Auditing Guide (AICPA 1983). However, a review of the literature suggests there have been major changes in sampling practices over the last three decades. Key findings from previous research include:

    • Limited research evidence exists on the extent of the use of statistical and nonstatistical sampling for tests of controls and tests of details, and how use of these methods has changed over time or across client characteristics or other environmental factors.
    • Little research evidence also exists as to the effectiveness of audit sampling relative to other audit procedures or the effectiveness of nonstatistical audit sampling relative to statistical audit sampling in providing sufficient audit evidence.
    • When auditors select samples statistically (e.g., randomly) and evaluate the results nonstatistically, research suggests they may be prone to decision biases.
    • Auditors often underestimate risks in order to minimize the extent of testing in tests of details, which could potentially compromise audit effectiveness.
    • Several studies find that auditors may not consistently project sample misstatements as required by auditing standards, which could lead to incorrect acceptance of accounting populations. However, more recent research suggests that when decision aids such as templates are used, auditors do usually project misstatements observed in the sample to the population.  
    Category:
    Auditing Procedures - Nature - Timing and Extent, Auditor Judgment
    Sub-category:
    Conclusions Based on Samples, Evaluation of Errors - Statistical and Non-statistical, Sample Selection – use of statistical sampling
  • Jennifer M Mueller-Phillips
    Differential Evaluation of Audit Evidence from Fixed versus...
    research summary posted November 24, 2014 by Jennifer M Mueller-Phillips, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.02 Sample Selection – use of statistical sampling, 09.0 Auditor Judgment, 09.03 Adequacy of Evidence 
    Title:
    Differential Evaluation of Audit Evidence from Fixed versus Sequential Sampling
    Practical Implications:

    The results have implications about situations in which others evaluate the auditor’s work after the fact, such as the audit review process or the examination of audit evidence by regulators, jurors, or judges. In such situations, decision makers need to evaluate the strength of previously gathered audit evidence, and to judge the extent to which the evidence supports a previously reached conclusion. Regarding the assessed sufficiency of audit evidence, the results suggest that evaluators of the auditor’s work could require larger sample sizes under sequential sampling than under fixed sampling, to support the same level of confidence in the auditor’s opinion. Although sequential sampling might in fact increase audit efficiency, the findings suggest that this benefit could be negated by subsequent unfavorable assessment of audit evidence from a sequential sampling plan.

    For more information on this study, please contact Marietta Peytcheva.

    Citation:

    Gillett, P. R., and M. Peytcheva. 2011. Differential evaluation of audit evidence from fixed versus sequential sampling. Behavioral Research in Accounting 23 (1): 65-85. 

    Keywords:
    sampling plan; audit evidence; evidence evaluation; stopping rules
    Purpose of the Study:

    The authors examine whether the assessed value of audit evidence depends on whether it was collected using fixed or sequential sampling. Opposing views are held by the two main schools of statistical theory: Bayesian statisticians maintain the value of audit evidence is the same, regardless of the sampling plan, whereas frequentist statisticians argue the sampling plan should affect evidence evaluation. This study tests empirically how using fixed versus sequential sampling plans influences the subsequent evaluation of audit evidence.

    Design/Method/ Approach:

    In two experiments, audit students and practicing auditors assess the strength of audit evidence obtained using different sampling plans. The experimental task involves testing of internal controls as part of the audit of the revenue cycle. The research evidence is collected in 2005—2008.

    Findings:

    Audit evidence obtained from a fixed sampling plan is invariably assessed as stronger, by both audit students and practicing auditors. This finding is consistent with frequentist statistical theory, but not with Bayesian theory. Participants in the first experiment who considered the fixed sampling plan (the plan more widely used in audit practice) were prone to consider additional factors (such as whether or not they had gathered the evidence themselves) in their assessment of the strength of observed audit evidence. Participants exposed to the sequential plan, however, did not respond to these additional factors but assigned generally lower strength to evidence obtained from a sequential plan. Qualitative data on the reasoning behind auditors’ observed preferences suggest that auditors perceive fixed sampling plans as unbiased. Sequential plans, in contrast, are perceived to leave room for bias. The main concern auditors report regarding the sequential sampling plan is that this plan presents samplers with an opportunity to influence the test results by increasing or altering sample size until the desired results are observed.

    Category:
    Auditing Procedures - Nature - Timing and Extent, Auditor Judgment
    Sub-category:
    Adequacy of Evidence, Sample Selection – use of statistical sampling
  • Jennifer M Mueller-Phillips
    Haphazard Sampling: Selection Biases Induced by Control...
    research summary posted October 20, 2014 by Jennifer M Mueller-Phillips, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.02 Sample Selection – use of statistical sampling, 11.0 Audit Quality and Quality Control, 11.09 Evaluation of Evidence 
    Title:
    Haphazard Sampling: Selection Biases Induced by Control Listing Properties and the Estimation Consequences of these Biases
    Practical Implications:

    These findings suggest that the properties of haphazard samples chosen from control listings are likely to differ from those of random samples.  Subconscious effort minimization and diversification behaviors, coupled with visual perception artifacts, yield samples that violate requirements for independence and equal selection probability.  These violations, in turn, are likely to produce biased error projections with difficult to discern risk properties.  Although widely used and specifically identified in audit standards as a sampling technique that can be employed to obtain a representative sample, haphazard sampling may not be a reliable substitute for random sampling.

     

    For more information on this study, please contact Thomas W. Hall

    Citation:

    Hall, T. W., A. W. Higson, B. J. Pierce, K. H. Price, and C. J. Skousen. 2012. Haphazard sampling: Selection biases induced by control listing properties and the estimation consequences of these biases. Behavioral Research in Accounting 24 (2):101-132.

    Keywords:
    nonstatistical sampling, haphazard sampling, sample selection bias
    Purpose of the Study:

    Audit samples represent an important type of evidence used to assess the error status of accounting populations.  As a result of its professional acceptance and lower cost, nonstatistical sampling historically has played a prominent role in audit sampling.  Haphazard sampling is a nonstatistical technique commonly used to emulate random sampling and consequently when used no explicit selection strategy should be employed.  However, a number of sampling experts have expressed doubts about whether haphazard sampling is a reliable substitute for random sampling.

     

    We hypothesized that haphazard samples differ from random samples because the haphazard selection process is influenced by: (1) auditor behaviors intended to minimize sample selection effort and to ensure a diversified sample composition, and (2) variations in the appearance of control listing entries.

    Design/Method/ Approach:

    We created two control listings representing a population of accounts receivable and a population of inventory items.  The accounts receivable control listing consisted of 22 pages with 792 customer accounts, while the inventory control listing consisted of 26 pages with 1,404 inventory items.

     

    We conducted three experiments in which participants were instructed to select haphazard samples from the control listings.  Participants in the first experiment were 75 students enrolled in either senior or master’s-level accounting courses at a public university located in the southwestern United States.  The second experiment utilized 40 university students in the United Kingdom who were enrolled in either senior or master’s-level accounting courses.  These students serve as effective proxies for entry-level auditors, who select most samples.  The third experiment utilized 53 audit seniors from two offices of a Big 4 audit firm located in the southwestern United States.  Because of time constraints, the audit seniors sampled only from the inventory control listing.  Upon completion of the sample selection process, all participants completed an exit survey.  The data collection was completed by the middle of 2009.

    Findings:

    As expected, we observed unequal page selection rates.  Most participants began the sample selection process on the first page of control listings.  Also, sample selections exhibited a high positive correlation with listing serial position, indicating that participants tended to proceed through the control listings in serial fashion.  Statistical analyses confirmed that participants exhibited higher selection rates for early pages, followed by declining selection rates for middle pages, with an upturn in selection rates for ending pages.  All of these results are inconsistent with the properties of random samples.

     

    Line selection rates also were unequal and consistent with expectations that visual perception biases influence sample selections.  Line entries with a low level of visual crowding tended to have higher selection rates than line entries with a high level of visual crowding.  Similarly, line entries with a high level of luminance contrast were selected more often than line entries with lower levels of luminance contrast.  Statistical tests confirmed that lines at the top and bottom of pages were overrepresented in each participant group’s samples. As with page selection, these results are inconsistent with the properties of random samples.

    Category:
    Audit Quality & Quality Control, Auditing Procedures - Nature - Timing and Extent
    Sub-category:
    Evaluation of Evidence, Sample Selection – use of statistical sampling

Filter by Type

Filter by Tag