Posts

Posts

  • Jennifer M Mueller-Phillips
    Training Auditors to Perform Analytical Procedures Using...
    research summary posted February 24, 2015 by Jennifer M Mueller-Phillips, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.01 Substantive Analytical Review – Effectiveness, 09.0 Auditor Judgment, 09.03 Adequacy of Evidence in Auditing Section Research Summary Database > Auditing Section Research Summaries Space public
    Title:
    Training Auditors to Perform Analytical Procedures Using Metacognitive Skills
    Practical Implications:

    This research furthers the understanding of auditors’ judgment performance in four important ways. We show that

    • Effective training in metacognitive skills increases auditors’ diagnostic reasoning by enabling them to control and direct their thinking.
    • Training in both divergent and convergent thinking provides significantly better results than only learning to think divergently. Because the former are better able to piece together all necessary facts.
    • The key to performance improvement due to training in both divergent and convergent thinking is a reduction in a psychological mechanism called “consistency checking.” Auditors trained to use both tend to avoid premature elimination of explanations, instead subjecting explanations they generate to subsequent, explicit evaluation. An important implication of this is that for auditors who try to do both kinds of thinking simultaneously rather than sequentially the best explanation for a problem might not be generated or might be prematurely discarded.
    • In the same amount of time that participants in the other training conditions took to arrive at their inferior answers, auditors trained to use both divergent and convergent thinking chose one of the correct solutions more often, generated better explanations, and eliminated more potentially time-wasting invalid explanations.

    For more information on this study, please contact David Plumlee.

    Citation:

    Plumlee, R. D., B. Rixom, and A. Rosman. 2015. Training auditors to perform analytical procedures using metacognitive skills. The Accounting Review 90 (1): 351-369.

    Keywords:
    metacognition; divergent thinking; convergent thinking; training; analytical procedures; ill-structured tasks.
    Purpose of the Study:

    Auditors encounter many ill-structured tasks. Due, in part, to their greater technical knowledge, partners and managers perform these tasks better than less experienced auditors. Partners and managers also have in their memories a diverse set of problem solutions gained from their experience that they can retrieve as needed to organize and solve ill-structured problems. Less experienced auditors do not have access to these additional experiences and may benefit from a more structured approach to thinking while solving ill-structured tasks. We believe that training less experienced auditors in in metacognition—consciously thinking about one’s thought process—will help close the performance gap. We chose to train less experienced auditors to use a sequential thought process comprised of two metacognitive skills: divergent thinking, where they generate explanations for unusual evidence, followed by convergent thinking, where they evaluate explanations generated and eliminate those judged infeasible. Training less experienced auditors in the proper use of these skills was expected to provide them with the problem-structuring knowledge that managers and partners acquire through their frequent encounters with ill-structured situations. 

    Design/Method/ Approach:

    Auditors with approximately two years of experience were randomly assigned to receive training in either divergent and convergent thinking skills, only divergent, or neither (a control). The training included four separate self-paced online sessions over two weeks. At the end of each session, we measured participants’ comprehension of the training and their ability to apply the specific skills addressed in that session. The fourth session synthesized the previous sessions and included a comprehensive analytical review case to measure whether the training resulted in better performance.

    Findings:

    We found that

    • In response to evidence inconsistent with their expectation, auditors who completed both divergent and convergent thinking training increased both the number and quality of explanations for that evidence. They focus more on generating explanations when performing divergent thinking instead of trying to sort out which alternatives ‘‘make sense.”
    • Training in both skills resulted in a greater ability to generate and ultimately choose one of the two viable explanations in the final case. Auditors trained to use both types of thinking had a fourfold higher likelihood of identifying a logically viable explanation compared to those receiving divergent thinking training alone—and a vastly better likelihood than those having neither type.
    • In a supplemental study, we asked participants about the process they used when generating explanations in the final case. Training in only one of these metacognitive skills leads decision makers to eliminate explanations while they are being generated, possibly eliminating a correct explanation.
    Category:
    Auditing Procedures - Nature - Timing and Extent, Auditor Judgment
    Sub-category:
    Adequacy of Evidence, Substantive Analytical Review – Effectiveness
  • Jennifer M Mueller-Phillips
    Two Decades of Behavioral Research on Analytical Procedures:...
    research summary posted May 28, 2014 by Jennifer M Mueller-Phillips, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.07 Interim Testing Procedures – Nature, Timing and Extent in Auditing Section Research Summary Database > Auditing Section Research Summaries Space public
    Title:
    Two Decades of Behavioral Research on Analytical Procedures: What Have We Learned?
    Practical Implications:

    The previous two decades of APs research have provided a number of important findings regarding the generation and evaluation of hypotheses for significant differences between the auditor’s expectations and the client’s reported results. The authors note that a vast majority of the research focused on performing preliminary and substantive APs. Therefore, this review emphasizes the need for more research on the use of APs during the final stages of the audit. The authors intend for this paper to lead to an increased recognition of the important of performing research on APs with the intent of improving audit practice. 

    Citation:

    Messier, W. F., C. A. Simon, and J. L. Smith. 2013. Two Decades of Behavioral Research on Analytical Procedures: What Have We Learned? Auditing 32 (1).

    Keywords:
    analytical procedures; auditing standards; risk assessment procedures
    Purpose of the Study:

    Audit researchers have conducted a substantial amount of behavioral research examining various aspects of the analytical procedures process. A comprehensive review of behavioral research on external auditors’ use of analytical procedures (APs) published over the last two decades is presented in this paper. With audit regulators recently revising AP standards and PCAOB inspections identifying more deficiencies related to firms’ performance of APs, this review comes at an important time. A review of this research will help identify areas where research has cited deficiencies and suggest ways to overcome those problems. 

    Design/Method/ Approach:

    This review was framed around four phases of the analytical procedures process: develop an expectation, establish a tolerable difference, compare the expectation to the recorded amount and investigate significant differences, and evaluate explanations and corroborative evidence. This approach allows the authors to discuss the phases of the analytical procedure process as outlined in practice. It also allows them to highlight what prior research has found and where future research is needed to improve the understanding and performance of APs. 

    Findings:
    • While the PCAOB’s inspections have repeatedly raised concerns with auditors forming insufficiently precise expectations and setting an appropriate tolerable difference, relatively little research has been conducted in these phases of the framework. 
    • The PCAOB has asserted that audit teams do not always investigate significant differences, a considerable amount of research has examined this phase, mainly testing issues related to the generation of hypotheses.
    • Auditors’ ability to generate hypotheses can be affected by numerous source factors, contextual factors, and auditor characteristics. 
    • The PCAOB has expressed concerns regarding auditors’ lack of obtaining corroborative evidence for client expectations during the fourth phase. 
    Category:
    Auditing Procedures - Nature - Timing and Extent
    Sub-category:
    Interim Testing Procedures – Nature - Timing & Extent
    Home:

    home button

  • Jennifer M Mueller-Phillips
    Was Dodd-Frank Justified in Exempting Small Firms from...
    research summary posted November 26, 2014 by Jennifer M Mueller-Phillips, tagged 04.0 Independence and Ethics, 04.08 Impact of SEC Rules Changes/SarbOx, 06.0 Risk and Risk Management, Including Fraud Risk, 06.06 Earnings Management, 07.0 Internal Control, 07.05 Impact of 404 on Fees and Financial Reporting Quality, 08.0 Auditing Procedures – Nature, Timing and Extent, 08.05 Evaluating Accruals/Detection of Abnormal Accruals, 08.06 Earnings Management – Detection and Response, 14.0 Corporate Matters, 14.01 Earnings Management in Auditing Section Research Summary Database > Auditing Section Research Summaries Space public
    Title:
    Was Dodd-Frank Justified in Exempting Small Firms from Section 404b Compliance?
    Practical Implications:

    Our study evaluates a provision of Dodd-Frank which provided permanent exemption from Section 404b compliance to non-accelerated filers. Our results show that these small firms did not improve their reporting quality to the same extent as large firms implying that the Dodd-Frank exemption will probably serve to keep the reporting quality of the exempted firms at lower than achievable levels.

    We also note that as part of the Dodd-Frank legislation, the SEC was given a mandate to investigate raising the Section 404b exemption requirements from $75 million to $250 million in market capitalization (Dodd Frank 2010). While the SEC eventually decided to leave the exemption criterion at $75 million, this matter is still considered to be an open topic (SEC 2011). Our study informs this ongoing debate.

    For more information on this study, please contact

    Anthony D. Holder, PhD, CPA

    Assistant Professor, Department of Accounting - MS 103

    University of Toledo

    Toledo, OH 43606-3390

    Email: Anthony.Holder@utoledo.edu

    Web:    http://homepages.utoledo.edu/aholder4/

    Phone: 1.419.530.2560

    Fax: 1.419.530.2873 

    Citation:

    Holder, A., K. Karim, and A. Robin. 2013. Was Dodd-Frank Justified in Exempting Small Firms from Section 404b Compliance? Accounting Horizons 27 (1): 1-22.

    Keywords:
    Sarbanes-Oxley; Dodd-Frank; earnings management; exempt filers
    Purpose of the Study:

    A major component of the Sarbanes-Oxley Act of 2002 (SOX) is Section 404b, which requires auditor certification of internal controls. However, not all firms were required to comply with this section. Fearing that compliance costs may be prohibitive, SOX allowed a temporary exemption to small firms called non-accelerated filers (typically those firms with market capitalizations under $75 million). Later, the Dodd-Frank Act of 2010 made this exemption permanent.

    Needless to say, both 404b itself and the small-firm exemption, remain controversial. At the heart of the issue, as with any regulation, is the cost-benefit tradeoff. In this particular instance, what are the potential benefits small firms would have obtained had they been subject to SOX Section 404b? By focusing just on the costs of compliance, we may be overlooking these benefits. We consider these foregone benefits an opportunity cost.

    The purpose of our study is to estimate this opportunity cost. We estimate the benefits lost by small firms, because they were not subject to SOX Section 404b.

    Design/Method/ Approach:

    Our sample contains listed firms (subject to SOX), divided into the large (accelerated) and small (non-accelerated) categories. Our data span the SOX period and are from 1995-2009. We measure reporting gains using two standard approaches, one measuring the extent of earnings management and the other measuring accrual quality.

    The reporting benefits foregone by small-firms can be understood by comparing the following two quantities:

    • Post-SOX reporting gains achieved by large firms (accelerated filers).
    • Post-SOX reporting gains achieved by small firms (non-accelerated filers). If these gains (or losses) are smaller than those achieved by large firms, we know there is an opportunity cost.
    Findings:

    We detect a significant deterioration in reporting quality for non-accelerated filers but not for accelerated filers. The result is invariant to whether we compare non-accelerated filers with all accelerated filers or only with small accelerated filers.  Our findings suggest a significant opportunity cost for the exemption. Although the consideration of the cost of Section 404b compliance is outside the scope of our study, our result concerning the opportunity cost suggests that it may have been premature to grant permanent exemption to the non-accelerated filers. This result is especially important, considering contemporaneous discussions to grant Section 404b exemption to even larger firms (up to a market capitalization of $500 million).

    Category:
    Auditing Procedures - Nature - Timing and Extent, Corporate Matters, Independence & Ethics, Internal Control, Risk & Risk Management - Including Fraud Risk
    Sub-category:
    Earnings Management – Detection and Response, Earnings Management, Evaluating Accruals/Detection of Abnormal Accruals, Impact of 404 on Fees and Financial Reporting Quality, Impact of SEC Rules Changes/SarBox
  • Jennifer M Mueller-Phillips
    Welcome to the day-to-day of internal auditors: How do they...
    research summary posted July 30, 2015 by Jennifer M Mueller-Phillips, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.11 Reliance on Internal Auditors, 13.0 Governance, 13.07 Internal auditor role and involvement in controls and reporting in Auditing Section Research Summary Database > Auditing Section Research Summaries Space public
    Title:
    Welcome to the day-to-day of internal auditors: How do they cope with conflict?
    Practical Implications:

    This study makes an original contribution to the development of new knowledge on internal auditing. It concludes that internal auditors tend to lack independence and audit committee members often exercise disturbingly weak power (on the internal audit function), as compared to the top managers. This points to the difficulty of applying an idealized conception of independence and purist governance principles to practice. That is, it encourages auditors to consider the appropriateness of internal auditing as a meaningful independent assurance device in operating the corporate governance "mosaic."

    Citation:

    Roussy, M. 2015. Welcome to the day-to-day of internal auditors: How do they cope with conflict? Auditing: A Journal of Practice and Theory 34 (2): 237-264.

    Keywords:
    audit committee, conflicts, coping, independence, internal audit
    Purpose of the Study:

    The internal audit function (IAF) was made mandatory in both private and public companies in North America in the early 2000s. This paper proposes a ''micro-level'' analysis of the way in which internal auditors express role conflicts in their day-to-day practice and how they perceive, manage, and resolve them. The study seeks to show that the independence of the internal auditor is often not up to the standards that are expected of the internal audit function (IAF), and therefore unlikely to play an effective governance oversight role compatible with the ideal of the new public management governance reform.

    Design/Method/ Approach:

    A field study was conducted involving semi-structured interviews with 42 internal auditors working in 13 public sector organizations. Interviews were conducted between May and October 2010. Each interviewee had 10-15 years of experience in internal auditing, with 20-25 years of professional experience total. Data was interpreted in the light of a theoretical analysis framework designed especially for this study. Organizational and social context was taken into consideration for each interview.

    Findings:

    Overall, the results indicate that internal auditors have a relative lack of independence (as compared with the Institute of Internal Auditors’ standards.) More specifically:

    • While internal auditors are strategic in managing conflicts, they do not consider the cumulative effect that their coping behavior has on their lack of independence.
    • The audit committee does not greatly influence internal auditors’ coping tactics at any stage in the audit process.
    • Auditors use “pragmatic” behavior because they are embedded in a specific organizational and social context that they are ‘‘forced’’ to take into account.
    • The analysis casts doubt on the audit committee’s ability and commitment to consolidate and ensure internal auditor independence.

    Ultimately, the study concludes that internal auditors behave as if the IAF were a means for managerial control, instead of a governance mechanism.

    Category:
    Auditing Procedures - Nature - Timing and Extent, Governance
    Sub-category:
    Internal auditor role and involvement in controls and reporting, Reliance on Internal Auditors
  • The Auditing Section
    Why Do Auditor’s Over-Rely on Weak Analytical Procedures? T...
    research summary posted April 13, 2012 by The Auditing Section, tagged 08.0 Auditing Procedures – Nature, Timing and Extent, 08.01 Substantive Analytical Review – Effectiveness, 09.0 Auditor Judgment, 09.03 Adequacy of Evidence in Auditing Section Research Summary Database > Auditing Section Research Summaries Space public
    Title:
    Why Do Auditor’s Over-Rely on Weak Analytical Procedures? The Role of Outcome and Precision
    Practical Implications:

    Analytical procedures are used frequently and increasingly are relied upon as substantive evidence. Based on this study, auditors are insensitive to the impreciseness of the analytical procedure when the results are favorable and may be a cause for over-reliance on weak evidence.  Performing a stronger, more precise analytical procedure caused participants in the favorable outcome situation to become more aware of the weakness of the initial procedure and re-evaluate their evidence strength rating. Further, evidence suggests that having auditors consider the possible weaknesses of an analytical procedure prior to performing the procedure will cause them to rate the strength of the evidence from a weak analytical procedure lower. Overall, this suggests a need to better train auditors in performing and interpreting analytical procedures.

    In a discussion of Glover et al.’s paper, McDaniel asks whether the findings may indicate that auditors in the unfavorable outcome (i.e. there is a material difference) are under-relying on the evidence rather than that auditors in the favorable outcome (no material difference) are over-relying on the evidence. Glover et al. respond that the over-relying of the evidence is of concern to regulators and the alternative does not explain all of the results.  McDaniel also notes that the case study was of a company in the financial industry but that the participants were not required to have any financial industry experience. Glover et al. note that the interest income item is the issue which is not specific to the industry or complicated.  McDaniel also notes concerns about a potential “anchoring” effect as the participants performed their analytical procedures based on prior year working paper results.  In response, Glover et al. discuss this feature of an audit. 

    Citation:

    Glover, S. M., D. F. Prawitt, and T. J. Wilks. 2005.  Why Do Auditor’s Over-Rely on Weak Analytical Procedures?  The Role of Outcome and Precision.  Auditing: A Journal of Practice & Theory 24 (Supplement):  197-220.  

    McDaniel, L. 2005.  DISCUSSION OF Why Do Auditor’s Over-Rely on Weak Analytical Procedures?  The Role of Outcome and Precision.  Auditing: A Journal of Practice & Theory 24 (Supplement):  221-228. 

    Glover, S. M., D. F. Prawlitt, and T. J. Wilks. 2005. REPLY TO DISCUSSION OF Why Do Auditor’s Over-Rely on Weak Analytical Procedures?  The Role of Outcome and Precision.  Auditing: A Journal of Practice & Theory 24 (Supplement):  229-232.

    Keywords:
    outcome; evidence quality; substantive analytical procedures; evidence assessment;
    Purpose of the Study:

    In 2000, a Public Oversight Board panel viewed audit work papers and determined that 20% of the time substantive analytical procedures were weak and provided insufficient evidence to support the conclusion. This study aims to look at one of the possible reasons why auditors’ over-rely on weak, unreliable analytical procedures.  The authors hypothesize that auditors do not consider their existing knowledge about the quality of the procedure when the outcome indicates that the balance is “fairly stated.”

    Design/Method/ Approach:

    The authors performed two experiments prior to 2005 where a material misstatement exists and a “weak, unreliable” (highly aggregated) analytical procedure is used.  In experiment 1, senior associates from one Big 4 accounting firm were asked to perform an interest revenue analytical procedure at the annual grand total level and compare the results to the client’s unaudited balance.  The balance is manipulated so that some participants’ results indicate there is no significant difference (i.e. favorable outcome) and the other participants’ results indicate that there is a significant difference (i.e. unfavorable outcome).  Participants evaluated the strength of the analytical procedure and concluded regarding a misstatement.  Additional disaggregated computations (interest revenue calculations broken down by type of loan and performed quarterly vs. annual basis) were then provided. Participants responded to the procedure strength of the aggregated analytical procedure.  In experiment 2, different senior associates from one Big 4 accounting firm were asked to document the weaknesses of the analytical procedure prior to performing the procedure.

    Findings:

    Experiment 1

    • Auditors attribute more evidential strength to the results of weak analytical procedures if the results indicate no material difference than the identical procedure where the results indicate a material difference.  In addition, auditors in the  avorable outcome are more likely to assign a lower risk of material misstatement and assess the balance as fairly stated, than those in the unfavorable outcome.
    •  After viewing the stronger, disaggregated analysis, auditors in the favorable outcome were more likely to revise their prior conclusion but auditors in the unfavorable outcome did not. Further, auditors in the favorable outcome were also more likely to downgrade their evidential strength assessment of the initial analytical procedure.
    • Altogether, the authors believe this indicates a potential for over-reliance on weak high level analytical procedures and that in situations where analytical procedures indicate no significant difference, auditors are less likely to realize their procedure may produce imprecise expectations and deem it to be a stronger procedure than it really is.

    Experiment 2

    • Auditors who were told to consider the potential weaknesses of the analytical procedure before performing the analysis were more likely to rate the strength of the evidence as lower than those who were not.
    • The prompt to consider potential weaknesses did not reduce the evidential strength assessment as much as requiring the additional analytical procedure in Experiment 1.
    Category:
    Auditing Procedures - Nature - Timing and Extent, Auditor Judgment
    Sub-category:
    Adequacy of Evidence, Substantive Analytical Review – Effectiveness
    Home:
    home button