The results of this study suggest that auditors’ precision assessments may not be well calibrated for relevant precision factors. Thus, auditors may benefit from additional guidance indicating the factors that should be considered for assessing the precision of analytical
procedures.  Furthermore, audit firms might want to consider integrating some of the findings of this study into future training sessions and/or decision aids that would assist auditors in improving their precision calibration. An insensitivity to important precision factors may lead to over-reliance on analytical procedures, negatively affecting audit effectiveness. Because the allowance for loan losses is an estimate, the results of this study provide insight into factors that could influence the potential effectiveness of audits of estimates. Understanding how auditors evaluate analytical procedure precision for estimates is particularly critical in that analytical procedures may be the only source of assurance for testing these accounts.

McDaniel, L.S. and L.E. Simmons. 2007. Auditors’ assessment and incorporation of expectation precision in evidential analytical procedures. Auditing: A Journal of Practice & Theory 26(1): 1-18.

The precision with which auditors form expectations during analytical procedures is important. The precision of an expectation is a measure of the closeness of the developed expectation to the actual amount and refers to the quality of the expectation, and thus, the quality of the analytical procedure. Professional standards clearly indicate that auditors should be able to form more precise expectations for accounts that are more predictable (income statement relationships generally are more predictable than balance sheet relationships) and when the information related to the account is more disaggregated (i.e., detailed). However, the Public Oversight Board’s (POB) Panel on Audit Effectiveness has found evidence that auditors rely on analytical procedures that do not provide the desired level of assurance, suggesting possible difficulty in assessing precision. To address this finding by the POB, this study investigates auditors’ abilities to assess expectation precision and incorporate their assessments into judgments related to substantive analytical procedures, as required by professional standards. A first step toward developing more effective guidance is obtaining a better understanding of why auditors sometimes fail to effectively apply analytical procedures. As such, the authors aim to answer the following two questions: 

• To what extent do auditors’ precision assessments reflect the level of account predictability and the level of detail of the data used to form expectations? 
• To what extent are auditors’ judgments consistent with their precision assessments? According to professional standards, the following judgments should be consistent with auditors’ precision assessments:

(1)   The level of assurance expected by auditors to be provided by the analytical procedure.

(2)   The range of the difference between the expected and recorded amount.

(3)   The likelihood that the difference between the expected and recorded amount is due to misstatement versus nonmisstatement causes.

The authors gathered their data experimentally at a firm training event for audit seniors and above (the training event occurred sometime during or prior to 2005). The participants were asked to review workpapers which included analytical procedures for two different accounts – the allowance for loan losses (a less predictable account) and interest income (a more predictable account). The expectations for the analytical procedures documented in the workpapers were either based on more or less detailed information. After reviewing the analytical procedures, auditors were asked to assess the precision and the level of assurance provided, provide an expectation range for the account balance, and judge the amount of difference between the expected and recorded amount due to misstatement.

• Overall, auditors assess precision higher when the data forming the expectation are disaggregated (i.e., more detailed) versus
  aggregated (i.e., less detailed). 
• Auditors also assess precision higher for the more predictable account (interest income) versus the less predictable account (the allowance for loan losses). 
• Auditors’ precision assessments for the allowance for loan losses are not significantly different between the disaggregated and aggregated analytical procedures. This finding suggests that in assessing the precision of analytical procedures for less predictable accounts, auditors do not consider the effects of data aggregation. 
• The results also show that for both accounts auditors judge the level of assurance obtained from the analytical procedures consistent with their precision assessments (i.e., higher assessed precision corresponds with a higher level of assurance). 
• Counter to the authors’ expectations, auditors’ precision assessments were not related to their judgments of the range of differences between the expected and recorded amounts or the likelihood of misstatement.

Based on the interviews and problems identified, the authors conjecture that potentially suboptimal auditing methods are being used to evaluate complex estimates which are an important and growing part of the financial statements. This may be negatively impacting audit quality. More specifically, auditors over-rely on management estimates because they lack the knowledge and incentives to behave otherwise. This possibility has direct consequences for auditor professional skepticism because increasing professional skepticism may be less effective unless auditors are also given the requisite knowledge to properly use it. These problems are reinforced by auditing standards and regulators which generally outline/criticize the current auditing methods without suggesting new or better ones.  

Griffith, E., J. Hammersley, and K. Kadous. 2015. Audits of Complex Estimates as Verification of Management Numbers: How Institutional Pressures Shape Practice. Contemporary Accounting Research 32 (3): 833-863.

Complex estimates are increasingly important to financial statements and of growing concern to both regulators and investors. While auditors have well-established procedures for auditing more objective account balances (i.e., valued at historical cost), little is known about the process auditors use to evaluate more subjective, complex estimates. This article conducts interviews with experienced audit personnel to determine how auditors evaluate such estimates, determines the problems with such approaches, and uses “institutional theory” to theorize the reason such problems exist and persist. The authors consider the influence of both audit firms themselves and regulators (i.e., information from PCAOB inspection reports) on auditors’ complex estimate audit procedures.

The authors conducted semi-structured phone interviews with experienced audit personnel. Participants are from 6 large accounting firms with at least manager level experience. Interviews were conducted between October and November 2010. The authors analyzed the audit process steps discussed by participants for complex estimates and coded these steps according to the PCAOB auditing standards related to accounting estimates (AU 342 and 328).  For steps that could not be appropriately classified into ones discussed by the auditing standards, the authors developed additional classifications.

While auditing standards allow for different approaches to evaluating complex estimates (e.g., testing management process, preparing independent estimate, etc.), the authors find that auditors usually just test management’s process (i.e., verifying inputs such as historical cost, understanding who and how estimate is generated, testing controls surrounding process, and testing sensitivity of assumptions used).  

Based on institutional theory, the authors theorize two key reasons that auditors mainly use management process verification when auditing complex estimates instead of other (potentially more creative and skeptical) approaches. The reasons are:

• Both audit firm policies and professional standards generally emphasize management process verification techniques over other potential techniques. Additionally, regulators (i.e., PCAOB) reinforce/encourage this behavior because inspection findings largely focus on problems with auditing management’s process instead of suggesting alternative, superior auditing methods.
• Audit firms employ valuation specialists who have the necessary knowledge to more critically analyze complex estimates. This fact means that financial statement auditors generally do not have the necessary knowledge to critically analyze management’s models or develop an independent expectation. When auditors do use such specialists, they over-rely on their work.
• Given the lack of guidance regarding complex estimates, firms tend to use practices that have been previously legitimized. For auditing of complex estimates, verification (which works well to audit less subjective accounts) is used to audit more subject complex estimates. Auditing standards also mainly emphasize verification.
• Given inspection pressures, firms find it safer and more legitimate to mimic each other’s policies and procedures for auditing complex estimates instead of develop new ones.

http://www.auditingresearchsummaries.org

Incorporating Big Data into an audit poses several challenges. This article establishes how Big Data analytics satisfy requirements of audit evidence, namely that it is sufficient, reliable, and relevant. The authors bring up practical challenges (such as transferring information, privacy protection, and integration with traditional audit evidence) and provide suggestions for addressing them in incorporating Big Data into audit evidence. They also suggest that Big Data can complement tradition audit evidence at every level of audit evidence: financial statement, individual account, and audit objective.

Yoon, K., L. Hoogduin, and L. Zhang. 2015. Big data as complementary audit evidence. Accounting Horizons 29 (2): 431-438.

This paper frames Big Data in the context of audit evidence, specifically looking at the requirements for something to be considered audit evidence, to provide an argument for the usefulness of Big Data to auditors. The authors address the sufficiency, reliability, and relevance of Big Data analytics; they then outline potential challenges to using Big Data for adequate audit evidence.

The authors summarize existing literature on audit evidence as it applies to Big Data. They perform no original analyses, but rather discuss the characteristics of Big Data analytics as they relate to regulations and research findings.

The authors address:

• Sufficiency: The authors suggest that—when used appropriately—Big Data analytics can meet sufficiency requirements for audit evidence. They provide the example of using an employee’s emails to identify motivation or rationalization of fraud to demonstrate Big Data supplementing traditional audit evidence where traditional methods may be deficient in sufficiently documenting audit conclusions.
• Reliability: Big Data, being typically from a third party and massive in nature, is argued to be generally reliable for audit evidence. They note that Big Data can validate things such as shipping terms to independently verify cutoff.
• Relevance: The relevance of Big Data is primarily driven by the timeliness of its availability. Traditional audit evidence is often gathered after-the-fact, however Big Data-based auditing can analyze current trends to provide timely information. They provide several examples, such as using management’s discussion of forecasts. Research has linked overly optimistic press releases to fraud, so using Big Data techniques on earnings forecasts may assist in assigning fraud risk.
• Integration with Traditional Audit Evidence: The authors acknowledge that Big Data may not always easily bridge into traditional audit evidence, however they provide a discussion of weighting evidence—as you would traditional audit evidence—so that more weight is given to the more sufficient, reliable, and relevant evidence.
• Information Transfer: Access to data provides benefits which may be leveraged based on economies of scale, however clients may restrict access to proprietary data. The authors suggest specifically contracting for use of internal data.
• Information Privacy: A common fear of releasing information is that it may be used for a secondary purpose. The authors acknowledge this and suggest that auditors should cooperate with information providers and ensure that information is anonymized.

www.auditingresearchsummaries.org

The results have implications about situations in which others evaluate the auditor’s work after the fact, such as the audit review process or the examination of audit evidence by regulators, jurors, or judges. In such situations, decision makers need to evaluate the strength of previously gathered audit evidence, and to judge the extent to which the evidence supports a previously reached conclusion. Regarding the assessed sufficiency of audit evidence, the results suggest that evaluators of the auditor’s work could require larger sample sizes under sequential sampling than under fixed sampling, to support the same level of confidence in the auditor’s opinion. Although sequential sampling might in fact increase audit efficiency, the findings suggest that this benefit could be negated by subsequent unfavorable assessment of audit evidence from a sequential sampling plan.

For more information on this study, please contact Marietta Peytcheva.

Gillett, P. R., and M. Peytcheva. 2011. Differential evaluation of audit evidence from fixed versus sequential sampling. Behavioral Research in Accounting 23 (1): 65-85. 

The authors examine whether the assessed value of audit evidence depends on whether it was collected using fixed or sequential sampling. Opposing views are held by the two main schools of statistical theory: Bayesian statisticians maintain the value of audit evidence is the same, regardless of the sampling plan, whereas frequentist statisticians argue the sampling plan should affect evidence evaluation. This study tests empirically how using fixed versus sequential sampling plans influences the subsequent evaluation of audit evidence.

In two experiments, audit students and practicing auditors assess the strength of audit evidence obtained using different sampling plans. The experimental task involves testing of internal controls as part of the audit of the revenue cycle. The research evidence is collected in 2005—2008.

Audit evidence obtained from a fixed sampling plan is invariably assessed as stronger, by both audit students and practicing auditors. This finding is consistent with frequentist statistical theory, but not with Bayesian theory. Participants in the first experiment who considered the fixed sampling plan (the plan more widely used in audit practice) were prone to consider additional factors (such as whether or not they had gathered the evidence themselves) in their assessment of the strength of observed audit evidence. Participants exposed to the sequential plan, however, did not respond to these additional factors but assigned generally lower strength to evidence obtained from a sequential plan. Qualitative data on the reasoning behind auditors’ observed preferences suggest that auditors perceive fixed sampling plans as unbiased. Sequential plans, in contrast, are perceived to leave room for bias. The main concern auditors report regarding the sequential sampling plan is that this plan presents samplers with an opportunity to influence the test results by increasing or altering sample size until the desired results are observed.

http://www.auditingresearchsummaries.org

The study provides evidence that the format of the audit evidence (using the business model evidence versus using the chronological evidence) impacts the level of fraud risk assessment.  This indicates that using the business model provides an advantage in evaluating and assessing the potential risk of fraud.  The advantages of using both forms of evidence may enable auditors to determine the nature, timing and extent of procedures to corroborate management’s explanations for fluctuations beyond the context of fraud.  Therefore, the findings of this study have broader implications, as it can be applied to other audit procedures to ensure that auditors are obtaining sufficient evidence throughout the audit as well as for an assessment of fraud risk.


For more information on this study, please contact William F. Wright.
 

Wright, W.F., and L. Berger. 2011. Fraudulent Management Explanations and the Impact of Alternative Presentations of Client Business Evidence.   Auditing:  A Journal of Practice and Theory 30 (2): 153-171.

When planning an audit, auditors perform analytical procedures and inquiries of management regarding any significant fluctuations in the key financial information.  Auditors will assess these significant changes based on these procedures and use the information to assist in planning the nature, timing and extent of audit procedures.  In the context of fraud, the explanations for fluctuations will be based on evidence that either emphasizes the client’s business model or knowledge of the auditor’s chronological presentation of audit evidence.  The chronological presentation of evidence is based on given knowledge of the client’s business, prior year results compared current year information, and other items such as key ratios.  The business model explanation provides linkages between the client’s business strategy and objectives to support the fluctuations.  The use of the chronological information evaluates the comparison of the year over year financial condition and results as well as trend analysis.

Since auditors use these as audit evidence, the study uses this evidence for generating expectations and for making risk assessments regarding fraud.  The authors hypothesize the following when management provides a fraudulent explanation: 

• There will be a more accurate expectation of the non-fraudulent account balances using the business model versus the chronological method of performance.  If a fraudulent error explanation was not provided as an explanation of the fluctuation, there will not be a difference in the precision of expectations between the two presentation formats.
• There will be a more effective fraud risk assessment for a significant fluctuation when using the business model versus the chronological method of performance.  If a fraudulent explanation was not provided as an explanation of the fluctuation, there will not be a difference in the fraud risk assessment.
   

The authors perform an experiment with 73 participants, 42 of which were auditors who had recently graduated from a Canadian university less than a year before. The remaining participants were graduate students.  The authors used a randomized 2x2 between-subjects design.  The factors that were evaluated included whether management’s explanation was or was not fraudulent as it related to the unexpected fluctuation in sales and the two client business evidence presentation methods:  business model or chronological.

• As predicted, the study finds that when provided a fraudulent explanation, the users of the business model approach have higher assessment of fraud risk. The auditors realized the inconsistency in management’s explanation when using this approach.
• When there was no seeded fraud, the business model and the chronological method of audit evidence did not yield any differences in judgment in the auditor’s fraud risk assessment.
   

http://www.auditingresearchsummaries.org