Discussion of “The effect of task complexity and expert system type on the acquisition of procedural knowledge: Some new evidence”

Mascha [Int. J. Account. Inf. Syst. (2001).] hypothesizes that differences in task complexity between studies in accounting and psychology explain differences in their conclusions regarding the benefits of feedback in acquiring procedural knowledge. Overall, the work was done carefully and it contributes to our understanding of learning while using expert systems. I comment specifically in three areas. First, I comment on whether procedural knowledge, which cannot be expressed verbally, or simply knowledge is the construct of interest. Second, I pose some questions regarding how task complexity was operationalized and the implications of that operationalization for the conclusions. Finally, I propose an alternative experimental design that I find more complete, which would eliminate some alternative explanations for the results.     

The influence of domain knowledge and task complexity on tax professionals'' compliance recommendations

This study examined how procedural knowledge and outcome expectations interact with task complexity when tax professionals develop recommendations for clients. Practicing tax professionals participated in two experiments. Results suggest that outcome expectations about whether a position can be defended are positively associated with aggressive recommendations. As complexity increases, professionals with more procedural knowledge (a) favor less aggressive recommendations and (b) rely more heavily on their outcome expectations. These findings illustrate the importance of accounting for knowledge as a two-dimensional construct.     

The effects of understanding rules and a worked example on the acquisition of procedural knowledge and task performance

Previous research (for example, Bonner and Walker, 1994) has investigated the impact of alternative forms of instruction and practice on both the acquisition of procedural knowledge and task performance. Responding to the call to consider alternative forms of instruction, this study reports the results of an experiment designed to distinguish between the effects of understanding rules and a worked example. Worked examples (such as previously prepared working papers, analysis sheets and spreadsheet templates) are of particular interest as they constitute an important source, frequently used by novice accountants, for understanding and performing various tasks.
The results indicate that understanding rules increase task performance through the acquisition of procedural knowledge, while a worked example provides task-specific heuristics that increase task performance but not procedural knowledge. The different benefits of the two forms of instruction emphasise the importance of matching instruction methods to learning objectives. Furthermore, combining the two forms of instruction lead to the highest levels of task performance even with limited practice and no feedback.
JEL classification: I20; M41     

The effects of feedback type on auditor judgment performance for configural and non-configural tasks

This study examines the effect of four different types of feedback (outcome, task properties, cognitive, combined task properties/cognitive) on the risk assessment judgments of auditors on two tasks, one requiring configural cue processing and the other not requiring configural cue processing. Task properties feedback and combined feedback improved performance on both tasks. Outcome feedback was more effective for the non-configural task, while cognitive feedback was more effective for the configural task. We also found that, on the configural task, the effect of cognitive feedback was heightened by lower levels of participant self-insight. Finally, combined feedback was particularly effective in transferring knowledge across tasks. These results have direct practical application for the use of feedback in audit practice. From a theoretical perspective they also help to reconcile earlier mixed results in accounting and psychology.     

Can knowledge based systems be designed to counteract deskilling effects?

Recent calls in the information systems research community argue that we know intelligent systems deskill users, and future research should focus on how to design systems that do not deskill, rather than continue to examine whether the phenomenon occurs. This should be a wakeup call for public accounting firms focused on implementing restrictive audit support systems, which leads to de-skilling of novice accounting professionals. Our research focuses on redesigning knowledge-based systems to facilitate expertise development and counteract the de-skilling effects that result from use of such systems. Specifically, we manipulate the design of the system interface by providing information cues in a screen format consistent with expert knowledge representations and manipulate automatic provision versus voluntary use of explanations for users during task completion. Results show that after using the knowledge-based system to complete a series of reenacted client engagements over a three-day period, both the interface design manipulation and automatic provision of explanations had a positive effect on novice accounting professionals’ development of expert-like knowledge structures. The results of the study have important implications for the development of knowledge-based systems intended to support accounting professionals’ (and other knowledge workers’) expertise development processes.     

Experience, instruction and knowledge acquisition: a study in taxation

This study examines the effects of direct work experience, gained through internships and cooperative educational experiences, on subsequent knowledge acquisition. In particular, theoretical and empirical research in education, cognitive psychology and accounting suggests that experienced individuals develop organizational knowledge structures which allow them to better assimilate new information into memory than inexperienced individuals. Consistent with this notion, the research investigates whether the benefits of prior direct work experience on knowledge acquisition is a function of the nature of the learning tasks (structured versus unstructured) used in subsequent educational experiences.As hypothesized, the results indicate that inexperienced subjects were assisted to a greater extent in the knowledge acquisition process by structure oriented tax return preparation tasks than by relatively unstructured tax research assignments. Also consistent with expectations, the opposite was true for experienced subjects. When experienced subjects were provided with instruction supplemented by unstructured tax research assignments, they demonstrated a greater level of tax knowledge than when assigned structured tax return problems. Further, the knowledge level of experienced subjects was found to be greater than that of inexperienced subjects only when unstructured tax research assignments were provided. These findings suggest that all benefits of direct work experience on the knowledge acquisition process cannot be realized without the specific consideration of such experiences on the design of educational and firm training materials. Finally, the study also indicates that the benefits of direct work experience on subsequent knowledge acquisition are generally greater for individuals with low versus high learning aptitudes.     

The interplay between cost accounting knowledge and presentation formats in cost-based decision-making

Most studies on cost-based decision-making examine the profit impact of cost reports that rely on different methods to allocate costs. In practice, firms’ cost reports often employ the same cost allocation method with subtle variations in the way that the cost data are presented. This paper examines experimentally the profit impact of a cost report’s presentation format in relation to a decision maker’s level of cost accounting knowledge. Using a customer profitability report prepared using activity-based costing and presented in either a tabular or a graphical format, participants analyze a complex pricing and resource allocation task that affects firm profitability. The results suggest a strong relation between presentation format and cost accounting knowledge. Specifically, decision makers with a low level of cost accounting knowledge attain higher profits when they use a graphical format in comparison to a tabular format. More surprisingly, graphs (versus tables) have an adverse effect on profits for users with a high level of cost knowledge. This result has broad implications: in order to facilitate the decisions of a variety of users of accounting data (e.g. managers, external investors, etc.), firms may need to adapt the presentation format of their accounting data to the level of accounting sophistication of the users.     

Investigating the effects of computer mediated interruptions: An analysis of task characteristics and interruption frequency on financial performance

Financial and accounting tasks require high levels of concentration as well as cognitive capacity. Today, advanced technology can help facilitate the effective and efficient completion of such tasks. At the same time, however, these same technologies can interrupt work flow and create work-related stress, thus having a deleterious effect on task performance. These interruptions can be characterized across a number of different dimensions, including frequency, complexity, duration, and relevance to the primary task, to name a few. This study examines the effects of interruption frequency, task complexity, and individual characteristics on cognitive load and subsequent decision-making performance on financial tasks. As hypothesized, the results indicate the significant influence of interruption frequency and order of task complexity on cognitive load which influences performance. This research has implications on the design and use of information systems by accounting professionals in order to reduce potential negative effects.     

Impacts of Additional Guidance Provided on International Financial Reporting Standards on the Judgments of Accountants

This pioneering study examines the impact of the provision of additional guidance on International Financial Reporting Standards (IFRS) as a “decision aid” on the accuracy of judgments of the accountants. To extend the prior research on accounting judgment and decision-making, we also examine the interactive effects of task complexity and additional guidance on the judgments of accountants. The results provide evidence that those accountants who are provided with decision aid in the form of additional guidance on IFRS make more accurate judgments than accountants who are not provided with such guidance. Furthermore, the study provides evidence that this additional guidance improves the judgments of accountants when they undertake tasks which they find complex. The results indicate that additional guidance on IFRS needs to be provided and suggests that accountants should exploit any guidance which is currently provided in IFRS and by the International Financial Reporting Standards Interpretations Committee.     

Can computerized decision aids do “damage”? A case for tailoring feedback and task complexity based on task experience

The theory of technology dominance [Arnold, V., and Sutton, S.G., 1998. The Theory of Technology Dominance. Advances in Accounting Behavioral Research. 1: 175–194.] predicts that mismatches between the skills of decision-aid users and the complexity of the tasks the decision aid is designed to assess will result in a lack of reliance on the decision aid and, ultimately, a potential for deskilling. The cognitive science literature on cognitive load suggests that varying the level of feedback provided to decision-aid users might moderate the risks that result from under-reliance. The current study presented participants, who were divided between two levels of skills, with decision aids in which the level of feedback was varied. These aids were used to solve cases of differing complexity. The results show that individuals with a high level of skill in understanding internal controls deskilled when they were assigned to non-complex cases. Further, this study found that with complex cases, a lower level of feedback was helpful and with non-complex cases, a higher level of feedback was helpful.     

Effects of Feedback and Task Predictability onTask Learning and Judgment Accuracy

This study examines the effect of different types of feedback on task learning and judgment accuracy across different levels of task predictability. The results of a laboratory study show that outcome feedback, alone, and in combination with task properties feedback, promotes judgment accuracy for both high and medium levels of task predictability. The beneficial impact of outcome feedback resulted from learning effects. Specifically , the outcome feedback improved judgment accuracy because of improved task knowledge and, in contrast to previous psychology research, it did not cause a deterioration in judgment consistency where task predictability was less than perfect. The results suggest that the negative effects of outcome feedback on judgment accuracy found in the psychology literature, where task predictability is less than perfect, may be limited in accounting settings where judges have experience with the task.     

Procedural v. conceptual: A developmental view

Much of the current discussion in accounting education is directed at the resolution of the choice between the procedural and conceptual approaches to upper-level courses. This paper provides a theoretical framework for examining the issues in this area. The framework is derived from the work of an expert, William G. Perry Jr., on the intellectual and ethical development of college students. The author argues that the debate must be expanded to include pedagogical as well as content-based issues. The Perry scheme is described and its implications for accounting education are examined.     

Recognizing and Revising Unconvincing Explanations

Expert system explanations sometimes fail to satisfy their users. The standard assumption is that these failures arise because the user did not completely understand the system's explanation. However, users may also be left unsatisfied because they are unconvinced by the explanation provided. Advice-giving systems should therefore eventually be extended to recognize when and why an explanation has failed to convince the user and to elaborate or revise explanations to make them more convincing. This paper provides an initial model of this process for systems that give advice about choosing between multiple alternatives for accomplishing a particular task. In particular, we discuss the model underlying a prototype system to advise novice computer users about how best to accomplish tasks on the UNIX operating system.     

Organizing tax instruction: Evidence regarding how students organize tax knowledge

An important part of accounting curriculum reform is a movement away from traditional methods of organizing accounting courses toward improved organization schemes. Two basic approaches to organizing the content of tax courses are a transaction-based scheme and an entity-based scheme. This paper reports the results of an experiment that investigates which scheme is more consistent with how incoming masters in tax students organize tax knowledge and how different schemes affect the manner in which graduating students organize tax knowledge. Ninety students drawn from two full-time masters in taxation programs participated in the study. Subjects individually completed three separate tasks that measured how they organize tax knowledge at both the start of their degree program and three semesters later at or near the end of their program. The results for all three tasks indicate that incoming tax students organize tax knowledge around the type of transaction involved. The results for two of the three tasks indicate that the students' transaction focus did not change during their graduate tax program, whereas the results for the third task indicate that the students' transaction focus increased during their graduate tax experience. The implications of these findings for improving tax instruction are discussed.     

Interpretation and application of “new” and “complex” international financial reporting standards in Fiji: Implications for convergence of accounting standards

This study extends prior research on accounting judgment and decision making research by examining the effects of ‘new’ and ‘complex’ accounting standards on judgments of professional accountants. It examines whether there are differences in judgments of professional accountants in Fiji when interpreting and applying selected International Financial Reporting Standards (IFRSs). A significant within-country difference in judgments of professional accountants has serious implications for convergence of accounting standards. The results show that interpretation and application of accounting standards are affected by complexity of the accounting standard and professional accountant's familiarity with the standard. The study also finds strong support for an interactive effect of familiarity with the accounting standards and complexity in accounting standards on judgments of professional accountants. Furthermore, the results show that differences in judgments exist between the Big 4 and non-Big 4 professional accountants when provided with new accounting standards that require complex judgments. The results of this study are of interest to stakeholders at a time when IFRSs are increasingly being adopted throughout the world and standard setters are struggling to promote compliance with those standards.     

IN-QUOTES: a knowledge-based system for supporting decision making in weakly structured domains

In complex and weakly structured domains, decision makers often employ multiple techniques, including quantitative modeling and reasoning from past experiences, to address the problem at hand. As such, there has been a call for more research on developing systems that merge problem-solving approaches, such as reasoning from past experiences, with other paradigms to provide support for both the unstructured and structured aspects of the decision-making process. Development of these systems is dependent on acquiring and modeling the knowledge and expertise inherent in the process and then representing and implementing it in an appropriate form. However, in weakly structured domains, knowledge acquisition may be better described as knowledge ‘co-creation’ in which the expert and system builder work together to understand the process and lend as much structure to it as possible. In this paper, we propose that the integration of principles drawn from the paradigms of case-base reasoning, expert systems, and object-oriented programming facilitates this process by providing a powerful approach to acquire and model knowledge in a weakly structured domain. We demonstrate this approach through the development of a system designed to assist a decision maker in the performance of a difficult, somewhat unstructured design and planning task. © 1998 John Wiley & Sons, Ltd.     

Formative assessment in accounting education and some initial evidence on its use for instructional sequencing

Formative assessment (FA) provides instructors and students with feedback to improve learning. Across a variety of education settings FA is one of the most effective classroom interventions for improving student learning outcomes. Yet the accounting education literature is almost devoid of any work related to FA. One barrier for developing FA in accounting education is the significant background knowledge needed to implement FA successfully. The purpose of this paper is to provide some of that background. The paper includes conceptual discussion from the general education literature to explain how FA improves learning and discussion of research that has identified features that affect the efficacy of FA practice. One of these features is how instructors use FA data to adjust instruction. An empirical study illustrates that accounting educators can use FA data to inform a decision about instructional sequencing. Instructional sequencing principles have been used to develop new accounting curricula and courses, and accounting education research has used empirical data to inform an instructional sequencing decision. The current study tests, using a sample of introductory accounting students, alternate versions of two FA tasks to determine which version is better for identifying deficiencies in student learning outcomes. Results suggest that one version of each task is better for identifying deficiencies, but an adjustment to instructional sequencing may be needed to ensure efficacy of one of the tasks, depending on how the FA practice is implemented.     

Measurement of knowledge structures acquired through instruction,experience, and decision aid use

This study investigates a method for measuring knowledge structure development in novice accountants by extending Bonner and Walker [Bonner, S. and P. Walker. 1994. The effects of instruction and experience on the acquisition of auditing knowledge. The Accounting Review 69: 157–178.], which examined the effects of instruction and experience on the acquisition of declarative and procedural knowledge. We employ Pathfinder network scaling, a recently developed and validated measure of knowledge structure acquisition, to determine whether the combinations of instruction and experience previously found to be effective in promoting declarative and procedural knowledge acquisition result in the development of expert-like knowledge structures. Further, we examine the effectiveness of a simple decision aid in promoting the acquisition of expert-like knowledge structures through experience with the decision aid. Results from two laboratory experiments indicate that Pathfinder-based measures of knowledge structure can effectively capture the effects of training and decision aid use on the development of expertise. The findings suggest that assessments of knowledge structures are valuable tools for measuring the effectiveness of training programs, and such assessments can be applied in decision domains where traditional measures of knowledge acquisition are insufficient or infeasible. Finally, the results indicate that properly designed decision aids can impart expert-like knowledge structures to novice decision makers, and these knowledge structures are the key to expertise.     

Human information processing and its relevance to accounting: A laboratory study

The purpose of this research was to empirically test emerging behavioral science theories which have import in our understanding of the complex information processing and decision-making behavior of users of accounting information. A multi-stage laboratory experiment which utilized accounting students as decision-makers was used to test several research hypotheses. In the experimental tasks, the subjects knew the nature of the decision and had the opportunity to seek information at a cost to aid them in deciding on an alternative. The findings are supportive to the applicability of general and idiosyncratic information processing theories to managerial environments.     

Refining the behavior of multiple expert systems: a concept and empirical results

Expert System Refinement (ESR) is introduced as a means to automatically refine the performance of one or more expert systems. The ESR concept is based on Holland’s learning classifier systems and a method for integrating multiple expert systems. Through users’ feedback about the usefulness/correctness of the integrated expert system’s recommendations, ESR enables behaviors of both individual expert systems as well as the integrated system to improve over time. The ESR concept is tested on a German Credit Database. This empirical evidence suggests that the ESR concept can be usefully applied in automating the process of expert system refinement and multiple expert systems integration. © 1998 John Wiley & Sons, Ltd.