Students� experience in Principles of Accounting I and II (prerequisites for both junior level courses) were coded as �no critical thinking courses� (CT=0), �one critical thinking course� (CT=1)2 or �both critical thinking courses� (CT=2).  The exam scores, prior academic achievement (overall GPA and accounting GPA from Principles I and II) and background variables for the participants were comparable except that Intermediate Cost Accounting was a daytime class yielding a mix with younger, full-time students and higher cumulative GPAs (see Table 2). 

Table 2

Participants Attributes by Junior Level Course Enrolled:  Mean (Std. Dev.)

      Control group.  All sophomore accounting students at the implementation school received the critical thinking experiences leaving no cohort group to serve as a controlled comparison group.  The only source of students without exposure to the critical thinking materials was transfer students from other schools and students who took principles prior to the new courses.  About 50% of the students enrolling in junior accounting courses at this school are transfers from local community colleges as well as nationally competitive schools.  Transfer students� exam scores at the junior level prior to implementation of the critical thinking implementation show that they perform on par or better than native students [Springer, 2002 #102].  None of the schools sending transfer students had critical thinking materials in their sophomore accounting courses [Springer, 2002 #102].

      Measures of performance.  All three exams in Contemporary Accounting Information were multiple-choice format and examined the students� ability to apply the technical terms and concepts to novel scenarios.  Intermediate Cost Accounting exams were 50% computations and procedures and 50% essays on how to respond to business situations based on evidence given. 
　

Results

      Data screening.  Although accounting GPA and overall GPA were significantly correlated (Pearson�s r=.53), the moderate level of this relationship indicated that multicolinearity would not be a threat to the regression model.  Assumptions about homogeneity of variance on all variables and normality of the residuals for each regression were met.  Missing data caused the case to drop from the particular analysis.  Missing data was most prevalent on post-baccalaureate students who are not required to apply formally or provide any academic or testing history for registration (n=29 in Contemporary Accounting Information and n=2 in Intermediate Cost Accounting).  No other systematic pattern for missing data was found.

      Nonequivalence of Control Group.  The control group (no critical thinking courses) differs from the treatment group (those with critical thinking courses) in several non-trivial ways.  First, the students with critical thinking are younger, full-time student with higher GPAs and recent experience in Principles of Accounting (see Table 3).  The model therefore controlled for age, lag time from Principles of Accounting I (first course in the two course sophomore sequence) to the junior course, number of credit hours enrolled, accounting GPA and overall GPA.

Table 3

Participant Attributes by Critical Thinking Experience:  Mean (Std. Dev.)

      Regression Analysis.  Exams scores were analyzed in a standard multiple regression model with overall GPA and accounting GPA representing academic achievement, critical thinking courses = 0, 1 or 2 and age, number of credit hours enrolled and number of months lag from Principles I as covariates to control for nonequivalence between control and treatment groups.  The results of this regression (see panel A, B, C and D of Table 4) show that number of critical thinking courses was a significant predictor of exam scores for all three exams in both courses after controlling for prior academic achievement (overall grade point average and accounting grade point average), age, number of credit hours enrolled and lag since Principles I.3

Table 4

Regression Analysis on Exam Scores.

      As a post hoc test to see if the effect was uniform across higher and lower achieving students, the interaction between grades and critical thinking course experience was added to the model.  The interaction term was negative on all exams showing the exam differences were greater for the lower achieving students.  The interaction term was marginally significant on Exam 1 (t = -1.91, p = .058) and the final exam (t = -1.63, p = .106) but not on Exam 2 (t = -.799, p = .426).  Figure 1 illustrates the interaction for Exam 1 showing that the effect of critical thinking materials at the sophomore level is greatest for the lower GPAs.

Figure 1

Exam 1 scores predicted by number of critical thinking courses (CT = 2, 1 or 0) and Overall GPA 
 
　

      As an additional post hoc investigation, the regression was run on each course separately with no change to the results.  The R squared, however, was almost double in the Intermediate Cost Accounting model ( .666) compared to the Contemporary Accounting Information model ( .361)

Discussion

      Students completing critical thinking courses outperformed students enrolled in traditional courses on all exams in both subsequent courses.  The average exam advantage falls between 1.436 points (out of 100) to an average of 7.531 points (out of 100) after removing exam variance due to prior academic achievement and background variables (see Table 4 Panel D).  This average effect size is small to moderate and was significant for all three exams in both courses.  The effect size, however, was not even across all participants.  The lower achieving (GPA) participants experienced a larger than average effect size (see interaction in Figure 1) which was as much as a full letter grade.

      Given the scale of this implementation, the number and variety of instructors involved, the diversity of the student body and the delay of the measure of learning advantage, the elements driving these results are powerful.  Whatever the general ability garnered in the treatment, the advantage it conferred persisted over at least a full semester and was not sensitive to the imperfections of large-scale implementations (Springer & Borthick, 2004).

      Students outperformed on different tasks (theory versus computational) and in different content areas (see Table 1) and the advantage was measurable months after completing the treatment courses.  This hints that the critical thinking work either developed or selected for qualities that improved general academic performance in a stable way.  This study replicates the learning advantages found by Springer & Borthick (2003) with new participants and on different content.  These results also show learning gains on a wide variety of students extending the results of a similar curriculum with honors students only (Stone & Shelley, 1997).

      Interaction of grade point average and critical thinking cases.  Critical thinking course experience diminished the importance of prior academic achievement (i.e., grades) in predicting exam performance (see Figure 1).  This points towards the cases as a developer of ability rather than as merely a filter that allows only the most qualified students to advance. 

      This interaction suggests that the critical thinking demands benefit the lower achieving students disproportionately, helping them to achieve at levels typical of students with higher grades.  An alternative explanation is that critical thinking coursework developed all students but higher achievers reach a ceiling in the grade distribution so their growth was not apparent in the exam data.

      Intermediate Cost Accounting versus Contemporary Accounting Information. These two courses placed different demands on students both in content and in style.  Contemporary Accounting Information required extensive theoretical readings (482 pages from articles and book chapters) and all multiple-choice exams.  Intermediate Cost Accounting required computations, procedures, casework and chapter readings, a similar structure to the critical thinking courses, and computation and essay exam questions.

      Even though the size of the main effects were similar in both junior courses, the R squared for the regression for Intermediate Cost Accounting was much higher (.666 versus .361).  This means that the grades, cases and background covariates explained much more of the exam variance in Intermediate Cost Accounting than in Contemporary Accounting Information.  Students may have enjoyed a cumulative effect of both procedural and thinking transfer in intermediate cost accounting helping to create a fuller explanation of performance.  Or, the similarity of the Intermediate Cost Accounting assessments to the sophomore course demands reduced random error in the model.

      Potential sources of the learning advantage.  This study used course exams to measure performance.  Without a direct measure on critical thinking ability, this study cannot discern if the materials designed to elicit this particular ability met this goal. Nonetheless, the learning advantage on downstream courses has been mapped to a course with direct content overlap (Springer & Borthick, 2003), to two courses with minor content overlap (See Table 1) and to a more diverse student population than previously tried (Stone & Shelley, 1997).  What is contributing to these desirable general academic outcomes?

      Starting with the least likely, students with critical thinking experience worked on the business simulations in groups, typically lasting most of the semester, giving these students access to a community of learners.  This community could have persisted over the semester break(s) and created support and discussions that improved exam performance (Bruffee, 1999).  Arguing against this possibility is the fact that both of these junior level courses created groups in the first week of classes (for in-class discovery learning exercises in Contemporary Accounting Information and for casework in Intermediate Cost Accounting) and so groups were available to all students.  In addition, the school where the participants attend is a regional commuter school with few facilities supporting group meetings or opportunities to be at common facilities together.

      Another explanation for the advantage is that assuming the role of business advisors in a variety of authentic settings might be highly motivating to students.  Such intrinsic motivation typically results in higher exam performance and general academic achievement (Sankaran & Tung, 2001).  Students with intrinsic motivation levels as a persistent trait, and not just a temporary response to an interesting project, would already have this trait controlled for in their academic achievement scores.  Additionally, one might expect different motivation levels in the reading-only course contrasted with a course containing more cases activities and these differences were not found.

      Ackerman (2003) suggests character traits as an important variable in student performance.  He conducted a series of studies showing that trait sets have knowledge and ability advantages in four different domains.   Building on Richard Snow�s theory of aptitude-and trait complexes (Snow, 1989), Ackerman (2003), correlated typically co-occurring sets of personality and interests (trait complexes) with performance and found that the enterprising/social traits score consistently higher in four content areas than the �conventional/clerical traits�.  Ackerman�s trait theory suggests that the advice-giving business simulations might have attracted a more enterprising/social student while the traditional courses (bookkeeping and financial statement preparation) may have attracted a more conventional/clerical students.  This study did not capture any trait data but stereotypes of accountants do fit the conventional/clerical profile.  If the business simulation appealed to a different trait set, monitoring the trait sets of the control versus treatment group may be important in future work.

      The performance effect may also be due to the active learning nature of the critical thinking work.  Using authentic cases yields greater insight into practical problems, helps students understand the value of applying theory, fosters retention of ideas and increases personal ownership of concepts.   Instead of memorizing solutions to existing problems (Bruer, 1993; Peterson & Walberg, 1979) students construct their own representations of the situations.  �[W]hen you have invented an idea for yourself, it is much more a part of you than when you memorize a description of it from someone else� (Dykstra, 1996).  While active responses from students may have spawned better understanding of the topics in the business simulation, it is unclear how students� mastery formed general abilities that improved performance in new content areas.  Transfer experiments rarely find students able to use prior learning to solve new problems unless the new problems are highly similar or students are prompted (Determan & Sternberg, 1993).

      Another potential source of the exam score differences is that the critical thinking courses required different mental activities and learning strategies than a traditional sophomore course, which might have led to better study and thinking skills.  For instance, the intervention courses included from 5 to 12 writing assignments each semester.  The self-regulatory demands of writing such as planning and evaluating, may influence cognitive processes (Graham & Harris, 2000).  By putting them in the role of expert having to consider context and varying perspective, the students may have attended to how information is related (Gentner & Rattermann, 1993) leading them to integrate information rather than merely attending to surface details.  In a curriculum change similar to the critical thinking courses, Shelley and Stone found students completing the �educating for expertise� curriculum had better problem-structuring skills (Stone & Shelley, 1997).  Actively engagement in the critical thinking courses may have similarly strengthened their mental model-building skills.  Students may also have come to recognize the need for thinking to solve a complex problem (Celuch & Slama, 1999), one that cannot be solved by formal logic alone (Dewey, 1938) and this revelation may have improved their subsequent studies.  Students crafting business advice would wrestle with various explanations and arguments, and this process could have acted as a form of comprehension monitoring (Gott, Hale, Pokorny, Dibble, & Glaser, 1993).  After practicing comprehension monitoring during their sophomore work, they could have continued this practice in future courses.  Finally, the use of a deeper learning strategy (aimed at finding the meaning and connection of the idea and procedures) is a better predictor of performance than academic ability (Crooks, 1988).  Students could not succeed in the sophomore course without explaining the meaning of the concepts to the client and connecting the ideas with each other to form a complete overall conclusion.  And, given the larger gains from lower achieving students with presumably weaker studying and thinking skills, the results hint that these improvements may have been among the lasting results of the critical thinking courses.

      The exam score advantage could also be the result of the relative weaknesses of the traditional methods compared to the critical thinking courses.  Spiro et. al (Spiro, Feltovich, Jacobson, & Coulson, 1991, 1994) argue that initial over-simplification impedes later advanced learning because students have not developed the cognitive flexibility to advance.  Traditional courses where surface learning of isolated facts and procedures leads to successful completion will not precipitate deep learning strategies (Crooks, 1988). 

      Given that the sophomore course combined as many good practices as possible, the learners may have a web of advantages due to strategy changes, selected traits, thinking improvements, deeper content knowledge and changes to their epistemology.  Future research that isolated these variables would add to the understanding of the source of the advantage.

      Importance of findings and future study.  Finding learning differences of half to a full letter grade three to eighteen months after an intervention is rare in transfer studies (Barnett & Ceci, 2002).  While the implementation team reported that some instructors were less than diligent in grading of the simulations (e.g., giving everyone 100% of the points for episodes) (Springer & Borthick, 2003), the effect was nonetheless stable across a wide variety of instructor and student ability.  Finding such long-term effects across such a variety of possible combinations of students and faculty ability in live classes and in multiple downstream college courses constitutes one type of general transfer called for in the literature (see Exhibit 1) (Barnett & Ceci, 2002).

      It is unlikely that such a successful implementation would relax some or all of the course features to permit detail study of the component parts.  The absence of random assignment of learners between control and treatment groups is typical of interventions with demonstrated effects (Willis, 2001).  Nonetheless, replicating this study with variables to measure critical thinking, motivation, personality traits and learning strategy would increase the generalizability of the results.  Including pretests prior to the critical thinking courses might permit discerning whether the results are due to learning gains during the critical thinking courses or self-selection for traits associated with better downstream performance. 
　

Exhibit 1

Taxonomy for far transfer applied to critical thinking courses (Barnett & Ceci, 2002) 
　

      Weaknesses of the study.  While this study demonstrated the superiority of the critical thinking courses, it did not reveal the source of the advantage.  While the interaction of grades and cases (Figure 1) hint that the courses improve student abilities rather than just allowing the most capable students to progress, no pretest established changes in student thinking ability, strategy use, motivation or other habits.

      Another weakness is the non-equivalence of the control group.  Students enrolling in Contemporary Accounting Information and Intermediate Cost Accounting who did not have critical thinking courses (transfers and those taking the courses prior to the treatment materials implementation) are not comparable to students with critical thinking experience in a number of ways (see Table 2).  Although variables were entered into the model to compensate for the differences noticed, the groups may be different in other ways that impact exam performance.

      Although this study used courses with little accounting content overlap, both courses contained some content similarity (see Table 1).  Tracking students with critical thinking experiences in accounting into a non-accounting course would permit a stronger conclusion about any general abilities acquired.

      Conclusion. Students completing sophomore accounting courses as critical thinking outperform students with traditional sophomore accounting courses even on subject matter different from the critical thinking courses.  The advantage, measured from three months to 18 months after the critical thinking course was completed, persistent for a full semester on all exams in two different downstream courses.  The effect size ranged from a few points out of a hundred to a full letter grade and is worthy of follow-up to detect the source and nature of the causes.  Experiments showing the impact of the potential individual traits and strategies would contribute greatly to understanding the mechanisms and theories underlying by these results.

      Finally, an interaction between prior academic achievement and exam scores showed that the poorer performers gained the most from the intervention.  More challenging curriculum is typically reserved for more capable students as was the case when this type of curriculum was first tried in accounting (Stone & Shelley, 1997).  The assumption that lower achievers might not be ready for the challenging work was not true for sophomores admitted to a university with a very diverse student body of average ability.  Students at the lower range of the grade distribution responded strongly to the critical thinking materials making this a particularly promising type of curriculum for a range of students to develop average students towards the highest goals of college�general ability to achieve across tasks and topics. 
　

References