innov manag_proquest_diss19.pdf
TRANSCRIPT
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
1/171
ONLINE BUSINESS SIMULATIONS: A SUSTAINABLE OR DISRUPTIVE
INNOVATION IN MANAGEMENT EDUCATION?
by
Jason Scott Earl
MARY F. WHITMAN, DBA, Faculty Mentor and Chair
MAUDIE GALLOP HOLM, PhD, Committee Member
CLARK GILBERT, DBA, Committee Member
William A. Reed, PhD, Dean, School of Business and Technology
A Dissertation Presented in Partial Fulfillment
Of the Requirements for the Degree
Doctor of Philosophy
Capella University
June 2012
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
2/171
All rights reserved
INFORMATION TO ALL USERSThe quality of this reproduction is dependent on the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscriptand there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
All rights reserved. This edition of the work is protected againstunauthorized copying under Title 17, United States Code.
ProQuest LLC.789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
UMI 3517084
Copyright 2012 by ProQuest LLC.
UMI Number: 3517084
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
3/171
© Jason Scott Earl, 2012
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
4/171
iv
Abstract
The focal goal of this research was to extend the empirical effort on business
simulations as a form of experiential learning by providing the first empirical analysis of
business acumen and knowledge application skills. Disruptions in technology are
providing more opportunities to improve the simulation gaming learning experience and
a number of pedagogical innovations are beginning to emerge which will drive the way in
which business simulations are used in the future. The purpose of this quantitative,
experimentally-based research study was to investigate the use of online business
simulations as a disruptive technology by measuring the change in participants’ business
knowledge and business acumen compared to traditional corporate training. A sample of
65 participants was randomly selected from a company population of 720 employees and
managers. This quantitative based research study demonstrated the disruptive nature of
online business simulations when it comes to gains in business knowledge by measuring
a 2.55 standard deviation difference in the normalized gains between traditional training
and business simulation training. Baseline tests against a control group and traditional
training group using MANCOVA to account for multiple variables and covariates imply
that online business simulations enhance both business knowledge and business acumen
on a staggering scale and over a very short period of time.
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
5/171
iii
Dedication
This work is dedicated to my wonderful wife and mother of our five young
children. Thank you, Natalie for all of your love and support over these last 15 years. I
would have never left the private equity world and gone into academia without your faith
in me and our purpose in life together. If nothing else, I have learned from this long
journey that true teaching is leadership and most teaching is bad leadership. Thank you
for your example and personal leadership in our own family.
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
6/171
iv
Acknowledgments
Someone once said that defeat is bitter, but only if you swallow. I would like to
acknowledge the support of Dr. Mary Whitman who believed in me and on more than
one occasion, kept me from swallowing that bitter pill. I would also like to acknowledge
my dissertation committee members who have served as great examples to me and as a
source of personal inspiration. I hope that my life can be a small reflection to my own
students of the great principles that they have taught and, more importantly, lived.
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
7/171
v
Table of Contents
Acknowledgments iv
List of Tables ix
List of Figures x
CHAPTER 1. INTRODUCTION
Background to the Study 2
Introduction to the Problem 4
Statement of the Problem 6
Purpose of the Study 8
Specific Variables 8
Rationale 9
Research Questions 10
Null Hypotheses 11
Significance of the Study 12
Sustaining vs. Disruptive Innovation 14
Definition of Terms 15
Assumptions and Limitations 18
Nature of the Study 19
Organization of the Study 21
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
8/171
vi
CHAPTER 2. LITERATURE REVIEW
Background on Experiential Learning 24
Nature of Business Simulations 28
Benefits of Experiential Learning 30
Educational Effectiveness of Simulations 31
Simulations and Corporate Training 33
Knowledge Application 35
Business Simulations as Knowledge Application Tools 36
Knowledge Application & Simulation Performance 38
Reflection 41
Reflection in Experiential Learning 43
Reflection Using Business Simulations 46
Debriefs Within Business Simulation 48
Business Simulations as a Disruptive Innovation 50
Business Simulations as a Disruptive Force Today 55
Andragogical Support for Business Simulations 58
CHAPTER 3. METHODOLOGY
Philosophy and Justification 65
Primary Research Question 66
General Linear Model for the Primary Research Question 68
Null and Alternative Hypotheses 69
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
9/171
vii
Research Design 70
Justification and Methodology 70
Research Design Strategy 71
Sample 75
Setting 80
Instrumentation and Measures 79
The Business Simulation 82
Dwyer and Ganster’s (1991) Autonomy/ Work Control 84
Field Testing 85
Business Simulation Performance Scores 85
Data Collection 86
Variables 87
Data Analysis 87
Validity and Reliability 92
Ethical Considerations 97
CHAPTER 4. RESULTS
Training for a Semiconductor Company 101
Participant Demographics 102
Normality Testing 103
ANOVA Analysis 109
MANCOVA Analysis 111
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
10/171
viii
Correlation Analysis 112
Comparison of Means 114
Hypothesis Testing 115
Alternative Hypotheses 117
Primary Research Question 118
Secondary Research Questions 118
Supporting Research Questions 119
Summary 121
CHAPTER 5. DISCUSSION, IMPLICATIONS, RECOMMENDATIONS RESULTS
Summary and Discussion of Results 124
Research Questions 124
Disruptive Innovation 129
General Discussion &Theoretical Implications 130
Practical Implications 131
Limitations and Recommendations for Future Research 135
Conclusion 137
REFERENCES 140
APPENDIX A. PRE-EXPERIMENT SURVEY & ASSESSMENT 152
APPENDIX B. POST-EXPERIMENT SURVEY & ASSESSMENT 157
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
11/171
ix
List of Tables
Table 1. Summary of Variables Used in This Study 21
Table 2. Instructional Methods Used by U.S. Organizations 33
Table 3. Null and Alternative Hypotheses 69
Table 4. Descriptive Statistics for Survey Items ( N = 65) 104
Table 5. Descriptive Statistics for Dependent Variables ( N = 65) 105
Table 6. Test of Homogeneity of Variances for Dependent Variables 106
Table 7. Test of Normality for Change in Business Knowledge 107
Table 8. Test of Normality for Change in Business Acumen 108
Table 9. ANOVA for Pre/Post-Test Results 109
Table 10. Tests of Between Subject Effects for Experimental Treatment 112
Table 11. Pearson Correlation Coefficients 113
Table 12. Group Statistics for Dependent Variables 114
Table 13. Independent Samples Test for Equality of Means 115
Table 14. Analysis of Variance Between Groups 120
Table 15. Summary of Hypothesis Testing 123
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
12/171
x
List of Figures
Figure 1. Conceptual Framework 20
Figure 2. Venn Diagram Depicting the Focus Area of this Study 23
Figure 3. Impact of Disruptive Innovation 56
Figure 4. Impact of Disruptive Innovation on Military Training Industry 57
Figure 5. General Linear Model for Primary Research Question 68
Figure 6. Research Design Schematic 72
Figure 7. Overview of the Research Design 74
Figure 8. Comp-XM
®
Comparative Standings 81
Figure 9. Foundation®
Business Simulation Performance Score. 86
Figure 10. Demographic Variable: Level of Autonomy 103
Figure 11. Normal Q-Q Plots and Boxplot for Change in Business Knowledge 107
Figure 12. Post-Test on Business Knowledge vs. Experimental Treatment 110
Figure 13. Post-Test on Business Acumen vs. Experimental Treatment 111
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
13/171
1
CHAPTER 1. INTRODUCTION
There has been a great divide between the gaming community and business
educators over the past twenty five years (Anderson & Lawton, 2009; Gosen &
Washbush, 2005; Wolfe, 1985). The gamers have dismissed educational simulations as
boring and irrelevant while business management educators have dismissed gaming and
simulations as trivial and pedagogically unproven (Aldrich, 2009a, xxi). Both appear to
be right and yet both may have missed an opportunity which lies within an engaging
business simulation and its potential impact on the world of management education
(Anderson & Lawton, 2009). Many business professionals have argued that the K-12 and
higher education systems are failing, myopically trapped in a nineteenth-century world of
“learning by knowing,” while the twenty-first-century world requires the judgment and
skill of “learning by doing” (Aldrich, 2009b, p. 12). Disruptions in technology are
providing more opportunities to improve the simulation gaming learning experience and
a number of pedagogical innovations are beginning to emerge which will drive the way in
which business simulations are used in the future (Faria, Hutchison, & Wellington., 2009,
p. 485). One of the major challenges with research in this field is that nobody has shown
definitively that simulation training works in the business world any better than
traditional instruction through workbooks or lectures (Davies, 2003, p. 36). The purpose
of this quantitative, experimentally-based research study was to investigate the use of
online business simulations as a disruptive technology by measuring the change in
participants’ business knowledge and business acumen compared to traditional corporate
training.
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
14/171
2
Background to the Study
In many respects, Bloom’s Taxonomy has been the anchor for assessing whether
learning occurs in business simulations (Bloom, Englehart, Furst, Hill, & Krathwohl,
1956). A substantial share of the early research on simulations focused on the attitudes
(affective domain) of participants exposed to the pedagogy. Much of this initial research
focused on comparing the general perceptions of students regarding cases, lectures, and
simulations (Anderson & Woodhouse, 1984; Blythe & Gosenpud, 1981). Subsequent
research expanded into attempting to assess what is learned from participating in a
simulation and almost all of these studies relied on perceptions and self-reports of
learning, rather than more objective measures (Anderson & Lawton, 2009, p.211). When
focused only on those studies that aim to examine participants’ affective reaction to
simulations, it is evident that students like simulation exercises and view them more
positively than either lectures or case discussions (Burns, Gentry, & Wolfe, 1990; Faria,
2001; Gosen & Washbush, 2004). It is worth noting that these relative comparisons have
been made by students experiencing different pedagogies within a course. There is a
dearth of studies employing experimental designs with control groups or where
comparisons are made between participants’ attitudes in one section of a course that is
solely lecture based versus those in a class that is solely case discussion based or solely
simulation based. Most business classes are taught today with one main pedagogy (e.g.,
lecture, case study or simulation) and almost no experimental studies exist that compare
learning outcomes under alternative pedagogies (Anderson & Lawton, 2009, p. 208).
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
15/171
3
Wolfe (1990) identified this problem over 20 years ago, yet the gap still exists.
Researchers continue to use self-assessments rather than more suitable tools because self-
assessments are much easier to employ. As a consequence, studies on the educational
merits of simulations often are measuring the affective domain, not the cognitive domain
they purport to measure (Anderson & Lawton, 2009, p. 197). Using perceptions tends to
be advantageous to those who wish to claim the superiority of simulations over
alternative pedagogies because simulations almost invariably are rated positively by
students. The downside of using perceptions is that evidence based on perceptions often
is dismissed by scholars because it lacks suitable rigor. However, studies that attempt to
go beyond perceptions to more objective measures of learning more often than not use
tools best suited for measuring lower levels of learning on Bloom’s taxonomy (Anderson
& Lawton, 2009, p. 209). The decisions required to effectively run a business simulation
often tap analytical, synthesis, and application skills of Bloom’s taxonomy (Bloom,
1956). This has led some researchers to believe that using simulations is a powerful and
disruptive form of learning because it is taking place at the higher levels of Bloom’s
taxonomy (Smith, 2006).
Disruptions often bring significant changes to an industry and these disruptions
create opportunities for those organizations willing to adopt and champion disruptive
technology (Smith, 2006). Christensen (1997) has highlighted the sometimes devastating
impact in the corporate environment of what he refers to as disruptive innovations (p. 41).
Successful, well managed firms that dominate their markets have sometimes gone into a
sharp decline or even collapsed when a new technology disrupts the pattern of their
market segment. Other firms and organizations, however, have handled such transitions
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
16/171
4
smoothly, maintaining their position of dominance in the market by employing specific
techniques to integrate the new and disruptive technologies into their operations
(Christensen & Raynor, 2003, p. 57). Traditional research universities enjoy a dominant
position in the higher education market, but they are beginning to feel the impact of
disruptive innovations such as online universities, distance education, and continuing
education units as semiautonomous incubators (Archer, Garrison, & Anderson, 1999,
p.137).
According to Smith (2006), changes in the underlying technology for online
simulations have improved to the point that they are now more powerful than many of the
established pedagogical tools in the field of management education (p. 8). This
disruption in the field of management education is very similar to the innovation model
that Christensen first proposed in his dissertation and built upon in The Innovator’s
Dilemma (Christensen, 1997). This theory of disruptive innovation allows for a detailed
lens to be focused on business simulations in an attempt to identify them as sustaining,
complementary, or disruptive innovations. Grüen-Yanoff and Weirich (2010) argues that
these technological and economical forces which are at work in the education industry
will create a tsunami of change throughout management education. Thus, they predict
this change will allow for the spread of more cost-effective, more powerful, and more
accessible simulations across the field of corporate training and management education
(Grüen-Yanoff & Weirich, 2010, p. 45).
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
17/171
5
Introduction to the Problem
Of great concern to management education scholars is the identification of factors
which will contribute to rapid changes in business and how business leaders in the future
will be taught (Smith, 2006). Although many educators do not see simulations as a threat
to traditional education programs today, there are signs showing a significant impact
from simulations already (Andersen & Lawton, 2007). The advent of flight simulators
and computer games has finally introduced a technology and learning media which is
interactive, low-cost, and scalable. Today, authors are creating “virtual velds” where
participants can repeatedly practice skills, instead of just hearing about them (Aldrich,
2009b). As individual managers and employees gain more autonomy or “work control”
over day-to-day business decisions, the ability to take advantage of these disruptive
technologies is expected to increase dramatically. An empirical study performed at a
manufacturing facility involving corporate training demonstrated that experimental
interventions that aim to augment worker control over their tasks and work environment
increased productivity and innovation (Dwyer & Ganster, 1991). It may be that
education reformers are signaling the end of the age of learning “how to know” rather
than “how to do” or “how to be” in a complex, interactive world (Aldrich, 2009a).
Davies stated that part of the problem is that nobody has shown definitively that
simulation training works in the business world any better than workbooks or lectures
(2003). The challenge is determining whether these simulations are truly disruptive
innovations to management education or simply sustaining innovations.
According to Christensen (1997), the main reason why so many successful and
dominant institutions fail due to innovation within their industry is their inability to
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
18/171
6
recognize the difference between sustaining and disruptive innovations. Sustaining
technologies are typically sought after by these successful institutions because they
improve the performance of established products for their most lucrative clients. This
strikes many innovative professionals as paradoxical, because the excellent business
practice of listening closely to their customers does not lead to disruptive innovation,
only incremental sustaining innovations. Technologies, in the sense that Christensen uses
the word, may refer to either “hard” technologies that result in new types of physical
goods or “soft” technologies that result in new ways of organizing work or providing a
service. Interactive simulations are referred to as veld technologies based on the
learning-to-do skills. It is possible that these technologies will successfully challenge the
institutions of management and corporate education in the near future (Aldrich, 2009a,
p.212).
Statement of the Problem
D. Goleman, R. E. Boyatzis, and A. McKee (2004) suggest that one of the largest
mistakes in management education is to assume that simply acquiring more information
(e.g., business knowledge) will automatically lead to becoming a more effective manager
or leader. They state that the development of competencies in cognitive or intellectual
ability (e.g. business acumen) leads to outstanding performance. Goleman et al also
suggests that self-directed learning is an effective method of achieving sustainable
changes in both business knowledge and business acumen (2004, p. 274). Online
business simulations are quickly becoming an effective tool for self-directed learning
while enhancing these management competencies (Segon & Booth, 2009, p. 112). The
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
19/171
7
problem thus becomes how do business management educators significantly increase the
level of both business knowledge and business acumen for learners today?
Because of the open-ended content of most business simulations, participants can
have a hard time articulating what they have learned during their game-play experience
(Anderson & Lawton, 2007). For example, there are no well-conducted studies that
actually investigate the learning effects of business simulations on both learners’
knowledge application skill and business acumen (Anderson & Lawton, 2009, p. 209). In
fact, almost all studies have focused on the students’ attitudes and perceptions of their
experience with the simulation (Aldrich, 2009, p. 220). The real gap in the research and
literature is the use of business simulations in the world of adult learning or andragogical
approach to learning (Anderson & Lawton, 2009). There is a need to determine whether
simulations are a disruptive innovation because “the holy grail of research within
management education is to empirically demonstrate that andragogical techniques lead to
better learning outcomes” (Knowles, Holton & Swanson, 2005, p. 235). Recent advances
in computer simulations allow potential business leaders to practice and rehearse these
business management skills in a context rich environment removed from real life, thus
allowing a participant the opportunity to strengthen their skills in an atmosphere of safety
(Sidor, 2008). Using computer based simulations, participants have the ability to review
business management skills and potentially modify their real world application skills.
This forms the following study’s primary research question: Do online business
simulations provide an increase in knowledge or business acumen for participants, which
is on the order of a disruptive innovation?
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
20/171
8
Purpose of the Study
The purpose of this quantitative study was to investigate the use of online
business simulations as a disruptive technology by measuring the change in participants’
business knowledge and business acumen compared to traditional corporate training.
This research was undertaken to prevent management educators from continuing to miss
the opportunity for creative learning by adopting inappropriate educational strategies that
are of little relevance to practicing managers (Burns, 1995, p. 284). Gosling and
Mintzberg (2004) have stated that business management is neither a science nor a
profession, neither a function nor combination of functions. Business management is a
practice and this research has a high level of relevance at this time because it allows
participants to appreciate the experience of making business decisions within a given
context. Management may use science, but it is an art that is combined with science
through craft (Gosling & Mintzberg, 2004, p. 19). This topic has been of repeated
interest in both simulation and gaming journals as well as business journals focused on
policy, research, and management education (Segon & Booth, 2009).
Specific Variables
Participants in the simulation group ran an online business simulation, titled
Foundation®
by Management Simulations Inc. which simulates five years of running a
company in a high-tech industry (Foundation®
, 2012). This “simulated” industry is very
similar to the semi-conductor industry where all participants work as managers and/ or
employees. A sample of 65 participants were randomly selected from a semiconductor
company population of 720 employees and managers for this research. The two
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
21/171
9
experimental groups (case study and business simulation) responded to prompt questions
during the three day training period. In an attempt to investigate these research questions,
this study also looked into which of the following variables contributed most to
participants’ business simulation performance:
(a) Number of years of industry experience
(b) Level of education (i.e., undergraduate, graduate)
(c) Area of expertise in the company
(d) Level of autonomy/ work control at the company
(e) Previous business simulation experience
Rationale
The following topics have been addressed by some scholars as fertile areas for
research: Do participants improve their grasp of interrelationships among the various
functions of business (marketing, finance, production, etc.) as a result of participating in a
simulation? Are the interpersonal skills of participants improved through participating in
a simulation? Do participants in simulations really develop a greater appreciation for the
difficulty of implementing what may, on the surface, appear to be rather straightforward
business concepts? Are business simulations really effective devices for integrating
participants into business programs, and are they effective at improving retention rates?
(Anderson & Lawton, 2009, p. 212).
Based on these research questions, both practical and theoretical reasons were
present for conducting this study. From a practical perspective, knowing the benefits of
engaging students in reflection activities may be very helpful in order to improve
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
22/171
10
business acumen and knowledge application skills for the participants. Understanding
the impact that this experiential activity can have on knowledge application and
simulation performance will provide information that is valuable to both designers and
administrators of future simulations. Furthermore, if the types of prompt questions used
in this study; (a) strategy questions for business acumen and (b) financial and/or
accounting questions for knowledge application, turn out to be effective in promoting
participants knowledge and simulation performance, then simulation designers and
business instructors will have a basis from which to make informed decisions when it
comes to designing a better learning experience. On a theoretical level, this study
integrated a number of the precepts of experiential learning, reflective learning, and
online simulations in order to provide information on the factors that promote
participants’ knowledge. This combination of instructional techniques clearly
demonstrated the impact of this teaching innovation. Past research has shown that a two
standard deviation gain in a pre/post-test of interactive-engagement compared to
traditional instruction may place this innovation on the order of disruptive scale for
education (Hake, 1998).
Research Questions
This research attempted to address the following research questions:
Primary Research Question:
1. Do online business simulations provide an increase in business knowledge orbusiness acumen for participants, which is on the order of a disruptiveinnovation?
Secondary Research Questions:
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
23/171
11
2. Are knowledge application skills in business positively correlated with onlinebusiness simulation performance scores?
3. How does the change in knowledge application skills with traditionalcorporate training compare with online business simulations?
Supporting Research Questions:
4. Do participants who are taught using an online business simulation gain asignificant increase in business knowledge and business acumen?
5. Do participants who are taught using traditional corporate training gain asignificant increase in business knowledge and business acumen?
6. Is the level of education or industry experience positively correlated withonline business simulation performance scores?
7. Is the level of participant autonomy/ work control positively correlated withonline business simulation performance scores?
This study used a MANCOVA technique for data analysis, based on three
experimental groups. The full general linear model and associated hypotheses are
discussed in Chapter 3. The null hypotheses based on these research questions are listed
below:
Null Hypotheses
Ho1. There is NO difference across experimental groups for business knowledgeafter adjusting for previous simulation experience and autonomy/ work control.
Ho2. There is NO difference across experimental groups for business acumenafter adjusting for previous simulation experience and autonomy/ work control.
Ho3. NO correlation exists between participants’ business acumen and business
knowledge.
Ho4. Participants who engage in an online business simulation will NOTdemonstrate higher business knowledge after experiencing the businesssimulation.
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
24/171
12
Ho5. Participants who engage in an online business simulation will NOTdemonstrate higher business acumen after experiencing the business simulation.
Ho6. There is NO difference between the simulation group, the case study group,and the control group based on their change in business knowledge.
Ho7. There is NO difference between the simulation group, the case study group,and the control group based on their change in business acumen.
Ho8. NO correlation exists between participants’ business simulationperformance and their level of autonomy/ work control in the company.
Ho9. NO correlation exists between participants’ business simulationperformance and their years of industry experience.
Ho10. NO correlation exists between participants’ business simulation
performance and their level of education.
Significance of the Study
This study provided key insight into what happens to participants in a business
simulation by measuring the overall increase in business knowledge and business
acumen. Several researchers have documented the benefits of using business simulations
in management education (Chapman & Sorege, 1999; Lefebvre, 1997; Segon, 2009).
The most common reported benefits included practice in an environment without risk,
increased creativity, more focused competitive analysis, increased cross-functional
understanding, and increased subject content knowledge. Although case studies and
practitioner experience support these benefits, little empirical evidence is offered on the
change process that an individual participant experiences (Scherpereel, 2005, p. 389).
Specifically, research has not studied adequately the effects of engaging students in a
business simulation and measuring the change in business acumen or knowledge
application skills. Anderson and Lawton (2009) refer to this as a “dearth of studies
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
25/171
13
employing experimental designs” ( p. 196). In addition to the possibility of simulations
operating on the order of a disruptive innovation, there is virtually no existing literature
investigating the order of magnitude impact that business simulations have on
participants when it comes to quickly and effectively mastering business management
skills (Anderson & Lawton, 2009, p. 211).
This study added to the body of knowledge on the nature and impact of business
simulations by employing five methods of measurement: (1) a pretest and posttest
designed to assess the participants’ knowledge application skills, (2) a pretest and posttest
designed to assess the participants’ business acumen, (3) participants’ responses to
different prompt questions, (4) business simulation performance scores, and (5) survey
questionnaires. The survey questionnaires were designed to investigate the participants’
academic and work-related background as well as their individual level of autonomy or
work control within the organization. A sampling of 65 managers and employees at a
semiconductor manufacturing facility were randomly assigned to three different groups.
These groups consisted of (a) the traditional instruction (case study), (b) the online
business simulation group, and (c) the control group. Participants in both the traditional
instruction group and online business simulation group were prompted to respond to
strategy questions which evaluated their business acumen. These same participants were
also asked to answer knowledge application questions which evaluated their business
knowledge. Participants in the control group were not prompted to respond to either
strategy questions or knowledge application questions.
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
26/171
14
Sustaining vs. Disruptive Innovation
Christensen’s (1997) theory of disruptive innovation was built upon the concept
of radical and incremental innovation initially proposed by Dewar and Dutton (1986).
Christensen proposed that disruptive innovations are different than radical innovations in
that they have the value-destroying characteristics of radical innovation; however, these
innovations work much more slowly and methodically up through an industry’s value
chain. Christensen’s theory states that disruptive innovations have a beginning point that
is actually much lower on the performance scale than similar existing technologies in the
same market. According to Smith (2006), the reason why low performance innovations
are so disruptive is due to their ability to meet the needs of a niche market that is
unaddressed by the current leading products and technologies (p. 4). Christensen (1997)
states that these disruptive innovations often grow in underserved markets because of
their low cost and consequently, are perceived as insignificant and less profitable to the
industry leaders due to much lower margins. This disdain held by industry leaders is
typically due to the small size of these niche markets and the small profits that are
available from them. Consequently, disruptive innovations move slowly up the value
chain and often take time to destroy the value of established products and technologies
(Christensen, 1997). In fact, almost all growth from these innovations is in completely
new markets. These disruptive innovations erode the value of formerly successful
institutions by systematically stealing away customers from the bottom of the value chain
and gaining more and more market share over time. As these disruptive innovations
improve the quality of their products or services to customers, they eventually meet the
expectations of a large customer base which is being served by the industry leaders and
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
27/171
15
suddenly become a real threat. While business simulations have been recognized as a
separate pedagogy for instruction, there is little empirical evidence in the research
literature on the magnitude of this innovation when it comes to measuring the increase in
business knowledge or business acumen (Faria et al., 2009, p. 485).
Accordingly, this research contributed to both the management education and
disruptive innovation literature. By investigating the order of magnitude impact that the
business simulation had on participants based on the pre-test and post-test analysis, it was
possible to determine whether online business simulations are simply a sustaining
innovation for educators or truly a disruptive technology that will change how managers
and business leaders learn in the future.
Definition of Terms
The following definitions were used in this study:
Assessment
The art and science of testing individuals to determine what they have learned or,
as is more often the case, what they have not learned (Grüne-Yanoff & Weirich, 2010).
Business Simulation
Computer-based role-playing game which makes use of high fidelity simulated
environments and involves decision-making in the research & development (R&D),
marketing, operations, human resources (HR), and finance departments of a company
(Grüne-Yanoff & Weirich, 2010).
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
28/171
16
Business Strategy
This is a particular game-play-strategy or group of unique business decisions
which are in complete alignment in order for participants to achieve the highest possible
score on simulation performance (Grüne-Yanoff & Weirich, 2010).
Disruptive Innovation
A term coined by Clayton Christensen and used in business and technology
literature to describe innovations which improve a product or service for non-consumers
in ways that the market does not expect. This is typically done by lowering price or
designing for a different set of consumers. Few technologies are intrinsically disruptive
or sustaining in character; however, it is the strategy or business model behind the
technology that it enables, which creates the disruptive impact (Christensen & Raynor,
2003).
Experiential Learning
A learning model which begins with the experience, followed by reflection,
discussion, analysis, and evaluation of the experience (Albert, 1970).
Game Based Learning
A learning method which combines educational content and elements of computer
games (Aldrich, 2009a).
Knowledge Application
The process of selecting appropriate business knowledge suitable to the challenge
at hand, and making connections between selected business knowledge and specific
strategies (Sarin & McDermott, 2003).
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
29/171
17
Online Business Simulation
An instructional method based on a representation of a physical or social business
reality in which participants compete for certain outcomes according to an established set
of rules or constraints. The competition can be (1) among themselves as individuals or
groups, or (2) against some specified standard, working as individuals or cooperating as a
group (Szczurek, 1982).
Prompt Questions
Questions that prompt participants’ reflection and guide the process of their
knowledge application during the course of the simulation. Two different types of
prompt questions will be used for this study – the Strategy Question and the Knowledge
Application Question.
Reflection
An important human ability, in which a person recaptures his or her experience,
thinks about it, mulls over it and evaluates it (Boud et al., 1985). Reflection typically
takes place after an experiential learning experience. These reflection periods are often
referred to as “debriefs” with an online business simulation.
Simulation Performance
Measured at the end of each round (or year) and defined by the company’s
Balanced Scorecard which is a combination of metrics for the Company involving
financial ratios (i.e., return-on-equity and stock price), learning and growth of people,
customer satisfaction, and internal business processes.
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
30/171
18
Sustaining Innovation
A sustaining innovation allows for increases in performance at typically higher
costs in the same market without effecting non-consumers from other markets.
Sustaining innovations allow a given technology to continue to improve in their own
market, but they do not directly impact other markets (Christensen & Raynor, 2003).
Assumptions and Limitations
In order for this study to proceed, certain assumptions were required. First, the
study assumed that valid and reliable data existed and could be obtained. Second, the
study assumed that all managers at the semiconductor manufacturing facility would
participate in the Capstone (Management Simulations, Inc.) business simulation. Third,
participants would accurately complete the survey questionnaire. Fourth, all three groups
of participants would have a similar level of knowledge of business management related
to managing a growing business in a highly competitive industry.
As with most empirical studies, a number of limitations began to emerge. Some
of these limitations had an effect on the validity and reliability of results, as described
below.
1. This study used a specific type of business simulation which is in the domain
of managing a growing business in the semi-conductor or electronic sensor
industry; hence it may be difficult to generalize results to other types of
business simulations.
2. This study used a relatively small sample size of 65 business managers in the
semi-conductor or electronic sensor industry, both limiting the kinds of
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
31/171
19
quantitative analyses that can be conducted and reducing the generalizability
of results.
3. Analyses were limited to mainly quantitative methods. While inferential
statistical procedures showed significance with numerical data, there was a
need for some qualitative analysis to shed light on the participants’ learning
experience with a business simulation. This study allowed for some limited
qualitative analysis in a short-answer post-test survey.
4. This study focused on a group of managers and employees at the same
electronic sensor manufacturing company. Consequently, it will be difficult
to generalize results to other managers or employees working in other
industries.
Nature of the Study
A true experimental study was used to investigate the possible cause-and-effect
relationships by exposing two experimental groups (business simulation and traditional
instruction) to one or more treatment conditions and comparing the results to a control
group not receiving the treatment.
Figure 1 highlights the conceptual framework that was used in this study, showing
how each of the variables relates to the Research Questions and Hypotheses.
The conceptual framework reiterates the problem that was used for this study.
There are no clear empirical studies on whether online business simulations are a
sustaining or disruptive innovation when compared with traditional instruction. The
problem established the framework for the study and allowed for both business acumen
and business knowledge to be measured in a true experimental study.
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
32/171
20
Figure 1. Conceptual Framework. The research questions and hypotheses in this figuretie out with the dependent, independent, and demographic variables in order to define theproblem in this study.
Characteristics of this experimental design required rigorous management of
experimental variables and conditions by direct control/manipulation or through
randomization (Isaac & Michael, 1997). The independent variables in this study were
participation in an online business simulation or participation in traditional instruction.
There was also a control group which did not participate in either. The primary
dependent variables were the change in business knowledge and change in business
PROBLEM
Are Online Business Simulations a Sustainingor Disruptive Innovation?
DEPENDENT VARIABLES
- Business Simulation Score
INDEPENDENT VARIABLES
- Online Business Simulation- Traditional Instruction
DEMOGRAPHIC VARIABLES(Impact Performance)
SimulationExperience
Autonomy/W-Control
IndustryExperience
EducationLevel
WHY?
Reflection Activity:1. Level of Engagement2. Ability to Apply Learning3. Peer Instruction4. Bloom’s Taxonomy
∆∆∆∆ in Business
Knowledge
∆∆∆∆ in Business
Acumen
RQ4
H1-H4
RQ4
H2-H5
RQ6
RQ7
RQ2
H3
RQ1
H6
H7
H9H8H1-H2
RQ3
RQ5
H10
Level of Autonomy in
the CompanyH8
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
33/171
21
acumen based on the pre/post-test for each group. The intent of this study was to make
ten predictions about the relationships among the variables identified in the null
Hypotheses. A summary of all variables is found in the table below (Table 1).
Table 1.
Summary of Variables Used in This Study
Organization of the Study
Following this introduction, the next chapter includes a thorough review of
relevant literature in the areas of experiential learning, online business simulations, and
disruptive innovation. This review supported the conceptual framework used in this
study and helped illuminate the gaps in existing works which represents opportunities for
future research. In chapter three, the research methodology is described
comprehensively, including the research design, experimental approach, review of
measurement instruments, and the data analysis procedures. Reliability and validity
concerns are also discussed. Chapter four reports results from the experiment as well as
from the subsequent data analysis which was conducted according to the procedures
outlined in chapter three. Finally, chapter five includes a discussion of results and their
Variable Type of Variable Measurement Level
Pedagogical Nature of Training Experimental Treatment Independent Ordinal
Change in Business Knowledge Dependent Comp-XM® Ratio
Change in Business Acumen Dependent Comp-XM® Ratio
Knowledge Application Score Dependent Foundation® Ratio
Level of Autonom y at Com pany Independent Mediating Work Control (Dwyer) Ratio
(Continuous Covariate)Simulation Experience Independent Moderator Survey Item Nominal
(Dichotomous Covariate)
Area of Expertise Descriptive Survey Item Nominal
Industry Experience Descriptive Survey Item Ordinal
Level of Education Descriptive Survey Item Ordinal
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
34/171
22
implications. The study concludes with recommendations for future research based on
the results from the experiment.
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
35/171
C
Corporations, ma
and collegiate business p
2004). Many educators a
to apply their knowledge
integrative, and iterative
empirical studies that act
participants’ knowledge
fact, almost all studies ha
experience with the simu
helps fill an important re
business simulations, exp
Figure 2. Venn diagram
23
APTER 2. LITERATURE REVIEW
agement education institutions, development c
ograms use business simulations to train and te
ssume that business simulations will help partic
to solve real-world problems based on their “in
nature” (Anderson & Lawton, 1997). However,
ally investigate the learning effects of business
pplication or increase in business acumen (Mil
ve focused on the students’ attitudes and perce
lation (Anderson & Lawton, 2006). The resear
earch gap by examining the relationship betwe
eriential learning, and disruptive innovation.
epicting the focus area of this study
nsulting firms,
ach (Summers,
ipants learn how
eractive,
there are no
simulations on
er, 1998). In
tion of their
h in this study
n online
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
36/171
24
Overall, the educational merits of business simulations have been subject to
considerable debate (W. Biggs, 1990). While many universities use simulations, only a
small fraction of the faculty members within those institutions integrate simulations into
their coursework (Summers, 2004). There are studies which indicate that other forms of
pedagogy are just as effective or more effective than business simulations. Anderson and
Lawton (1990) argued that only a very weak link between participation in a simulation
and learning has been shown. These researchers also pointed out that valid, reliable
instruments to assess mastery are rare, and valid measures of higher level learning of
objectives are almost nonexistent. Kayes (2002) noted that while countless management
scholars and practitioners see ‘experience’ as central to management learning, the notion
of experience has received critical attention. Kayes explains that criticisms of so called
experience-based learning arise for both empirical and theoretical reasons (p. 137).
Based on the gaps identified in the existing literature, this research study seeks a
causal relationship between online business simulations and experiential learning as a
possible source of disruption when compared to more traditional methods of corporate
training. This study therefore connects experiential learning theory from psychology
(Lewin, 1935) with the developing field of disruptive innovation (Christensen, 2008) in
order to support or refute the business case for significantly higher levels of adult
learning through the use of online business simulations.
Background on Experiential Learning
The use of business simulations as a form of experiential learning and disruptive
innovation has evolved significantly over the last six decades. The evolution and
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
37/171
25
development of experiential philosophy has been significantly influenced by five
individual researchers: John Dewey, Kurt Lewin, Jean Piaget, , David Kolb, and Clayton
Christensen.
Dewey's (1938) Experience and Education focuses on the conflict between
traditional and progressive education. The essence of Dewey’s work was that truth and
knowledge are not absolute but rather continuously evolving over time. Dewey
explained that experiences directly influence knowledge and what is come to be known
as truth. He further argued that experience should be incorporated into the education
process and that all education should be participatory in order to be experience-based.
Several of Dewey’s ideas have made their way into traditional educational programs over
the last sixty years, particularly at the primary and elementary levels. As individuals
grow, they find that new experiences conflict with earlier learning and knowledge.
Experiential educators in higher education are addressing these challenges by using an
innovative approach that incorporates the best of traditional and experiential
methodologies (Ruben, 1999).
Kurt Lewin is often referred to as the founder of American social psychology and
his work has laid much of the foundation for modern educational and organizational
development work. Lewin’s (1935) research on experiential learning and group
dynamics had a profound influence on the discipline of social psychology and
organizational behavior. Lewin’s studies on group dynamics and the methodology of
action research led to laboratory-training. This training is now considered one of the
most significant educational innovations of this century when it comes to the process of
learning and change (Kolb, 1984). Lewin described the change that takes place during
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
38/171
26
learning as a three-stage process; (1) “unfreezing” or overcoming inertia and dismantling
the existing mindset, (2) a period of confusing and transition where old ways are being
challenged but there is no clear picture of the new mindset that will take place, and (3)
the third and final stage he called “freezing.” Lewin (1947) describes this new mindset as
crystallizing and former comfort levels are returned to the learning participant.
Jean Piaget, a renowned French psychologist and epistemologist, is another major
contributor to experiential learning. The essence of Piaget's (1973) work is based on the
description of how intelligence is shaped by experience. Piaget stated that learning is the
product of interaction between the person and their environment. The growth and shape
of intelligence is impacted by decisions and the realization of consequences for each
decision. In this interaction, the ability for the person to act and experience consequences
is key. In Piaget’s studies of children, from infants to teenagers, this research
demonstrated the importance of abstract reasoning and interaction with the environment.
The ability of the child to manipulate symbols comes directly from the infant’s actions in
exploring and coping with the immediate concrete environment.
David Kolb (1984), an American educational theorist, laid the foundations of
modern experiential education theory based on the idea that knowledge is gained through
both personal and environmental experiences. Based on Lewin’s earlier work, Kolb
developed the experiential learning cycle which has been widely reproduced based on a
four-stage model of learning. Initially called “The Lewinian Experiential Learning
Model,” this model is now primarily recognized as Kolb’s (1984). Kolb states that in
order to gain genuine knowledge from an experience, certain abilities are required; (1) the
learner must be willing to be actively involved in the experience, (2), the learner must be
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
39/171
27
able to reflect on the experience, (3) the learner must possess and use analytical skills to
conceptualize the experience, and (4) the learner must possess decision-making and
problem solving skills in order to use the new ideas gained from the experience. This
theory offers a fundamentally different view of the learning process from that of the
behavioral theories of learning based on an empirical epistemology or the more implicit
theories of learning that underlie traditional educational methods (Kolb, 1984).
Christensen’s (1997) The Innovator’s Dilemma laid the foundational research
which helped to explain why successful companies and institutions often fail to invest
aggressively in disruptive technologies. When applying this theory to higher education,
Christensen (2008) states that the more “student-centric” classrooms become, the more
demand there will be for new technologies. Christensen’s research also suggests ways to
identify innovations which are about to disrupt entire industries. The primary method for
identifying these innovations is that they begin as products or services which are much
simpler and cheaper than the existing competition. Also, these innovations generally
promise much lower margins and very little profit. The secondary method for
recognizing disruptive technologies is based on the fact that they are typically
commercialized in new and/ or insignificant markets (Archer, 1999). The third way that
disruptive innovations are recognized is based on the perspective of the leading firms’
most profitable customers. Typically, these high-end customers, who are willing to pay
much higher prices, don’t want or can’t use such inferior products or services. In almost
every case, a disruptive innovation is initially embraced by customers who mean very
little to the industry leaders in that particular market. The great irony of this theory of
disruptive innovation is that those companies which are the best at listening to and
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
40/171
28
serving their most profitable and most successful customers are rarely able to build a case
for investing in disruptive technologies. When they finally do, it is often too late .
Christensen’s (1997) research highlights the fact that disruptive innovations in the
past were technologically straightforward and did not rely on a specific business model.
In fact, Christensen argues that many of the early disruptions in the computer industry
consisted of off-the-shelf components put together in a product architecture that was
simpler than prior approaches. These innovations typically offered less of what
customers in established markets wanted because they were much lower in product
performance. These disruptive innovations also began in emerging markets and
consequently, offered a different package of attributes which were considered
unimportant to the industry leaders. Christensen (2008) argues that through experiential
learning in a virtual environment, assessment–the art and science of testing individuals to
determine what they have learned–can be revolutionized . One of the potential areas for
this revolution in the virtual learning environment is online business simulations.
Nature of Business Simulations
Clearly distinguishing simulations from simulation games is quite difficult and
debatable. While a simulation imitates reality and is often used to predict what would
happen in a given scenario, the word “game” suggests playfulness and competition.
Simulation games combine all of these characteristics. Many researchers now use the
term simulation game (Jacobs & Dempsey, 1993) to describe a new class of games that
make use of high fidelity environments. Simulation games have also been defined
variously as a combination of simulations and games with competition (Heyman, 1982)
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
41/171
29
and as a subset of games (McGrenere, 1996). One of the more complex definitions is
proposed by Szczurek (1982). Szczurek referred to such educational tools as an
instructional method based on a simplified model or representation of a physical or social
reality in which participants compete for certain outcomes according to an established set
of rules or constraints. The competition in simulations is often against some specified
standard, where participants can work as individuals or cooperate as a team.
The origin of the business simulation dates back to 1955. In that year, the Rand
Corporation developed an exercise called Monopologs (Jackson, 1959). Monopologs
required its participants to perform as inventory managers in a simulated Air Force
supply system, thus providing decision-making experience without the risks associated
with the consequences of a wrong decision. The Air Force continued the use of
Monopologs for many years and reported it to be a highly successful training device.
One of the first practical and most successful business simulations for the masses was
Top Management by the American Management Association (AMA) in 1956. It was
used in numerous management seminars (Meier, Newell, & Pazer, 1969). Additionally,
the consulting firm of McKinsey and Company developed the Business Management
Game in 1957 for use in its management seminars (Andlinger, 1958) and the University
of Washington became the first university classroom user of a business game when a
simulation developed by Scheiber was used in a business policy course in 1957 (Watson,
1981).
The increased usage of business simulations can be measured in several different
ways. First, the number of simulations available in the market has increased dramatically
over the last ten years. Second, the number of organizations and journals devoted to
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
42/171
30
business games and simulations has also risen significantly (Faria, 2009). An email
survey of 14,497 business faculty members across all disciplines at American Association
of Collegiate Schools of Business member schools led to only 1,085 respondents. Of
those who responded, 30.6% were current business simulation users, 17.1% were former
simulation users and 52.3% had never used a business simulation in their coursework. In
earlier work, Faria (2004) had estimated that 95% of AACSB schools (The Association to
Advance College Schools of Business) and 86% of all business schools in the United
States were using business simulation games. Surprisingly, business simulations showed
the highest use in business policy and marketing areas.
Benefits of Experiential Learning
Experiential exercises, including role-plays and simulations, have been widely
used for educational purposes. Learning objectives are thought to be accomplished by
providing realistic, but controlled, environments in which students are guided only by
implicit rules. Although there are distinctions between different modes of simulation, the
design of most simulations allows students to be exposed to stimuli that encourage them
to acquire the key concepts of the subject area being taught (Druckman & Ebner, 2008).
Cherryholmes completed the earliest evaluations of simulation learning outcomes (1966).
He evaluated five hypothesized topics–interest, learning, retention, critical thinking, and
attitudes–with six studies using complex simulations conducted over periods of time
ranging from one day to 12 weeks. The results were clear: Only interest in the material
being learned by the simulation participants improved significantly (compared to case
study and other conventional classroom approaches); negligible changes occurred on
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
43/171
31
learning and attitudinal variables. However, there were convergent findings when “the
task of designing a simulation before playing it, either re-designing an existing game or
constructing one of their own” (Cherryholmes, 1966, p. 7). Based on finding from
Cherryholmes research, a design opportunity will be provided for in this study, where
participants will be able to choose the financial outcomes and ratios that are most
important to their chosen strategy while running the business simulation.
Educational Effectiveness of Simulations
Pierfy (1977) reviewed the findings obtained from comparative evaluation studies
reported during the 1960s and 1970s. With regard to learning, 15 of 21 studies showed
no difference between simulations and other instructional techniques. With regard to
retaining the information learned, 8 of 11 studies showed that the students participating in
simulations retained the information longer than those exposed to other instructional
techniques. On interest, 7 of 8 studies reported that students showed more interest in the
simulation activities than in more conventional classroom tasks. In their update of
Pierfy’s review, Bredemeier and Greenblat (1981) concluded that simulations are as
effective as, but not better than, other instructional methods on learning the subject. They
stated that simulations are more effective when used only as aids to retaining the learned
material and in instilling a positive attitude toward the subject matter. Also supporting
these results is research completed by Ellington, Fowlie, and Gordon (1998) who found
that simulations have an advantage over traditional methods in motivation, participant
involvement, and commitment. A question suggested, but not answered, by the studies is
why the learning impacts are modest. Wolfe and Crookall (1998) have also asked why
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
44/171
32
the field of experiential gaming has made little progress. Regarding educational
effectiveness, a variety of suggestions for significantly improving the contribution of
simulations has been given. Examples include clarifying learning objectives (Bredemeier
& Greenblat, 1981), providing more conceptual background on the subject prior to the
simulation activity (Druckman & Robinson, 1998), creating time for reflection on the
events and getting feedback (Mclaughlan & Kircpatrick, 2005), and providing
participants with conceptual maps and graphics that reflect the simulation’s purpose
(Druckman & Ebner, 2008).
In contrast to just game playing, actual simulation design contributes to analysis
by identifying critical elements (roles, goals, resources, and rules) leading to new
analytical questions (Ebner & Efron, 2005). Attention to the design process remains a
strong focus, as evidenced by the simulation-building exercise featured at the 2007
International Simulation and Gaming Association conference at Nijmegen in the
Netherlands (Durckman & Ebner, 2008). These observations suggest the hypothesis that
simulation designers learn more about the concepts being simulated than do simulation
role-players. Crookal expounded on this hypothesis when he stated the key features of
the design process: (a) Design is concrete – you can touch the results; (b) it is creative –
you develop an object, and (c) it is involving – you develop understanding in a passionate
and intimate way (1995, p.161). When participants have the ability to make changes
within the simulation and design their own experience, the learning about relations
between different concepts goes up significantly and approaches synthesis (Greenblat,
1998). In the case of an online business simulation, the participants in this research study
will have the opportunity to design their own performance metrics for their own company
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
45/171
33
and specific industry of competitors. This interactive engagement with the simulation
may lead to higher levels of motivation, participant involvement and commitment.
Simulations and Corporate Training
Organizations have a wide variety of methods available for training their
employees and business simulations have begun to enter this market. Table 2 reports
U.S. organizations’ usage of selected instructional materials and methods from Training
Magazine’s 2003 survey (Galvin, 2003). Computer-based games and simulations have
low usage rates, with 1% of respondents always using them, 9% often using them, 47%
seldom using them, and 44% never using them.
Table 2
Instructional Methods Used by U.S. Organizations (Galvin, 2003)
Instructional Method Often or Always Never or Seldom
Instructor-led classroom 91 9
Self-study, Web based 44 56
Performance Support 44 56
Public seminars 42 58
Case studies 40 60
Role-play 35 65
Non-computer-based games, simulations 25 75
Self-study, non-computer based 23 77
Virtual classroom with instructor 21 79
Computer-based games, simulations 9 91
Experiential programs 6 94
Virtual reality programs 3 97SOURCE: “Industry Report” (2003, p. 31)
Percentage of Respondents
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
46/171
34
Despite the low percentage of business simulations used in 2003, Summers (2004)
outlines three specific advantages to using computer-based simulations in corporate
training which will allow it to grow exponentially in the future. These three advantages
are; (a) specific knowledge, (b) learning on demand and (c) lower costs (Summers, 2004,
p. 226). Summers also argued that many new technology companies are introducing
business simulations to corporations based on the benefits of learning on demand (p.
228). This research poses the question: “Will the new technology companies come to
dominate or even replace the old?” If so, this would be a case of creative destruction, a
concept posed by Schumpeter (1911/1989).
Alternatively, the new technology companies which introduce these simulations
to corporate training programs may simply increase the number and variety of products.
When the theory of disruptive innovation (Christensen, 1997) is cast on this dilemma, the
outcome depends on whether the new technology is superior and whether the new
technology companies can consolidate the industry (Summers, 2004, p. 232). Online
business simulations have rapidly penetrated business schools; however, the superiority
of such experiential learning methods has not been proven in the corporate training arena.
A simple increase in the number and variety of training methods for management
education would indicate a sustaining innovation for the industry, while a significant
consolidation of the corporate training industry would indicate a disruptive innovation
(Christensen, 2008).
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
47/171
35
Knowledge Application
Application of theory is an on-going issue in higher education (Falkenberg,
Russell and Ricker, 2000). The fact that most of what participants learn is intended for
application to problem situations in real life is indicative of the importance of knowledge
application as a learning objective, especially in the field of management education.
Bloom, Engleheart, Furst, Hill, and Drathwohl (1959) developed a system that classified
learning into six levels. These levels arranged in a hierarchical order to reflect
progressively higher levels of learning. They are, in ascending order: basic knowledge,
comprehension, application, analysis, objective synthesis, and objective evaluation.
Bloom (1956) showed the components of knowledge application in the problem-
solving process of answering questions. The process involves six steps, in ascending
order: restructuring and classifying situations, selection of abstraction suitable to problem
type, the use of abstractions to solve a problem, and solution to a problem. This process
shows that in order to solve a problem through knowledge application, there are certain
steps to be followed. Learning how to apply knowledge means learning how to follow
these steps effectively. Bloom (1956) distinguished knowledge application from
knowledge comprehension by saying that a demonstration of comprehension shows that a
student can use the abstraction when its use is specified; while a demonstration of
application shows that he/she will use it correctly, given an appropriate situation in which
no mode of solution is specified.
Bloom (1956) stated that comprehending an abstraction does not certify that the
individual will be able to apply it correctly. Thus, participants need practice in applying
their knowledge to real-world problems in order to make their knowledge more useful for
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
48/171
36
real-world decision making rather than remain inert. In this study, “knowledge
application in a business simulation” refers to the process of selecting appropriate
business knowledge suitable to the problem at hand, and making connections between
selected business knowledge and business strategies to solve a complex problem. Agyris
and Shon (1974) argued that the only way for organizational effectiveness to increase
over time was through individuals learning from experience. Online business simulations
build upon this concept because the application of learning is based almost completely on
the participants’ experiences. Agryis and Shon (1974) stated that these types of
experiences provide insights during the learning process and allow for specific
knowledge application tools to be developed leading to consistent learning outcomes .
Business Simulations as Knowledge Application Tools
One major problem that comes from learning with business simulations is that it
is not always clear that learners will leave with exactly the same conclusions, mental
models, and learning outcomes. In fact, it is not clear that learners will be able to apply
what they have learned in future real-world situations. From their experiment with
business school students, Mandl, Gruber, and Renkl (1992) confirmed that students using
a computer-based simulation had serious deficits in knowledge application and problem
solving using their previous knowledge. The results of the experiment clearly show that
business school students have considerable deficits in using their own declarative
knowledge that they acquired in business school, and they are not able to use their
knowledge as a tool in the real world. This is largely due to the fact that those students
gained their business knowledge mostly through traditional methods, like lectures, case
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
49/171
37
studies, and textbooks, and had not had enough opportunities to practice applying their
knowledge to real-world problem-solving either in a specific situation or in a simulated
environment. As a result, their knowledge was not sufficiently conditioned to relevant
application conditions, and remained inert.
Training students to apply their knowledge requires very different methods of
instruction than training them to memorize information or understand relationships within
a business context (Reigeluth & Moore, 1999). In business education, the most
commonly used instructional methods are those of linear formats, such as
lecture/textbook format and case analysis. These linear formats can be more efficient
than the experiential learning method for communicating a large number of concepts to a
large number of students. However, these formats do not do enough to encourage
creativity, problem solving, decision making, risk taking, and knowledge application. In
addition, most knowledge that business school students acquire from their lectures and
textbooks is what Anderson (1985) called the “declarative knowledge”: various business
concepts and principles are taught in a declarative form rather than an experiential form.
Therefore, when students learn business knowledge, they often treat new information as
facts to be learned rather than knowledge to be used. As a result, many business school
students have difficulties in using their knowledge as a tool in their business decision
making (Mandl et al., 1992). Since their knowledge is not based on actual experience, it
often remains inert.
Not all business knowledge needs to be taught in a procedural form. However,
most of those management-related concepts and principles consist of knowledge about
how to do things, which is what Anderson (1985) called the “procedural knowledge.”
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
50/171
38
Procedural knowledge is goal-oriented performance knowledge that can be executed
efficiently. Therefore, when learning this kind of management-related knowledge,
students need opportunities in which they can transform their declarative knowledge into
procedural knowledge so that their knowledge can be more readily available for real-
world problem solving rather than remaining inert. One important instructional goal of
business simulations is to help students transform their declarative knowledge into
procedural knowledge. Business simulations are designed to help students practice in
applying their knowledge, which is mostly in a declarative form, into specific action-
oriented problems in a relatively safe, controlled, and simulated environment.
Knowledge Application & Simulation Performance
Business instructors have promoted simulations as a means of accomplishing a
wide range of learning objectives, including improving interpersonal skills
(VirtuaLeader), improving general decision-making skills (Capsim), and helping
individuals to understand themselves (Second Life). Anderson and Lawton (1997)
outlined learning objectives common to simulation exercises. These include outcomes
such as increased knowledge of facts and concepts of the business discipline; improved
analytical skills, critical thinking, decision making, and interpersonal relations; enhanced
ability to simultaneously manage interrelationships; and a better understanding of
business dynamics.
While business simulations are believed to have the potential to stimulate learning
at all levels of learning objectives, many scholars argue that simulations are best suited to
facilitate learning at the higher level of Bloom’s Taxonomy of learning objectives:
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
51/171
39
application, analysis, and synthesis of knowledge (Anderson & Lawton, 1997). In fact,
researchers in business education have frequently used Bloom’s Taxonomy as a
framework for guiding their thinking on the areas where simulations are likely to have the
greatest impact on learning (Bloom et al., 1959). Since business simulations require
participants to act in the role of managers, it would seem likely that, if business
simulations excel in any area they would be strong in application (Anderson & Lawton,
2002). However, objective evidence for business simulation effectiveness at these higher
levels of Bloom’s taxonomy has been lacking.
Measuring the higher levels of Bloom’s taxonomy of learning objectives has
proven to be a difficult task. A lack of reliable and valid instruments has hindered
attempts to measure the learning occurring at the higher levels of Bloom’s taxonomy
(Anderson & Lawton, 1995). Thus, while Bloom’s taxonomy provides a useful
framework for the purpose of establishing learning objectives, the framework has not
been as helpful for assessing student learning, especially in the context of business
simulations. However, Anderson and Lawton (2002) conducted a study that has been
directed exclusively at the efficacy of simulations as a pedagogy for learning about the
application of business concepts. The premise underlying their research was that if
simulation performance does reflect learning associated with analysis and application,
those students who apply the concepts that are critical to the discipline should outperform
those who do not apply the knowledge presented in the course. The results of their study
showed a significant relationship between the application of concepts presented in a basic
marketing course and performance on a marketing simulation game. The greater the
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
52/171
40
number of concepts that students utilized in the management of their simulation
company, the higher their performance scores.
These findings provide a powerful validation for simulations as learning and
teaching tools. In answer to the question of whether applying the principles and concepts
of a discipline results in positive results in a simulation, the answer appears to be an
emphatic “yes” (Anderson & Lawton, 2002). Their study demonstrated that simulations
are useful tools for operating at the application level of Bloom’s hierarchy, the level at
which traditional classroom lectures are thought to be weak. De Jong and Van Joolingen
(1998) point out that an appropriate design theory for instructional simulations may arise
based on this higher form of learning. These researchers also defend that “discovery
learning” with simulations can take its place in learning and instruction as a new line of
learning environments based on technology where more emphasis is being placed on the
learner’s own responsibility ( pp. 19).
Other research that investigated the link between knowledge application and
simulation performance is found in a study conducted by Wolfe and Luethge (2003).
Wolfe and Luethge investigated whether students, who were involved more in the
simulation and applied more business knowledge over time, performed better than
students who were less involved and applied less business knowledge. They examined
the following in this study; (a) the degree to which simulated companies in a business
simulation need to be run by active, knowledgeable, and engaged players and (b) the
extent to which strategic involvement affects performance. The results of this study
found that less involved students, who applied less business concepts, performed poorly
relative to more engaged students. The results of their study indicated that good
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
53/171
41
performance in a business simulation is not the result of luck or random guesses and that
a business simulation rewards intelligent, planned decision-making practices.
Reflection
The word “reflection” appears frequently in the literature relating to experiential
learning. Boud et al. (1985) defined reflection as an important human ability in which
people recapture their experience, think about it and evaluate it. Dewey (1910) pointed
out that all genuine education comes through experience and that reflection can assist in
this process. The importance of reflection goes back to Dewey’s early writing, but there
has been increased interest in researching and using reflective processes in adult teaching
in the last twenty years (Salmon, 2001).
Bruce (2001) stated that reflection is described as contemplating the results of a
given experience within the overall context of the impact on the individual. Boud, Keogh
and Walker (1985) argued that only when this reflective process leads to a significant
change in behavior, can it be called reflective learning. Costa and Garmston (1994)
stated that reflective learning is the ability to mentally wander through a recent personal
experience. This mental process of reflection includes the following; (a) drawing forth
cognitive and emotional information from visual, auditory, kinesthetic, and tactile
sources, (b) linking information to previous learning, (c) comparing the results that were
anticipated and intended with the results that were achieved, (d) searching for effects and
finding connections among causal factors, (e) acting on and processing the information
by analyzing, synthesizing, and evaluating, (f) applying learning to contexts beyond the
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
54/171
42
one in which it was learned and making commitments to plans of action, and (g) the
metacognitive process of thinking about thinking.
Some researchers argue that reflection is essentially an independent activity. An
online business simulation allows for ideal period of reflection in between rounds of
company decisions. This period of reflection or debrief allows participants to ask critical
questions about their company performance and challenge their own basic assumptions
about strategy and execution (Davies, 2003; Foreman, 2004). Other researchers stress the
importance of collaboration with others in terms of the reflection process (Rose, 1992).
Lin (1999) argued that students’ reflection can be enhanced in a reflective social
discourse, and defined reflective thinking as actively monitoring, evaluating, and
modifying one’s thinking and comparing it to both expert models and peers.
Reflection on experience is based upon the metacognitive theory developed by
Flavell (1987), who argued that becoming aware of oneself as a learner allows the student
to reflect, monitor, and revise the process and products of his own learning. The term
metacognition itself emerged from the early work of Flavell who referred to it as
knowledge concerning one’s own cognitive process and products or anything related to
them (Flavell, 1976).
J. Biggs (1985) discussed the role of metacognition in learning, utilizing the term
“metalearning” to define the application of metacognition to student learning. More
particularly, he also defined metalearning as students’ awareness of their learning and
control over their strategy selection and employment. According to J. Biggs (1988), a
metalearner is one who is aware of their motives, task demands and personal cognitive
resources and exerts control over strategies used. J. Biggs (1988) also stated that these
-
8/17/2019 innov manag_ProQuest_Diss19.pdf
55/171
43
reflections invite