rigor versus relevance: why would we choose only one?

RIGOR VERSUS RELEVANCE:WHY WOULD WE CHOOSE ONLY ONE?�

JOHN T. MENTZERThe University of Tennessee

In business academia, we have long debated the relativeimportance of rigorous research versus research that isrelevant. It is hard to understand why we continue this

debate, when the answer is right in front of us: Whywould we choose only one? How can research be con-sidered ‘‘good’’ if it is not relevant to the discipline understudy? How can research be useful if our methods are

not rigorous enough to allow us to be confident inour results?

This is not a new issue. The gap between scholarly re-search and practice in business has existed for decades.Academics argue that the rigor is so important to what

they do that relevance is not important. After all, we are‘‘scholars.’’ Not surprisingly, practitioners complain thatthey never read the academic journals since they are notapplicable to ‘‘real world’’ situations and problems.

Why is this so? Why would ‘‘scholars’’ want to studysomething that was not relevant to their chosen disci-pline? Other sciences do not suffer from this ‘‘academicschizophrenia.’’ A medical researcher would never dream

of making the statement, ‘‘My work is more scholarlybecause it is so rigorous that it has no relevance tomedical science.’’ Such a statement borders on the absurd.All good scientific disciplines define what their discipline

of study includes and what it does not include, and thenconduct rigorous research that is relevant to those de-fined phenomena.

This is no less true in the business disciplines, including

supply chain management (SCM). Over 60 years ago,Lewin (1945) stated that, ‘‘nothing is so practical as agood theory.’’ The central mission of business scholarsand educators is to conduct research that contributes

knowledge to a scientific discipline (like SCM) and toapply that knowledge to the practice of management(Simon 1967). To do this, we need to design our researchso that it provides understanding of the practical prob-lems facing a profession, and we need to develop good

theory so that our research advances the knowledge thatis relevant to both the discipline and practice (Van de Ven1989). As Van de Ven (1989, p. 486) put it, ‘‘Good theoryis practical precisely because it advances knowledge in a

scientific discipline, guides research toward crucial ques-tions, and enlightens the profession of management.’’

Relevance must, by its nature, apply to a discipline.Disciplines, such as SCM, define what phenomena theyaddress and what phenomena they do not address. In the

development of any discipline of study, it is commonand healthy to have ongoing debate that provides defi-nitions of the categories and elements of the discipline —i.e., what it is and what it is not. For example, the broad

area of social science eventually evolved into (at least) theseparate disciplines of psychology (the study of individ-uals), sociopsychology (the study of individuals ingroups), sociology (the study of groups) and anthro-pology (the study of cultures). Similarly, scholars should

continue to debate the definitions of SCM and comparethem with its practice to determine what research is rel-evant to SCM, and what research is not.

Rigor is the constant examination of whether research

can actually support and justify the claims it makes. Rigordoes not imply the use of increasingly complex meth-odologies just to prove we can use them (Mentzer andFlint 1997). It implies use of the appropriate theories and

methods to avoid concluding something the research didnot actually reveal.

If rigor and relevance are so inextricably intertwined,why do we continue to have a gap between scholarly

research and practice? I believe it is because, in SCM re-search, we forget the role of the scholar in the study ofany discipline. Mentzer and Schumann (2006) provide aprocess model of marketing scholarship that is readily

applicable to SCM scholarship (Figure 1). The processmodel (as adapted here) describes a framework that at-tempts to (1) depict the interactive environment whereSCM scholarship occurs, (2) label and organize the

different types of feedback necessary for the SCM schol-arly process, (3) identify and describe the origination ofinputs within various forms of feedback and (4) dem-onstrate the flow of SCM scholarly activity. It incorpo-rates all four Boyer (1990) dimensions of scholarship

(discovery, integration, teaching, application). This is truein SCM in equal measure as it is in medicine, physics,psychology, or any other scholarly discipline.

Feedback, viewed within the scholarly process and

largely obtained within various types of learning envi-ronments, contains inputs that are the antecedents ortriggers to scholarly thinking. Scholarly thinking, in turn,is disseminated through three traditional scholarly roles:

researcher, teacher and application/service provider. In�Invited Comment.

Volume 44, Number 272

the business world, these inputs originate with individ-uals as they interact within either the traditional learningenvironment (e.g., university class on campus) or within

the everyday business world, which itself is a significantlearning environment, and a significant laboratory inwhich ‘‘SCM truths’’ can be observed and tested.

The model suggests that input to the SCM scholarly

process is derived from sources that are either initiallyexternal (serve as independent stimuli), or result directlyfrom, and are dependent upon, the SCM scholarly pro-cess (feedback forms). Initially, external independent

sources are typically found in the form of a SCM scholar’sobservation of the SCM phenomena of interest in his/herenvironment (the phenomenon driven learning envi-ronment), or through a scholar’s interpretation of exist-

ing knowledge (the learning environment containing thebody of knowledge to date–both SCM and relatedknowledge). The SCM scholar digests these inputs anddevelops initial insights and perceptions about the phe-

nomena of interest (as in Boyer’s dimensions of discov-ery and integration). These insights and perceptions are,in turn, disseminated through different channels to ap-propriate audiences and/or become part of the scholar’s

research (the ongoing scholarship of discovery). Dis-semination audiences might include other scholars, stu-dents and/or SCM practitioners.

The outcome of these initial insights and perceptionsthat becomes part of the SCM scholar’s research may leadto additional theoretical conceptualization. This con-

ceptualization may be facilitated through a groundedtheory approach to the phenomena of interest (e.g.,Strauss and Corbin 1990), and/or may be adopted fromexisting SCM and other related theories. With iterations,

a theoretical framework evolves and is eventually trans-mitted to an interested SCM audience through someform of presentation and/or publication in the SCM lit-erature (conferences presentations, proceedings, journal

articles). This output becomes a documentation of thescholar’s insights and perceptions, but also serves as acatalyst for further conceptualization (insights and per-ceptions) and subsequent theory testing (research) by the

scholar and others. The SCM scholar considers the resultsfrom theory testing in light of the existing SCM and re-lated knowledge bases. How do these results comparewith existing empirical evidence? Do they confirm or

conflict with existing findings? What are the implica-tions? The research process itself provides feedback to thescholar and the SCM discipline (through the publicationprocess), and produces further insights and perceptions.

The SCM scholar’s insights and perceptions are alsodisseminated through the instructional process. Ascholar’s ideas are shared with his/her students and this

SCMScholar

Integrative Process:Insights andPerceptions

Application/Consulting/

ServiceTeaching Conceptualizaton/

Research

Inputs ofStudents

Inputs ofPractitioners

PublicationIn the SCM Literature

Observations ofSCM

Phenomenon

ExtantLiterature

Observations

DocumentedBody of SCM Knowledge

Anecdotal

IntegrationWith Existing

SCM Knowledge

Adapted from Mentzer and Schumann 2006

FIGURE 1A Process Model of Supply Chain Management Scholarship

Rigor versus Relevance: Why Would We Choose Only One?

April 2008 73

process results in student reaction and response. It isimportant to note here that today’s SCM student rangesfrom the undergraduate major in his/her first introduc-

tory SCM course, to the corporate SCM executive in asenior management executive education program. Thestudent’s perspective is most likely to evolve from ques-tioning and examining the validity of the scholar’s orig-

inal insights and perceptions. Do the insights andperceptions of the scholar make intuitive sense? Do theysurvive a test of logic? Do SCM examples exist within thestudent’s experience that either confirm or contradict

these insights and perceptions? Has the student’s readingof the body of SCM and other related literature uncov-ered perspectives the scholar missed? The resulting stu-dent validation or refinement of an insight or perception

may come from the existing body of SCM and other re-lated knowledge/literature, but is most likely based uponanecdotal evidence. Thus, students may arrive at agree-ment with the scholar, or they may find SCM examples intheir own life experiences that cause an adjustment to the

insights and perceptions (e.g., theory adjustment or fal-sification). These student ‘‘observations’’ serve as inputsthat provide important feedback to the scholarly process.

Furthermore, as students transition from the classroom

to the world of SCM practice, these initial insights andperceptions of their teachers are brought forth to theworkplace, communicated through students to co-work-ers (e.g., ‘‘My SCM instructor told us . . .’’), and tested in

this new learning environment. Again, these observationsare anecdotal, but test the validity of the scholarly in-sights and perceptions and, if systematically monitored,provide inputs back to the overall SCM scholarly process.

If SCM scholars are diligent, they make an effort to trackthe results of their dissemination of insights and per-ceptions by actively seeking feedback from former SCMstudents. Thus, the workplace provides a quasi laboratory

in which to test the SCM scholar’s ideas. In this manner,Boyer’s (1990) scholarship of teaching is a two-way, dy-namic process in which the student learns from thescholar and the scholar receives ongoing feedback on the

validity and relevance of what he/she is teaching. Suchfeedback, and its resultant adjustment of insights andperceptions, is fundamental to the scholarship of teach-ing. In other words, teaching by itself is not scholarship–it is only when the insights learned from teaching are

channeled back into the body of knowledge that teachingachieves the state of scholarship.

Finally, the scholar is often asked to provide his/herexpertise in an application/service capacity. This expertise

incorporates the insights and perceptions gleaned fromthe scholarly process. The act of providing SCM knowl-edge again invites a response from a learning environ-ment. In this case, the response comes either from

individuals or from the SCM environment as a whole.These responses take the form of observations about thephenomenon of study, are anecdotal in nature and,

again, are part of a feedback loop to new or adjustedthinking on the part of the SCM scholar. The applicationof these insights and perspectives either elicits the ex-

pected results in SCM practice or it does not. Whenviewed from the perspective of field research, applica-tion/service provides evidence of the limits to externalvalidity (where the insight works and where it does not)

and relevance, and the opportunity to generate addi-tional hypotheses to be tested within the theoreticalframework (McGrath 1982). Thus, the scholarship ofapplication and service is only achieved when the scholar

treats such activity as field research for their program ofstudy and, as such, uses such anecdotal insights as feed-back to his/her continued insights and perceptions.

The questions that result from this process model of

scholarship are twofold. One, how do scholars followthis process to provide relevant, rigorous contributions tothe SCM body of knowledge? And two, how do practi-tioners incorporate this knowledge into practice? Eachquestion is addressed in the next two sections.

GUIDELINES FOR SCHOLARSFirst and foremost, SCM articles should be written

clearly. The old adage that ‘‘the more obtuse the writing,the more likely it is to get published,’’ is simply not true.

Clearly articulating the purpose of the article, telling thereader a story of how the literature and/or qualitativeresearch led to the theory and hypotheses, how they weretested and what we learned from all this is essential for

our research colleagues and practitioners to understandwhat was done, why it was done, and why it is relevant.However, complex issues should not be oversimplified.The writing must be clear, but the rigor maintained and

adequately explained (i.e., not oversimplified). Thus,clear, accurate writing leads to greater likelihood ofpublication and impact.

Good research is grounded in existing theories and

practice, but also contributes something new to an im-portant issue. The real measure of any research is itsimpact. Does it address an important question in thediscipline of study? Does it address the question with

sufficient rigor that we can trust the results? Can we un-derstand what the findings mean for future research andpractice? The answers to these questions ultimately de-termine the impact on theory and practice.

Does the research have sufficient antecedent justifica-

tion? This implies three things. First, that the concepts wedefine in the research (called constructs because theirdefinitions are constructed from theories) are logical der-ivations of existing theory and occurrences in practice (this

is called theoretical correspondence, i.e., the definitionsand titles we give to the constructs correspond to thetheory). Second, the measures of our constructs are con-sistent with the definitions and names of our constructs

(this is called operational correspondence, i.e., our

Journal of Supply Chain Management


measures (operationalizations) of our constructs corre-spond to their definitions/names). Third, the relationships

between our constructs (called hypotheses) are logicallyconsistent with existing theory and occurrences in practice.

Is our methodology rigorous? This means, first, that ourmeasures are valid. There is not space in this paper to go

into the excruciating detail of how we test for variousaspects of validity, but the short answer is validity an-swers the question, ‘‘Did we measure what we said wemeasured?’’ It also means we picked the right methods totest our hypotheses, or answering the question, ‘‘Did we

test what we said we tested?’’We have to eventually tell the reader what it all means.

Do the conclusions stick to the context in which the datawere gathered and not try to draw conclusions beyond

what we tested? Do we properly articulate the limitationsof the study, and as a result, lay out what future researchis needed to overcome these limitations? It is importantto recognize that all research is flawed. Methodological

research strategies fall into four generic classes (I, II, IIIand IV in Figure 2). These classes differ according towhich one of three research goals (A, B and C in Figure 2)

is maximized (McGrath 1982). The three researchgoals are:

A. Maximum Generalizability, i.e., the ability to gener-alize to the population(s) of interest,

B. Maximum Precision/Control, i.e., precision in con-trol/measurement/manipulation of variables,

C. Maximum Realism of Context, i.e., existential realism,or whether or not the research ‘‘(takes) place in set-tings that are existentially ‘‘real’’ for the participants(or the objects of the system of interest)’’ (p. 74).

Research goal A addresses one dimension of external

validity, i.e., the ability to generalize to a populationcontingent on how much the chosen sample representsthe population. Research goal B addresses the constructvalidity of a concept, as reflected in the convergent and

discriminant validity of some particular set of opera-tionalizations of the concept (McGrath and Brinberg1983). Research goal C addresses a second dimension ofexternal validity, i.e., that of realism, or whether or not

the context of the research closely matches some real

Point of Maximum Concern withGenerality of Actors

Point of Maximum Concern with Precisionof Measurement of Behaviors

Point of Maximum Concern withSystem Characterof Context

A

C

B

I = Settings in Natural SystemsII = Contrived and Created SettingsIII = Behavior not Setting DependentIV = No Observation of Behavior Needed

Source: McGrath (1982, p. 73)

I

I

IIII

III

III

IV IV

Formal Theory Computer Simulation

Field Studies

Field Experiment

ExperimentalSimulations

Laboratory Experiment

Judgment Tasks

Sample Surveys

FIGURE 2Methodological Research Strategies


April 2008 75

world counterpart (Lynch 1982). Thus, any single studyis going to lead to future research to better understandother aspects of the phenomena under study.

Finally, our work should be timely and timeless. Timelyrelates to the issue of relevance, i.e., does the researchaddress phenomena that practitioners are presently fac-ing? However, we should also be cognizant of the larger,

timeless aspects of the issues we are studying. For ex-ample, it is timely to study the strategic and operationaleffects of transponders on containers that are linked withlow earth orbiting (LEO) satellites to track containers in

global supply chains. This is certainly timely to whatsupply chain practitioners are facing today. However, thetimeless issue of supply chain visibility, and its impact onglobal supply chain risk management, has been with us

for decades and will be for decades to come. Drawing ourparticular research study from this broader, more time-less, theory of the necessity to keep track of materials intransit to manage risk gives our study greater breadth oftheoretical underpinnings and implications. This is pre-

cisely the job of the scholar — to address timely issueswith timeless theories and perspectives.

GUIDELINES FOR PRACTITIONERSFor practitioners to achieve full value from scholarly

work requires recognition by scholars and practitioners

of what parts of the scholarly article should be written forpractitioners, and what parts should not. The middlesection of any scholarly work should primarily be dedi-cated to rigor. It is the authors’ demonstration to the

editor, the reviewers and other researchers that the workwas done correctly (it addresses the details of what wasdescribed in the previous section of the paper as ante-cedent justification, theoretical correspondence, opera-

tional correspondence, validity and methodology). Therole of the editor and the reviewers is to make certain thisrigor is accomplished so the practitioner can trust thatwhat is said in the rest of the paper is (with a reasonable

level of confidence) correct.The realm of the practitioner in a scholarly paper should

be the abstract, introduction, literature review and con-clusions. It is in these sections that the authors establish

relevance — i.e., whether the paper is important, new andimpactful on what we know about the discipline. Readingthese sections tells the practitioner what they can takeaway from the article to apply to their practice. ‘‘Scholarly’’practitioners take it as their responsibility to report back to

the discipline (through communication with the authors,professional presentations, or their own papers) whetherthe conclusions of the paper actually work in practice, andif so, the conditions under which the conclusions worked

and did not work. This, in turn, adds to what we all knowabout the discipline we are studying (represented as one ofthe feedback loops in Figure 1).

Finally, it is the joint responsibility of scholars and

practitioners to understand each other. Practitioners

should constantly provide to scholars the issues that areimportant to them to give relevance to the work of thescholars. Scholars, in turn, should regularly discuss their

research with practitioners to get a ‘‘reality check’’ on thetimeliness and relevance of their work. This requiresscholars to attend ‘‘practitioner conferences’’ and practi-tioners to attend ‘‘educator conferences’’ and interact

with each other. It was to facilitate this type of two-waycommunication that we at the University of Tennesseestarted the Supply Chain Strategy and ManagementForum over 10 years ago. One of the primary purposes of

this Forum is to bring together practitioners and aca-demics twice a year to exchange ideas. Practitioners reg-ularly provide insights into the issues they are facing. Theacademics involved in the Forum regularly report back to

the practitioners how they address these issues: usuallythrough speakers at the Forum; changes in undergradu-ate, MBA, doctoral, or executive development curriculumat the university; or research projects. For the last in thislist, it is the responsibility of the academics to make the

research timely and timeless so that it answers the prac-titioners’ immediate issues, but also contributes to thescholarly body of knowledge. Presently, we have over 50companies involved in the Forum–testimony to the fact

that our work can be both rigorous and relevant. Weencourage other schools to consider creating similar ve-hicles for practitioners and scholars to interact and im-prove the relevance and rigor of our collective work.

CONCLUSIONS — IT IS NICE TO NOT HAVETO CHOOSE

For research done well, we do not have to choose be-

tween rigor and relevance. Researchers, editors and re-viewers should pay attention to rigor. Researchers,editors, reviewers and practitioners should pay attentionto relevance. Researchers and practitioners should dedi-

cate themselves to regularly interacting to understand thetimely and timeless issues faced in our discipline. If wedo all these things, our journals will produce much morevaluable and scholarly knowledge and application about

the discipline of SCM. This will lead to the extinction ofthe ‘‘gap’’ between scholarly research and practice.

REFERENCESBoyer, E.L. Scholarship Revisited: Priorities of the

Professoriate, The Carnegie Foundation for theAdvancement of Teaching, Princeton, NJ, 1990.

Lewin, K. ‘‘The Research Center for Group Dynamics atMassachusetts Institute of Technology,’’ Sociometry,(8), 1945, pp. 126-135.

Lynch, J.G. Jr. ‘‘On the External Validity of Experiments inConsumer Research,’’ Journal of Consumer Research,(9:3), 1982, pp. 225-239.

McGrath, J.E. ‘‘Dilemmatics: The Study of ResearchChoices and Dilemmas.’’ In J.E. McGrath, J. Martinand R.A. Kula (Eds.), Judgement Calls in Research, SagePublications Inc, Beverly Hills, CA, 1982, pp. 69-102.

Journal of Supply Chain Management


McGrath, J.E. and D. Brinberg. ‘‘External Validity and theResearch Process: A Comment on the Calder/LynchDialogue,’’ Journal of Consumer Research, (10:1), 1983,pp. 115-124.

Mentzer, J.T. and D.J. Flint. ‘‘Validity in Logistics Research,’’Journal of Business Logistics, (18:1), 1997, pp. 199-216.

Mentzer, J.T. and D.W. Schumann. ‘‘The Theoretical andPractical Implications of Marketing Scholarship,’’Journal of Marketing Theory and Practice, (14:3),2006, pp. 179-190.

Simon, H.A. ‘‘The Business School: A Problem inOrganizational Design,’’ Journal of ManagementStudies, (4), 1967, pp. 1-16.

Strauss, A. and J. Corbin. Basic of Qualitative Research:Grounded Theory Procedures and Techniques, Sage,Thousand Oaks, CA, 1990.

Van de Ven, A. ‘‘Nothing Is Quite So Practical as a GoodTheory,’’ Academy of Management Review, (14:4),1989, pp. 486-489.

John T. Mentzer (Ph.D., Michigan State University) is the

Bruce Excellence Chair of Business at the University ofTennessee in Knoxville, Tennessee. His research hasfocused on the contribution of marketing and logistics tocustomer satisfaction and strategic advantage in supplychains; the application of computer decision models to

marketing, logistics and forecasting; the management ofthe sales forecasting function; and mega trends affectingthe future of logistics and supply chain management. Hewas recognized in 1996 as one of the five most prolific

authors in the Journal of Academy of Marketing Science, andin 1999 as the most prolific author in the Journal ofBusiness Logistics. He was awarded the Academy of Mar-keting Science Outstanding Marketing Teacher Award in

2001, the Council of Logistics Management Distin-guished Service Award in 2004, and the InternationalSociety of Logistics Armitage Medal in 2007. He is a pastpresident of the Academy of Marketing Science and the

Council of Logistics Management. Dr. Mentzer receivedan MBA from Michigan State University.


April 2008 77

rigor versus relevance: why would we choose only one?

Documents