Are we there yet? Growing maturity in qualitative tools and methods

Christopher Thorn

Assistant Scientist and Director of Technical ServicesWisconsin Center for Education Research

University of Wisconsin-Madison

Are we really in the back seat?

How would we know?

• The qualitative research SIG in AERA is celebrating its 20th anniversary this spring. It’s also bigger than two of the divisions

• Take a look at the long history of CAQDAS, a long list of software, and the level of the discussion on the related listserves

• The criticism of tools as distancing from data is a decade out of date

Maturity of the tools

Basic Functionality of CAQDAS Tools

• CAQDAS Networking Project– Structure of work – The Project– Closeness to data and interactivity– Exploring the data– Code and Retrieve functionality– Project Management and Data

Organization– Searching and interrogating the database– Writing tools– Output

Maturity of Evaluation & Review

• AERA Standards for Reporting on Research Methods – guidelines that discuss the underpinnings of quality

• The Qualitative Report’s Quality in Qualitative Research References provides a 22 page list of scholarly references

• Journal and grant proposal reviewers are requiring explicit discussion about the links between questions and methods used

Maturity of Literature

• Early 90’s - Pulling it all together• Specialization of the late 1990s

– Organizational Studies – Mental Health– Program Evaluation– Social Policy Reform– Education

• Reintegration of the mid-00’s– Cross-cutting issues of quality and rigor

Maturity of Training• Exploration

– The rise of artisans– Teach me the basics– You are not alone

• Specialization– The rise of guilds– Training widely

available, but often narrow

• Reintegration– Scope and sequence

Maturity of Infrastructure• Availability of

Archives• Standards for

Storing & Transmitting Data

• Standards for Secondary Use

• Scholarly Discourse on Secondary Use

Maturity of Desire• Transfer data and analytical coding from

one tool to another – leverage different strengths

• Storage and transmission standard– Ability to share primary data– Ability to share journals and memos– Ability to share coding and coding frameworks

• Real-time simultaneous access for coding and searching

Maturity of the Market• Desktop vs Teams

– QSR is using technologies that would allow web services and multi-user

• Enterprise services– Atlas services are already

there, this a is a place QSR could head as well – discovery inside the enterprise

• Real Easy vs Real Research– Market segmentation

Trends• Social Network

Analysis– Visualization of groups

and processes– New notions of mixed

methods

• Tools for federation of data and analysis– Search and storage

• Web 2.0 tools overlapping with collaborative qualitative analysis

Is it the right metaphor?• I think I got out of

the back seat a while ago

• Maybe I don’t have the right vehicle?– Hybrid– Landau– Nascar– Horseless carriage

• Maybe another picture be better?

A New Era of Collaboration• The tenor of the new

AERA publication guidelines should provide a platform for high-level dialogue and professional engagement

• It’s time to get out of the car and work together

A few other “there yet?” images