Qualitative Analytics
By
Jerry W. Thomas
Much has been written about how to conduct qualitative research (that is, the
techniques of moderating and interviewing), but comparatively little has been
published about the far more important task of analysis and reporting. The purpose
of this primer is to share some basic ideas on how to achieve the greatest learning
and the most profound insights from qualitative research.
The term “qualitative research” generally refers to relatively
unstructured, nondirective discussions or interviews (such as focus groups,
depth interviews, and ethnography or observation) to explore a topic. Typically,
sample sizes are small, fewer than 50 respondents, and sampling methods are
rarely scientific.
In recent years, data-collection methods have expanded from in-person focus
group discussions, depth interviews, and observation to include online chats,
bulletin board forums, online communities, and Web monitoring. Some researchers
would also include neuromarketing research (physiological measures of human
reactions) as a part of the qualitative domain. As more is learned in future
years, perhaps neuromarketing research can evolve into a quantitative science.
The terms “moderator” or “moderator-analyst” or “analyst”
will be used in this paper as descriptors for the person who conducts a qualitative
project, regardless of project type.
While this article will primarily discuss in-person qualitative, most of the
analytical concepts apply equally to online methods. Decisions at every step
in planning, executing, and interpreting qualitative research influence the
validity, reliability, and relevance of the ultimate analysis.
A Time And A Place
The first step to good analytics is to use qualitative research only when and
where it is appropriate. Qualitative research techniques are completely interactive,
and that allows us to attack problems with thousands of variables, whereas most
quantitative research is limited to fewer than 200 variables. Qualitative research
is recommended…
- When you need to understand human motives. Qualitative can answer the “why”
of human behavior (for both consumers and business-to-business).
- When you are studying intricate systems or processes. For example, what
are all of the steps and actions, sources of information and influence, involved
in buying a new car?
- When you are planning large-scale quantitative research and need to understand
the consumer’s language, perceptions, issues, and range of answer choices
to help design the questionnaire.
- When marketing issues are especially complicated and confusing. The interactive
nature of qualitative allows the moderator-analyst to navigate through very
complex issues.
- When sample sizes are very limited. If you cannot talk to large numbers
of people, then get as much information from each individual as possible.
Not All Respondents Are Equal
The analytic output of qualitative research is influenced by how respondents
are chosen. Respondents are not equal in analytical value. Some are articulate,
some are not. Some are knowledgeable, some are not. Some are aware of their
own motives, while others are not. Some recruiting guidelines:
- Try to make the sample as representative as possible of category users,
including heavy users. Heavy users tend to know much more than light users
or nonusers.
- Don’t include so many screening hurdles that you end up with a very
low-incidence and nonrepresentative sample. Who cares what .005% of the target
audience thinks?
- Don’t do all of your qualitative research on the “focus group
circuit” in Los Angeles, San Francisco, Chicago, New York, Atlanta,
and Dallas. Choose markets based on target-market considerations, not flight
schedules.
- Consider screening the sample in some way to ensure that respondents can
communicate, but beware of introducing bias. I once discovered that cowboys
in the panhandle of Texas who dip snuff do so to avoid talking, so screening
for good communicators would have destroyed the greatest single learning from
the groups. In general, you will learn more from better communicators than
poor communicators.
Free The Moderator
Planning for the actual conduct of the qualitative research is a crucial step.
Much of the analytical potential is determined at this stage. The moderator
and the client must invest substantial time in sharing information and ideas
to ensure the research objectives are correct and precise. This upfront planning
should involve all of the key stakeholders in the client company. It’s
essential that the moderator understands the situation, the background, the
issues, the politics, and the risks related to the project.
It’s also essential that the client gives the moderator the freedom to
design the interview guide to achieve the objectives. Some clients like to dictate
exactly what questions will be asked in what order, subverting group dynamics
and discussion into a moderator-led “question and answer” interrogation,
which greatly diminishes the reliability and analytic value of the groups. The
dynamics of a free-flowing discussion reveal patterns, issues, and motivations
that mere questions would never uncover. Let the moderator design the guide;
give the moderator the freedom to maneuver, to deviate from the guide, depending
on events in the session.
It’s critically important that the client and the moderator build trust
and confidence in each other, so that the qualitative research can be conducted
in an emotionally relaxed environment. A stressed or distrustful client creates
a stressed moderator, who creates stressed respondents, and that inhibits the
group discussion.
It’s generally best to limit the number of observers to no more than
five or six. The greater the number of observers, the less anyone in the backroom
will see or hear. With a large number of observers, the backroom clown or comedian
will have an audience to entertain. The greater the number of observers, the
greater will be the stresses on moderator and respondents.
Eyeball To Eyeball
The moderator should also be the analyst. Sitting eyeball to eyeball with respondents
gives the moderator information and clues that are not visible from behind a
one-way mirror. A good moderator will begin to make inferences as the qualitative
research unfolds, and will test those inferences in subtle ways (by taking the
opposite point of view, for example, to test how strongly people feel on an
issue). It’s not always possible for the moderator to be the analyst,
but at a minimum, the moderator should have inputs to the analytic process.
The Wallflower Moderator
The philosophy and style of the moderator are key determinants of analytical
outcomes. The ideal moderator is soft, low-key, nonthreatening, warm, accepting,
and quiet. The goal is to let respondents tell their stories with minimal interference
and bias from the moderator. Most questions and most probes risk “leading
the witness.” So, fewer words from the moderator mean less bias. A great
moderator can guide a group discussion with very few words.
The goal of the moderator is to make respondents the center of attention.
The moderator should be a wallflower. The moderator’s job is to make respondents
feel totally accepted, relaxed, respected, confident, and competent. Moderators
must downplay their own importance by the way they dress, how they act, and
what they say. Even if an expert on the topic, good moderators will typically
act as though they know nothing.
Moderators must be open to and accepting of all answers and information, and
must not take anything for granted, or jump to conclusions, or carelessly assume
that something is true before vindicated by sufficient evidence. Objectivity
is absolutely essential. If a moderator is highly opinionated on a subject,
that moderator should not be permitted to conduct qualitative research on that
subject. If a client is highly opinionated on a topic, that client should not
be permitted to influence the qualitative research plan or the interpretation
of results.
Searching For Black Holes
As the qualitative research progresses, some of the most important dimensions
involved in analytics are:
Unconscious Motives
While we all like to think of ourselves as logical and rational, the truth
is we are heavily influenced by deeper instincts, genetics, and emotional forces
that we are not fully aware of and don’t fully understand. These unconscious
factors include our fears, sublimated sexual drives, anxieties, status needs,
dependencies on others, and dominance aspirations. The moderator-analyst searches
for these underlying motives like an astronomer looks for black holes—inferring
their presence by studying what is happening around the edges and by measuring
their influence on nearby celestial bodies.
Symbolism
As moderators study the transcripts or videos, they are sensitive to the functional
or utilitarian benefits of a product, service, or brand, but also sensitive
to its symbolism. For example, a dress provides functional benefits (keeps one
warm, prevents sunburn, hides nakedness), but it can also express symbolic values,
such as one’s social or economic status, one’s tribal affiliation,
or one’s search for a new mate. All products and services provide functional
and symbolic benefits, and the analyst must fully understand both. Much of modern
marketing is based on understanding and exploiting products’ symbolic
values.
Psychological Projection
Often we see in others what we cannot see in ourselves, or refuse to admit
about ourselves. The person who accuses others of gossip mongering is most likely
the gossip. Good moderators incorporate projective techniques into the interviewing
process, overtly and covertly, to help reveal or confirm respondents’
deeper motives. For example, the moderator might encourage respondents to talk
about their friends or neighbors, or about imaginary people or situations. The
moderator might use pictures, collages, or inkblots as stimuli, or ask respondents
to engage in role-playing, draw pictures, or make up stories.
Language
The analyst pays close attention to each respondent’s exact words and
phrases, in much the same way as a lawyer or a detective. The details of language
reveal a lot about a respondent’s motives, aspirations, emotions, behavior,
biases, and perceptions. For example, if a respondent says, “I have to
go to work at 8 a.m. in the morning,” that is a far different response
compared to the person who says, “I go to work at 8 a.m. in the morning.”
The “have to” is the difference, and the analyst must strive to
understand. Does the “have to” mean that person doesn’t like
to go to work or doesn’t like his or her job? Or, does it mean that respondent
feels guilty about leaving the children at home?
Metaphors
We use many different metaphors as we talk. For example, when talking about
“love,” we often use the metaphor of “falling.” We don’t
say, “I got myself in love.” No, we say, “I fell in love.”
What does the metaphor of falling tell us about love? It suggests that love
is sudden, unplanned, accidental, uncontrollable, and possibly dangerous. Nicely,
since love is an accident, perhaps we don’t have to accept responsibility
for our behavior either.
Omissions
The analyst is not only analyzing what was said, but is trying to identify
and understand what was not said. These omissions can be clues that lead to
new hypotheses and new understanding.
Slips of the Tongue
Do respondents exhibit a pattern of “slips of the tongue,” (that
is, using an unintended word or term)? Often these “slips” reveal
deeper feelings, issues, or prejudices. For example, if a respondent mentions
“Honda” by accident when intending to say “Toyota,”
it reveals something about how these two brands are perceived. The analyst must
be very attentive to these “slips.”
Emotions
When respondents suddenly burst out in laughter, the analyst circles back to
review what triggered the outburst, and looks for the underlying causes. Often
laughter is a way of dealing with stress or taboos. What were the stresses or
taboos that led to the laughter, and how do these relate to the product or service
under study? Tears also tell a story. Many years ago, while conducting a series
of focus groups on jewelry, I was startled by female respondents starting to
cry soon after each discussion would begin. As the groups unfolded, it became
obvious that jewelry is more than “just glitter.” Jewelry is a symbol
of, and proof of, a man’s love for his woman.
Intuition
What does the moderator “sense” while interviewing respondents?
How do these respondents make the moderator feel? Are they being open and honest,
or are they hiding the truth? Are the respondents aware of their motivations,
or are they deluding themselves? Are these nice people, or not so nice? Are
the respondents selfish, or not so selfish? Are they stylish, or not so stylish?
Very often these kinds of learning can only come from actually conducting the
focus groups or depth interviews. That’s why it’s so important that
the moderator also be the analyst.
Observation
Fish are probably not aware of the ocean. People are like the fish, but the
ocean we all swim in is our culture. Often we are unaware of exactly what we
are doing and why—because we have never known anything else. A little
observation can go a long way.
A study of residents at a large real-estate development in a Central American
country illustrates the point. Telephone interviews from the U.S. indicated
that the country was very safe and that no one felt security was an issue. When
the moderator arrived to personally observe the apartment complex and interview
residents, he was shocked to discover that the complex was surrounded by rows
of military-type razor wire with guards posted at entrances. Respondents in
the country had adjusted to living in a highly insecure environment, and they
considered the situation as normal.
Observing people in their homes, or while they are shopping in a store, can
raise new questions and lead to new understanding. Observation does not have
to be in person.
With digital cameras ubiquitous in many countries, respondents can be asked
to submit photos of their refrigerators, their bathrooms, their garages, their
neighborhoods, or their friends. These photos can lead to new questions and
new insights.
Clues And Artifacts
There are no shortcuts, no easy ways to analyze qualitative data. True, software
that purports to analyze qualitative data is available, but its value tends
to be limited. No software can take the place of an observant, analytical human
mind. Analysis involves listening to tapes, watching videos, and reading transcripts
of respondents’ verbatim statements. Respondents’ answers are often
confusing, inconsistent, entangled, and complicated. The analyst spends many
days poring over videos or transcripts again and again, striving to understand
what it all means. Instant analysis is no analysis at all.
As moderators read transcripts and watch videos, they are searching for clues,
like a detective at a murder scene. Or, like an archeologist excavating a site,
sifting for artifacts. These “clues” or “artifacts”
are called hypotheses; that is, “facts” that appear to be relevant
and true. These “facts” are organized into sections and logical
order. Once these hypotheses (i.e., all the clues and artifacts) are logically
grouped, the analyst reviews them multiple times, looking for relationships
and linkages among the hypotheses and thinking deeply about what the hypotheses
mean. Through a deliberate process of inductive reasoning, the analyst arrives
at a set of conclusions based on the hypotheses. The last step is developing
the marketing recommendations based on the hypotheses and conclusions, as well
as on the industry, category, and brand knowledge of the moderator-analyst.
Dog And Pony Show
In the hands of a good moderator-analyst, qualitative research is one of the
most powerful analytical tools available to researchers and marketers, but the
full value of the analysis is often lost—even in the best of written reports.
Bring the moderator in to do a “dog and pony show” for all the key
stakeholders at the same time in the same room, so that questions can be answered,
ambiguities can be resolved, and marketing implications can be discussed. The
goal is for all stakeholders to leave the room with the same understanding of
the results. It’s a way to help stakeholders achieve consensus and agree
on next steps, which typically involve tests or experiments to confirm the hypotheses.
The ultimate value of qualitative research is determined by the quality of
the analysis. Get more from your qualitative research by working with moderators
who understand the power of qualitative analytics.
Copyright © 2010 by Decision Analyst,
Inc.
This article may not be copied, published, or used in any way without written
permission of Decision Analyst.
About the Author
Jerry W. Thomas (jthomas@decisionanalyst.com)
is President/CEO of Dallas-Fort Worth based Decision Analyst. He may be reached
at 1-800-262-5974 or 1-817-640-6166.
Additional Resources from Decision Analyst
Related Services
Related Case Histories
Related White Papers
