Qualitative By the Number – Blog Two: Six Biases in QLMR Part One
Part One – First Three Biases
For many years, quantitative research [survey research] has been King in the research universe because “metrics” can be counted and validated. Qualitative research [focus groups, IDIs, etc.,] has been in the shadow of survey research and is often maligned and mistrusted because of several factors:
- Low sample sizes [often less than 40 people interviewed in a study]
- Convenience samples [keep recruiting until quota for study is reach]
- Variations in moderator qualifications, style and experience
- Respondents may not tell the truth to the interviewer
- Subjective data collected and analysis and reporting are also subjective
And yet, qualitative research is still alive, and well-used by those who understand the biases and work within them.
These two blogs point to six key biases that are present in qualitative research and asks readers to see, that even with their existence, the power and value of qualitative research is a valuable tool for decision-makers.
Some famous quotations:
“Know your enemy.”
“Keep your friends close and your enemies closer.”
As you read the biases outlined below, keep these quotations in mind!
SIX BIASES PRESENT IN QUALITATIVE RESEARCH
[First Three]
- The data collected cannot be projected to a universe of similar respondents.
Suppose the study design calls for 8 focus groups. Four with users of Hellman’s Mayo and 4 users of Miracle Whip. The data generated from 32 respondents cannot be projected to all users of Hellman’s or Miracle Whip.
It can report what respondents said about each product, or reactions to package design or ingredients. Those insights can provide language for ad campaigns, issues to explore more deeply in a survey or uncover drivers for purchase behavior to support strategy planning by a client.
- The moderator is the instrument of research.
Therefore, his/her, tone, pitch, pace, speed, and volume along with hairstyle, make-up and clothing will influence how respondents answer Qs and probes.
If a middle-class moderator is hired to talk to low-income residents, about a proposed change in the food stamp program, a smart researcher will leave her normal six gold rings at home and just wear one of them and dress more like a school teacher than a fashionista!
A good moderator will work to manage personal behaviors that draw attention to themselves and aim to make respondents the “hero” of the discussion rather than serve as a lightning rod to keep attention on the moderator.
- The same Qs are not asked in every group in the series the same way Qs are asked on a survey, which could include skip patterns.
If respondents answer Q 7 on the moderator’s guide and also wind up talking about an issue planned to be explored later in Q 10, and a spirited group discussion follows around the theme planned for discussion in Q 10 – a good moderator doesn’t waste time asking Q 10 as they move through the guide.
If the study design called for six focus groups – there is a high probability that one or more Q’s will be skipped for reasons such as the one above, or because the moderator decides that the sacrifice of one question leaves time to explore an unexpected direction. As well, some groups talk longer on a theme than another group, and to manage time, some Qs don’t get air play. As long as the core questions critical to reaching the study objective are asked across the series of focus groups, that is good enough to understand respondent perceptions, opinions, beliefs and attitudes [POBAs] for inclusion in the final report.
Summary
Question: Given the three biases outlined above, what is the key reason that QLMR is so important?
Answer: If you know how someone thinks, or what they believe…you have tipping points to provide clues to future purchases decisions, beliefs and behaviors.
THE REMAINING THREE BIASES WILL BE OUTLINED IN THE NEXT BLOG POST ALONG WITH A FAMOUS EXAMPLE!
Written by: Naomi Henderson, CEO & Co-Founder