A prompt about 'unprompted' survey results

0
There's no such thing as an "unprompted" survey response: it's an oxymoron

We repeatedly hear researchers refer to statistical survey results as “unprompted”. Sadly, it’s an oxymoron that can only serve to impoverish, if not seriously misguide and bias, the insights to be gained from the survey.

“Prompted versus unprompted” response language even turns up in peer-reviewed research papers, along with the gibberish of supposed “unprompted questions” in marketing textbooks. Oh dear. Can they even hear themselves talking?

At ResearchSquirrel we argue that “unprompted” language should be excised from the research lexicon. Here’s why.

An example

Say your company markets manual shaving razors and one of the things you want to assess is the market power of your brand, compared with competitor brands. One headline metric for that is brand awareness. We can ask survey respondents about awareness of razor brands in different ways, for example:

“Thinking about razors, what brands are you aware of?”

or

“Are you aware of the Neroom razor brand?”

The first way is category prompted, because we asked the respondent about brand awareness by framing their thoughts about the product category. We then let their minds ‘fill in the blanks from there’ as it were.

The second way is brand prompted, because we asked the respondent about brand awareness by framing their thoughts about, and specifically identifying, a particular brand (and usually asking about each of them... or at least the major brands). For this article I've made up the Neroom brand as a market leader in the manual razors category.

There are good reasons for asking one way or the other, though generally at ResearchSquirrel we prefer the category prompted approach, for reasons beyond the scope of this article.

Where does “unprompted” come in?

Some researchers refer to the first way of asking (above) as “unprompted” meaning that no actual brand was prompted, and the second way as “prompted”. But that’s a sleight of hand that can lead to ill-informed thinking and misinterpretation.

The stimulus-response pair

There's no such thing as an unprompted prompt.

Here’s the point: both questions were prompted, but by different things. In fact in survey research, as anywhere, it’s impossible to have an “unprompted response”. It’s an oxymoron. By definition, you administer a prompt (stimulus — the question), and record the response (the answer).

Participants are not called respondents for nothing. You can't get a response to analyse and interpret, without a stimulus.

Why does it matter?

Perhaps it’s sounding a little academic. But it’s far from that. It’s central to valid and penetrating interpretation of research data.

“Unprompted” language can invite less critical and often quite invalid interpretation of the data.

Let’s say 87% of the market said they were aware of Neroom when prompted by category (razors). By referring to the result as “unprompted”, the research client may come to assume at some point in their time-poor and complex working milieu, that the reported figure means more generally that “87% of the ‘unwanted hair’ market automatically and naturally thinks about Neroom”. After all, that's what “unprompted” means, doesn't it?

Far from it! It means when prompted to think about razors (and only razors), 87% of the market can, whether immediately or after some mental effort, bring to mind the Neroom brand.

Every question frames the responses

To refer to any result as “unprompted” is to misunderstand or misrepresent market research, as though question stimuli are devoid of any influencing context.

Unless you guide or frame respondents’ thinking, you'll just get “random stuff” responses.

On the contrary, unless you guide respondents' thinking in respect of what you want to know, a statistic for “my next-door-neighbour's stereo was too loud last night”, “I wonder what Jenny's doing next weekend”, and countless other random thoughts will turn up in your data. What you actually want to know probably won't.

To efficiently obtain the information we're seeking, we have to shape and guide respondents' thinking. The skill is to frame the stimuli with as few assumptions about our own interests, perspectives, perceptions, expectations and preferences as possible, to get to the nub of what's actually in respondents' hearts and minds. Nevertheless, questions are always framed.

Framing is an interpretation anchor

If you framed the respondent’s thinking around the razor category, then that’s the context in which the result must always be interpreted. That and no further.

Every survey prompt comes pre-loaded with its own framing, which influences the response you get.

To illustrate how critical framing is, let's switch to an objectives stimulus: “unwanted-hair control”. That's closer to an actual value proposition. If Neroom's real market is people who want to control and remove unwanted hair, their sponsored research has only asked about one method of achieving it: Neroom's. As a razor supplier, that’s their interest. It's not necessarily respondents'. It’s an understandable supplier perspective that can cloud research insights.

Another stimulus framing

What if respondents were instead asked:

“Thinking about how you might control or remove unwanted hair, what brands are you aware of?”

Respondents will also recall brands of electric razors, depilatory cream, waxing strips, electrolysis studios and feminising hormone tablets, for example. These are all potential solutions to the respondents' problem that lie outside Neroom's product category. And Neroom may enjoy only a 42% — and shrinking — brand awareness in this (objectives) market.

Referring to a result as “unprompted” is itself a framing, and a misleading one.

But you wouldn’t be focused on that critical insight if you were revelling in the warm and fuzzy result of 87% “unprompted” Neroom brand awareness. Nearly everyone naturally thinks about Neroom, hurrah!

No they don't. The “unprompted” tag is itself a framing — in this case of interpretation rather than of stimulus. We argue it's a poor and easily misleading one.

Conclusion

Referring to any survey result as “unprompted” can inadvertently lead to invalid assumptions about what the figure means. In this simple example, “unprompted” brand awareness may be easily misjudged as representing a brand's market power in solving people’s problems. It can lead to quite misguided investment decisions.

Ultimately, the oxymoronic notion of “unprompted” survey results should be ditched altogether. Proper descriptive anchors such as “category prompted”, “brand prompted” and “objectives prompted” are the appropriate practice. Results are then far more likely to be interpreted correctly, avoiding damaging and even ruinous thinking sneaking into business decisions.


Share This Post:

Got a topic you'd like us to consider? Ask here...