The danger of generic questionnaire templates

It happens a little too often: the client who asks us to tweak a questionnaire template they've retrieved from... somewhere or other.

The wrong end of the research design stick

Starting with a preset template is like choosing your house paint colours before you've decided what kind of house you want.

In terms of questionnaire design, using someone else's template is to start at the wrong end of the stick—even if there are no copyright issues.

The design process gets muddied and corrupted if you start with a template (free as it might be) that wasn't developed precisely for your own unique environment and particular decision-making needs. Starting with a generic template easily leads to all sorts of assumptions that may not be true even if the template is otherwise well-written (which many aren't); to undesirable biases.

It's like choosing your house paint colours before you've even decided what kind of house you want.

Fundamental and serious flaws

Some of the biases of generic templates we've seen include:

• Not actually measuring what the template title suggests it does.
• Failing to qualify respondents in meaningful and appropriate ways.
• Asking questions whose answers will be so general as to hide any insights that could lead to effective action.
• Clumsy question repetition that asks the same thing in different ways, unnecessarily exhausting respondents.
• Badly-worded questions that leave respondents confused; or just as bad, think they're answering a question you didn't ask.
• Heavily biased questions (or their coded answers) that prejudice responses in a particular direction.
• Poor question order that leads to unwanted bias in following questions.

An example to illustrate the risks

Even supposed 'expert certified' generic templates can be rubbish.

To illustrate the piont, we've critiqued a free online 'customer satisfaction quetsionnaire' template; one that even claims to have been certified by a research expert. We doubt it: take a look for yourself at the dozen-plus serious issues we found.

The right end of the stick

A questionnaire is properly and professionally designed from your own specific and proper research plan, which includes context, objectives and methods. That's the approach we take at ResearchSquirrel.™ It means that the questionnaire maximises opportunities for deep insights which directly address your own most compelling issues, while minimising possible bias effects.

(In fact, your research opportunity might actually be best solved using a methodology other than a pre-coded, structured questionnaire. If that's the case, we'll recommend those other methodologies.)

Conclusion

Remember: a top-quality, professional questionnaire that feeds directly into effective decision making is founded on your own custom detailed and proper research plan, not on someone else's notion of what 'general questions anyone might ask.'

 

Keywords: 

Bias

|

Market research

|

Questionnaire design

• Log in or register to post comments