Free questionnaire template critiqued

Very satisfied | Somewhat satisfied | Neither satisfied nor dissatisfied | Somewhat dissatisfied | Very dissatisfied

It can be useful to garner a headline attitude metric, though ‘satisfaction with our company’ may or may not be driven by product satisfaction—the assumptive purpose of this questionnaire template. For example, if the company has recently run a well-liked ad campaign or opened several major new stores in convenient locations; or alternatively has been in the news about a terrible worker injury or for financial mismanagement, that may drive the answer. While you certainly will get an overall measure you won’t know why, so what action can you take to improve?

Also note that positive answers appear on the left of these scales. This will highten positive (acquiescence) response bias compared with negatives on the left.

Reliable | High quality | Useful | Unique | Good value for money | Overpriced | Impractical | Ineffective | Poor quality | Unreliable

Wasteful duplication: Most of these product attributes are also asked in following questions: Reliable [Q4] | High quality [Q4] | Useful [Q3] | Good value for money [Q5] | Overpriced [Q5] | Impractical [Q3] | Ineffective [Q3/Q4] | Poor quality [Q4] | Unreliable [Q4]. Here, the attributes are merely binary (high/low), whereas more sensitive 5-point scales are used in later questions.

Attribute assumption: ‘Unique’ is the only attribute not asked in any way elsewhere. It may or may not be an important attribute for your customers. Indeed in some categories, product compliance with technical standards and normative expectations may be more important than ‘difference’. The ‘unique’ attribute assumes a differentiation marketing position. Alternatively, 'unique' might indeed be an important attribute, but of a non-product factor (not a product factor as only asked in this question) such as purchasing, acquisition or disposal.

Double-barrel bias: But most of all, the question asks about multiple ‘products’ in general. Unless your company sells only one product, how do you know which are being judged? Do you have a stellar product and a problem product? Should you take action on one, some or all of them? What will you do if a respondent ticks both ‘high quality’ and ‘low quality,’ which this question allows them to do?

Extremely well | very well | Somewhat well | Not so well | Not at all well

Double-barrel bias: Again, which product/s does the answer relate to? Do some products meet needs very well and others poorly? Are there components of the products that meet needs but other components that don’t? For example, are the paper towels you sell both soft and strong, but the pack doesn’t fit your customer’s home dispenser properly?

Scale bias: The mid-point statement ('somewhat well') is positive rather than properly neutral.

Very high quality | High quality | Neither high nor low quality | Low quality | Very low quality

Double-barrel bias: Again, which product/s does the answer relate to? And which product components? Many products these days are blends of goods (physical items) and services (e.g. delivery, maintenance). Is the physical item excellent but the customer service around it terrible… or vice versa? This question won’t tell you.

Scale assumption bias: It also assumes that ‘very high quality’ is best, but it may not be. Budget products such as disposable pens or razor blades are ideally based on modest quality.

Double-barrel bias: According to the IEEE there are nine major dimensions of quality. The dimensions remain unpacked in this question (but two of which appear in Q2) so how do you know which dimensions are already working well, and which need improvement? What will you fix if say, half your respondents say 'high quality' and the other half lower quality?

Excellent | Above average | Average | Below average | Poor

Double-barrel bias: Again, which product/s does the answer relate to?

Attribute bias: The 'value for money' attribute is not appropriate for aspirational and luxury brands in which the customer is after ego-satisfaction and prestige value, not monetary value. And in other categories, different attributes may be equally imporant. There are no questions about any other attributes, such as availability (location and timeliness), accessories and consumables, nor other aspects of satisfaction related to non-product experience such as purchasing, ownership and consumption.

Extremely responsive | Very responsive | Somewhat responsive | Not so responsive | Not at all responsive | Not applicable

Product bias: The question wrongly assumes that ‘products’ is the only context worthy of ‘responsiveness,’ when responsiveness to purchase orders, account queries, service and repair, etc, can be equally if not more important.

Validity problem: Secondly, what does ‘responsive’ mean to the customer: Time to first response? Accuracy of response? Usefulness of response? Time to resolution? Fairness of the resolution? Method/s of contact? Etc, etc. Without a handle on which are important to your customers, you won’t know what specifically to do to improve your service.

This is my first purchase | Less than six months | Six months to a year | 1 – 2 years | 3 or more years | I haven’t made a purchase yet

Context problem: The 'you' of this question is ambiguous if the buying unit is potentially more than a single individual. For example, if other members of the respondent's household have purchased from the company for longer than the respondent alone, what is the correct response? If the respondent is a company buying officer, are they answering in respect of their own experience, their company division, or the entire company?

Validity problem: The question also falls short on understanding this customer's value in terms of past purchase behaviour or in tying their attitudinal responses to a particular behaviour (identifiable purchase occasion). ‘Time-since-first-ever-purchase’ is certainly not the best metric, especially when it’s the only behaviour metric. What about customers who first bought from you a long time ago, didn't buy from you again for years, and have only recently returned? There is no metric for how much the respondent has purchased overall nor how much in the most recent purchase, either of which might provide insights markedly different from time-since-first-purchase. In any case, respondents have difficulty in accurately recalling such historical timelines.

Recruitment error: Someone who “hasn’t made a purchase yet” by definition is not a customer, so how can they be relevant to a ‘customer satisfaction’ survey? Their participation will invalidate the research.

This question could be usefully adapted partly as a screener question.

Extremely likely | Very likely | Somewhat likely | Not so likely | Not at all likely

This question is a behavioural mate to attitudinal question 1. It gives you a headline metric of overall likely future purchasing.

Double-barrel bias: The questionnaire fails to identify which items or categories the respondent is interested in and why, or why they may not buy again (e.g. are they moving out of the district, for example, versus having just joined the opposition’s loyalty program)? What specific executive actions will you take if your average response is ‘Somewhat likely’? You won’t know what to do.

Logic error: By definition, if the respondent said they ‘haven’t purchased yet’ to Q7, they can’t purchase “again” as this question asks.

Not at all likely:  0  1  2  3  4  5  6  7  8  9  10   :Extremely likely

This is often a good question to confirm how entrenched the customer’s attitude towards your company is and to what degree they are an evangelist or enthusiast rather than merely accepting the brand or even rejecting it. It is a simple proxy metric for 'loyalty.'

[Open-ended entry]

It’s often useful to have an open-ended question like this in order to collect ‘unexpected’ or more in-depth responses.

Insufficiently precise stimulus: This question appears without appropriate context and may well be interpreted from the rest of the questionnaire as only about the company's products. In reality would you want to know about the customer’s recent argument with your accounts receivables department? Would you want to know that one of your sales representatives didn’t show up for her last appointment? Does the fact your customer car park is always full count? Would you want to know if the customer is delighted because you made the last delivery two days ahead of schedule? More guidance needs to be given as to what contexts you are expecting them to think about. Given the vagueness of the question, might the respondent tell you how worried they are about global climate change or the grave concerns they hold about Uncle George’s upcoming hernia operation?

Desired-action failure: There was no question in the template to establish whether the respondent still has an outstanding issue with the company and whether the respondent would like to be contacted (and how): a significant lost opportunity.