Not ‘Just’ a Survey: How to Ask Better Questions

A survey can be a powerful starting point for user research and data gathering. Designers have access to a variety of survey-building tools ranging from free Google Forms to more robust database systems such as Airtable. Much like the user interview, a survey is best executed when there is a distinct flow for the user. A good survey requires:

  • A clear introduction/explanation
  • Warm-up questions (ex: demographic input)
  • Body/focus questions
  • Cool-off questions/dialogue
  • Wrap-up/completion confirmation

Beginning the survey with a defined outline can help designers determine what questions they need to be asking, rather than starting with common questions and rewording to suit their needs. Breaking a survey into the five sections also allows designers to establish limits within each section. As noted in Chapter 10 of Understanding Your User, it is important to ensure that the survey does not become so long that users will lose interest and not complete it, or worse, proceed with “satisficing.” Satisficing is a decision-making process by which individuals exert the minimum satisfactory effort. In the context of a survey, this could mean answering all the questions as quickly as possible for the sake of completion rather than considering each option to complete the survey with accuracy. To prevent users from disengaging, designers can enact several strategies to ask the questions that matter to them and their audience.

Keep it short

The first rule of writing better questions is to keep them short. Unless long-form questions are an absolute necessity (i.e. seeking feedback on a specific scenario) short form questions will yield more accurate results. Short does not mean lacking information; a designer’s job is to make the user’s role easier in both their participation in the survey and through the delivery of a more refined product.

Keep it focused

We are often tempted to write questions the way we would speak, which means we may combine questions for the sake of expedience. In conversation, it may feel natural to ask: “What time will you be there, and where should I park?”

However, in a digital survey, users experience the questions without the nuances of audio or face-to-face interaction. Therefore, to maintain momentum and reduce the demand of a user’s cognitive facilities, asking a single question at a time will yield better results.

Be neutral

As with asking focused questions, the temptation to write how one would speak can also translate to biased questions. For example, if conducting a survey on users of an existing application, the designer may want to know how long it took a user to find/employ a certain feature. However, in a survey, wording the question with “how long” may raise the user’s defenses and cause them to skew answers towards their ideal representation of self (i.e., the user may want to appear as an early adopter and claim that they found and used the feature right away.)

Rather, designers must frame the question so that they still gain the answers they seek without causing the user to feel defensive. An alternative wording for the same question would be: “when did you first use [feature]?” with answers ranging from immediately to I haven’t used this feature.

Change it up

Another essential component of asking better questions is asking a variety of questions. Recalling Scantron tests, questions with repetitive responses styles can cause users to lose interest and answer consistently (all ‘C’) or erratically (at random) to complete the survey as quickly as possible. Designers can subvert this temptation by varying the question types from Likert scales to lists and graphic selection matrices. Introducing a break in survey screens can also prevent the “wall of text” effect and ease the user into feeling that they are making tangible progress.

Conclusion

Gathering accurate survey data is an essential foundation for user experience design, but is an exercise in user-centric design itself. Keeping your user interested long enough to complete the survey and engaged enough provide the information you need is critical in delivering a more refined product to them.

Resources

  • “Chapter 10: Surveys.” Understanding Your Users a Practical Guide to User Requirements Methods, Tools, and Techniques, by Catherine Courage and Kathy Baxter, Elsevier, 2005.
  • Beadell, Stephanie. “Surveys 101: A Simple Guide to Asking Effective Questions.” Zapier, zapier.com/learn/forms-surveys/writing-effective-survey/.
  • Sauro, Jeff. “12 Tips For Writing Better Survey Questions.” MeasuringU, 11 Oct. 2016, measuringu.com/survey-questions/.
  • Zheng, Jill. “What Common Survey Questions Are Asked Most Often?” Survey Monkey, www.surveymonkey.com/curiosity/what-survey-questions-are-asked-most-often/.
  • “How to Write Good Survey & Poll Questions.” Survey Monkey, www.surveymonkey.com/mp/writing-survey-questions/.