The Simple Way to Design a Standout UX Research Survey

Survey design is complicated.

I dread putting surveys together. The reason I don't look forward to creating surveys is simple: people have high expectations of the information they will glean from them, and they rarely live up to these expectations.

Surveys are too often used to get answers to questions better suited to other methods, such as qualitative interviews. At times, people use questionnaires as a cop-out. Because of this usage, I often steer people away from them.

Sometimes, though, surveys can be helpful for a study. My favorite way to use surveys is to supplement other methodologies. If necessary, I will use them in isolation. I only do this when I have no time or budget to use them in conjunction with other research techniques.

Regardless, surveys are challenging to design. Generally during user research, you're trying to get at a deeper meaning behind someone's actions or motivations. With this method, you're attempting to ask people to answer questions with a predetermined list of answers. There is no probing, digging, or explanation happening during a survey. If you ask for a reason, you aren't always guaranteed to get one. At least not the deepest level of description.

It's necessary to define the goal and objectives of the survey. By brainstorming goals and objectives, you're better able to design the study that will get you the type of results you will need—and mitigate everyone's expectations.

Jump to a section:

• Define your goal
• Quick tips for designing
• The dark side of surveys

So we would generate a list of different answers. Ideally, these answers are based on past research insights. Then, we leave an "Other" option for users to write in their response if they want to.

The survey might look like this (for an online board game platform):

Q: Which feature would you love to see us work on next?

• Worldwide online competitions
• Customizable board game characters
• Shareable record chart of wins/losses
• AI board game challenges (play against the computer)
• Practice tournaments
• Other: Open text

We're not trying to figure out why a person wants a particular feature. Ideally, we have done the qualitative interviews in which our users explained what they needed and why.

We're trying to get a larger sample size to prioritize better what to work on next. Surveys don't give you the why; they give you the what.

Typically, surveys are a bit more complicated than the above example and involve many more questions. In this case, there are many more pieces of the puzzle to consider. You have to think about question structure, number of questions, flow, and many other considerations. A longer survey requires much more brain power and time.

It becomes much harder when we want to use surveys to understand behavior or opinions. However, it's not impossible. Again, I recommend doing qualitative research in addition to sending out a questionnaire or poll.

To illustrate this, let's take another example. The online board game platform was so successful that we decided to start an entire online board game store and community.

We're looking to understand the following from our survey:

1. How people would like to purchase board games online
2. What people expect of a board game community
3. What they think of other online board game communities
4. Get their opinion on a prototype of the online store

That is a lot to ask of a survey, but I'm going to the extreme for this example. I would hope that either alongside or before and after, we would conduct some qualitative user research to understand our potential users even better.

Before I sent out this survey, I would include a screener that asked if people were board game users—and if they had purchased board games online in the past. This would help ensure I get helpful answers from relevant participants.

I'll always ask about past behavior before asking about anything future-based. Gathering people's past experience is my biggest piece of advice for getting the most relevant and insightful information from users. Always ask about the past before the future, because past behavior predicts future behavior.

Questions for understanding behavior

How do you prefer to purchase board games?

• In-store
• Online
• Other
• Logic: Why do you prefer to purchase board games that way? (Open text)

With which stores have you purchased board games online?

• Multiple-choice with "other" option

How often do you purchase board games online?

• 1-2 times a month
• 3-4 times a month
• 5+ times a month

Why do you purchase board games online?

• Either: Multiple choice with answers from previous research and "other" option OR open text

What is your main goal when visiting an online board game shop?

• Multiple-choice with other

Is there anything you look for on an online board game store that is missing?

• Yes/No
• Logic, if yes: what are you looking for?

Have you ever returned a board game you purchased online?

• Yes/No
• Logic, if yes: Why did you return it? (open text)

Questions for gathering opinions

Think about a time when you purchased a board game online. How was your experience?

• Scale: Very good, good, neutral, bad, very bad
• Why (open text)

Think about a frustrating experience you had purchasing a board game online. What happened to make that experience frustrating?

• Open text

What is the most difficult part of buying a board game online?

• Either: Multiple choice with answers from previous research and "Other" option OR open text

What do you look for when going to an online board game store?

• Open text

What would make you purchase a board game online?

• Open text

How would you improve the experience of purchasing a board game online?

• Open text

What do you like about purchasing board games online?

• Either: Multiple choice with answers from previous research and "other" option OR open text

Questions for prototypes

What are your initial impressions of this website?

• Open text

What is the most important information on this page?

• Open text

What information is missing?

• Open text

What information is confusing to you on this page?

• Open text

What would you change to improve this page?

• Open text

Would you use this page?

• Yes/No
• Logic: Why or why not?

Here we have a mix of open-ended and closed questions that touch on behavior, opinion, and prototype. If I were offering an incentive, such as a gift card or discount to the store, I would ask users all of these questions. If I couldn't compensate, I'd try to pick and choose 7-10 questions from this bank.

Once you feel comfortable with wording, you can use different question types, such as matrix or forced rankings. I stuck with multiple-choice, scale, and open text because they're the easiest to start with. I recommend exploring other options as well.

Now that you have your questions...

Here comes the much more fun part of the entire process: sending out the survey. If you don't already have a tool for surveys, here are three of my go-to's:

Typeform

• Beautiful UI
• Easy to use
• Mobile version
• Logic flows
• Can be expensive for smaller teams

Google Forms

• Free
• Various question types
• Logic flows (although difficult at times)
• Not as intuitive
• Not as beautiful

HotJar

• Easy to use
• Good UX/UI
• Logic flows
• Can be expensive

Then it is finally time to send your survey out to potential respondents! If you're posting using a tool, then I try to use a catchy title. It helps a lot if you can offer compensation for taking the survey. If so, I always include that in the title, such as: "Help us improve for a 15EUR gift card."

If you aren't able to offer any money (and that is okay), you can try titles like: "X Company needs your help!" or "Help us improve for you!" Once you get them with a title, always make sure to include a small description of the survey topic, as well as an approximate amount of time it will take. Include an email address for any comments or questions.

If you're sending the survey in an email, you have some more room for explanation. I use the following template with a subject line similar to what I mentioned above:

Hey [name], (if you don't know the person's name, you can say: "Hi there!")

We're working on [the topic and goal of research]. It would be great if you could complete a quick survey—it will take you [average time] minutes. In exchange for your time, we will offer you [compensation]. We will use your feedback to improve our product and make it more useful for you. Thanks for your help!

Best,

[your name and role in a company]

Voila. You have sent out your survey! What do you do now? Wait for the responses to roll in. If they aren't, make sure nothing is broken and that you asked the right people for the questions you wanted to be answered. Also, check to see if the survey is too long, especially if you aren't offering an incentive. 

Brownie points if you respond to each user who took the time with a thank you and explain how their participation helped you.

Interested in seeing a few examples of standout surveys?

• See how the team at Relish Works conducted a large-scale survey and series of moderated interviews to connect with their audience across the restaurant industry and share key findings across their organization
  
• The Lenovo team ran a week-long survey to learn more about the technology experiences and needs of their D/deaf and Hard of Hearing (DHH) users. Explore their approach and see the impact their findings had on the organization.
• The team at Salesforce Ignite surveyed 900 people to find the value proposition behind a new product. See how they were able to find key opportunities to delight their customers and have the data to back up their findings.