Try These Tactics When Working With Unengaged Research Participants

As a user researcher, I have a few different nightmares.

The first time I ran a generative research session, I completely bombed it. I dialed in and started the call. My voice was so high-pitched and shaky, I'm surprised the glass in the room didn't shatter. Right from the start, the participant mentioned a specific feature I was not familiar with. It took me over five minutes to ask the same question before comprehending what she was explaining. The participant was frustrated. About 15 minutes into our scheduled 90-minute call, I gave up—nightmare one.

Nightmare two occurred way too recently in my career. About two and a half years ago, I was researching how people book a trip online. I had fought for this project and was so excited to begin. The first participant came in, and I went through my normal warm-up. Then I got into it and asked my first question, "So, I'd love it if you could walk me through the process of booking your last trip." The participant responded with, "Oh, I didn't book it. My partner did."

We had about 85 minutes left, and I had no idea what to talk about. I had made a mistake and forgot the most important recruitment criteria.

Now, finally, nightmare three tends to be a recurring nightmare that a lot of researchers face. This nightmare has to do with the unengaged or difficult participant.

What unengaged looks like

This participant signs up for research for some reason but doesn't typically have anything to say when asked for feedback. Unengaged participants tend to:

• Love one-worded answers
• Use the phrases "It's fine" or "It's good."
• Think internally and not share their thoughts
• Feel uncomfortable when asked certain questions
• Ask a lot of questions on if they are "doing it right."

These participants are tough to handle because they can cause nightmare one, where you completely freeze and have no idea what to say next. They will give you general information that you know won't positively impact your insights. It is really disheartening to sit with a participant for an hour and not have anything to show afterward.

However, there is a light at the end of the tunnel! After hundreds and hundreds of hours spent with participants, I have started to pull together some common unengaged responses and what to do with them. Since pulling together this list, I have seen a rise in engagement with my participants. I have gone from clenching my jaw and hoping for the best to try this list of strategies. Not only does this help me relax more during interviews, but it also helps the participant and my stakeholders!

How to respond to unengaged responses

I know it can be difficult to face someone who doesn't give you more than one- or three-worded sentences. Or someone who stares back at you with a blank face.

Truth be told, I have encountered quite a few of these participants. I used to want to yell, "Why did you sign up for a research session?!" Now, I take much more calm approaches.

One-word answers

It can quickly become frustrating when participants employ the one-worded answer response.

When you are faced with this type of participant, there are a few different ways to face this.

• Ask only open-ended questions. Instead of asking questions that can be answered with one word, ask engaging questions. Check out my article on open-ended questions for some other ideas. For example:
  • "Walk me through your thought process of what you think about this."
  • "Tell me about the last experience you used something similar?"
  • "Explain how you feel about this screen."
  • "Talk me through your step-by-step process."
• Start with a warm-up. Instead of jumping straight into asking your questions, use some warm-up questions beforehand. This gets the participant into the rhythm and mindset of having conversations with you versus an interview format. My favorite warm-up questions are:
  • "What are some of your favorite hobbies?"
  • "Have you tried anything new lately? Like a hobby, movie, TV show, type of food?"
  • "What do you like to do in your free time."
• Ensure you have a proper introduction. Your introduction should be the first thing you start with as soon as the participant hops on (of course, after asking them how they are). This introduction will help you build rapport with less engaged participants. In this introduction, I always:
  • Stress that this isn't a test
  • Remind there are no right/wrong answers
  • Tell them they can stop at any time if they feel uncomfortable

"It's fine" or "it's good."

Sure, we'd love it if there was only praise and no complaints when we show prototypes to participants. However, as a user researcher, your job is to collect actionable and constructive feedback. Gathering this type of feedback can be almost impossible if you are met with the wall of "it's fine" or "I have nothing really to say."

This is so tough because our job is to get actionable feedback, and there is nothing actionable about this! In this scenario, there are a few approaches you can take:

1. A proper introduction. Again, introductions are key before just diving right in. For this participant, you should emphasize:
   1. How important honest feedback is
   2. That they won't hurt anyone's feelings because no one on the call designed the interface (even if it is a lie)
2. Continue digging. Even if it feels obvious and boring, keep digging past what they are saying. For example:
   1. "What do you mean by fine/good?"
   2. "Explain what you would do on the page?"
   3. "Tell me one thing you would change."
3. Be honest. My job is to come back to the team with a balanced set of views, so if I had to press you for something you don't like about this, what would it be?"
4. Asking for a friend. Sometimes people don't feel comfortable giving criticism, so I often ask, "how would you improve this for a friend or family member?" This helps remove them from the situation of criticizing their end.

Thinking internally and not sharing thoughts out loud

One of the worst situations a researcher can get into is not having participants think aloud during a usability test. If the participant is silently scrolling, you are just sitting there and hoping they verbalize the thoughts you know are in their head.

I have been in this kind of usability test, and it is not helpful. There are a few approaches you can take during this situation:

1. Mention this in your introduction. Whenever I start a usability test, I mention how important it is for the participant to think aloud. I explain what thinking aloud means, give an example, and tell participants why it is important for them to share their internal thoughts.
2. Do a think-aloud exercise. I started this a few years ago and have never looked back. Before starting a usability test, I have the participant do a think-aloud exercise to understand what I am looking for. I ask them, "walk me through how many windows are in your house/apartment, starting with when you walk in your door." If they internally count and answer with "seven windows," I say, "it would be great if you could walk me through how you got to that answer and tell me what you see." This exercise helps prime them to share their thoughts.
3. Remind them. Have no qualms about reminding participants during the session! I use many words that continuously prompt them to open up and share, such as "explain what you are seeing.." or "talk me through what you are thinking..." If they seem to go silent, jump in and ask them, "hey, what are you thinking right now?"

Seeming uncomfortable

Sometimes you might get the vibe that your participant feels uncomfortable during the session. This uncomfortable feeling can lead to a lack of rapport, the participant not answering questions properly, and the participant shutting down.

1. Look out for body language. When participants feel uncomfortable, they can display certain types of body language. For example, people might lean away from the camera (or moderator), or they might look down quite a lot. Of course, these are not always indicators of uncomfortable feelings, but just keep them in mind.
2. Make sure you are sensitive to the topic. If you are talking to a vulnerable population about a sensitive topic, make sure you are properly empathizing with them. Even if you don't think the topic is particularly sensitive, it may be to the participant, so be sure to make not if someone is hesitating to answer. You can always ask the participant if they want to continue or move on to a different topic.
3. Too many stakeholders. Some people get stage fright, and this is no different in user research sessions! If 25 different screens are popping up filled with stakeholders during a research interview, it may feel unnerving for the participant. I always try to broadcast my screen so that participants can not see how many stakeholders are watching. I always tell them there are observers, but never how many.
4. You can stop or take a break at any time. In my introduction and during the interview, I remind the participant they can stop or take a break. This knowledge helps the participant feel more comfortable during the session and gives them an out if needed.

Other types of responses

Besides unengaged participants, there are a variety of difficult scenarios you can find yourself in as a user researcher.

Off topic/rambling participants.

It is exciting to find a person who wants to share a lot, but, oftentimes, ramblers can be off topic, which can lead to very unproductive conversations. Try to steer the conversation back to the relevant topic by using segues such as, "that is really interesting, and I want to circle back, but for the interest of time, could we focus on…" or, "I would love to talk about this more if we have time, but there are only X minutes left..." And doing this over and over again to try and get as many useful insights as possible.

The participant is yelling at you.

If the participant is yelling at you, try not to interrupt them since that generally doesn't help the situation. Try to divert the conversation by saying, "I understand what you are saying, and I want to get back to that later on, but I would like to focus on this…" Another option is actually trying to listen by going on the journey with the venter and using some quotes for insight. If the person is being verbally abusive, you have every right to end the session.

Participant states a subjective emotion.

Whenever the participant states a subjective emotion or concept, always dig into what that means. For example, when a participant says something is "confusing" or "helpful," ask them what exactly they mean. Getting participants to clarify subjective emotions is important because what "confusing" or "frustrated" or "easy" means to me is very different than what it means to you. You want to create a shared understanding with your participant.

Not all is lost when you face these participants. Instead of giving up on unengaged participants (or barely holding in your internal screaming), you can get them talking. My final recommendation is to think about research sessions from the participant's point-of-view. They may have never done anything like this before, and they may feel nervous. Consider what it is like to be on that side and empathize with them. Remember that not every research participant and session will be perfect, so sometimes it is okay to throw your hands up and move on.