6 Common Mistakes to Avoid During Qualitative Interviews

One might think that one-on-one interviews are an easy method to master. Ideally, you are having a conversation with a person who has used your product/service (or sometimes someone who hasn't). This concept can seem simple—as you’re just talking to another person.

However, there are a lot of rules and best practices to juggle while you have this conversation. You need to make sure not to:

• Ask yes/no questions
• Lead the participant
• Create an environment for social desirability bias
• Interrupt
• Be too obvious during usability tasks
• Use positive words
• Get distracted
• Forget any necessary questions

The above is only part of what you must keep in mind during interviews.

After many years of interviewing and assessing others, I have found sneaky ways to mess up interviews, even if you feel like a seasoned interviewer. Thankfully, we can put safeguards into place that help us avoid (and recover from) these mistakes.

Here are some of the most common issues I see from experience interviews and how to fix them:

1. Diving right in

Building rapport during sessions is complex, especially if you are remote. But we all know, making a participant feel comfortable tends to lead to a better interview and richer data.

I used to dive right into the topic after doing a quick introduction to myself. My first questions would be about that person's role or something related to the subject. I wouldn't give participants a chance to warm up and get into the mindset of opening up to me.

It can be challenging for people to feel relaxed during an interview session, so a warm-up at the beginning of the interview can help. Rather than diving into the subject and discussion, take a few minutes to get to know the person and get them familiar with the types of questions you will ask them and answers you expect.

For example, I like to start with:

• What are some new hobbies you've started recently?
• What do you like to do in your free time?
• Is there anything you've enjoyed watching recently on Netflix?

2. Interruptions

Whenever I assess interviews of students, I immediately notice and flag interruptions. As I mentioned, getting participants to speak freely is difficult, and interruptions can ruin the sense of rapport you have built.

Not interrupting can be particularly tricky when you are remote and dealing with a slow connection. Whenever you interrupt a participant, you are telling them that what they are saying is not as important as something you might have to say. Even if they don't seem to care, this can cause them to stop elaborating on their points.

The participant should always be talking more than the moderator. My rule of thumb is that the transcript should be eighty percent participant and twenty percent moderator, and for that twenty percent, it’s mainly questions.

To achieve this, I count up to three in my head every time a participant finishes speaking. These three seconds give them time to decide if they want to continue and ensures they have entirely made their point. Silence is your best friend!

3. Positive acknowledgments

After a participant makes a point, you generally acknowledge that you have been or are listening. These acknowledgments are critical for indicating active listening and make the participant feel heard. However, as much as possible, they should be neutral. I am so guilty of this and do it all the time.

For example:

Participant: I started looking for a mattress randomly one night. I think it is because my husband and I kept complaining that we were tired and not sleeping well. We couldn't figure out what it was. And then, suddenly, I was like, maybe I'll look for another mattress. Maybe that is it!

Me: Mmmhmm (YES!)

Participant: And so I went ahead on Amazon, which I didn't actually expect to work. But they had a bunch of mattresses. So I went onto Amazon because they have actual reviews from people instead of direct websites where they always seem to say every bed they are selling has five stars!

Me: Great! (NO!)

Whenever we are responding to participants, the language should be as neutral as possible. Instead of "yes," "good," "great," or "perfect," we can use words like "okay" and "mmhmm" to indicate our attention.

4. Sneaky leading questions

The sneaky leading question is similar to positive acknowledgments in the sense that we might not even notice we are using non-neutral language. A common mistake I see is using language that goes toward either the positive or negative.

For example, I often hear:

Participant: We ended up finding a few mattresses on Amazon, which was great, but then I realized I wanted to try them out. So I ended up looking up the direct seller of the beds to try them out, which was time-consuming.

Moderator: How difficult was that for you?

Framed this way, we are assuming that the participant found the task difficult. The only indicator they gave us was time-consuming. Instead of asking about a level of difficulty, we can neutralize the question by asking, "How was that experience for you?" or "How was it looking up the direct sellers?"

Another common sneaky leading question I see is along these lines:

Moderator: Let's switch over to talking more about budgeting when it comes to groceries and restaurants. How important is it for you to keep a budget for these things?

This type of question can invite a sense of social desirability bias, which causes participants to respond in a way that makes them look good to the moderator. As humans, we want others to see us in a good light and, even if we don't care about a budget, we might want to seem more fiscally responsible, causing us to answer untruthfully. Similar to before, keep your language as neutral as possible.

5. Awkward transitions

I get asked a lot about how to properly transition between topics during an interview, especially if the participant is rather long-winded. Sometimes trying to switch to a new topic or guide a participant back can result in awkward interruptions and abrupt questions. It can be challenging to navigate the person smoothly on to the next subject or back on track.

Here are a few transition phrases I use:

• "Thank you for sharing. If we have time later, let's come back to that."
• "Since we have a limited time, I'd like to get back to X first and then talk about Y."
• "Thank you for sharing that! Before we talk more about that, can we go back to X topic first?"
• "Thank you for sharing. I'd like to move on to the next topic on..."
• "The next set of questions will focus on..."
• "I'd like to transition to the next topic on..."

6. Trying to relate

When a person is talking, one of our first instincts is to try to relate to them and find common threads, to make us feel more connected. I have experienced this a lot myself, especially when participants are talking about something difficult.

In the past, I’ve used phrases like, "I can imagine..." or "I can't imagine..." or briefly brought up a similar experience I've had. It's okay to do this sometimes but best to keep it to a minimum.

I stay away from those phrases because they can end up belittling the participant's experience. Even if the intent is to empathize and connect, the participant may not take it in that way.

For example, I remember interviewing participants about travel, and the most recent experience a participant had was traveling to a close relative's funeral. I scrambled, not ready for such a response, and tried to relate to the participant, shutting the person down completely. Instead, I give a neutral acknowledgement or express empathy through an "I'm sorry you went through that."

Improve your interviews with self assessments

When I set about trying to improve my qualitative interviews, I didn't know where to start. I knew I used too many positive acknowledgments, but I didn't know the scale or what else I could focus on.

With this in mind, I created a tool to assess my interviews based on Steinar Kvale's criteria of a good interviewer. After each interview I conduct, I listen to the entire session and use this template to evaluate myself.

The template includes the ten principles I use to measure and assess my research interviews and touches upon the points above. Once I listen to my last three interviews, I pick two points to work on in the upcoming interviews.

For example, if I notice I am using positive acknowledgments and sneaky leading questions too often, I will improve those areas in future sessions. Once I feel more comfortable, I will move on to other areas of improvement.

Bonus points if you can get colleagues or your manager to evaluate your interviews as well! At one organization, I set up a bi-weekly meeting where all of the researchers listened to and assessed each other's interviews. It was a fantastic learning experience for everyone!