From Chaos to Clarity: How to Prioritize Your Qualitative Insights

Qualitative research can be incredibly fun and exciting. You’re able to walk into sessions and walk out with a new understanding of participants' lives.

It’s fascinating to understand what and how people think about different concepts. From travel to hospitality and social media to body image, I never get tired of talking to people about how various topics impact their lives.

When I first started conducting generative research, I was in a state of bliss. I couldn't believe that doing something so enjoyable was my job and something I got paid for. Yes, synthesis was complex, but I managed to get through it and even included debriefs with stakeholders in my process.

What's actually important?

After 10-20 interviews, I had mounds of data after synthesis. How was I supposed to sort through all of this information and present the “most important” information?

Plus, importance is subjective and not one participant told me what was most important to them when describing painful experiences or goals they needed to achieve.

I assigned the importance to insights and presented this to my team for a while, but it felt disingenuous. And, when asked seemingly simple questions like, "How do we know this is what we should focus on?" I struggled to answer. I also couldn't argue with that old, dreaded phrase of, " But we only spoke to fifteen people."

I set out to fix this recurring issue. I wanted my insights to be valid and reliable, and I wanted to be confident when presenting to my teams.

Generalizing qualitative insights

I knew that what I wanted to do was generalize my qualitative insights across a larger population. Yes, I was speaking to the right sample size of people for my method, but I wanted to be sure that these insights could be valid and reliable across the entire segment.

With that as my goal, I knew I had to dip into my previous training in quantitative methods. Surveys would allow me to reach a more extensive population and understand if the broader audience aligned with my insights.

This approach would give me that level of confidence when I was presenting to colleagues. I could assuredly say what we needed to focus on next and give my teams the direction they asked for.

After some research, I decided to try two different methods of surveys: a feature request survey and the opportunity gap survey.

✔ Feature request surveys

The feature request survey was my first iteration on applying my insights to a wider population. I still use this approach today, however, less than the opportunity gap survey. I find it efficient and effective, especially when I have to get a prioritized list of features to my teams quickly.

This survey is most helpful when your team asks the question, "What should we do next?" or "What features do users need the most?"

When teams come to me with these questions, I conduct a quick generative research study if I don't already have enough research to put together a list. In this study, I aim to learn about the goals users are trying to achieve on our platform and the main pain points they are encountering.

Let's dive into an example about a weather forecasting app (because, hey, maybe we can leave our houses a bit more now).

Say my team approached me asking what they should build next, based on the user's needs. In this study, I would focus on understanding:

1. What people are trying to do and learn on our app
2. What pain points people are encountering on our app

After conducting the research and synthesizing, I have clusters of information. The larger the groups meaning more participants have felt/thought that particular sentiment, are considered the top themes.

Let's say the major themes are:

1. Goal: People want to understand what the weather will be like during hours they’ll be outside
2. Pain point: It’s annoying not being able to see yesterday's temperature as a comparison for how it will feel today
3. Pain point: No notifications when the app predicts significant weather changes can be irritating because people don't feel as prepared for the weather
4. Goal: People want to understand how they should dress on a given day for the weather and bring "just-in-case" clothing or accessories, such as an umbrella or rain boots
5. Pain point: Sometimes, when people are rushing, they forget to check the weather, and not having an accessible overview can be frustrating when they are running late
6. Goal: People want to understand what the road conditions will be like with regards to the weather
7. Pain points: People do not understand what a 50% chance of rain means and what they should prepare for. There was a lack of context that one might get from watching or listening to a professional weather forecaster.

Armed with this knowledge, I build a feature request survey by turning these into ideas. First, I use the user story format to convert them and then take all the extra wording out for the survey.

These would look like:

1. Goal: As a user, I want to know the weather for the exact hours I will be outside so that I can plan accordingly for what I need to wear/bring
2. Pain point: As a user, I want to be able to see yesterday's temperature so that I can better compare what today's weather will actually feel like
3. Pain point: As a user, I want to know when the weather will change significantly and get enough notice so that I can be adequately prepared for last-minute weather changes
4. Goal: As a user, I want to understand how I should dress or what I should bring with me based on the weather, so I feel more prepared
5. Pain point: As a user, I want to be able to quickly scan and check the weather for the day when I am in a rush, so I can know exactly what I need without thinking
6. Goal: As a user, I want to know what the road conditions will be like with regards to the weather, so I know if I need extra time to reach my destination
7. Pain points: As a user, I want to know what the different weather terms mean, like I am listening to a personal forecast so that I can prepare for the day ahead

After writing through these user stories, I would set up the feature request survey. In this survey, I ask users to rank the features they would like to see, and which are most important to them.

Sample feature request survey

I would send this survey to the appropriate sample size and use frequency to indicate what we should focus on next, resulting in a chart that could visualize the most highly ranked feature ideas.

Something to keep in mind with feature requests surveys is that they are technically future-based and asking people what they want. These two concepts can be tricky to capture through user research, which is why the opportunity gap survey may serve you better.

✔ Opportunity gap survey

The opportunity gap survey is part of the jobs-to-be-done framework. However, I’ve also used it for other generative research sessions.

This survey analyzes the gap between how important each insight is and how satisfied people are with current solutions. These results give us more context than the above feature request survey, but it does take longer. We can use this information to compute the opportunity score of each insight.

Let's take the example from above and turn that into an opportunity gap survey. First, you need to conduct research to figure out the insights, and then you ask each survey respondent two questions per insight:

1. Importance: How important is it for you to be able to [insight]? (1 to 5 scale, 1 = “Not at all important”, 5 = “Extremely important”)
2. Satisfaction: How satisfied are you with how you can currently [insight]? (1 to 5 scale, 1 = “Not at all satisfied”, 5 = “Extremely satisfied”)

Sample opportunity gap survey

Once you receive your responses, you can calculate the opportunity score or plot the answers on a graph like below:

The work you should prioritize are insights with a low level of satisfaction and a high level of importance. Afterwards you can focus on those with a high level of importance and a mid-level of satisfaction (ex: importance three or above and satisfaction at around three). You can use this type of graph to present to teams, showing them what to prioritize next.

This survey is modified from the original jobs-to-be-done model, which focuses on job steps and outcomes, but has helped me immensely when prioritizing insights for teams.

Overall, surveys help us along in our journey as user researchers, and these techniques have aided me throughout my career in giving teams the best information possible. Give these a try if you are struggling with how to prioritize during your next generative research study!

Interested in more articles like this? Check out…

• Foolproof Qualitative Analysis Tactics—For Whether You Have a Month or an Afternoon

• Qual or Quant? Research isn't either-or anymore
• Insights Activation Roundup: How to Make an Impact with Your Findings