Hybrid Research: The "Just Right" Mix of Generative and Evaluative Insights

What is hybrid research?

Hybrid research is a combination of generative research and evaluative research. It helps you simultaneously understand your users, as well as how a product/service is performing.

Now, this is not magic and, since it covers both spaces at once, it does not go as deeply into a place of understanding or evaluation. Any time I see this mix of understanding a thought process, while evaluating a product, my mind goes to hybrid research.

When you have a mix of goals, trying to understand both behavior and attitude, hybrid research may be the best type of research to apply. It can be a creative way to achieve many goals and objectives within a research study, giving your teams great information on how to move forward best.

Hybrid research methodologies

If you decide on hybrid research, what comes next? The best thing you can do is look back at your research objectives.

Let's take an example to illustrate when hybrid research would be relevant, and how you could incorporate this type of research into your practice.

Problem statement: Understanding and redesigning the navigation of an e-commerce website that sells clothing online.

Research objectives:

• Understand users' mental models about e-commerce online clothing websites
• Discover how users are currently using the website and when they need access to particular pages/features
• Uncover the limitations and pain points users are facing in different situations with the website

For each of these objectives, you can assign a research methodology (or two) that would work best to understand what you are trying to learn. That way, I can understand what I may need from hybrid research and make the best decision on methodologies. Below is how I would work through the above objectives:

• Mental models: 1:1 interviews
• Discover current usage: Usability testing, contextual inquiry (also site analytics)
• Uncover limitations/pain points: Usability testing, contextual inquiry
• Potential improvements: Usability testing, card sorting

With this, I can see the different types of research methodologies I would conduct to answer these objectives. In an ideal scenario, I might run 1:1 interviews, usability tests, and card sorting sessions all separately. I would also use analytics to assess trends in current behavior. However, as mentioned, most times, we don't have the time or capacity to do all of the above.

In this circumstance, hybrid research would be a life-saver. Specific hybrid methodologies are perfect for these situations.

• Walk the store: This is my favorite hybrid method. And is a mix between 1:1 interviews, contextual inquiry, and lite usability testing. When I do a “walk the store,” it lasts about 90 minutes. I start the session with 30 minutes of generative research (understanding mental models) and then dive into the product for 60 minutes. During the 60 minutes, I have the participant take me through the last time they used the product. While I observe them using, I ask them how they are feeling, what problems they have encountered, and then might ask them to complete specific tasks we want to learn about more. The conversation evolves quite naturally, and it ends up giving a play-by-play of usage, as well as underlying thoughts.
• Card sorting (open): This is my (close) second favorite hybrid method. This method asks users to organize items into groups and assign categories to each group. You do this by giving participants cards, either with information or blank, for them to sort in a way that makes the most sense to them. It is especially great for understanding how users combine and classify information (mental models), such as for navigation (information architecture).
• Concept testing: With this method, you give participants a rough approximation of a product/service to understand if they would need it. A concept test can consist of sketches, paper prototypes, or low- or high-fidelity prototypes. This method helps you understand how users think about a product (mental model), and how they may use it (paint points, flow, or missing features).
• Participatory design: This is an enjoyable hybrid method where you give participants materials (ex: markers, cut-outs) to construct their ideal experience. With this method, participants express what matters to them most and why (mental models). It also gives you an idea of how users could imagine the usage of a product (flow, features, information architecture).
• Surveys (with open-ended questions): Although potentially controversial, I believe surveys can be a hybrid method. I use many open-ended questions in surveys to gauge an understanding of how people are thinking of particular ideas or concepts. You can use items like, "how would you improve the current experience" or "tell us about your last difficult experience" to achieve a combined understanding of mental models and evaluations of a product.

Concerning the research problem above, I would pick walk the store and card sorting as my two hybrid methodologies. By using these, I could understand users' mental models and how they are using the platform while reimagining an improved navigation and flow.

And this is why I love hybrid research methods. Although you may have a lot of different questions to answer for your team, you can find ways to combine methods creatively. By doing this, you can get the best information for your organization, without sacrificing time (and your capacity).

What else can you do?

If hybrid research is not your thing, and you have the luxury of time, another methodology to use in this scenario is the triangulation of data. Triangulation of data means using more than one method to collect data on the same topic. With triangulation, you can help to assure the validity of the research.

As I mentioned in the ideal scenario, I would run 1x1 interviews, usability tests, and card sorting sessions all separately. I would also use analytics to assess trends in current behavior. Additionally, I would compare the existing website to heuristics and complete a heuristic evaluation.

The 1:1 interviews would help me to determine how people believe an e-commerce site would work and understand their overarching goals in coming to the site. The usability tests and heuristics would greatly help in identifying pain points, barriers, and problems in the flow of the website. The card sorting would then help to reimagine improved navigation and flow for the site. And, finally, the analytics (ex: google analytics) would show me trends in behavioral data that either support or disprove insights from qualitative data.

Again, this whole process might take me a lot longer to conduct than a hybrid method, which is why having access to both is fundamental. We can then choose the best set of techniques depending on what the team needs to learn and the timeline associated with the project.

I believe hybrid methods are an area of user research that could positively evolve over the coming years. It combines expertise and creativity and can help user researchers reach goals more effectively and efficiently.