Set the Table for These Multi-Course Concept Tests

Evaluative research—like sourcing feedback on in-flight or pre-production concepts—is a primary research goal for UXRs, designers, and product folks. Choosing between paths often requires MORE than simply a usability test or a few interviews...a confident decision would combine these approaches, maximizing the diversity of data.

And there are lots of questions surrounding a concept test:

• In what format should I present my concepts?
• How do I want participants to access it?
• Should I focus on breadth or depth?
• Do I want to prompt participants or do I want their free thoughts?

This recipe moves those questions from "either/or" to "yes/and." It's flexible, molds to your team's specific needs, and can be used with a variety of participants and timelines. Flexing between Diary and Express offers the choice between longer, deeper work, and quick-turn.

So whether you have two days or two weeks, this recipe will offer a breadth-and-depth design to answer both your generative (how might someone use this) and evaluative (which do they prefer) questions. The use of different dscout tools also offers a rinse-and-repeat approach if your design team comes back with a different direction; you won't need to recruit over and over.

Jump to a step:

• What you'll need
• Preparation
• Serving

What you'll need

This recipe works best with a few things in hand:

1. A visual or audible prototype
2. 3-5 Likert-style questions
3. A discussion guide
4. A general sense of recruitment criteria

Preparation

Step 1: Determine a design

There are a few key decisions you’ll want to make upfront to ensure a smooth research project and end results that will be relevant, useful, and impactful.

First, choose your course of action. Do you want participants to see your concept(s) and be prompted to answer specific reaction questions? Do you want to capture natural moments over a period of time, and then show your concept(s)? It's helpful to keep in mind the different strengths of each of dscout's three tools: Diary, Express, and Live.

Depending on your ideal course of action, here are some course combinations for inspiration:

In this recipe, we’ll outline a two-course meal—i.e. a two-mission concept test—starting with a Diary or Express mission followed by a Live session. Feel free to mix this up, fitting it to your own research needs! Remember: Diary and Express are unmoderated tools, while Live is moderated. Adjust your needs accordingly.

Second, decide the type and scale of data you’d like to collect. Are you hoping to create charts or graphs to show users’ reactions? Do you want a few supporting quotes? Knowing your ideal outcome will help you ask the right questions and prevent data overload.

Finally, consider whether you want current users, prospective users, the general public, or all of the above! To make your decisions, ask yourself what your end goal is: is the purpose of your project to discover what your users want? Is it to uncover what might lead to greater adoption? Is it to understand what the general population wants or doesn’t want? Once you have an idea, you’re ready for Step 2.

Step 2: Recruit

You can either bring your own participants or use the dscout platform to do your recruiting.

Skip this step if you’d like to use an Express mission (see the next step).

Step 3: Run your first mission

Depending on the depth of your research, you can use a Diary or Express mission to ask your closed-ended questions and (if you choose) show your concept. If you're on the fence about which tool to choose, here is a quick comparison and potential impacts on questions:

Express missions are best suited for fast research or getting gut reactions from participants without going into too much detail. They’re also more affordable, with incentives between $.50 to $5. If you have more than 1-3 concepts to share, we recommend using a Diary mission.

Use dscout Express to ask a few qualifying questions to narrow your pool down to the user-base you’d like to hear from. Then, show your concept and add scale- or ranking-style questions to collect reactions. Add a single open-ended question or 30-second video prompt to ask why they rated it that way.

Express sample outline

Diary missions are best suited for more in-depth research, giving participants more than one day to complete. Diary missions are also more suitable if you plan to ask multiple open-ended questions.

When using a Diary mission, you won’t need to ask qualifying questions; instead, jump straight into your question set.

Diary sample outline

Step 4: Review entries to select Live session participants

Whether you choose Diary or Express, you’ll have lots of entries to review! Use these entries to choose who you’d like to invite to your Live mission.

There are two main factors we recommend considering:

1. Expressiveness
2. Concept ratings

Select scouts who are expressive and detailed in their responses to ensure they’re engaged in the research, optimizing your chance for a great interview. Additionally, be sure to select a variety of scouts with differing ratings of the concepts to hear from all sides.

Step 5: Conduct Live interviews

If you’ve already shown your concept(s) to participants, you may use this time to gain further insight into their reactions. Otherwise, you may use a Live interview to show your concept(s) and get reactions recorded in real-time. Either way, be sure to have a discussion guide ready to go—you can even practice fielding your Live mission by using the test mode.

Serving

Sharing the insights is just as flexible as the recipe itself. You could simply share the "winner" based on participant scores, however there are "extra mile" options to consider that can all be created using features native to dscout:

• Create a clip playlist from video collected during interviews or missions
• Offer collaborators access to mission or session data to poke around themself
• Word clouds of sessions might be used to highlight pain points or delight moments