Sometimes you feel “between” discovery research and usability testing. Your team has a few concepts in mind, but they aren't fully formed ideas. They want to test to see if they are going in the right direction, but the concepts aren't usable.
Concept testing encourages you to put early-stage ideas in front of participants to determine which direction the team should go in or whether or not your team is going in the right direction. These tests will help you understand how people feel about your concepts/ideas.
Here are three sample dscout mission designs for concept testing in dscout—plus some upfront advice on when to use each one.
First: Decide on a method
Concept testing can work in a variety of ways. Choose the one that’s best for your bandwidth and for what your team needs.
Moderated concept testing using Live
This lets you get in-depth feedback and insights about one or two concepts. You can understand how people are interpreting and interacting with the concept, get detailed design feedback, and dig deep into emerging areas of interest. Running Live interviews are rewarding, but time- and resource-intensive.
Unmoderated testing using Diary
Gets you a bigger sample size than Live, and requires less resources. It also allows you to test a wider range of concepts without eating up too much of the scouts’ or researchers’ time. Diary is also good for comparing multiple concepts against each other with both qualitative and quantitative metrics.
Unmoderated testing using Express
If you are looking for quick, broad directional insights, Express will allow you to ask a brief series of questions about your concept(s) and get you feedback in record time, all with minimal effort on your part.
An Express mission is a great first stop along the way to a bigger study. Use a quick-hit survey to help crystallize which concepts warrant further examination, either with a Diary or a Live mission.
Next: Decide how you’re going to present your concepts
Another big decision worth making before you design your study: How will you communicate your concept(s) to the users? Our teams at dscout have done this in a variety of ways:
Moderated: share your screen
Perhaps the most straightforward way to share concepts. Use Live to interview scouts, and ask them to respond to
You also have the option to add stim to your Live missions, which will give scout some control, which may be useful if there’s a lot for them to explore, or if it’s an interactive wireframe or prototype that you want them to look through.
Stim in the unmoderated/Diary dscout tool allows you to upload still images to survey questions. This is a good option if your concept is relatively simple to get across, or if it can mostly be described in text with some visual aid. Program in a question with a text description of your concept and use stim to illustrate your idea.
An external website
Create a private micro-site with separate pages for each concept, including images, descriptions, use cases, or anything else you want scouts to respond to. You can include relevant links in your dscout study, or you can send Live participants a link ahead of time if you prefer for them to spend time reviewing the concept before your call.
Method 1: Moderated concept testing
Moderated concept testing is a good way to get detailed, in-depth feedback on your concepts. It can be time- and work-intensive, but will be worth it for the level of fidelity you will get on your samples.
Recruit and sample size
For in-depth concept feedback, you will likely want people who are already knowledgeable about your product (or about similar products) and who are going to be articulate.
Your sample size doesn’t need to be large for this study, and will be in part determined by how much bandwidth you have for interviewing. We suggest 5-10 people per key segment you want to look at.
Method 2: Unmoderated concept comparison
Unmoderated concept testing has the advantage of reaching a lot of scouts with less time and effort. Your feedback won’t be quite as detailed as it is in moderated testing, but will still give you a broad sense of what people are responding to in concepts.
Designing an unmoderated concept study also allows you more space to ask foundational questions relating to your concepts. This is optional, and your team may have already done this work. But survey-style testing will allow you to kill two birds with one stone.
Recruit and sample size
Who you recruit will depend on the segments you are trying to reach with your new concepts. Consider:
- Power customers who know a lot about your product
- Light customers (the concepts might get them to increase their usage)
- Prospective customers (the concepts might get them to switch to your product)
A mix may also be beneficial. Also consider any key segments your company wants to examine.
Sample size can vary depending on your bandwidth. The sample can be as small as a moderated study (~5-10 scouts per segment), but can also be much larger without much additional effort on your part. Making your study automatic will decrease on scout management needs, making this an even lighter lift.
A larger study (25+ scouts per segment) can have the added benefit of allowing some quantitative reporting (though these numbers will still be representative or definitive).
Karen is a researcher at dscout. She has a master’s degree in linguistics and loves learning about how people communicate with each other. Her specialty is in gender representation in children’s media, and she’ll talk your ear off about Disney Princesses if given half the chance.