Comparing Concepts: Two Study Designs for More Nuanced Feedback

Transcript

Hi, I'm Ben from dscout. You know, designers are more empowered than ever because of trends like democratization and embedded team structures to conduct their own research. That's especially true with concept testing. Whether it's early feedback on a completely new idea or pressure testing, something in the wild, the best experiences anticipate user needs and account for the context that informs them. Very often, designers turn to unmoderated research. That is where participants complete any activities autonomously on their own.

This approach can however, lead to blind spots, especially if designers only ask about direct interactions with a screen. What might be happening before or after a user lands on a page, opens a modal or attempts any action? It is these moments that help designers capture the context that is key to delighting folks. So in this video, I'll outline two approaches to concept testing, both using a remote frame where participants complete activities using their smartphones or a desktop computer. These will help you capture contextual feedback to create more human centered designs and experiences, no matter your development stage or schedule. By the end of this video, you'll have two new ways to source human insight, improving your design practice more generally.

Let's start with the classic rapid concept test, where feedback might be time sensitive. Maybe you're working within an agile or sprint cycle, and you have only 24 to 48 hours. You may have executed research like this before. The opportunity here is to expand your aperture a bit, instead of only getting feedback on the concept, which I recognize is critical, ask yourself what other pieces of information might be useful to you and your stakeholders.

Using a mobile frame also, lets us play with location, which depending on your use case or industry could offer a more exclusive look into the lives of your users. Maybe you're in the mobility space, so you'd love to see them on their new commute schedule. Or maybe you work in retail. And so having them review a concept as they're in aisle is really helpful. For this example, I'm using a tool called dscout Express/ as the name suggests it's a fast way to capture feedback from a diverse group of people. For designers, specifically, the value add is in the kinds of questions that we can create and the speed at which the feedback is obtained. Let's take the example of comparing light and dark modes of an app. I want some feedback on which versions folks prefer and why? With Express, I can program closed, ended questions to recruit only regular users and then ask them a few starter questions to build context.

What do they currently like about the design on the app and why? How do they rank the design compared to other characteristics, like the feature set, the price, the ratings, et cetera? I can also ask about frequency? Ask them for a screenshot on arguably the most important real estate in the world, their home screen. All of this before I've even shared with them, the concept on which I want feedback. I give them a checkpoint, which is just a way of saying, "Hey, heads up, we're going to do something next." In this case, it's review a concept. Then I show them the concepts here.

Now, in this example, I'm embedding static concepts, but if you had something that was interactive, like a prototype that you wanted them to navigate to, you can drop a URL and do that as well. Here, I just have my dark and light modes of the app. They'll be viewing this on a mobile screen. So again, I can see how they would view it normally. So again, I'm getting a lot more context than simply having them look at a screen and tell me what thing they prefer. I ask them, which they do. Then I have them use a selfie video to tell me why. I might've even asked them to show me a screen recording of their phone. I might be able to add them to this concept even if it is static to have them walk me through what it looks like in their app environment.

I closed by asking if this concept would help them increase their usage and by how much? Then finally I close with any feedback that they have for the design. It's only 10 questions. You can see that if I were to get this sort of feedback from say a hundred people, I as a designer would feel really good. Not only about the decision that I'm making between, in this example two concepts, but you may have more, but there's also a whole host of feedback that I can share to contextualize, bolster, and add rigor to my design decision. Just think about how it bolsters your story, boosts your recommendation as a designer. You're doing research. You're asking questions that you would normally do in the lab, but you're doing quickly and at scale with a rapid feedback tool like dscout Express.

There are other times when the concept is still in flight or situations where deeper dive data might really help refine and sharpen the direction you want to take. In those instances, we might opt for a few more kinds of research activities with users, as opposed to just a short feedback survey. For something like this I'm going to use dscout's Diary Tool, which allows multiple research activities to be strung together in a single project. For our light and dark mode example, here's what I'm building, I'll start with a benchmarking activity, asking for moments of natural use. For each submission, users will take a screenshot, tell me what's happening and use a screen recording video to take me to the action. I'm learning a lot here about the, in the wild ways, these folks are using the app already, which helps me frame their feedback.

I can program all the same questions here as I did in the rapid test example. Only this time I'll get multiple moments from the same person which helps paint a fuller picture of an individual user's path. As an aside, any part of a diary mission can be on mobile or desktop, depending on your specific use case or design needs. After the benchmark, I'll have them do a bit of what I did in the rapid test that is show my concepts, ask for a winner, and have folks describe why it was their choice. I could have if it made sense, show them one concept at a time and ask for feedback on each instead of picking a single winner. This might help if you believe both concepts have a place in your experience, but at different ship days.

Finally, in part three, I'll wrap up with a co-creation exercise. This is a chance for me to empower my users, asking them how they'd improve or advance the design? I like to have folks take pen to paper and actually draw their ideas, and in a selfie video, give me a tour of their finished product. If you recruit a solid group of users, this activity can produce a wealth of novel ideas for your team even if you don't exactly use the designs themselves. Diary lets you message these folks, place them into groups, and offers all the same analysis tools found in Express to help you mind this empathy, rich, deeper concept feedback.

So there you have it, two ways that you can weave contextual feedback with your ongoing concept feedback practice. Seeing all of this helps, but as a designer you'd probably like to play around for yourself. Click the link below to be added to this project, to check out its capabilities. Thank you so much for watching. I'm Ben from dscout.