Here’s How to Make your Study Design Remote-Mobile Ready

Over the past five years at dscout, I've supported hundreds of research projects—from 48-hour prototype feedback sprints to multi-month homebuyer journeys.

And time and time again I’ve seen the same thing stall processes, derail data-collection, and hold up projects amidst ever-shrinking deadlines. Teams come to the table with research briefs, plans, and documents that aren’t ready for the platform they’re planning to actually do research on.

And this is understandable. Sometimes, we’re concerned about the ballooning number of stakeholders involved in a project—stakeholders who best understand traditional research briefs and are looking for assurance that their questions will be answered. Sometimes, we’re just not sure exactly how to translate our goals and processes to a remote-mobile setting.

In any case, remapping our research blueprints will help us collect the data we need on the efficient timeline remote research facilitates. We’ll go through the most common “remote study design flaws” I see—and tactics you can take instead to ensure your next project is executed smoothly.

A note: we definitely think of remote-qual through a “dscout” lens—but many of these tips can help improve your remote project implementation on any platform. Some dscout specific vocab to be familiar with, in case you’re not a user: we call our research participants “scouts” and our studies “missions.”

Three common study designs that are incompatible with remote mobile research

Traditional surveys

Sometimes, we just have a ton of questions we want answered. We see the remote research as a way to supercharge our reach, boost our sample size, and heighten our resultant conclusions.

Believe it or not, these are the very characteristics that make surveys difficult to fit into remote, mobile platforms. Think of the participants' thumbs! Surveys are often brought to us at dscout because they’re inherited from another researcher, team, or stakeholder set; it may even be an old survey that produced some "OK" results in the past and is being recycled to save time.

Discussion guides

In a traditional research environment, a good in-person discussion guide can be invaluable. But when we go about the (usually difficult) process of designing one, we generally do so assuming we’ll be in the same room as an interviewee.

We lay things out believing that we can branch into different ideas and explore different lines of questioning based upon a participant's response. As a result, this doesn't naturally fit with remote, unmoderated, mobile research platforms like dscout Diary (although we do have an in-house moderated interview tool called Live that facilitates conversations a discussion guide would benefit).

(Almost all) video prompts

It’s difficult to find the sweet spot between “too much data to analyze” and “not enough data to be confident in your results.” When researchers say they want as MUCH data as they can get, 9 times out of 10, they’re not being realistic. Who can sit through 15 hours of participant videos? I always pause when customers tell me "We just want a lot of videos!"—because there's so much more context awaiting with a few other question types.

Converting your research plan for smooth remote-mobile implementation

Whatever your research document, there are three strategies worth considering before leveraging a remote research platform generally, and a mobile qualitative one like dscout, specifically: 1) Chunk it up 2) Determine your trigger and 3) Start with analysis.

Let's cover each in turn.

Chunk it up

dscout's Diary tool affords discrete research activities called Parts. Each part has its own set of questions, and participants can be required to complete a “part” as many times as you’d like (yup, just like a diary).

The most successful projects I see in dscout use these parts to their advantage. A common framework includes an intro, outro, and a part in the middle participants complete multiple times.

It's this second part that harnesses the magic of remote qualitative inquiry; it’s where you'll take advantage of experience sampling methodology. Instead of mashing each and every question you have in your research document into a single, ever-scrolling part—why not chunk it up? This will provide you a lot of flexibility. For example, you can design a “multi-moment part” to collect:

• Steps in a process that move along a broad journey (show us all the important things you do to get your kids ready to go back to school...)

• Usage moments with your app, product, or experience (show me every time you log into your banking website...)

• Daily check-ins offering a window into your customers' lives (log and reflect on ____ every morning…)

• Slices of life when a particular behavior or action is taking place (complete an entry every time you go out for lunch…)

Paired with an intro part (where you ask a few questions to get to know your sample) and an outro part (where you ask your sample to reflect, co-create, or ideate on an concept), you've got an impactful and digestible study design that will answer your stakeholders' bigger questions without all of their smaller ones explicitly included.

Determine your trigger

The "trigger" lets participants know when to submit a moment in your mobile qual study. The trigger also sets the altitude and/or focus of the study. Types of triggers include behavioral (e.g., anytime you want to snack), emotional (e.g., moments when you're feeling productive), activity (e.g., when you use our app), and others.

You can't (shouldn't!) ask everything in any remote study, so choose your trigger wisely. This allows you to fast forward through the “not-very indicative” stuff and focus on the data that’s most meaningful and impactful.

Another way to think about triggers is the phrase “show me...". At dscout, we often phrase our triggers using "show me" language because it grounds participants in an actual, in-the-wild moment, and helps fight recall bias. Partnering a "show me..." sentence with a trigger makes for clear instructions for your participants, which in turn produces on-target data you can more easily (and quickly) make sense of.

Here are some specific triggers we recommend frequently:

• If you’re most interested in moments when a participant is using a product or service... use an "action" trigger. (i.e., Every time you use a service, open dscout on your phone and show us that experience).

• If you’re most interested in the emotions that drive a participant’s behavior...use an "emotive" trigger. (i.e., Every time you feel particularly happy or frustrated with a service, open dscout on your phone and show us that experience).

Start with the end

Making sense of your data is the most interesting and difficult part of research. Set yourself up for success by thinking about your analysis and synthesis timelines as you design. Remote qualitative data is thick and multifaceted, containing different input types (e.g., photo, video, open and closed-ended questions). Here are some things to consider when designing that will save your future-self time:

• Consider sample size: It is qual, after all, so strive for samples between 15-25, depending on your goal. You can still maintain sampling rigor through balancing and segment selection, it'll just be on a smaller scale. Leave the law of large numbers to your data science counterparts.

• Leverage closed-ended questions: Speaking of quant, strive to include closed-ended questions where possible. For moments where you and your team know the universe of potential selections (e.g., part of the app being used, strength of felt emotion, location) using a closed-ended question can act as a participant-generated tag or code, allowing you to filter for it later.

• Use video wisely: The video question should be saved for the question that's most important to you and your team; the rich description and ability to see with your participants cannot be overstated. As such, use your video prompt to ask the deepest, most complex question. If you’re using dscout, our auto-transcription will make it easier to search for critical words or phrases and your participants' energy and imagination will be put to best use through video.

• Limit open-ended questions: Although it can be tempting to program 10 open-ends asking about every aspect of a moment, be judicious. Remember that participants are typing on a tiny screen with their thumbs! How much depth and rigor will they offer after the fourth open-end? Combined with your video prompt, which should afford all manner of depth, home in on two to three open-ends that participants can quickly answer. Strive for two to three sentences each. This will save their thumbs and your eyes come analysis time.

Summing it up

Conducting qualitative user research remotely offers flexibility and time-saving benefits when you can't be in-field. And because you're not in-field, dscout and other platforms require some tweaks to your research design.

Most traditional qualitative research is augmented and benefited by a remote component, but it takes a shift in approach:

• Time and attention become paramount: How will you capture attention with your instructions and value time with your questions?

• You can't ask everything: Start with your outputs, analysis, and stakeholders. What will resonate most while capturing the "Whys?"

• You’ll want to utilize a multitude of inputs: Mixed methods doesn't mean all survey questions. Leverage the bevy of question types to paint the full picture.

Remote research platforms like dscout, if used strategically, can serve as an additional set of hands on your team. Keeping in mind these strategies will get you rich but digestible data that you can use to make a real impact.