During a recent webinar with Jen Romano-Bergstrom, we walked through a mobile app research project focused on user experience. With a mobile-first methodology and platform, dscout is perfect for understanding mobile experiences of your users. Participants are able to share their screens and add context to interactions with your app, while the diary format makes it easy to collect many entries for rich data.

In the Q&A portion of the webinar, we fielded several questions from attendees about the different types of app research dscout may be useful for, as well as inquiries about how to design projects that capture users’ perceptions and actual experiences.

What follow are a few client-tested app-focused research designs, each examining a different aspect of researching the app experience. Within each example, you'll also find a quick-start template, offering a way to leverage these approaches right away (and if you need a hand, a dscout research advisor could help you further personalize it to your specific research needs).

These five examples should provide out-of-the-box frameworks for running studies of your own that get you closer to the experiences of actual users. It'll be helpful to understand the types of questions you can ask in unmoderated research with dscout as you consider the research approaches below.

Appy researching!

App performance tests/bug hunts

Having trouble matching bug reports with in-the-moment examples? Ask scouts to document any time they perceive your app acting not as it should, capturing the context as they do.

Quick-start template:

Part 1 | Intro + Baseline: Get a sense for how your app has performed in the past, and investigate how scouts have engaged with it to date. Have scouts answer just once.

• Which device are you using?
• Which version of the app are you using?
• How often do you update apps, generally? [Provide ranges]
• Of this list, which have you experienced? [Closed-ended list of app problems]
• Which is the MOST debilitating to your app experience? [Same list as above]
• Which areas of the app do you experience issues most?
• How have you remedied these issues in the past? [Closed-ended list or open-end]

Part 2 | Bug Diary: Ask scouts to track, capture, and explain bugs or performance issues with your app as they happen, in-context. Have scouts repeatedly answer these questions.

• Give this bug a title.

• What part of the app are you using?
• From 0 to 10, how debilitating is this bug to using the app?
• Describe what you're feeling in this moment in ONE WORD.
• How frequently would you estimate this occurring?
• Which of the following best describes the bug? [Closed-ended list of performance markers]
• In a photo/video entry, please SHOW us the bug. [Screenshot or screen recording]
• What action was taken to address the issue?
• How is your device connected? [Closed-ended list]

Part 3 | Recap: After tracking a set number of performance and bug entries, ask scouts to reflect on the worst offenders and anything else you or your team would like to know. These questions are typically asked only once (like a follow-up survey).

• Of the bugs and performance issues you tracked, which seemed the worst to you?

• Rate its negative experience overall, from 0 to 10.
• Of the following list, which can you live with? [Closed-ended list of bugs]
• In a video, describe one “wish” for our app. This can be a fix or a new feature.

First-time user journeys

What's it like to search for, download, and ultimately use your app for the first time? Recruit scouts to capture every step of the journey. Are you overlooking something in your onboarding flow?

Quick-start template:

Part 1 | Search and Download: Use this part as a baseline, investigating pre-usage attitudes and exploring how easy it is to find and install your app. (If it’s not yet public, you could send scouts a beta version of the app via TestFlight.) These questions are best answered just once.

• In one sentence, what do you expect this app to do?

• What information is missing from the description of our app?
• In a video, describe your first impressions—what's noticeable and why? (Or, ask scouts to use a screen recording to capture themselves navigating to your app in a store, narrating thoughts along the way.)
• Describe your perception of the app in three adjectives.
• Do you use another app to do the same job? If so, which one and how do you like it?

Part 2 | Use Journey: Ask scouts to submit each step in the onboarding, setup, and usage process. For each moment, they'll answer questions about ease, and show you how it went.

• Either give scouts a series of steps to show you, OR have them self-generate the steps to begin using their app. In this first question, they'd either select the step to show OR title the step they're creating.

• In a screen recording, show us this step and narrate what it's like, pointing out any wins or wishes along the way.
• Rate this step's EASE, from 0 (not easy at all) to 10 (extremely easy).
• Rate this step's COMPLEXITY from 0 (not complex at all) to 10 (extremely complex).
• Describe how this step could have been easier.
• How important do you believe this step is for users? [closed-ended response scale]

Part 3 | Reflection: Use this part to have scouts explain what they wish were better or describe whether or not they'll continue using your app, and for what reasons.

• In a selfie video, describe whether or not you'll continue using the app beyond the study, and why.

• What is one way the app will improve your everyday life?
• What is one way the app could hinder your everyday life?
• (If exploring paid options) What would you pay for an app like this? [closed-ended ranges]
• Describe the ideal user for this app.

Jobs to be Done app research

The Jobs framework is tailor-made for app research. Curious what role your app plays in users' lives? Ask scouts to capture moments using your app and describe what prompted them, what job it's performing, and the app’s perceived effectiveness at solving that job. If you're hoping to fulfill a certain NEED of users, have them document who or what they hire for that job and why. Moments-based research is perfect for Jobs approaches and is easy to spin up.

Quick-start template (feel free to use both or just one part, depending on your needs):

Part 1 | App's Jobs: This is a quick way to generate Jobs statements about your app. Learn how, when, and why your app is being hired. Scouts will answer these questions multiple times, for each moment they use it.

• (Depending on score) How could it be better/Why is it so effective?
• What job is the app fulfilling in this moment?
• How frequently does this job come up? [Closed-ended]
• How effectively is the app doing in this job? [Closed-ended]
• In a video [or screen recording], show us this job being done and explain how it's going.
• What alternatives do you have when doing this kind of job?

Part 2 | App for Hire: Here, you can focus scouts' on a specific job and surface the apps they hire for that job. Scouts will answer these questions multiple times, as before.

• What app are you hiring for (insert job)?

• How is this app performing this job? [Closed-ended]
• How often do you hire this app for that job. [Closed-ended]
• In a screen recording, show us how you use this app to accomplish this job, narrating along the way.
• Name and describe one way this app could perform this job more effectively.

Feature feedback

Do users "get" a new feature you're rolling out? Are there applications or use cases the design team hasn't thought of? Put it to the test: Arm scouts with the feature and have them use it in-field!

Quick-start template:

Part 1 | Baseline: Here, you might want to know how users currently do what your new feature does, or better understand what kind of user they are. Create a profile-style survey part to begin.

• How often do you (insert feature goal) in general? [Closed-ended]

• In a video, describe the last time you (insert feature goal) and what app you used to do that. How effective and easy was that experience, and what could have been better?
• If you were able to (insert feature) in (app name), how often do you think you'd use it? [Closed-ended]
• From 0 (not at all) to 10 (extremely), how useful would you find a feature like that?

Part 2 | Feature Test: Deploy or point scouts to the feature of interest, and ask scouts to use it naturally. You could always require a certain number of moments per day, too. Scouts should capture multiple moments to showcase the breadth of use cases and build reliability.

• Give this moment a title.

• Where are you? [Closed-ended location responses list]
• In a video, record your screen and show us why you chose to use this feature in this moment. Make sure to narrate along the way so we know what's going on!
• How much of a fit is this feature to this moment and your goal? [Picklist: Not at all to very much]
• Describe how this feature could have been better in this moment?
• Did/would you consider using another app in this moment? Which and why?

Part 3 | Verdict: Will scouts adopt this feature? Ask them in a survey-style part to close things out. Again, things bubble to the surface following a moments part, so feel free to get creative with your questions!

• In a selfie video, describe if you'll keep using this feature.

• In a few sentences, describe a new way to use this feature.
• Rate this feature's usefulness/ease/fun/etc. from 0 (not at all) to 10 (extremely).
• What is ONE way we could design this feature to make it more useful for you.

Hits, misses, & wishes

For foundational research, dive into how dedicated users explore your app with a "hits, misses, and wishes" framework. You'll ask scouts to show you a certain number of moments (or keep it wide open and let them decide), giving you and your product teams a quick way to filter to the right moments. What irritations do they deal with that don't rise to the level of a "bug." What is your app's magic power?

Quick-start template:

Part 1 | Hits, Misses, & Wishes: This is our time-tested app experience framework, which focuses scouts' entries on three categories. Scouts can answer the same questions for each, or leverage skip logic to show them different question streams depending on the moment.

• Is this a hit, miss, or wish moment? [Closed-ended]

• Rate the intensity of this (hit, miss, or wish), from 0 (low) to 10 (high).
• How frequently would you say you experience this? [Closed-ended]
• Where are you? [Closed-ended location responses list]
• In a screen recording, show us your hit, miss, or wish, and narrate along the way. What made this moment a hit, miss, or wish? What should we know?
• What part of our app best describes this hit, miss, or wish? [List of app parts]
• In a few sentences, describe the impact this hit, miss, or wish has/ will have on your experience with our app.

Part 2 | Reflection: After completing the usage diary, scouts usually have deeper insights you can surface. Dig deeper with a reflection part, focusing on overall perceptions or concept testing a new idea.

• Overall, how would you rate our app on the following scale. [hate it — love it]

• To what extent could our app be an integral part of your everyday life [not at all — extremely]?
• Sum up our app for us: In a video, describe our app's “superpower.” Why did you choose that superpower and how does it work?
• If you could change one thing about our app, what would it be and why?

From these approaches, you can see how powerful remote qual research can be for understanding app experience. Now, how do you mine insights from these studies? It can be easier than you might think. Watch our webinar with Jen Romano-Bergstrom to see her demonstrate her top tips for effective analysis in dscout.

Ready to start building missions in dscout? If you’re a new user, register for a free researcher account. Current dscout users interested in trying one of these study types should talk to their research advisors!

‍

Ben has a doctorate in communication studies from Arizona State University, studying “nonverbal courtship signals”, a.k.a. flirting. No, he doesn’t have dating advice for you.