Remote Research: 7 Experts' Advice for Methods, Management, and Mechanics
Starting, transitioning to, or optimizing your remote research workflow? Take a few tips from UXR leaders at Wells Fargo, LinkedIn, Credit Karma and more.
In the wake of COVID-19, many of us at People Nerds have ramped up our remote UXR practice.
In light of that shift, we contacted some researchers we admire—adept at conducting remote or distal user research—and asked: What they wish they had known before starting their own remote research practices and What advice they’d give someone just starting (or restarting) now.
We breakdown their deft guidance below.
1. Test often, share often.
Amanda Andres, Associate VP—Business Initiatives Consultant @ Wells Fargo Advisors
1. Make sure you test your remote study design. Test it with a colleague. And test it again with someone else who is in your target population. There is nothing worse than having to throw out data and speculate what people would have done or said if something wasn’t broken or wacky with your study design.
2. If you are a member of a team or work group, it is imperative to establish remote-friendly UX/design research team/community “hygiene” rituals.
Some ideas to consider:
- Virtual weekly team Kanban to see what everyone has going
- Virtual monthly or quarterly team meeting to share in detail what everyone is working on and have time to mingle and talk afterward in breakoff calls/skype chats.
- Virtual bi-annual or annual RIPs (Research Insights Forums) to bricolage projects and topics and form new interpretations or research questions.
2. Play to your strengths and match your methods.
Amanda Stockwell, President @ Stockwell Strategy
You can make nearly any UX research method remote, though we all know that remote research has some constraints.
Still, remote research can be very powerful. For one, it allows you access to a wider breadth of people. International customers? No problem. People with mobility limitations? All good! With most remote research methods, you can access pretty much anyone with an internet connection. Which leads me to my next point…If you’re looking to do usability testing, you might actually get more realistic contextual feedback. WHAT? Blasphemy? No.
When you run remote usability sessions, you’ll have people use their own equipment, in their own setting, with their own connections. Things that look great on a brand new Mac in a usability lab can render pretty differently on an 8-year-old flip phone with rural-Maine level cellular connections. Or you’ll see real-time interruptions. Did their boss/kid/pet just come on in needing something? These particular days might have more distraction than usual.
The key, as with most research efforts, is to match the methods to your goals and practice pragmatic flexibility. Some methods lend themselves particularly well to remote work: usability tests, interviews, and of course, diary studies or experience samples. But that doesn’t mean you should force a square-shaped research objective through a round method or push a method beyond its limits.
One other note; if you’re going to go remote, you might want to consider what you’ll be missing and look for creative ways to fill gaps. Maybe you can run extra sessions with different sorts of users, try hybrid or overlapping methods, or re-run studies to compare results. There are no exact right or wrong answers, but it pays to be a little experimental, especially if you or your organization is new to remote research.
3. Keep a watchful eye on your remote research “mechanics.”
Anna Lee Anda, Lead UX Researcher @ Zendesk
Some advice for starting off and planning a project that is remote:
- Use a document tool that is useful in collaborating and documenting comments and changes made to your project. We find Google Docs is helpful with this.
- Share your project with the appropriate stakeholders. You can tag them in Google Docs or start a group channel in a tool like Slack.
- Schedule and execute a remote kickoff meeting to run through any final details or discuss and debate lingering questions.
And some tips for conducting the remote research:
- Send a reminder to your participants regarding software/ hardware requirements (e.g. station yourself in a quiet place, double-check for any sounds issues when using a tool like Zoom).
- In the reminder, we find it helpful to send a primer—things they need to think about, collect, or prepare. This primer could include a rough agenda of what the session will look like.
- During your session, you should remind the participants the session might be recorded and how your time together will be used. I find it helpful to remind participants that feedback and questions will be covered at the end of our session so we do not get away from the objective.
- When doing research from home, let your loved ones and housemates know you are about to have a call so they don’t barge in mid-session.
- If you use a chat tool like Slack, consider keeping communications open during your sessions. This will be important if your moderator forgets anything.
- Record the session, with permission of course, and ensure a team member is able to take notes in case there are technical issues.
- If you need to collaborate with the participant you can share your screen and hand over remote control- keeping your browser clear, hiding bookmarks/tabs, and closing anything else confidential. Or you can use Google Slides by sending a link to the participant and having that open on your own screen so you can see what they are doing. Later on, you can remove the participant as a collaborator so they don’t make changes to the document after the call.
4. The same rules apply—but how we follow them changes.
Tatiana Vlahovic, Staff UX Researcher @ Credit Karma
Similar to field or lab research, consider how to help your participants protect their privacy and prepare for their remote interview. For example, if you plan to ask participants to share their screens or surroundings, let them know in advance so they aren’t caught off guard and don’t show you something they aren’t comfortable with.
Prior to the interview, tell your participants if you’ll need them to do the interview over a specific device or to log into existing accounts on websites/apps. This helps your interview timing stay on track because your participants will be ready to go. For instance, if your participant doesn’t know that you’d like to observe their workflow on desktop, they might join the remote interview over mobile.
Lastly, video chat software and internet connections sometimes fail. Consider having a backup method to contact your participant (e.g., phone or another video chat system) and allocate buffer time for technical troubleshooting.
5. Managing participant (and stakeholder) expectations correctly is crucial.
Janel Faucher, Senior UX Researcher @ Udemy
While adapting your research practice to be fully remote certainly offers many advantages and opportunities to consider in your study design, it also introduces new challenges to think through. Here are a few tips I’ve found useful:
- From a recruiting perspective, remote research gives you more access and flexibility to broaden your pool and include international participants. Whether you’re doing remote moderated or unmoderated research, make sure to clearly state what language the session will be conducted in and set clear expectations in the screening/invite process. In addition, it’s important to be thoughtful about different time zones and to schedule the sessions at a reasonable time for the participants to increase sign up and show rates.
- If you’re looking to observe/collect participant data over time via a longitudinal study, lean towards over-reminding. People get busy and often forget the task they’re meant to do and when it is due. Put yourself in their shoes and consider how often you might need to be reminded to do something that doesn’t directly relate to your work or personal life. Hint: A LOT!There are some great tools that can automate this process including dscout, Ethnio, and Calendly. Having plenty of reminders and building rapport is equally important for remote moderated interviews. Depending on the recruit, you can build rapport via email, text, or over the phone.
- Remote methods are great because it is easier for stakeholders to participate in sessions since they’re often working from home and don’t need to account for commute time. However, be mindful of each participants’ comfort level and privacy they may need. For example, if you’re conducting a remote moderated interview with multiple stakeholders dialed in, participants’ may feel more nervous because they’re being watched, and that can impact the interview. Get a sense of how many stakeholders will be joining before the interview and plan accordingly. Also, at the beginning of each session, always introduce everyone on the call and state their purpose.
Lastly, always send research etiquette reminders to stakeholders joining the call — arrive to a session at least 5 minutes early to ensure a timely start, mute your microphone and turn off your camera unless actively participating in the session, and send any follow-up questions via Slack/email vs. jumping into the session unannounced.
6. Manage your methods and be ready for rigor.
Matthew Doty, Founder & Principal @ XD Go!
For diary studies (and other asynchronous methods)….
- Get ready to spend some serious time reviewing/analyzing responses! The first time I did the analysis for a diary study it took me about 3x longer than I originally estimated. Give yourself the space to dig in!!!
- Set clear analysis objectives and hold yourself accountable to them. The good news about diary studies is that you get a plethora of rich insight. The bad news about diary studies is that you get a plethora of rich insight! Without a clear set of analysis objectives, it’s VERY EASY for people nerds to get lost in all that richness.
- Bring people along for the analysis ride! Conduct analysis with a few trusted partners on the project who represent key perspectives. While this may seem counterintuitive to saving time in the short term, it will PAY OFF IN SPADES in the long run. Specifically…
- The extra people will perceive critical insights in the moment that would have otherwise been lost in translation.
- The others in the room can also help hold you accountable to analysis objectives.
- The extra hearts and minds will come in handy later in the project when you need advocates for the customer’s perspective.
For remote Interviews (and other real-time methods)…
- Include Observers! Just like you would invite key project stakeholders to observe interviews or focus groups from behind the glass at a facility, get those same people “in the room” with you during each remote session. Whether the observers are physically in the room with you or virtually dialed into the session you’re running, having observers in the moment will help you glean critical insights in the moment that would have otherwise been lost in translation as well as pivot tactics and improve from session to session
- For heaven’s sake, do a dry run! This may seem like a no-brainer but as we get more and more confident as researchers, this is something that can fall by the wayside (often in the name of efficiency). With remote research, we introduce a whole host of new things that could go wrong so do not skip this step.
For when you’re just getting started…
- Make sure you are pursuing remote research for the right reasons. One of my first questions to anyone looking to start a remote UXR practice would be “Why?”. Remote research carries with it many inherent benefits such as increased efficiency, reduced costs, increased reach. At the same time, it can be tempting for some to view remote research as a way to cut corners or otherwise eliminate critical aspects of gaining human-centered insight. The answers to “Why” can help reveal and resolve these types of misunderstandings. For example, because remote research is so convenient, it is easy for organizations to want to “put all of their eggs in that basket” and do all research remotely. By taking this approach, however, these organizations unwittingly cut themselves off from the benefits of live observation
- Select the right tools for the job. Remote research, depending on the methods used, requires certain capabilities (e.g. cameras, apps, screen sharing, strong/consistent internet connection, virtual collaboration tools, etc). Since no singular tool can do it all, and any tool that claims otherwise will disappoint on at least one of these fronts, you need to experiment with what combinations of tools work best for what you’re trying to achieve. My remote research suite looks like this:
- dscout: Foundational platform for diary studies and live interviews.
- Optimal Workshop: Organizational/perceptual exercises
- SurveyMonkey: Quantitative attitudinal feedback
- Mural: Collaborative post-it-like exercises for remote workshops
- Get ready for the rigor. Like I mentioned above, it can be tempting for some to view remote research as a way to cut corners or otherwise eliminate critical aspects of gaining human-centered insight. While it’s true that remote research eliminates certain complexities of live, in-person research, it still requires serious effort and rigor when it comes to analysis of observations and synthesis of those insights into clear, meaningful recommendations that inspires stakeholders to action. You still need to invest the time, energy and talent in these areas.
7. Recognize that empathy is more essential than ever.
Aryel Cianflone, UX Lead @ LinkedIn + Founder @Mixed Methods
- The most important variable to consider during remote research now is context. We’re all trying to navigate a very uncertain situation and as researchers, we need to be sensitive to that. Let participants know that they’re welcome to have kids join the session, reschedule, or bring up any new time constraints that may have arisen. We’re all in this together, so make sure your participants feel that.
- Be flexible, be clear. Participants are going to have changing circumstances, so be willing to shorten or break up sessions, reschedule, or otherwise adjust accordingly. With observers, you may be using new technology or processes to allow them to join in so just be clear about the etiquette and expectations.