January 27, 2022
January 27, 2022
To many, starting at a new company is an intimidating experience. Oftentimes, when you start as an in-house researcher, there is a certain "lead-in" time to learn the product and get familiar with how a system works. But if you don't get this leisure or are a consultant, it can be terrifying to face a completely new product, especially if it is particularly complex.
About six years ago, I started at a company called Olapic, a user generated content platform that helped companies promote social media content from their customers.
Before joining, I had never heard of user generated content. I didn't know the role of social media managers existed, and I had no idea how they would use our platform. I had come up against a mental model and workflow I couldn't understand.
I would ask participants how they use a system and, although they would explain, I would get lost in the jargon and fail to understand their workflow. I then faulted myself for not knowing the product well enough to follow the conversation.
While that was part of the problem, I realized it wasn't about my perspective of the product or system. Because, even if I did fully understand it, I still didn't know how users were using it. I could only assume or draw from my own limited experience.
In my quest to better understand users' experiences (and realize that I am terrible at people explaining things to me without visual cues), I finally asked someone to share their screen. Although this might seem obvious to some (it certainly was after I asked), it was a game-changer for me and became a go-to method.
In a “Walk-the-Store” interview, the participant has complete control and we are walking through the product from their perspective.
I first started this when I was back at Olapic. In my role there I conducted many remote interviews, and at the time it wasn't as intuitive to have someone show me their screen.
The first few sessions I facilitated remotely were challenging and ineffective. I couldn't follow their train of thought, even if I had the platform open on my computer to follow along. I also felt so distracted trying to keep up with their words that my questions were off and irrelevant.
At one point, I finally asked the participant to share their screen to walk me through what they did on the system rather than tell me. Show don't tell echoed in my mind as I wondered why I didn't do this before.
As soon as the participant shared their screen, I knew I had hit gold. After that interview, I went away and redid my research plan and discussion guide. I then reviewed the recruitment emails, made sure participants consented to share their screen before the session, and sent them a quick tutorial video on sharing a Zoom screen.
I was lucky that my research question and goals aligned with the Walk-the-Store approach. This method gives you a specific outcome and insights, so it is essential to ensure that you expect this outcome before moving forward.
Walk-the-Store interviews are what I would call a hybrid research method. It includes the generative component of contextual inquiry, where you watch someone in their modus operandi.
You want to observe how they use a system "in the real world." Walk-the-Store allows you to watch the participant work more "naturally" compared to the scenarios and tasks of usability testing.
However, there is also an evaluative component to Walk-the-Store interviews. You are watching them complete their tasks and where they run into pain points. When you see this pain point, you dig in to understand it better, as you might in a usability test. You're evaluating how the product is working for the participant.
Whenever I have used Walk-the-Store interviews, I have specific hybrid goals and outcomes in mind. The most common goals for Walk-the-Store interviews are:
There is a combination of observing, learning, and evaluating in these goals, making it a hybrid approach.
Deliverables are crucial to think about when considering a method. Unfortunately, I mistakenly chose methods that led to information that I couldn't properly use. Ending a project without actionable outputs for teams is not a great feeling, so I make sure to reverse engineer the project when considering a new methodology.
Some Walk-the-Store deliverables include:
If you have these goals and deliverables in mind, Walk-the-Store might be an excellent approach for you to try.
Walk-the-Store interviews follow a similar process for other 1:1 interviews, but there are some key distinctions to call out. I'll use an example to highlight how I have conducted Walk-the-Store interviews in the past.
I am working with a hospital with an internal system where doctors write and collect patient notes and track rounds, patient check-ups, and medication. The hospital wants to know where they should improve their product and what new features they should build next.
I take this project from a researcher lens and create the following goals:
My expected deliverables for the project are:
Armed with this plan, I decided on using the Walk-the-Store method and began the following steps, including recruitment and research.
Since I am using Walk-the-Store, a hybrid method, I need to ensure proper saturation. While evaluative research can get away with a smaller sample size, this hybrid method requires more participants. I determine the following recruitment criteria:
I choose 20 doctors, ten in different segments, to understand how different types of doctors might respond to the system. In this project, I identified that the most negative feedback was coming from emergency department doctors, and the department using the system the most was neurology. With this in mind, I begin with these two segments.
Once I've recruited the participants, I start conducting research. I break my discussion guide down into the different areas of the system:
In each of these sections, I ask questions that will help me guide the above goals:
I start with general usage to get the conversation going, so they can dive into the last actions on the system, and then I probe through the other areas. With this type of research, participants will often organically mention the different areas of the system, so you might not ask your questions in such a linear way.
Show me the last time you opened the system; what did you do? Is this a typical daily experience?Explain your previous experience; how did it feel for you?Talk me through a negative experience you recently had with the system?
- What did you do to "fix" that experience?
- How would you have expected it to go?What are you trying to accomplish when you come on to the system?
Walk me through the last time you wrote patient notes. What did you do?What were you trying to accomplish when you were writing the patient notes?Describe a frustrating experience with patient notes.
Show me how you track rounds.What were you trying to achieve when you were tracking rounds?Describe a frustrating experience with tracking rounds.
Walk me through the last time you monitored patient medication. What did you do?How was this experience for you?What were you trying to accomplish when you were monitoring patient medication?Describe a frustrating experience with monitoring patient medication.
Within this discussion, I would follow up with the participant and let them show me while I interject with as few questions as possible. Then, when I finish conducting the research, I would synthesize my determined deliverables. If you feel a bit lost with the synthesis in general, check out this article.
Overall, Walk-the-Store interviews can bring you deep and rich insights that you can easily share with your colleagues. Seeing is truly believing, and this approach gives you an inside look into participants' minds.
Nikki Anderson-Stanier is the founder of User Research Academy and a qualitative researcher with 9 years in the field. She loves solving human problems and petting all the dogs.
To get even more UXR nuggets follow her on LinkedIn, or subscribe to her Substack.