Rolling Research: Your Key to Creating a Culture that Incorporates User Insights

The demand for research is on the rise across organizations: more teams are asking for it, more kinds of questions demand it, and fewer decisions are being made without it.

One way to respond to that demand is to create "always on" or "rolling" research programs. These involve research in more decisions, loop in more teammates, and contribute to a user-centric culture. People Nerds recently brought together operations and researchers experts from Facebook (Akilah Bledsoe) and Github (Zara Logue) to discuss the hows, whens, and whys of this burgeoning research approach.

This conversation is recapped from a deeper-dive webinar on scaling iterative research practices. Stream it on-demand here. 

Schedule, schedule, schedule

In order for any research practice to iterate and flow, it needs structure. How often will data be delivered to stakeholders? When (and how) can (and should) others pose questions for the program to tackle? How are needs prioritized? These and other questions related to structure, deadlines, and scheduling are imperative to spinning up any impactful (and therefore long-lasting) rolling research program.

"...it's research on a schedule. It's at a set cadence. You can plan for it. Researchers can plan their research around it and through it. Honestly, [rolling research] is something that's stable and folks in the org or the research team know it's always happening. They can always plug in to make sure everyone knows we've got users here we can ask questions to. They can make our products better." - Akilah

---

There's no "wrong" way to setup a structure, really; it's about settling on a cadence and socializing that to the wider organization. Create an intake form or a regular set of office hours when you and your team (if you have one) will accept or meet with stakeholders who might want research access. Set parameters for typical fieldwork, giving flexibility for the inevitable shift in focus. The goal is transparency and consistency across execution; change is fine (this is iterative research, after all), but stakeholders need to know the structure will be honored.

"Maybe [it's] with a single team that wants to be doing things on a regular basis, and therefore it's more iterative. That same team is getting insights about a particular topic over and over. Or maybe it's more of a program to be open to the entire organization and say, 'What do you have to put in front of customers? Let's do it.' I think we came to the conclusion that it's more about establishing a structure and trying to fit different pieces into it, and seeing which pieces going to work best for your particular situation." - Zara

With a structure in place, shareouts and delivery dates become habitual. Stakeholders who asked the questions bring their teammates, learn and then ask further questions, creating the iterative part of the rolling research program. If you've scheduled thoroughly, stakeholders will know when they can next ask questions (and how to do it). Schedules not only help triage questions going in—they also keep stakeholders accountable when its socialization time.

Roll up to strategy...over time

A pleasant by-product of an efficient rolling research program is democratization: getting other "non-researchers" involved in the process. By leveraging structure, templates could be created for oft-repeated questions (e.g., "I want to compare these concepts" or "I want to see this user journey") such that "researchers" spend less time programming and can serve as approvers or QC analysts, ensuring the work is "field-ready" and then pressing play. Stakeholders interested in being part of the program can shadow and then take more ownership.

(On weaving in a stakeholder) "I facilitated the first week. We talked about what came out of it, we did do a little "finding stock"  and then the next week I was going to be on vacation. His job during that first week was to watch me moderate a couple sessions. Then, he moderated his own, where I was giving him feedback via Slack as he was asking the questions. It's very much about: can we train people? Can we find people who are really adaptable to this type of role? We're not so much giving up our expertise, but we're offering a place where people really can take part in the questions." - Zara

Structure gets other teammates involved, as it provides guardrails for engagement and expectations for effort required to "get answers" or insights. Might your designers or marketing folks benefit from designing a survey, conducting an interview, or co-observing with you? Their worldview (organizationally) may make them well-suited to ask questions and create activities; they certainly have an idea of what they're hoping to answer. Rolling research programs' structure can empower others to get involved, which can leave more time for UXRs to think about strategy questions: those thornier exploratory and discovery ones that require the expertise of a human-centered thinker.

"The more people you have that you can lean on to minimize the work that you're doing around either the setup or the synthesis, the better. I think the danger in these programs is that you're doing these things really quickly and then no one is absorbing what actually comes out of them. For us it's been really important to have the product manager, maybe the engineering lead, and definitely the designer be involved in the sessions..." - Zara

Honesty is the only policy

To be clear, rolling research is not about taking on ever more questions from a broader set of stakeholders about an ever-evolving basket of problems...there is only so much a team can answer. If you have access to a research operations team or person, leverage them to help take care of the instrumental aspects: recruitment, incentives, privacy, and other pre-research to-dos. If you don't have an operations team, act as your own, and ask yourself, "What kind of time commitment do I have for a 'rolling' research program this quarter/month?" A rolling research program that stops rolling after a week because the researcher(s) is/are swamped doesn't set a strong precedent to engender future engagement and excitement.

"...it also comes down to how much work you need to do and how much headspace you have as a researcher—or how much you can lean on an embedded ops team if you have that, to help support it. Looking at your resources is the next thing. Do I need to go to a vendor to recruit participants, or do I have an embedded operations team that can help me do that? Looking at who you've got to help you get it off the ground helps you keep forking down the road." - Akilah

You may have to pause your rolling program to focus on other, more pressing matters, so be honest with yourself and your would-be stakeholders as to the time and energy you can commit to an "always-on" program. Smaller teams (or shops of one) should see point one and really structure the program in a way that fits with ongoing and existing organizational cadences (e.g., two-week product or engineering sprints; once-a-quarter exploratory work).

"As much as you can automate for yourself also, using tools, Calendly, appointment scheduler, plugins, that kind of stuff. They automatically send a Zoom link, they automatically send your NDA for you, those are tools that have been really helpful to me..." - Zara

Share, socialize, repeat

With all of this research "happening," the data (and ultimate insights) need to go somewhere for others to consume. As such, repositories become an important aspect of any rolling, iterative research endeavor: How best to socialize the work being done so that folks 1) Can better leverage the insights to make decisions and 2) Reduce rework? As with many aspects of establishing a rolling program, start small and stay consistent: Think about where and how stakeholders typically consume research today (if at all) and try to meet them at and in those channels. Intra-communications platforms (e.g., Slack or Teams) or Wikis are solid places to start. Socializing and making the publication a habit is key.

"...we're trying to experiment with different ways of letting people know what we're doing. I feel like it's still evolving. It's a little bit of Slack and a little bit of internal team posting and just telling people three times what you're doing so that they remember." - Zara

Structure and clear expectations (once) again become imperative: What, where, and how data will be shared needs to come from you and your team after cleareyed consideration to your week, month, quarter, and OKRs.

"Be really clear about the expectations for the output of this ahead of time. If it's just you, don't tell folks you're going to give them a big deck. Tell them you're going to get a 'key findings email' or your 'top 10 findings from the sessions.' Keep it easy on yourself. The best way to do that is set expectations about what you can do, what your timing is, and how you're going to proceed." - Akilah

Roll with the punches

The greatest threat to a successful rolling research program is perfection; that drive to make the interview guide just right, the prototype exact, or the survey flaw-free. This is time that could (should!) be spent weaving in stakeholders, launching research, and digging into results. Maybe a rolling research program is your time to get scrappy and deliver results from smaller samples, but more often. Research that no one reads isn't helping anyone, least of all the researcher whose time is spent working on it. Stretching yourself can make a rolling research program sing.

"This is definitely not something that's impossible to setup. It's definitely something that you can do; you want to take some time to sit back and think, 'How can I make this work for me and who could I pull along on this journey to make this something that's sustainable?' It's totally doable."- Akilah

"Put perfectionism aside and be willing to put something together and see how it goes. Be really clear about the fact that if this doesn't work out we'll try something else. We'll give this two tries and then if it's not valuable to the team, if it's not valuable to you, then maybe we don't continue doing it. Being really clear about expectations and being willing to be scrappy on the setup is key." - Zara

Rolling research can be a gateway to a whole host of outcomes for research and your team organizationally. From looping in more teammates to more, scrappier outputs, rolling research can create an ethos internally that questions should be asked—and answered—from a user-centered, data-driven perspective. From opening space for strategy questions to naturally creating democratization opportunities, rolling research and its iterative nature can hold several benefits.

We thank Akilah and Zara for some of their time. If you'd like to check the entire webinar and learn more, head here.