How to Start a Rolling Research Program (An Easy 9-Step Process)

I spent months convincing some of my teams about the value of user research. It had been a grueling task, but there was a light at the end of the tunnel. Finally, my teams saw the value and importance of user research. We were talking to customers about real problems and then testing the solutions. After six months of hard work, I was finally starting to see the results.

Then suddenly, I had flurries of research requests coming in. I set up an intake process and learned to prioritize, but it didn't seem like enough. I wanted to do more for my teams and to continue setting them up for success.

I was scrolling through messages with my product managers and designers, and they kept asking me, "How can we squeeze in more user research?" It was, and still is, one of my favorite questions to hear my teams ask, but I was stumped. I went into our war room and sat, looking at the whiteboard. I sketched out how much research we were doing in the next two months.

And that is when I saw it. As a researcher, our work ebbs and flows. We have times when we are at full capacity, but then we have some downtime where we can support projects, but nothing is coming in. There were gaps in the schedule, really nicely timed gaps. The downtime perfectly aligned with the idea to start a rolling research program.

We talked about the ins-and-outs of rolling research with leaders Github and Facebook. See how they set up their programs in our recent webinar, which you can stream on demand.

What is a rolling research program?

A rolling research program is a lightweight way to ensure research is constantly recurring at your organization. In this program, you are conducting frequent, simple usability tests.

I set up rolling research programs in the following way:

• I allow two teams to test two different designs
• We speak to 7 participants per test,
• The tests are a total of 75 minutes each (30 minutes per test + intro/outro time)
• There is a mandatory debrief of 40 minutes (20 minutes for each team) after each session
• Each overall session takes no more than two weeks to complete

Generally speaking, these tests should be on prototypes with simple flows and with teams asking easy questions.

For example, I would use a slot at a rolling research program to assess the usability of a check-out flow or get quick feedback on early-stage concepts. I would not be using these sessions to dive into any generative research or test complex flows and designs.

There are many advantages to starting a rolling research program:

1. Rolling research time by taking the complexity from scheduling. Each rolling research session is set at the same cadence with the same time frame, so teams know what they need to do by a specific deadline.
2. By having teams on a regular schedule, they will constantly be putting work in front of users, supporting fast product development. This cadence works very well with agile teams working in sprints.
3. By having a consistent way to get feedback from users, you constantly reinforce a research-centric organization and mindset. Instead of them waiting for extensive, information-laden reports, you can get them quick input and insights.
4. Having small tests available once every month can make room for those small yet important projects that don't quite make the priority list. It can also help encourage designers and developers to make iterative changes to improve.
5. As a user researcher of one, starting this type of program can help you support multiple teams by enabling them to run multiple tests in one session.

When to start a rolling research program

Rolling research programs aren't for every team or company. Before you start one, make sure that your organization and team can support the work. Usually, rolling research is best for:

• When you have many simple requests that come in, but they are too complicated for unmoderated testing.
• You have little capacity to do a full-blown project but can help with quick insights.
• You have a team that understands how to work with insights and can take the brunt of the work themselves (see below for support levels).
• Teams/companies that have relatively easy access to customers (and can recruit within a week).
• Those who have the budget to spend to incentivize people for quick recruitment.

How to set up a rolling research program

Before diving into a step-by-step guide, there are a few ways you can support your rolling research program. I recommend thinking about the level of assistance you want to give to teams for this program. Here are the three levels of support I have used in various research programs:

1. Full support indicates that you are doing everything to run the research project, from script development to recruitment and analysis and (lean) report-writing. The team needs to be present in the interviews and analysis for this to be effective.
2. Medium support means that you are conducting around 50% of the interviews, reviewing the documents (such as the interview guide), and helping with 50% of the synthesis sessions. I would recommend, at medium support, encouraging the team to write the report and having you double-check it to make sure they interpreted the insights correctly.
3. Minimum support implies that you are doing very little active work. At this level, you can recruit, review the interview guides, but are not conducting any of the interviews or synthesis sessions. I always recommend taking a look at the report, however, to ensure it is sound. You should reserve this level of support only for teams with a high degree of experience in user research.

Please keep in mind that, despite the support level, each of these steps should be lean. Ideally, you can complete a project in 1-2 weeks. Also, you can customize each session depending on the team's experience—supporting different processes for different teams, or offering different levels of support to account for different research competencies.

Now, let's dive into a step-by-step guide of setting up your rolling research program:

1. Determine a standard time frame and cadence. Most of the rolling research programs I have created happen once a month. Pick whatever rhythm works best for your organization, and make sure to stick to it!
2. Decide on how long each overall session will be. Each program session should span 1-2 weeks to ensure you research in a lean and efficient manner. Once a project takes over two weeks, you are no longer getting the value of a quick research program.
3. Build a schedule. Once you determine how long you want each overall session to be, you can build a plan. I always recommend setting clear deadlines for the team so they (and you!) know when to complete everything. For example:
   1. Week one:
      1. Day one: Recruitment starts
      2. Day two: Initial prototype review
      3. Day three: Building the interview guide
      4. Day four: Prototype and interview guide review
      5. Day five: Final tweaks, recruitment ends
   2. Week two:
      1. Day one: Any minor tweaks to guide and dry run
      2. Day two and three: Testing, debrief sessions
      3. Day four and five: Synthesis and lean report writing
4. Set standard recruitment criteria. To get users quickly, you should have conventional recruitment criteria set for these tests. For example, trying to find precise details and customer behavior might take too long to recruit for. Keep the recruitment as simple as possible.
5. Decide on the number of users (+ backup). Six to seven users are relatively reasonable to complete in two days and will get you the feedback you need for straightforward tests. I recommend always scheduling a backup for the last minute, just in case someone drops out. During this step, it is also helpful to determine an incentive amount for participants and the backup.
6. Create templates. This step helps you and your teams streamline the tasks to get ready for a session. Create templates for anything you can, such as recruitment emails, interview guides, consent/privacy forms, and debrief and synthesis boards.
7. Make a checklist. List out all the things you need to do for each session and create a checklist you can easily copy. This checklist will help you and your team, especially as they learn to run these sessions independently (at low support). As you go through this process, you will start to form (and add to) your list. Some steps on mine include:
   1. Have an alignment session with both teams to remind them how this works
   2. Create a chat group for the teams for communication
   3. Create a sign-up sheet for moderators, note-takers, and observers
   4. Start recruitment
   5. Send calendar blockers and links for interviews and debriefs
   6. Prototype meeting
   7. Review interview guide
8. Figure out deadlines for each step. Once you have the schedule ready, set hard deadlines for when each person needs to finish each step. For example, by Thursday of the first week, the team needs to be done with the prototype to finalize the interview guide.
9. Educate your teams. When I created my first program, I thought each step was obvious. However, I didn't properly communicate the different steps and why each was important to the overall machine of the program. I introduced the idea through a presentation and a reference paper for reminders on how the program worked.

It took me some time and practice to get this right, so don't feel discouraged if the first sessions don't go as perfectly as planned. However, all the planning and work that goes into a rolling research program is highly worth it and always extremely rewarding.