I used to hold colossal synthesis sessions after a study was complete.
In one project, we spoke to about ten people and had session notes and videos. The next logical step for me was to hold a synthesis session. I invited many team members to a room and shut us in for four (sometimes up to eight) hours. Of course, I was nice. I ordered pizza, had drinks (soda for morning sessions, beer for later afternoon sessions), and brought heaps of cookies and chocolate.
But people still felt overwhelmed. There was a lot of information to sift through. Ten sessions worth of video content and notes sat in front of each person. Not every stakeholder observed every interview. I tried assigning homework before the session, but people frequently came without reviewing the information beforehand.
I did my best to timebox. I had each person review the notes of the ten sessions for fifteen minutes. I then played some video clips for five or ten minutes. I even started tagging the notes, so people were reviewing my tags.
Then, when I put up the categories for affinity diagrams, they mostly had the work done for them in the notes I had tagged.
I loved synthesis, but soon was dreading the sessions. I kept banging my head against the wall, trying to understand how to include the teams. I didn't want to synthesize alone. I also didn't want to assign mountains of homework. I didn't want to tag all the notes before the session, completely biasing the workshop participants. Again, more headbanging.
I finally decided to try another method: debriefing after each interview session. It was the best decision I had ever made.
What is a debrief?
A debrief is the time you take after a research session to reflect on it and encourage deep learning and complex connections.
Think of it as downloading all the information you just learned. The debrief is a perfect time to reflect on what just happened during the session and bring many minds together. It is a nugget-sized synthesis session.
The way I now structure my process is by having a 30-minute debrief after each interview session. I then have a two-hour synthesis session at the end of the project, to wrap everything up, and find trends across interviews.
Why should you conduct debrief sessions
If you are fortunate enough not to have run into the above example of a synthesis session nightmare, that is fantastic! However, I would still recommend running these debrief sessions. They are a great way to:
- Immediately reflect on what happened during the session
- Get the team to interact with each other
- Find multiple perspectives on each interview
- Highlight key information from each session
- Teach people how to synthesize on a smaller level
- Make your overall synthesis session shorter and more effective
- Not have to assign homework for the overarching synthesis session
I'm usually all for something that makes my life easier in the long run and debriefs have done that. Debriefs have been just that. For example, once I started running debriefs, the teams were more engaged in the research, and our synthesis session went from five hours to two hours. Everyone was happy.
I also found the teams thinking about the research more frequently. They started talking about this session or that session with others in casual conversation. The debriefs made it much more manageable for stakeholders to ingest and interact with the research. Hearing teams talk about the findings in between sessions made the fire of user research grow throughout the organization.
How to run a debrief
I have a trick to get people to attend the debrief session. Of course, it doesn't always work, but it increases the chances people will stay. When I invite stakeholders to observe the session, I always include the debrief in the calendar invite.
For example, let's say we are doing a 60-minute interview with the user from 10:00 am - 11:00 am. I make the calendar invite for 10:00 am - 11:30 am, to include the 30-minute debrief. That way, when stakeholders accept the invite, their schedule is most likely clear for the debrief. It was shocking to me how well this worked!
Aside from tricking people into spending time with you, debriefs are relatively straightforward to run. You are there to facilitate and moderate the session, more so than leading. It is more important for others in the meeting to lead and participate.
Here is my step-by-step process:
Before the debrief
- Decide who to invite to the session. Invite the stakeholders who are observing that session (ex: designers, developers, product managers) because they will be able to participate. If more people are observing outside the team, I give them the option to join, but I highly recommend the people working on the project to be present
- Decide on what points you want to cover. It is essential to decide on what you want to include in the debrief. What is most helpful to reflect on from the interview? For example, pain points, surprises, questions we still have, key quotes, and key takeaways. I usually choose 3-4 topics because more tends to take too long.
- Create timeboxes for each activity. Once you decide on each point to cover, place a time on each one. For instance, give people five minutes to brainstorm pain points individually and then five minutes to discuss them together.
- Create a board (in-person = whiteboard; remote = Miro, Jamboard, Trello, Mural). Before going into the debrief session, create a template you can fill out. This template gives the team structure to fill out the appropriate information at the right time but allows for creativity. I create one board per participant.
- Always have a parking lot. A parking lot is where ideas go that don't fit into the predetermined topics. Instead of trying to shove everything into pain points or another topic, we put them in the parking lot. You can address the parking lot in the synthesis session.
- Invite people. Use the +30-minute calendar invite trick! Invite the stakeholders to the session. In my calendar invites, I give the time of the session, and when the debrief will start.
During the debrief
- Explain each section/point. Some stakeholders may not know what a "pain point" is, so be sure to explain what each section means.
- Go through each section individually (with timer). I go through each part on its own, allowing people to brainstorm and then discuss. For example, I focus on brainstorming pain points for five minutes and then talking about it for five minutes. Make sure to be strict with the timer here! Cluster similar ideas into groups, so you have some trends emerging at this point.
- Divert questions. If people have questions about the debrief method, encourage them to place issues in the parking lot, or ask you afterward. It is super distracting for others to brainstorming and listening to questions at the same time.
After the debrief
- Share the board. Make sure people can review the board, especially if they weren't part of the actual session. I have also seen stakeholders adding information they forgot about initially.
- Bring the board to the synthesis session. This board will help you so much during the more extensive synthesis session. Those who weren't able to attend all of the sessions will be able to glance at the highlights, and the team will already have a handle on the key insights.
- Ask for feedback. This step is crucial! For whatever reason, a debrief might not work for your organization. You may also need to change the structure, or find a different way to recap information. The most significant part is that your stakeholders see value in this debrief, so make sure to ask them if that is the case! And adjust when necessary.
Since the above steps can feel vague, I want to give you a concrete example and a Miro template to use. Of course, make these your own. If you have less time, include fewer topics, and if you want to add different sections, please do so!
Let's imagine I am doing 60-minute hybrid research on how users adopt pets from a rescue app. The team working on the project consists of a UX designer, a product manager, and three developers. We will do a 30-minute debrief after each research interview.
Before the debrief
- Decide who to invite to the session. I will invite all the stakeholders who are working on the project since I enjoy diverse perspectives on the research insights.
- Decide on what points you want to cover. For this particular debrief, I will pick three different topics to cover. I do this because I know the stakeholders aren't super familiar with synthesis or the topics. The focus areas will be pain points, surprises, and key takeaways. I will also include a parking lot section.
- Create timeboxes for each activity. I will give people five minutes to brainstorm each section and then three minutes to discuss their findings. This leaves extra time for explaining and questions.
- Create a board. Since we are in the times of COVID, I will create a Miro board to use. I will put all seven participants on this one board. I love Miro for the collaboration and the timer in the tool!
- Invite people. I use the +30-minute calendar invite trick! Calendar invite example:
- Opening questions: 10:00 am - 10:15 am
- Prototype 1: 10:15 am - 10:35 am
- Prototype 2: 10:35 am - 10:55 am
- Wrap up: 10:55 am - 11:00 am
- Debrief: 11:00 am - 11:30 am
During the debrief
- Explain each section/point. Once we start, I explain each of the different sections. I give myself 1-2 minutes to do this and take any clarifying questions.
- Pain points: a place where the user struggled with an action or was confused. Also, a situation where the user seemed disappointed or unsure
- Surprises: something that made you think, challenged an assumption or changed your perspective.
- Key takeaways: big findings that are important for the project
- Parking lot: a place to put anything that doesn’t fit into the above (including questions!)
I generally go in any order, starting with any of the above topics. Some groups find it easier to start with key takeaways more broadly, while others like to start with specifics such as pain points. Vary how you begin and ask the team for feedback.
- Go through each section individually (with timer). My schedule is as follows:
- Intro and explanation: 2 minutes
- Pain points - brainstorm: 5 minutes; discussion: 3 minutes
- Surprises - brainstorm: 5 minutes; discussion: 3 minutes
- Key takeaways - brainstorm: 5 minutes; discussion 3 minutes
- Wrap-up and other points: 4 minutes
While people are brainstorming and putting their sticky notes on the virtual board, I am clustering similar ideas. You can also tell people to put a +1 on sticky notes they agree with. Although, I prefer if everyone writes out their own. If I have time, I include my thoughts, as well.
After the debrief
- Share the board. I share the board with the entire team and encourage stakeholders to look before the synthesis session.
- Ask for feedback. Depending on the level of comfort of the group, I either ask for feedback directly and openly (ex: in Slack) or send an anonymous survey.
Overall, debriefing with your team members is a phenomenal way to start identifying patterns and to reflect on learnings. It truly sets you up for success once the synthesis session comes around!
Nikki Anderson is a qualitative user experience researcher with about 5 years in the field. She loves solving human problems and petting all the dogs. Read more of her work on Medium.