Stop Lights and Rainbow Charts: Two Engaging Templates for Qual Research Reports

I sometimes struggle to find ways to present user research that is both digestible and actionable.

Qualitative research tends to include words, quotes, and audio clips. These reports can become text-heavy and difficult or time-consuming for team members to read.

Lengthy reports can be counterproductive if you are working with a new team, coming in for a short time as a contract worker, or trying to convince people to actually read your research.

However, qualitative research, especially usability testing, is fundamental to share across a company.

After several attempts to share usability test results, I found that stakeholders weren't gaining the most value from my reports. I set out to find a different way.

I started by talking to the stakeholders involved. I asked them what information was most important to them to see in a report. With this, I could understand the hierarchy of information they needed and better prioritize my reports.

From there, I did my research on how to present usability test findings. Finally, I came to two different and useful reports:

1. Stop light (or traffic light) Usability Testing Reports
2. Rainbow Charts

Jump to:

• Report #1: Stop light reports
• How do I create a stop light report?
• Report #2: Rainbow charts
• How do I create a rainbow chart?

Another relevant read: How to Present Your Research So Stakeholders Take Notice and Take Action

Download the template here

In the above example, there are different components:

1. Each participant has a column, with a participant summary at the bottom
2. Each task has a row, with an average task summary over to the right
3. The three colors indicate whether a participant succeeded in a task, was unclear in the task, or failed the task These can also be severity ratings. The red is the most critical to fix, orange is the medium priority, and any green tasks are the lowest priority
4. The time for each task is recorded within the task participant bubble and averaged per task

There are a few things to do before using this sharing approach:

1. Make sure to define what passing, unclear, and failing mean for each given task — this is something you discuss with other stakeholders (designers, developers, product managers)
2. It is also beneficial to benchmark and average how long tasks generally take users. That way, this data is useful. For example, if task 4 on the homepage usually takes users 10 seconds to complete on the current product, and it is taking users over 30 seconds on the prototype, that is a huge red flag. If you don't have any benchmarking data, it might be worth doing some benchmarking before you go out to usability test
3. Get some buy-in! Before I started sending these out to my stakeholders, I presented the idea to them, and they gave me some great feedback that made the transition more comfortable and valuable for everyone involved.

A lot of the work comes during the study:

1. Have an observer help you with filling out whether a participant passed, struggled with, or failed each task
2. Write down the time taken for each task
3. Write down any notes associated with the participants while they are completing the task
4. After, cross-reference your notes and observations with others who observed the research session

If you don't have someone to help you during the research session, I recommend filling out the chart after watching a recording. The only vital portion to complete during the study is recording the time on task.

Ta-da! As you can see, most of the work happens before and during the study by defining the different levels and recording the data. I don't want to seem as though I am oversimplifying the process, but it is relatively simple. And, after all, we do want to keep this as simple as possible.

Overall this approach has helped me clearly and effectively share usability testing results with colleagues and is more valuable than a written report. It makes it easy for people who want to glance at the results, to understand what the priorities are (summary), but also gives the option of digging deeper into the results (notes) and includes actionable insights based on the test (recommendations). Download the template I use here.

Download a template here

In the above example, there are different components:

1. Each participant has a column, with a participant summary at the bottom, and each participant has a color assigned to them
2. Each observation has a row
3. The colors indicate whether the observation happened during that participant's session
4. Each observation has a potential solution in the last column

There are a few things to do before using this sharing approach:

1. Define the different opacities of the colors. Rainbow charts can signify more than if an observation happened or not. There can be different levels of observation, and I use color opacity to make this clear:
   1. Full color: the observation happened as stated
   2. 50% opacity: the observation occurred partially
   3. No color: the observation did not take place for this participant
2. Make sure you can take notes about the different observations. Each participant should have a spreadsheet in which notes about that participant get recorded.

Much of the work of rainbow charts comes after the study when you watch the recording of the session. While watching the video recording, you can fill out the table with the information:

1. Write down all the significant observations you noticed across the study; each has a row
2. Assign a color to each participant when the observation occurred for that particular participant. Determine the opacity of each observation as well
3. Write down any notes associated with the participants
4. After, cross-reference your notes and comments with others who observed the research session

Another way to use rainbow charts is to substitute assumptions for observations.

The stop light report and rainbow spreadsheet are tools to support collaboration. If these templates don't work for you, feel free to customize them to your team's needs. Find creative ways, like these deliverables, to share information and learn about users together!

There are many ways to share usability testing results, and I'd love to hear about your approaches or feedback!