June 10, 2021
June 10, 2021
The first time I got asked to test the content of a website, I was stumped. I had no idea how to test information, or words on the page, aside from asking participants what they thought.
Later, I figured I could conduct usability tests and, if users failed the tasks, there was not enough information. That is how I tested content for some time.
While there isn't anything inherently wrong with what I was doing, it usually didn't get the best feedback for my teams. I would present the usability issues we gathered. We could conclude that the information on the website wasn't quite right, but I couldn't provide more direction.
Whenever a content test came up, I felt apprehensive, and my team seemed deflated by the lack of clarity. I couldn't be afraid of content tests forever, so I dug deep into the world of dedicated content testing. Through research and practice, I quickly realized other great ways to test content and get more concrete answers for teams directly.
When users achieve their goals, our organizations win. We accomplish our acquisition, retention, and conversion goals. If our users cannot find what they need, they’re less likely to sign up, purchase something, or stay loyal to a company.
Content testing exposes whether or not users can adequately find and understand critical information; it’s key to bringing these insights to light, and putting debates on copy changes to rest.
Content testing can go beyond explaining whether or not the information in your interface is clear. It can tell you whether the tone of voice makes sense and if users can efficiently process and digest the text. If someone comes to your website and cannot understand the value of your product, or you come off sounding too jargon-y, the likelihood they will purchase starts to drop.
There are three main ways, outside of usability testing, to test content:
A cloze test is a fancy way of describing what some of us might remember as "Mad Libs." In this test, you take a sample of text, remove specific words, and ask participants to fill in what they believe the missing words are. Through this test, participants must rely on the context and their knowledge of your product.
Cloze tests are great for determining how appropriate and understandable text is for your audience. These tests are beneficial when dealing with highly complex topics, such as legal or healthcare information. However, you can still use a cloze test to assess the understandability of any website.
If you're interested in running a cloze test, follow these steps:
After scoring, you can use the following benchmarks:
A recall-based test assesses how memorable and comprehensive content was. There are two processes humans go through when reading: decoding, and then comprehension.
Decoding is seeing letters and putting them together to make words. Comprehension is about what the words mean in the context of each other. For instance, have you read a sentence, then got to the end, and had no idea what you just read?
This is why it is essential to differentiate understanding (ex: "Did you understand what you just read) and comprehension (ex: "What does that sentence mean to you?"). Recall-based testing can help with this.
Here are the steps to conducting a recall based test:
If participants are having a hard time recalling information, it would indicate a rewrite. After the post-test survey, you can go back and ask some open-ended questions about how the participants might improve the content.
Finally, one of my favorite ways to test content is the highlighter method. Here, you ask users to indicate what’s clear and unclear by highlighting text. This test can help determine precisely what needs to be rewritten. It can also reduce the amount of text on a page to focus on what is essential to the user. The highlighter test is excellent for determining value proposition and what information is necessary to help users achieve their goals.
Running a highlighter test is relatively simple:
First and foremost, define your goals! Sometimes content testing can get confused with concept testing or usability. In content tests, we are zeroing in on the text's comprehension and readability. If you want to test both content and usability, I would recommend separating the projects, if possible. Defining your goals will also help you choose what type of content testing to use.
The most common goals for content testing are:
Once you have defined your goals, you can choose your method. I recommend moderated testing for these approaches. With moderated testing, you can follow up with the user and also read body language cues. For instance, you can see someone hesitate while reading and ask them why.
After defining the approach, make sure you recruit the right people. If you are testing with people who aren't interested in your product/service, you won't get great feedback on the content. You should recruit people who are trying to achieve goals or solve problems with your product/service. You can do this through a screener survey or screening interview. For content tests, I recommend speaking to 10-15 people per segment.
Finally, always remind people that we are not testing them! These types of tests can feel much more like a test than usability sessions or 1:1 interviews. Explain that you are testing the content, not them!
You can determine what users truly understand versus what they say they understand through these content testing methods. This is such an important distinction. While people can claim to understand your text, if they don't truly comprehend it or find it helpful, they might not use your product/service. By constantly testing your content, you will ensure users have the best information to achieve their goals and, ultimately, your organization's goals.
Nikki Anderson-Stanier is the founder of User Research Academy and a qualitative researcher with 9 years in the field. She loves solving human problems and petting all the dogs.
To get even more UXR nuggets, check out her user research membership, follow her on LinkedIn, or subscribe to her Substack.