Finding the Right Scenarios for Internal Testing

User testing with employees gets a terrible reputation. For years, I refused to test anything internally, with the fear of skewing data, integrating biases, and considering inauthentic behavior as a genuine reaction.

There are many arguments against user testing with employees:

Brand loyalty

Employees are (generally) loyal to a brand, which may make them feel strongly positive (or negative) during the user test, regardless of what I put in front of them. They are less focused on the prototype or potential feature and are more likely assessing their feelings about the company.

Participants know each other

Internal participants may know or be friends with those involved in the study. This relationship could heavily influence behavior during a test. You don't want to tell someone you are friends with that his designs suck. Instead, you may participate in a white lie for fear of insulting or disappointing colleagues.

Bias

People working for the company have internal motivations and opinions on the best course of action for the business. If you are testing a feature aligned with those motivations, there may be more positive feedback, and the opposite can be true.

Overly technical responses

Tech knows tech, so sometimes, developers might give very technical feedback or say something is not technically feasible. It may be difficult for employees to step out of the company box they are stuck in and give innovative feedback.

Internal knowledge/jargon

Employees may have prior knowledge about the study and have prior knowledge about the company. They know the company's jargon and how the company thinks about ideas, which could make them perform or understand better than an "outsider."

I am not arguing with the above, as they are all great points against employee user testing. I know that it is significantly harder to observe authentic behavior when testing internally, which is the entire goal of user research.

However, I would like to provide a counterpoint. Like everything else, internal user testing has a time and place and can offer some benefits.

Benefits of internal user testing

Of course, we cannot view internal user testing the same way as user testing, and we need to take the "findings" with a large grain of salt. Nevertheless, internal testing can be beneficial when used in the right circumstances if you know what to expect and how to interpret what you hear.

Here are some (unexpected) benefits I came across when I finally gave in to internal user testing:

Internal user research is free

This is probably the most common pro of conducting internal user research. Conducting research takes a lot of time and money across recruiting, planning the study, talking to users, and compensation. There are ways to lower a user research budget, but very rarely are interview sessions free.

You can get this free feedback when dealing with internal research (under 99% of circumstances). You aren't going to get all the insights you may typically get with users, but you are getting feedback, nonetheless.

People are easier to recruit

Recruiting can take a lot of time. Employees are much more accessible than users and are more willing to give their time to the company. I have posted a message to our general Slack channel looking for interested employees and scheduled seven tests over the following two days.

It promotes UX research within the company

Inviting employees to try out prototypes or concepts increases the visibility of user research in the company. Although I have given many presentations on user research, involving people in research is an excellent way for them to experience what you do.

It opens up the opportunity for employees to understand what we do as researchers and why it is essential for us to talk to our users. In the past, this has led to more UX support, in general, and a larger budget.

Employees can give feedback and help the product team

I have seen product teams completely siloed in companies, which can lead to miscommunication and misalignment. When I first started trying internal user testing, we put something in front of an employee (an account manager), and they said, "Wow, you’ll have our users do that?!"

That moment opened my eyes to a whole new (and different) world of feedback that I could gather internally. Not only that, but employees feel more empowered to make a difference and contribute.

You test your user testing protocol before speaking with users

This is my favorite reason for conducting internal user research. I can run a pilot test of my research script, see which of my questions don't make as much sense, and where the flow may get confusing or awkward on my side. This process lets me know where the conversation might get stuck before I head into a session with a user.

When to do internal user testing

I don't use internal user testing for everything, but when I do it always compliments actual user testing.

I typically focus on user testing in these scenarios:

Feedback on a prototype before putting it in front of users

Internal testing is a great way to find bugs, typos, and generally confusing information before placing something in front of actual users.

The number of times I have found an issue with an Invision prototype or confusing copy during an internal has been numerous and allows for better feedback once you speak with your users

Testing out a research script

As mentioned, I appreciate running through my user test in a semi-realistic scenario before stepping into a user research session. Not only can I find problems with my interview flow, but it makes me feel more confident when it matters most

If you are building a product for employees

Some products we make are for employees, which is the perfect time to run internal testing—in fact, this officially changes it from internal testing to user testing.

When there’s internal resistance to UX research

Whenever I’ve started at a company that is resistant to the value of user research, I always start by running usability tests internally. Although the information is not the same, it can still highlight feature or concept ideas that we did not consider upon conception.

Testing with stakeholders can break down the barriers and open up the possibility of communication and discussion regarding "gut decisions."

Gathering feedback on a very confidential concept

Sometimes conducting tests on a confidential subject can cause recruitment and user testing issues, especially if it is highly sensitive and the company is worried about others finding out.

In this case, your employees have already signed NDAs, and are less likely to cause confidentiality problems than external participants.

How to set up internal testing at your organization

Once I saw how many benefits were possible with internal testing, I created different ways to set up regular internal testing:

Monthly test sessions

To start, I set up a recurring testing session at the end of each month. For these sessions, two teams could sign up for slots to get their ideas tested internally. This way, every month, teams could get feedback on prototypes or concepts before putting them in front of users.

During these tests, I would also try out the script I’d use on users to get in practice and ensure the flow of the session was sound. I also did this for external user testing, which you can read more about here.

Speed testing

A spin-off of speed dating, speed testing is when you have a few ideas to test and not much time to test them. I set up four prototypes/concepts, each getting a station, and recruited seven people to test the ideas.

Colleagues then rotated through each prototype/concept, getting ten minutes to try it and give feedback. I provided each person with a form they took with them during each round, prompting them on areas we needed input

Create a demo desk

After colleagues got used to the idea of regular internal testing, I put together a demo desk. A demo desk is an area that allows employees to stop by and play with any current prototypes/ideas we’re working on and give feedback.

To do this, I put our mobile testing devices and a spare laptop in the area. I then set up instructions on accessing the prototypes and a form to fill out with the feedback we were looking for. This form asked questions like, "How does the prototype feel?" or had specific tasks to complete

Random testing

Have a script and post ready if you need last-minute internal testing. If a team needs feedback outside of the regular testing cadence, I typically send a message to Slack, email it, and announce it at an upcoming team meeting.

I include what we're testing, how many people we need, and the deadline in this message. By including my Calendly link, colleagues can quickly sign up, and I can typically fill slots up within the week.

Remember, employees aren't the same as users

Internal user testing will not get you the same results as testing with your actual users. You can't expect to walk out of an internal user session with the same profound findings you would have if you had spoken to a real user.

Of course, it would be great if you were able to get some great, innovative ideas out of the session (especially if your internal stakeholders use the product). Still, I always tag those insights as "internal" and see if I hear these same ideas in external user testing.

Internal user testing is not a "get out of jail free" card from speaking with your actual users, so please refrain from only talking with employees. However, when used in the right circumstances and in conjunction with user testing, doing user research with your employees can be an advantageous experience.