Ideas

How to Set Up a User Research Framework (And Why Your Team Needs One)

Orient your team to "how UXR works" with this handy, concrete, stakeholder-ready, user research framework.

Words by Nikki Anderson, Visuals by Allison Corr

A buzzword you’ll encounter frequently in the product, tech, and UX world is "frameworks."

These days there is a framework for everything we need to get done on a day-to-day basis. There is scrum, agile, design thinking, lean, kanban, user stories, etc. The list goes on, and probably gets added to on a pretty frequent basis.

I struggle with frameworks because often, they’re considered magic bullets.

Take scrum, for example. Scrum is an incredible framework, which gives you loose guidelines and ideas on how to structure everyday processes in product teams. It’s a helping hand and can be iterated on and personalized for your team to enhance your workflow.

However, just because scrum worked at Spotify doesn't mean it will work that exact way at your company. It most likely won't. If we copy and paste this model as an answer, it will probably fail.

The framework isn't a Band-Aid; it is a guide. You follow what makes sense, try different aspects, and then mold it to your team.

That’s why I want to talk about a user research framework I have set up in the past. My user research framework is a guide to how user research could potentially work at a company.

Again, it is not an "end-all-be-all" answer. It's something that you can shape to fit your team or company. It is a jumping-off point to brainstorming what user research does, how it can be organized, and what it can bring to an organization.

What is a user research framework?

Primarily, I don't write user research frameworks for myself. I write them for other members of the team, especially if we are trying to hire new researchers, and also for other members of the company to understand what we do and how we do it.

Why is this important? Often, people on other teams don't fully understand what user research is or what user researchers do.

Although we are mainly part of the product team, we can positively impact other areas of a company. However, it isn't easy for people to reach out to us with questions or ideas when they don't know what we do. Also, a user research framework can highlight the value of user research to the product team and the broader organization.

Even though we work with products, sometimes colleagues we work with on these teams might not know how to best work with a user researcher. This framework can help others understand the depth of user research and how we can help organizations.

Here are the components I typically include in a framework deck:

What is user research?
Misconceptions of user research
User research principles and mission
User research responsibilities
User research process

What is user research?

User research focuses on deeply understanding our user's needs, goals, pain points, and motivations through an array of methodologies. It allows us to connect with users on an empathetic level and positively impact the success of a design/product/service.

Without talking to and understanding our users, we won't know what to build them.

Our most used user research methods

Misconceptions of user research

It is an exact science. There are two humans in a room together, talking about what is difficult. There will be biases, confusions, and incorrectly assumed insights.
It will give us concrete answers. It provides us with an understanding of our users so we can make informed decisions.
It will say, for sure, that users will purchase/use/love something
It is statistically significant. We are usually talking to 7-10 participants for usability tests, and around 15 for a generative study. This is not statistically significant because qualitative data is words, not numbers.
It is correct 100% of the time. User research might bring you in the wrong direction, or a researcher might incorrectly interpret the results. Incorrect insights don't happen often, but it is a risk we take with qualitative data.
It can efficiently test minutia. We don't need to check every single change or small component on a website. User research does not need to be used to find which button placement users prefer, but rather the holistic usability of a product.

User research principles and mission

Give teams the evidence to make the best decisions
Sometimes 5-7 users are enough
Do a little, often
Watch what they do versus just listening to what they say
Focus on goals and motivations over demographics
Involve everyone in the researchhttps://dscout.com/ds432admin/entries/peopleNerds/4147-how-to-set-up-a-user-research-framework-and-why-your-team-needs-one#
Represent users faithfully
Respect the privacy and integrity of the user
Conduct the best research we can in any given situation

Strategic mission:

We engage with customers before the development cycle to understand their overarching goals, motivations, and pain points. We use this information to work with all teams and departments to help craft a strategic vision based on customer's needs.

Tactical mission:

We start testing with customers as early in the development cycle as possible to give teams evidence to make the best decisions possible.

User research responsibilities:

Junior UX Researcher: Embedded in a team to carry out user research activities. They have some practical experience but need regular guidance and training to produce their best work and develop their skills. They generally work in combination with a more senior user researcher
UX Researcher: Embedded in a team and responsible for planning and carrying out user research activities. They can work independently on a team without too much guidance.
Senior UX Researcher: Able to plan and lead user research activities in larger teams and on more complex services. They build user-centered practices in organizations and align user research activities with more comprehensive plans to inform service propositions. They may supervise and develop other user researchers to assure and improve research practice.
Lead UX Researcher: Leading and aligning user research activities across several teams. They ensure that organizations take a user-centered, evidence-based approach to service design and delivery. They develop and assure good user research practices.

User research process and timeline:

Discover

Gather insights that inform the UX strategy. Establish the user's needs, goals, and motivations. Some research methods include:

Generative research
Competitive analysis
Journey mapping
Product walkthrough

Define and ideate

Based on the discovery, come up with several ideas to test with users. Show the concepts to users to see how they react. These tests ensure we are going in the right direction. Some research methods include:

Concept testing
Card sorting

Test and iterate

Test the solution, including the look and feel, features, and functionality. See how users react, and what can be improved in the next iterations. Some research methods:

Usability testing
Product walkthrough
Beta testing

Measure

Evaluate the UX to identify opportunities for improvement. Constantly be monitoring and testing what is working and what needs to be fixed. Some research methods include:

Benchmarking
Usability metrics
Surveys
BUX (Bad UX)

Sample generative research timeline:

A sample framework

The 3 E's Framework:

Ease of use
Effectiveness
Emotional impact (happiness, anger, frustration)

Ease of use

Ease of use is the ability of a user to access and use a given product/feature to complete a given task. If a user isn't able to easily use a product or feature, they are unable to experience the benefits, and the product/feature will, most likely, remain unused or create continuous frustration for the user.

We measure ease of use through usability testing, structured with a list of tasks; this list is repeated across multiple users, creating results that are reliable and comparable over time.

Measuring ease of use includes:

Time on task
Number of errors
Percentage of users who completed a task
Severity ratings: how serious a usability issue is

We can measure each of these metrics for individual tasks separately, but also combine them to estimate an entire workflow.

Each task is measured on a scale of 1-3:

Rating of 1: This task can be accomplished easily by the user, due to low cognitive load or intuitive pattern
Rating of 2: This task requires a notable degree of cognitive load, but can generally be accomplished after some effort/thought
Rating of 3: This task is difficult for the target user due to a significant cognitive load or a confusing interface. Some users would likely fail or abandon the task.

Then each task can be measured to give a score to the entire user workflow.

Sample workflow: Using our revenue tracking software, generate a report from the previous month that includes revenue amount, media that performs the best, and gaps in product coverage.

How would you find the amount of revenue generated from the previous month?
1. Rating 2
What would you do to find the media that performs the best?
1. Rating 3
Where would you find products that don't have content?
1. Rating 2

The overall score for workflow: 7

The total score can give us a benchmark based upon the current state of the built functionality and where difficult tasks lie within a workflow. The team can then prioritize revisions based upon where users had the most difficulty. Visual representation makes it quick and easy to spot the most significant issues.

Effectiveness

Effectiveness measures if a product, service, or feature is meeting the needs of users. It is determined in two ways:

Frequency of use of a specific feature/product area
The adoption rate of a particular feature/product area
Qualitative interviewing of how a user would ideally complete a task

Emotions

Emotions, such as anger or joy, can be measured through qualitative interviews and can also be seen through observations during usability testing (ex: "rage-clicking" or sighing). These insights will come in the form of quotes or notes that will help explain why a task or feature got a low rating score.

By bringing these three measurements together (ease of use, effectiveness, and emotion), we can paint a holistic picture that marries both qualitative and quantitative data, giving us more information to make data-driven decisions.

The outcome would be a combination of the three of these measurements, effectiveness, and emotions will supplement the ease of use quantitative data by including quotes, videos, or observations that tie to the different scores, giving additional evidence and context.

Final thoughts

Although writing up this information and determining the best framework for your team and company is time-consuming, having all of this knowledge documented is incredibly valuable to the entire organization. It is a fantastic way to start and grow a user research practice in an organization. Also, it can be very helpful in educating about user research, as well as socializing and solidifying user research contributions.

Nikki Anderson-Stanier is the founder of User Research Academy and a qualitative researcher with 9 years in the field. She loves solving human problems and petting all the dogs.

To get even more UXR nuggets, check out her user research membership, follow her on LinkedIn, or subscribe to her Substack.