Tried-and-True Methods to Prove the ROI of Your Research [Q&A]

The ROI of UX has been a hot topic for People Nerds lately, but impact is consistently at the top of the list of what UX and Research leaders want to talk about!

Today we’re really digging into the numbers behind “ROI” with a new friend of mine, jD Buckley.

Over the years, jD has pioneered ways to measure success, develop metrics strategies, created design benchmarking strategies, and built communities of research leaders to share ideas along the way.

Below we explore what metrics to connect to research impact, barriers to success, how to be agile, and how to keep your skills sharp in an ever-changing environment.

jD Buckley is a customer experience strategy and service design leader at a consumer financial services company.

Julie Norvaisas is VP of UX at Dscout.

Julie: What are some of the most successful tactics that you have developed to monitor impact and improve accountability for research activities and insights?

jD: I can’t commit to being 100% successful every time, but there are some things I’ve started to do consistently over the years that I’ve found allow me and my team to make strong arguments for our impact:

✔ Ruthlessly prioritize

Develop, document, and share your engagement strategy. For example, typical levels might include what I’ve named the “Olympic Package” consisting of specific criteria for Gold Level/Strategic, the Silver Level/Consulting and Bronze Level/Tactical. This ensures you have a triaging approach to prioritize which initiatives your team will and won’t work on.

✔ Find a research “angel” stakeholder

One way to gain support for connecting tactical and strategic research is to network and search until you can identify an issue that is important to an influential research “angel” stakeholder. A research angel stakeholder is similar to an angel investor.

At an organization, it’s usually a high-level executive with influence and/or budget. They sponsor and invest in the development of research as a strategic imperative across the organization, and champion your research team's value and efforts.

Ensure that your research angel stakeholder’s initiatives have short, medium, and long-term impact over time. This should include a broad, ongoing strategic initiative with a measurable impact on company KPIs, such as benchmark studies, developing behavioral archetypes, shedding light on sales issues, etc.

It may be painful in the short term to conduct the research and show value, but if you can achieve buy-in from this research angel stakeholder who will vouch for you and your team over time, it can supercharge your impact and buffer your team against critics as you innovate or introduce new ways of thinking.

✔ Address short, medium, and long-term concerns

This is going to sound counterintuitive, but if your organization insists that your team focuses on primarily tactical projects, find some way to insert a percentage of strategic initiatives as part of your research and insights portfolio. Even if that means identifying a team-sponsored project and allocating time to showcase the value of thinking longer-term.

If your organization only wants you to focus on future-facing or aspirational ideals, ensure you have some mechanism to gather impactful tactical short-term or foundational insights. I can’t emphasize this enough—make sure those insights end up on the roadmap and get released.

Finally, ensure you can demonstrate that your short-term/tactical projects or initiatives ladder up to the aspirational/long-term initiatives.

I’ve found that indexing on one extreme to the absence of another is rarely successful over time as markets, leadership, macroeconomic, and company priorities can often shift. Keep in mind the full spectrum of a user’s and a business’s concerns for today and tomorrow.

✔ Measure and track your impact

Connect the activities and metrics your team tracks to revenue or sales metrics. If you can’t find a satisfying connecting metric…invent it. For example, I’ve introduced a research impact metric I call RR2R/I and RR2R (Ratio of Research Recommendations to Roadmap and Ratio of Research Recommendations to Release).

A few places you could tie back to are…

• Business strategy metrics – Revenue, profit
• Sales – Monthly sales growth, CAC, CLTV/LTV, MRR, ARR, churn
• CX or service design metrics – NPS, CES, CSAT
• Product metrics – CTR, net credit sales, new accounts
• UX and UI metrics – Efficiency, effectiveness, satisfaction

✔ Include training, education, and sharing insights on team impact metrics

Make a research portal

You can create a research portal (think Google Sites, Sharepoint, etc.,) to centralize…

• Recordings of formal research presentations or lunch ’n learns
• Forms
• Templates
• Articles
• Videos to share insights
• External articles, reports, or trends

This portal can also act as an entry point to your research repository. Track downloads and visits with Google Analytics quarter-over-quarter (QoQ) or year-over-year (YoY).

Create a standard intake system

Even if you have a more informal project pre-kickoff meeting or process, include a central research or project request intake starting point. This creates a standardized way for teams to launch engagements with your team.

Additionally, this intake system allows you to track specific categories of requests and increases or decreases QoQ or YoY. It’s also a handy way to quantify the degree of demand for your team's expertise, and can support discussions when advocating for increasing the size of your team.

Collect metrics around your teaching and resource initiatives

Create a research repository and report out QoQ visits, traffic of new and revisiting users, as well as downloads of research reports. Track YoY attendance at monthly or quarterly lunch ’n learns.

You can also track attendance at workshops, co-creations, listening labs, or other collaborative and cross-functional events your team sponsors. Track attendance QoQ or YoY. Collect testimonials or any feedback positive or negative from attendees.

✔ Manage your team like you’re running a small business

Try a mini-workshop to create a team mission, vision, and team principles:

• Have your team collaboratively create a mission and vision statement.
• Brainstorm team principles.
• Make an elevator pitch. Allow everyone on the team to quickly and consistently explain the value of the team in 10-15 seconds when/if asked.
• Establish team goals. For example, quarterly, bi-annual, or annual goals mapped to company goals and KPIs.
• Make sure team annual goals map to company goals per initiative or project, so each team member understands the impact of their efforts on the business.

✔ There’s nothing wrong with PR

Create a Slack or Teams channel to announce and share out your team’s events, presentations, or the latest trends, tools, and industry perspectives. You can host lunch ’n learns with an occasional guest speaker to encourage folks to join in and engage with your work.

Other areas you can incorporate PR and promote evangelism…

• Project demos
• Co-creation workshops
• Customer/user listening labs
• Cross-functional partnerships

What are some of the counter-intuitive barriers you’ve found with your stakeholders as you experimented with some of this?

✔ Barriers between the design and research community

This is controversial, but I’m going to say it anyway. I’ve often struggled more with my own community of design and research people on subjects such as creating, connecting, and tracking business metrics to measure design and research impact.

I’ve gotten ostracized for advocating the value of balancing customer, user, and business needs. If we have no users or customers then we have no business, but if the business can’t sustain itself, then ’nuff said.

This has actually improved somewhat over the last four to five years, and especially the last two years. So many design and research tech folks have realized their company is not actually the same as their family, and they are better equipped to push back if they can quantify their impact.

That being said, you can be in a situation where you are tracking and articulating your impact and still run into problems. Do it anyway.

I’m a big believer in tailoring your teams and your own skills to the demands of the market. If you can’t gain any traction where you are, you’ll want to ensure your skills and those of your teams are valuable to someone or somewhere else.

✔ Barriers with executives

Several years ago, I worked at the innovation center of a well-known human capital management company known for payroll processing. For several years I ended up driving a UX ROI initiative, building a measurement program to quantify and measure the impact of improvements in the user experience to design expenditures.

A small UX research and design team did a herculean amount of work to identify the top tasks, establish a baseline of measurement prior to a design release, and measure improvements over time. This connected the design teams’ impact to company metrics such as reduction in support calls and NPS.

However, we ran into problems when we attempted to connect those improvements—or lack of improvements—in the users' experience to each siloed department's reported NPS improvements.

That’s when we learned executives were bonused on a certain percentage increase in NPS and there was apprehension about the potential of reporting out potentially conflicting inter-dependencies that contributed to NPS fluctuations or improvements.

There also started to be an expectation that design should be able to forecast QoQ which improvements in the design would drive what percent of NPS improvement across siloed departments.

How do you think that both the need for and the conversation around ROI/impact has changed in your years of tackling this?

Technology has shifted from an immature stage to a more mid-level stage. Along with technology maturity, the profession of design and research has also matured. Helpful and low-cost tools like design patterns, qualitative and quantitative research guidelines, scripts, templates, articles, YouTube videos, as well as best practices information are abundant.

The bar for what constitutes “impactful” research—whether that’s embedded in customer experience, service design, experience, or product design—has increased in even just the last three years.

In 2016, when people first started talking about UX ROI, it was controversial because design leaders and executives didn’t imagine it could be measured. Now, the conversation has shifted to whether design and research ROI should be measured.

The reality is that as areas of expertise become more mature and more sophisticated—and require more revenue and more nuanced skills to support them—research and design leaders should expect that executives (who often still don’t truly understand design and research) will want to be able to rationalize budget, revenue, and headcount.

What do you think are the most important skills to develop here for leaders? How can we gain them quickly?

Never get too comfortable with your skills or wherever you work. We live in an ever-changing world. The one thing you can count on is change. Ensure your teams understand, embrace, and expect to change. Manage ambiguity and expect to learn on the fly to be successful.

Along with encouraging teams to embrace change, focus on creating a culture where experimentation and failure are accepted and expected by modeling that behavior yourself. Model the behavior that at the end of the day there are things you can control and things that you can’t. Focus on the things you can control.

For example, your effort and passion for what you do, a thirst for learning, and seeing yourself and your team members consistently mature, evolve, and excel is key.

When you’re pushing or doing things for yourself because you understand the value and want to track and regularly measure your team's progress, it will feel less like drudgery. Finally, understand how to quantify your impact—if not only for where you are today, but also where you want to be tomorrow.

Wrapping it up

As the UX research industry has matured, so too has the need to implement more sophisticated proof points of your work's value. Not every company will appreciate or champion your work.

Do your best to…

• Work with your strengths
• Enable your teams
• Find advocates who will back you up in the organization
• Quantify your impact in a variety of ways QoQ and YoY

You may also like…

• Championing the ROI of UX Research
• Insights Activation Roundup: How to Make an Impact with Your Findings
• A Guide to Hiring and Growing Your UXR Dream Team