How to Democratize Your User Research Practice (Even Now)

There's never been a more necessary, or more challenging, time to pull "more hands on deck" for a research project.

As UXR matures within a company, the demand for research insights swells. In turn, researchers are challenged to conduct more research, faster, while still ensuring the quality of their work. Bringing "non-researchers" on board can help in this endeavor.

But it's easier said than done.

While the demand for insights may not have changed in our work-from-home, resource-scarce moment, our mechanisms for meeting it have. And as we work to democratize our research practice with new constraints, we increasingly find new strategies are needed.

Join our panel of experts as they discuss the art of scaling, democratizing, and restructuring their research processes, in the wake of new challenges and shifting goals.

The panel talks about:

• What "responsible" democratization means for today's human-centered org
• How the current moment is shifting the focus and approach of democratization
• Some strategies for getting started (or re-evaluating) with your own practice

Transcript:

Roy:

Which is to do the type of research that they can do. The type of research that's going to take a lot of time, a lot of investment, a lot of skill. But we also realize that to do that across the company is unsustainable.

Roy:

We know that PM's and designers also have curiosity about users. And so how can we set them up for success as well? So that's the way in which we approach it at Zapier now and similar to how we approached it when I was at Buffer as well.

Behzod:

I'll jump in because I actually have a question for Rani as a result of my own thinking on this. But I think structure is important because you need to... and to Roy's credit, I think he pushes a lot of people on this, but what is the purpose of research in your company?

Behzod:

And something that I talked to a lot of early stage folks about is making sure that while you're trying to build a practice intentionally, you're not trying to cover all of the surface area. And so something we thought about a lot at Slack was like, what role does research play for different parts of the company? And what is the right cadence and depth that we can serve them?

Behzod:

And that helped us drive both the work that we did as an embedded team, as well as the programs and structures that we built to support other parts of the company. And so, I think one of the challenges for a lot of folks early on is being focused enough that you prove that you're valuable and highlight the work that research can do while acknowledging that you can't do everything.

Behzod:

And so there is that trade off of like, "I'm going to be valuable and practice in this way, and I'm going to enable people to get answers over here," without letting that run too wild. So the thing I already asked Rannie, or as your experience has been, how do you think about the structures and where you're putting people on how you make those prioritization decisions?

Behzod:

Because you obviously have a finite set of resources and arguably an infinite set of problems but how you set up research for success in that space can be very different depending on your industry, your customer, your problem space, your size, et cetera.

Rannie:

Yeah. I think that there's this inclination to... and I think about this outside of Thumbtack as well, there's a tendency to jump to solutions and say, "Hey, we need a researcher. We need designers. We need to answer these questions. So whoever can answer them, let's go after and do it." And it's challenging, but to take a step back and say, "Okay, first things first, what are our needs? What are the biggest questions?"

Rannie:

And to have that conversation first and foremost, with the leaders within the organization, and to get aligned on where the biggest gaps are. I find that a lot of folks will just say like, "Well, this is on fire. This is on fire. We just need to scramble to get it addressed. And we have a deadline or we're shipping this thing, or we need this net for the experiment." There's a sense of urgency.

Rannie:

And I would push back on whether or not that's perceived versus real urgency. And what are those questions that we need to get to the core of first in order to set up the organization up for success? So one of the things that we do at Thumbtack is, put a big emphasis on the needs, what are the user's needs and what's the organization's needs, and where does that overlap?

Rannie:

So if you can imagine some type of venn diagram where it's like, "Here's what the user is facing, here's what we're facing." Where's that overlap? Now, let's go after that. And are there opportunities within that space that we can immediately address and/or outsource, or delegate to non researchers and that's within their skillset in order to address.

Rannie:

So I think it's taking a step back, going back to understanding the needs of the organization and our users. And then solutionizing from there. I find that that has been a really helpful approach. And I'd be curious if any of that resonates with other view as well.

Roy:

Yeah. I was going to piggyback on that because I think in general, to be honest, I've seen user research as she struggled to apply the practice of research to what we do with companies. Whenever I hear discussion around like, "Oh so, and so doesn't want to do research, why didn't we want more research? Or we need this resource or that resource," and step back and go like, "Oh, do you actually understand, you're saying on it?

Roy:

Do you actually understand the needs here? Do we actually understand what the core problem is in this whole discussion?" Because we're able to take a step back and use our skills, then you get to a point of really clear problem definition, which is a really big problem. I think we don't talk about that enough, problem definition is a real problem within companies.

Roy:

And then getting to a point of figuring out what solution may address this problem. But if you can't define that problem well enough, then our key superpower is not being applied to our work, which is tragic.

Michael:

Yeah, right. It reminds me of challenge is this whole, which I'm curious to get all of your takes on is, especially in practices in their earlier days. And I think this ties to democratization of, sometimes busy-ness is acquainted with value.

Michael:

We're really busy, we're doing a lot of stuff and I think I'm curious about separating some of these questions about problem definition and focus get down to this question of value versus tonnage. And I think a lot of organizations struggle about where to play in those, how to optimize for those different choices.

Behzod:

Yeah. I think that to Roy's point about problem definition, I would imagine that most people are not very attuned to the way that most parts of their organizations make decisions and the pace at which those decisions get made. And one of the conversations that I think happens a lot with folks, so I talk to you about democratization is, you have these different speeds at which decisions get made.

Behzod:

There's things that decisions that, look questions we need to answer in days and weeks and months and years, whether it's feature or products, business, or company, whatever that layer looks like. And most of, I think the frustration, especially for early stage folks is applying the wrong method or approach based on the speed of the decision that needs to be made or the layer that it's being made at.

Behzod:

And so you end up with this mismatch of people trying to do really long research for something that needs to be made really quickly. And to Roy's point, if you haven't spent time or energy doing that organizational introspection, or look, you don't even understand the problem space that your colleagues are working in or the amount of time or bandwidth they had to really reflect on those things.

Behzod:

And so if you're bringing a method forward or method focused approach, you're often applying the wrong solution to the problem because it's something that they can't use that's done in the wrong amount of time, in the wrong way to the wrong depth, whatever that is. And I think that Roy's point about problem definition is actually a bigger problem for a lot of folks in research.

Michael:

And so, how do you tie that to democratization? So how do you think that different types of decisions play well or differently with democratization needs?

Behzod:

I talked about this, the organization's metabolism for information, which is what I'm talking about, days, weeks, months, years, but also an organizational appetite for research, which is like, what is it that they need done to what that's? Is it, they need a broad survey being done? I think there was a really great Twitter thread that went back and forth about whether people need to answers or whether they need insights, answers being a relative position right now.

Behzod:

What is happening and insights being more of a framework or a lens for looking forward. And I really liked that discussion because I think there was, I think as a practice, want to be focused on delivering insights and delivering frameworks, and pushing on knowledge and helping people make decisions better and creating safe spaces where people can get answers, to Roy's point.

Behzod:

How can we let other people talk to customers and reduce that distance to their customers so they can get answers today, but are those more short term really well scoped questions, I think becomes a safer space for us to think about when and how to democratize rather than saying, "Okay, we're just going to open up our total place-based and anything we do, anyone can do," which is, I don't think any of us here are trying to push for.

Roy:

There's a point that I think was mentioned earlier too. When you have different models just in and of themselves, but where the state of your company, the maturity of your company and your UX practice, but they're like, what growth is looking like, what the full internal philosophy in the company looks like. That can shape a lot.

Roy:

So the companies I've worked for, Buffer smaller company, less than a hundred people. Growing, but not growing as quick as the company, CarNow workforce, just different pieces. And that's going to require different things. And so acknowledging that, acknowledging the philosophy that the product leaders and that UX leaders have towards democratization is really important.

Roy:

So again, we've talked about this issue of a blanket. Yes. Get more people to do research. It's not only that, it's getting back again to the problems, what are the key problems that we have here and who might be best positioned to tackle those problems based on the way that we function and what we're doing as a company, how it growing. There's so many variables here, which is why I get back to that issue of problem identification.

Roy:

If you can't do that, then you're not in a space to then take that step, you're talking about, Micheal, which is how do you then go about democratizing this and getting more folks to be part of the research game.

Michael:

Yeah, we've got a good one in the chat, which I think is probably something I'm sure that many of us have thought about. And it's referencing Molly Stevens who many of us know well at booking.com and the view of, and the concern around democratization as de-valuing the field, which I think is tied to a little bit, this conversation about structure.

Michael:

I think we all thought this would be a good part of the conversation. So I think this is maybe a good one that to dwell on. And obviously I'll share a brief perspective and I'm curious if you guys can fill in with some more information as practitioners. And I think I certainly have obviously seen and read many concerns around democratization and its concerns for the field.

Michael:

And I was a history major. I'm a history nerd. So I was looking back at other examples and it reminds me a lot of, there've been many fields that have gone through this challenge of democratizing. So Graphic Design is a great example, and there was a something called Desktop Publishing in the 80s, graphic design used to be guash, and cut and paste was a literal.

Michael:

You would actually cut things out of designs and retake them and had a lot of very specialized technical expertise. And it went through the desktop publishing revolution and the same concerns were raised by graphic designers and still are. Are you cheapening our skill set? Are you demeaning our field? Although while graphic design has exploded as a field, it's also raised these kinds of questions.

Michael:

And I to think about it in this, I'm curious, your takes on a little bit, the threatening side of democratization and how you handled it in your organizations. I'm sure these are conversations you have all the time and how do you think about that threatening component? Rannie, you want to make me feel that one first?

Rannie:

Yeah. So I don't know if folks have ever chatted with non-researchers conducting research, but it tends to be a really big empathizing building moments for non-researchers to realize like, "Holy shit, what you do is hard." And if anything, it has allowed us to advocate for more resources because non-researchers realize how much time and effort that intentionality is required in order to conduct these research studies.

Rannie:

And if anything has worked in our favor to advocate for more resources. So I'll put that out there. The other thing I will say is, I would try to compartmentalize a bit the different aspects of democratization. It's, do we democratize our process versus the practice of research and can anybody go and do it? I think those are two different things and we have to... there's this preciousness that we can tend to unfold.

Rannie:

It can only be a researcher who can speak to participants and it can only be a researcher who knows how to ask thoughtful questions. And I'd like to separate that out and say this is a skillset that we train on and we're thoughtful about. And in fact, I think it's part of our responsibility to help educate and advocate our partners to go and have... Bezat, you agree, go and have this type of skillset and in fact, it will make the organization better.

Rannie:

Now there's this other aspect about, there is some training that needs to happen. There is a process and thoughtfulness that goes with that process that I think Molly would agree is, we should hold that gold standard for the practice and make sure that we're not taking that lightly.

Ben:

Bezat, you've spoken at length and written at length about within any good customer centric organization, folks are talking to customers, sales are talking to customers, [crosstalk 00:14:38] are talking to customers, support are talking to customers. And I'm hoping that in answering the question about, focusing on this, we can tease apart feedback versus research.

Ben:

I know I'm keeping my eye on the chat and as a current caring researcher myself, I feel the skills that Rannie and I gleaned from our communication sciences study makes me a sharper critical both consumer of research and conductor of research. And it's happening a lot of different parts of the org.

Ben:

So Bezat, could you talk a little bit about how you're enabling those folks to look at the practices of feedback collection and applying the rigor that we all have to those practices?

Behzod:

Yeah. And I'll state my bias. I'm a rhetoric major, so I'm right there with you. And though a communication sciences loosely. I think for anyone who's not familiar, I wrote a piece called democratization is our job. So I'm going to just share my cards, put them all on a table. But I do think that that Rannie's point about this being our responsibility is true.

Behzod:

And I think people talk about the mechanisms in different ways, but there a really good comment from Andrew Wor who used to be at Uber on LinkedIn as a response to the piece that I mentioned, and we had a back and forth about yes, a conversation with a customer is not a research interview, but in many organizations, both of those things are having an impact on product strategy and the things that we are building.

Behzod:

And so it's irresponsible of us not to recognize that and to do something about it. And when I talked to a lot of folks who are heads of research or heads of design and are thinking about how to democratize their practice, we have very early conversations about quality bar, and I talk about how the first things they need to think about are, what are the guardrails you set out?

Behzod:

How do you prevent customers from being exploited? How do you think about embedding intentional practices into the systems and processes that you're building so that there are these safe places for people to play it? Because the reality is that, yes, a product manager or designer is not going to have the same competency from the get go that some of us who've spent a decade doing this are.

Behzod:

But if you think about the play spaces that they're in, or in the case of some of the things I've talked about at Slack, we have account who are doing research before they go on site and train people on how to use Slack. And it's actually disrespectful of me to our customers to see that different account managers are asking the same questions in different ways and are trying to compare results to better serve our customers.

Behzod:

It's a hundred percent worth my effort and my team's effort to work with them, to standardize those questions, think about how to be rigorous instruction, what they're asking. And then all of our account teams can use that information. And the rest of the organization can learn in a more rigorous way. And I think to Rannie's point, doing that helps people understand why our skills are special and where they can be most valuably utilized.

Behzod:

And in many cases it wins us more resources because we're not coming in and saying, "Y'all are totally wrong and we're just going to steamroll you into your jobs." We're saying we have a specific lens on how you're trying to accomplish something, and we want to respectfully and responsibly play with you in that space to make those things better.

Behzod:

I think Roy actually talks about this a lot in his writing online of like, what are the right systems that we set up so that people can meaningfully and intentionally engage in this process and gather feedback in a way that they're doing so with respect to the customer, with respect to the business, and then they're not generalizing are extrapolating that into thinking that they've boil the ocean or have some magical learning about how has app your customer works.

Behzod:

It's a very narrowly defined problem. And I think the conversation that I want us to continue pushing on, and the reason I keep talking about this is, it's not about it. It's really about how and when, and I think there is that responsibility that we have because these things are happening all around us, whether we want to fight them or not.

Roy:

And I'd love to piggyback on that because I think the there's I think a difference between what we hope and what we think versus what's really happening. Product managers, designers, marketers, they're really making decisions right now about what to do next, whether you liked it or not because that's part of their job. They have to make decisions. They have to make progress.

Roy:

And so I think it's a question of, do we embrace the challenge of helping them learn in a better way and progressively make more customer centered, user centered decisions? Or do we only embrace one? Do we only embrace one method of doing that, which is train up your XRS and they're going to be a filter through which information process. In reality, it sounds great. It just doesn't work.

Roy:

So in theory it sounds great in reality, it doesn't work. So I think that's the challenge that folks on the ground face, that people are making decisions all the time. Do I help that or do I shrink back and say, "This is the only avenue through which we can help." And there's something that we're facing at Zapier right now, where we haven't got to a stage where...

Roy:

We're really happy, lots of people are interacting with users. But we need more guidance and more rails there. Because something else we've seen is, even someone saying that they have a research background, what does that mean? Just because someone did use the research as a PM and another company, it doesn't mean they started that by themselves, just speaking to customers.

Roy:

It could be different to someone who actually went through a UXR training course. So you want to help, but I think the reality is that these decisions are being made, are we going to embrace that challenge of helping or are we going to swim in the UX lane? And that isn't, I think what Molly is fully saying, but I think it's worth talking about the reality of what's going on. And I think that's how I view this challenge.

Michael:

I want to build on that Roy and just say, I teach MBAs on an innovation program at Kellogg down the road. And so I'm sure we can all say, and I certainly can say in my experience, it's not the same thing as researchers doing research, MBA students doing interviews. So I think there is a difference.

Michael:

And one of the ways I've tried to square that difference, which I think is may or may not be helpful for people is to try to separate what I would call the craft or the experts research from a liberal art. Liberal arts were invented as a Greek idea of the tools that all citizens should know. And I think there's an interesting question about whether corporate citizens are designed citizens, whether they do need to know certain things about how to interact with customers and get that feedback in a way that's more rigorous.

Michael:

But that's really different than an undergrad English writer and writing a paper, super different than PhD writing a dissertation who's been deeply steeped and trained in methodology. And I guess back to maybe a question related to this, and I think we'll take it in two parts. One is about training, because we've had a lot of questions about what's good training and what does that look like?

Michael:

I think another one, it ties a little bit to this issue of back to this issue of focus and value, which is if you're expert researchers, it's that sense of are your expert researchers doing work that count when you compare it, are they doing work that is far better than the liberal arts undergrad in this case?

Michael:

And this question of where, where the value is in the field. So I don't know if anybody wants to try to take that one on or whether you guys want to hop right into what would be part of that liberal arts curriculum? I think those are two places that I think of the crowd would be interested in.

Ben:

And to add on briefly, my Michael's point is echoing something Sean said in the chat, quote, "Should engineers convince and persuade their leadership they need to write code, should accounts persuade and convince the people around them they need to use math?" And Michael and I were talking about a Photoshops, democratization of design and even Excels' democratization of accounting.

Ben:

When we were brought into organizations where we expected to be able to craft a basic presentation or go through a set of numbers and make sense of them, to Michael's point, is there a Corpus of research skills that we could all say we would want in the liberal art of UXR?

Behzod:

I'm conflicted about bringing up this example, but I think one of the things that the five of us talked about in preparing for this, and it's unrelated thing was, data science and analytics being a corollary here where a number of organizations I've been a part of and friends I've talked to, their companies have a SQL one Oh one or some intro to analytics that in the same vein of what we're talking about helps people get answers.

Behzod:

Where you can learn the syntax of SQL and you learn what are some of the canonical tables that people referenced and you by no means are expected to build models or dashboards or anything like that. But you're saying, when you need to understand how many people in the U.S. do X, Y or Z thing, there are a very simple set of commands that you can use in a safe pipeline, or that has been built for this explicit purpose.

Behzod:

And doing that actually helps me do my job better because instead of asking an analyst to go and do something that is incredibly low level for them, I'm able to get a quick answer and then continue on with my work. And so what I would push people to think about is not this conversation of like, are we giving away the core responsibility, but what are the ways, I think Rannie said this earlier, that we can invite people to participate in parts of that process that are meaningful and safe and valuable for them to understand.

Behzod:

I run some SQL queries and I am never going to build models. This is so far outside of my work. And now I need to have time and space to work with a data scientist to say, "Hey, here's what I'm trying to understand. Here's some of the things let's work back and forth to understand what the right way is." I'm not going to go to them and say, "I need this tomorrow," because I have no idea whether that's a reasonable request and doing that query actually helped me gain the understanding of what I was asking.

Rannie:

Yeah, there's this-

Behzod:

Yeah.

Rannie:

... Go ahead, Bezat.

Behzod:

No, you go for it.

Rannie:

There's this aspect of relying on our strengths and owning it. In having these conversations, I often hear from researchers about this insecurity that lives on is like, "Am I going to be replaced? By teaching other people this skill, am I still going to be valuable the organization?" I think that's what we're really tiptoeing around.

Rannie:

And I think taking a step back and saying, "Oh, what are our unique skillsets that we bring to the organization that we have had such so much time and training and thoughtfulness about that is unique aspect to our team?" And one of those is we're really comfortable with ambiguity as we tend to be like, "Ooh, a mess. Ooh, do I dive in?" And we want to scope down the problem. "What is the question? Okay, what's the question to the question? Okay. What, why, how do you get there?"

Rannie:

And then we get to the core. And we apply that to our research questions in our studies, but these are such transferable skills over to our teams where we can say, "Hey, what's the goal here? How are we thinking about this with the user pain point? How does this fit in and the overall roadmap, could you help me understand?" These are open ended questions that I'm asking in a way that's neutral, it's trying to triangulate with other teams other functions.

Rannie:

Where we can say, "Hey, I heard this thing from design, but I also heard this other thing from PM. And then here's the data that I'm seeing from the analytics team. How are these pieces playing together?" And then if there isn't a solid answer, which often there isn't, it becomes our role as a neutral researcher to play a bit of a matchmaker and also to fill in the gaps and say like, "Okay, it seems this is where the biggest gap is, and let's go in and address that." And that's where we can conduct a research.

Rannie:

So I do think that there's a lot of tying our unique skills and our training and educating our partners like, "Hey, here's how to use me. Here's how to use us. Here's where we haven't had a skillset before, but here's where we have it today. And will we have even more of this if you hire more researchers who have this type of skillset. A quantitative researcher is different than a market researcher. A market researcher is different than a qualitative one ethnographer."

Rannie:

And part of that is they don't know what they don't know and it's part of our responsibility to help educate where those gaps need to be filled and how to fill them.

Behzod:

Roy, you look you're going to say something?

Roy:

No, you go for it, man.

Behzod:

Okay. I have shared context with Rannie from our time at Facebook. So I think to answer this question about the liberal arts curriculum, one of the things that we used to do on the ads team and I think that that Rannie could speak to on the consumer side was, we would very often take product teams into the field.

Behzod:

And the first thing that we would have them do, even before they came to the lab or into the field, just have a conversation about what bias looks and how we would ask questions and how to engage and to listen meaningfully in these conversations. And then it would be how to engage meaningfully and to ask questions. And then it might be something like, "Okay, we're actually going to let two or three of you help co-moderator moderate, because there are specific things that you want to do."

Behzod:

And it was very much a crawl, walk, run, approach of saying, "What is this you're about to experience and how can I help you listen to it so that you don't sit in on one interview here, one piece of feedback and totally pivot a product strategy?" And that became then a conversation about, "Okay, so what is it that you would want to learn about?"

Behzod:

How do we deconstruct that into things that people can actually answer the same way that you think about how to break apart a survey question into things that are very discreet? And I think all of those working through the layers of what we've learned and what we've developed, that skills, are really important because the orientation that we bring, to Rannie's point is this idea of how to often watch and listen and observe and synthesize and make sense of.

Behzod:

And each of those things, there is an ordering process for understanding what that looks like, because you don't want people to just be a direct translation of, "I heard the customer say this, so I'm going to go do that."

Behzod:

Because customers have very often, less than 1% of an understanding of what your organization is capable of or strategy or whatnot. And so you have to help people that we're working with make sense of that process. I don't know if that was true when y'all did some of the emergent trips on the consumer side.

Rannie:

Yeah, yeah, absolutely. I can give an example. So here's an example. We were conducting research on the personalization of categories with trending news back at Facebook when trending news is still a thing. And I was working with a PM and I was like, "Okay. So the question is about, what are meaningful buckets of content for the user, for the consumer?"

Rannie:

And he's like, "Yeah, that's what we're looking for. What are those buckets? What should those buckets be?" And they're like, "Great, okay. I don't need to do any design knocks. We need anything. We're just going to do a card sorting exercise." And in his mind, he was like, "Wait, you're not going to show them the product?" I'm like, "Nope, we're not going to show them the product." I was like, "We're going to strip out the UI.

Rannie:

We're just going to focus on those content categories. We're going to talk about meaningful buckets of content. And I'll be able to translate those findings into actual insights for the product." And in his mind, he was very skeptical about this approach and raise lots of concerns. And we had to have a conversation about like, "Here's what you're going to look for when we have the sessions."

Rannie:

And after the first one, he's still skeptical, but after the second and third who was completely onboard and could completely understand exactly why we were taking this approach. So yes, I could easily teach that to a PM or designer how to go and conduct that. But I think the heuristics that we have as researchers in terms of how to approach a complex or complicated problem space, we have that tool kit. That's a lot more accessible than those without proper... not proper, but more sophisticated research training.

Michael:

Maybe Rannie, you could talk a little bit about what the training looks at Thumbtack. I know, obviously you've done a lot of work on training and training regimens, probably been through it a few times. So I'm curious, what the evolution looks like, where your training looks now for non-researchers to participate in democratization.

Rannie:

Yeah, this is credit and kudos to the research team over at Thumbtack. They were running with this before I even joined. So Cordelia, Jordan, Eric, [inaudible 00:32:12]. And it's part of that was, they were seeing a need to scale themselves as they were a small, but mighty team of three researchers.

Rannie:

And they were getting, to Bezat's point, an endless amount of questions. And there were only three of them within the organization. And having to figure out like, "How do we focus on the most complex questions, the most strategic questions, and uniquely use our skillsets towards those and prioritize those bigger ones that can have larger amounts of impact at the same time, without blocking cross functional partners for the smaller tactical, quick ask that they needed answered?"

Rannie:

So in terms of training, they held workshops and they invited non-researchers and they could also tailor it depending on the audience. There could be one for customer sales and success managers to ask really unbiased questions for, how has your experience? Versus you seem frustrated with that. It's, "How are you feeling about that new feature?"

Rannie:

It's even just reframing questions. And then for our non-researchers more on the product design side, having very clear questions and protocols, role-playing is another thing that we did is just seeing how, "Okay. Here's how to answer your question.

Rannie:

Let's try to talk about a different scenario or a feature. How would you ask a participant about that?" And we talked through best practices. And we're still seeing the fruits of that labor, and we're seeing a lot more respect for our research organization too, as to how we approach problems.

Michael:

Roy you're in an earlier incarnation at your new company. I'm curious, are you doing training? Are you feeling the need for training? How has your training evolved in Zapier?

Roy:

Yeah, right in the middle of getting things up and running on that side. So first when I came into introduced this thing called, all-hands research ride-alongs, so anyone in the company can jump in into a research call and just observe. So they're hidden from the participant and just an easy way for people to just get involved.

Roy:

And so in kicking that off, we have UX, all one-on-one training, which is just intro to research. And we also do an observer training. We use a tool called Lessonly for some of our async type training. So we have that in depth. So folks can learn how to observe sessions. Because there's also parts of the research that are very normal to us like jumping in on a call is a really big deal.

Roy:

There's some people who are just very arrested to jump into this thing. So even to get to some training on how to do that is really important. And then right now we're putting together modules the way it works for us, we have decided to do it in modules. A usability testing is pretty big in our company, a few different types.

Roy:

Interviews is probably even bigger at our company. So putting in place modules to train these folks in these methods. So that's a very active thing right now, because what we've seen is I think what everyone else has observed at different different folks come in with different levels of experience. And so that training is really important so that we know where everyone sits.

Roy:

The way I'm approaching this is, I have this picture of freeways and toll booth, tall arrays. On the freeway, you can drive as fast as you're allowed to. There's more freedom there, but they all guidelines, but then you eventually have to slow down and pay something. You have to invest something to get to proceed.

Roy:

And so we're getting to a spot of learning who is at what point, how far along are you? Do you have to go through three toll booths to get over there? Or you called to proceed? So we're writing that-

Michael:

[crosstalk 00:36:19] to your metaphor here. I feel like there could be provisional student driver, maybe he can-

Roy:

[crosstalk 00:36:26].

Michael:

... [crosstalk 00:36:26] toll booth set up, I believe at Thumbtack and Bezat, I'm sure you went through this at Slack. I'm curious if there's any particular wisdom you have about how to create that kind of sense of, people can do things, but where you put in meaningful controls or check.

Rannie:

Yeah. So we have two things at Thumbtack that I think has been helpful for us to be these traffic control cops of like, "Oh, that big question, are we're going to put that over here?" Or, "I think we can break it down a bit." So one is, we hold research office hours and that allows for folks on our research team, to meet with those within the organization who have these maybe smaller types of quick asks around like, "Hey, can you give me some advice on this question?"

Rannie:

Or, "I'm thinking about launching this survey, could I get some feedback on it?" The other is a bit more intentional about who and how, and when folks can speak with participants or survey participants. So within our participant recruiting practice, we have a research ops person. Her name is Maya, she's amazing. She basically gets all of the requests through a form that, hey, for anybody who wants to talk or interact with participants, they have to go through this form.

Rannie:

And that allows us to know who is conducting research and interacting with folks using our platform. And that doesn't mean it's a red light. It doesn't mean it is a hard block. Instead we describe it as a beaded curtain. You think a walkthrough, but you still got to walk through it. And it allows us to be this checkpoint of saying, "Okay, how many participants? Wow. A thousand looks pretty aggressive. Why a thousand?

Rannie:

Let's talk about that." "Okay. Eight?" "Okay. That sounds great. What about alternatives? Are you thinking about alternatives? What's the timeframe here? Oh, you're doing three checks, three surveys, three checkpoints?" We'll talk about why that's necessary. "Can you just get away with one? It dilutes the population pool of survey respondents. So let's talk about that."

Rannie:

And it becomes one, a checkpoint for us at a moment for us to educate on some of the intentionality behind the decision making and then three, it's just a building the muscle and the relationship ongoing. And it's a good practice overall for the company to spend money wisely. And also for them to realize some of these [inaudible 00:39:07] that have to be made with research protocols.

Behzod:

Yeah. I can't co-sign office hours or that visibility enough. At Slack, I actually had complete oversight and control over who got to use Qualtrics and they had to-

Rannie:

That's amazing.

Behzod:

... they had to read terms of use that I wrote that said, "You're not going to do a bunch of these things or else I'm going to take your access away," agree to it, and then I would grant them an account. But we also had office hours for exactly that reason because the entire point was to help make visible parts of our practice that are very often invisible.

Behzod:

And we were really lucky to have folks who had in prior worlds been doing research in some capacity. And so we were able to say, "Oh, look, you actually can write a really beautiful discussion guide. And you're an incredible moderator. You just need to think more structured about recruiting. Let's help you do that. And in fact, great, we've templatized all of these things, let's help you.

Behzod:

I'll check off on your pole trick screener, and then I can make sure to help you with scheduling and whatnot." But like, I think acknowledging that there are different parts of this process that are going to be people's strengths or weaknesses. Justin, freckled and Lena Black. Gosh, I don't remember his last name, at Asurion actually wrote this great piece about thinking about research as cooking and the importance of teaching people, kitchen skills.

Behzod:

You need to learn knife skills and how much you need to cook a chicken too before you can serve anyone a meal. And so there are these foundational pieces, I think that we often, to Roy's point take for granted that they're implicit our work and even just the act of saying, "Okay, we're going to have office hours. We're going to talk about why is it that you want a thousand people or you want to have enough confidence?

Behzod:

Actually let's have a conversation about what confidence and conviction looks in research. You don't need a thousand people." That I think for us helps at Slack, at least designed what the curriculum was that we wanted to start teaching more regularly because it was the conversations that we were having over and over again that we realized we should actually just standardize documents and scale.

Michael:

That's interesting. I think it's this question of where the expertise shifts in the field as a lot of what we're talking about. And I think many of the questions people have are tied to this. So in a couple of them, one, you can see how I think the work, Bezat's talking about is how his expertise can scale across many different studies or a lot of work people are doing without doing the work itself.

Michael:

So that's one example of how you might get that scale. There is a challenge still of for core researchers, democratization might not be right for your company and where it is. I think we started with those most mentions, but what should be clear in my opinion is that, if you're a card carrying research practitioner. Your work should be a lot better than my business school students that do a couple interviews.

Michael:

And by better, it means it has much more impact, it's being able to look at different questions and get too much more explanatory power. And so I think that inherently raises some of these existential questions that are hard to avoid around our field when we bring this topic up. Roy, did you have something you wanted to add?

Roy:

Yeah. I was going to add on the side of, talking about training and guidelines here and rails, there is a risk involved in democratizing research. You can get to a point where anyone and everyone is doing the thing and you run the risk of really poor participant experience, bad decisions and budget.

Roy:

On the point of like, you want a thousand people for your research, this is legit. I have seen folks who have run a survey, not in my current company. However, lots of, in doing consulting, who have run a survey with tens of thousands of folks recruited. And I've no idea why they recruited. So you have burned through budget because you didn't know.

Roy:

There's risk complying with the law. People who don't know this stuff will just go out and speak to people. So I think there's the important part of going. We encouraged us and you want this to happen. It happens differently in different contexts. But also think about the risk. You need to be careful about this so that you don't end up in a place where you're burning through money.

Roy:

You're creating really bad participant experience or worse. So I think that's a point that's worth throwing out there because the last thing I'd want is someone to just run out here and just say, "Open flood gates." And before you know it, you have no budget and you're in trouble because something happened to someone. So, yeah. Just wanted to mention that.

Ben:

And I think that leads-

Behzod:

[crosstalk 00:43:57].

Ben:

... Pardon me, Bezat, go ahead.

Behzod:

I was going to say, Roy again, continues to do this very eloquently, but I think similar to how we don't spend a lot of time talking about the problem, I think a lot of us take the fact that we could or should do research for granted without stopping to acknowledge that all research has a cost and it's a very significant cost.

Behzod:

It costs you time and energy to plan and do, it costs your participants time. It costs you energy and resources to gather, store, process cleanse and make sense of data. None of this is free and you shouldn't be doing these things if you don't have a good reason. And so the risk in doing this is that you are exploiting, wasting, creating something that is really problematic.

Behzod:

And I think just as you think about, who you're helping or what a problem is, you should be asking yourself, "Is research worth doing?" And I think for Slack, it was very easy early on for us to focus on the things that were going to have the longest shelf life would be the most impactful to the organization. And we realized that saying no to some of these things that we've later democratized or because they were okay and safe and worth doing.

Behzod:

And I think that that is a really another aspect of our practice that is very often under discussed because we just assume that we are researchers and so we are going to go out and research. And the reality is that there are some things that are not worth researching at this moment in time, in this way.

Ben:

With the closing time we have, we've talked a lot about structures and what democratization might look like, and there's a lot of good questions about what does the ideal democratized organization look like, which I think is an interesting question. Which Mike, is it just that we have... Mike, when I've talked about this, if given the option, would you just hire a thousand new XRS or is democratization a response okay?

Ben:

Bezat, you're saying that. I'm curious then, what's the ideal democratized... I'm using it as an end goal, but let's say you're getting very close to... you're further along in the democratized scale. I don't know if we can say it's a switch, it strikes me as a continuum. So what does it further along look like, Bezat? It's not hiring a thousand new XRS.

Behzod:

No. And I said this, new XRS, I think it's both a bad goal. I think it's also impossible. I think that, to Rannie's point, if you look at what we bring to the table, I think we are people who bring rigor and structure to a decision making process. And the end goal of democratization is to have a healthy organization that is learning in ways that are appropriate and respectful of the people that it's trying to serve.

Behzod:

And you're not sending out 10,000 person surveys because you just want to see what things look like, or you're getting usability feedback on every feature, because you don't have any sense of historical data or intuition or understanding of what a best practice is. I think that the balance that you want to strike is that your organization is healthy, looking inward and looking outward inappropriate ways.

Behzod:

And you have mechanisms to do that safely and respectfully. And I think when people hear me say that they're like, they're very on board. And when they hear me say, "Let's democratize," some of the pitchforks come out, but I don't think that all of us actually disagree on... maybe I'm wrong. I think many people agree on that end goal.

Behzod:

I think the journey to get there as hard and it's different, it's going to look very different for different companies, in different spaces, in different sizes. But all of us, I think, would to be part of healthy teams that are working well together in service of whatever we're doing. And I think democratization is a means to getting to that point.

Roy:

I feel Rannie touched on this earlier. More researchers, even five more researchers is not going to solve the problem.

Rannie:

You may recall if you could.

Roy:

For real. If that's what you think a remedy is, I think you haven't defined the problem well. You likely. There's this tiny, tiny [inaudible 00:47:54]. But likely if you think you need to find more researches to get research done better, you're probably not tackling the core problem.

Ben:

And Roy, you talked about in your recent presentation how you should spend time if you're a small team thinking about the research the operations of research. And so that leads me to a question I want to end with each of you, what is something that you would advise someone out there, whether they're at a great big org that has a lot of established silos and structures or they're in a smaller, maybe even one or two person situation?

Ben:

What is something that they can do next week to get themselves primed? Because it sounds there's a lot of forethought that, again, as researchers we're used to doing, what is something that they can be doing to get themselves in their org, on the road to democratize or determine if it's something that they should do?

Rannie:

Yeah. I can kick us off here. Define the Sandbox is what immediately came to mind. And what I mean by that is, understand, what are you working with? And it goes back to the problem definition of what constraints do you have, what constraints don't you have, what are the problems you're facing?

Rannie:

Let's prioritize those lists of problems and then really get crisp on that prioritization list and say, "Okay, what's must have nice to have, could live without, for the nice to have and could live without. We're not going to discuss that right now. Let's really focus on the must have. And how do we sequence this in a way where maybe you have one to two researchers on your team.

Rannie:

How do they focus on the big questions? What's going to take longer? What must we get 90% right? Maybe that's the core consumer segments. Maybe that's the largest pain point in Q3 or Q4, whatever that is. And then maybe for those smaller tactical questions, you say, "We can get this 60% right. We'll experiment along the way." It doesn't have to be a full researcher dedicated to this.

Rannie:

And then once you define that Sandbox, what exactly you're playing within, then you can build your castle. A need to use an analogy. But if you could build that castle and feel really great about that foundation, you can build more effectively on top of that. That's my advice.

Roy:

Yes. Somewhat close to that as well on my side. I think I advise UXR team of ones to understand the business. If you don't understand how this company makes money, then you don't understand how you can add value. And that's really important. It's where you add value. So I'd say, I'm fortunate that I work with an incredible head of research named Jane Davis.

Roy:

We work really closely together and she understands this stuff and we have conversations about this. Understand how the company makes money, understand how product works. Because if you work in a SAS company, product engineering is the heartbeat of everything. So you certainly don't understand how things work then, relationships you can build so that you can understand that well.

Roy:

Because if you don't have that context, then you're just firing blindly. You don't actually understand what the intersection points off for you to add value. And as Rannie, you're saying there, then make further decisions about where you're going to focus on and not. I'm really surprised by how many researchers I talked to who have zero idea about how product make decisions.

Roy:

What are the leavers that allow a company to make more money? That sort of thing. You have to understand the business, you have to understand how decisions are made for the upstream.

Ben:

I'm struck by how little-

Michael:

[crosstalk 00:51:57] underlying, I think there's a big tendency in the field to begin with of people not just learning the core functions of the business. Money sometimes as unseemingly part of our field. We don't like talking about it. We don't like mentioning it, but it is obviously a critical part of what we're trying to do. And it's an important way for us to translate to.

Michael:

Especially when doing strategic work, the work is more effective and it's more impactful. If we can translate our work out into the organization versus asking people to adopt all of our teaching on our principles or something in the field about being able to go the other way as well, which I think, I see often researchers somewhat reticent to do.

Ben:

Yeah. Before we close-

Behzod:

[crosstalk 00:52:44].

Ben:

... just briefly, Bezat, it's always struck me as interesting when our customers and discounters and folks like y'all are, espousing rightfully empathy for the customer, and don't often just shift that empathy toward their stakeholders. Folks who might not quote no research and the impact that can have happen. Bezat, take us after.

Behzod:

Yeah. I don't know if Rannie, you remember this, but I used to give a presentation at Facebook on how to present research. And I had the slide where it was methods, and it literally says, "In your appendix," because you are brought on very often as an expert. And the only people who want to know about your methods are people who want to replicate them or disagree with you.

Behzod:

And so the important thing about sharing, learning and helping drive decisions is doing it in a way that makes sense to the business. And I think both underline everything that Rannie and Roy said here, most people don't understand how their institutions work and they've under invested in what the organization... how they make decisions, how they preserve knowledge, how any of that works.

Behzod:

And what's the interplay between a lot of their partner to use. And they come in with a, "I know how to do X and I'm going to do X because that's what I was hired to do." Most of us need a much more creative and holistic view on our roles as researchers, not as people who talk to customers or do surveys. We help people make better decisions.

Behzod:

And there are infinite ways for you to do that in an organization, do the one that works best for you. And it's okay if that doesn't work for other people, because they're in different companies at different sizes, working on different things.

Rannie:

We tend to think of research as this external research that we can doubt, but there's internal research that we can bring to our organizations as well.

Ben:

And all of that goes on to socializing what we do and furthering jobs and the impact that we can have. Bezat, Roy, Rannie, Michael, thank you so much for some time. To all the attendees who stuck around, the 136, who're still there, thank you very much. Against, stay well. We hope to see you-

Rannie:

Thank you.

Ben:

... on another one, our next time.

Behzod:

Yeah, Thank you so much, Tom for all this.

Rannie:

Thank you.

Roy:

Yes, everyone.