From Out There to In Here: Turning the UX Lens Inward

User experience methods have been used to improve experiences, products, and services far and wide. How might they be used to improve internal experiences, like employee onboarding, work tool management and access, and even preferred meeting cadences? This panel of researchers begins to tackle these questions—sharing learnings from their own projects.

Specifically, this panel explores:

• Examples of "internal" UX research from each panelists organization
• Nuances of conducting this work compared to "external" versions
• The impact of sharing internal UX findings with teammates
• How conducting internal UX research can improve one's broader practice

Transcript

Katie Masciopinto:
And today we'll be talking about how to bring research inside of your organization. How can we optimize and adapt your research, both with and for internal audiences such as your stakeholders and collaborators, the folks who you see in the virtual and physical hallways of your organization each day. And today, we are so lucky to be joined by three fantastic panelists, Taylor, Angeline, and Lindsay. Thank you all so much for being here. I'm really looking forward to hear from you and learn from your experiences within this space. Now each of you has conducted original internal research within your organization. And so before we dive into the questions at hand, I would love if each of you could take a moment or two and introduce yourself, your role, and tell us a little bit about your journey with internal research up to this point. Angeline, if you don't mind, let's start with you.

Angeline Vu:
Yeah, totally. Hey, I'm Angeline. I am an experience researcher at Thumbtack. Normally I don't do internal research. I'm very much a consumer facing researcher that works with users within our marketplace, but I was given an opportunity to partner with our chief of staff and people analytics team at Thumbtack on a really cool project all around the future of work. So when Thumbtack pivoted to a virtual model after the pandemic, our team has been really committed into understanding what does it actually mean to work in a virtual first environment. And so as a UX researcher, I was really able to help partner with them and help them understand what a day in the life of a virtual employee looks like, how that feels, and kind of just develop more empathy for what that looks like.

Katie Masciopinto:
Awesome. Thank you Angeline. I know we'll talk about that diary study in more detail. Lindsay, I'll go ahead and pass it your way.

Linsey Feldman:
Hi, I'm Lindsay Feldman and I'm a user experience researcher at World Wide Technology. So like Angeline, I've done research with my fellow World Wide Technology employees, but that's not the main type of research I do. WWT is a, I work in a consulting capacity, so most of the research I do is actually for clients and with their own employees to help them build the better tools to help their employees work more efficiently and productively.

Katie Masciopinto:
Thank you Lindsey. And yes, I'm excited for the folks at home to hear a little bit more about those nuances of doing work in a consulting environment a bit later during our discussion. Taylor, if you wouldn't mind rounding this out.

Taylor Kim:
Sure. Hey everyone, I'm Taylor Kim and I am a senior UXR at Shopify. And the product that I work on is not Shopify the software, but Shopify the work environment. So there are some differences there and a lot of similarities. I'm excited to get into that. But just in the nutshell, the easiest way to think about the work I do is making sure that our users, which are employees in this case, so my colleagues, have a really fantastic experience. And we do that by taking the exact same product thinking we would take to when we're building products and we bring that right into when we build HR or talent specific products.

Katie Masciopinto:
Awesome. Thank you so much Taylor. I think that that's a really good segue into our first question because I'm guessing that there are folks who are dialing in, some who may have run internal research before within their organization, some who this might be a new style of research that they're interested in exploring in more detail. So I'm curious if you each could share what is a nuance or consideration that you take into mind when you're conducting this type of internal research, and what drives you to conduct internal research within your organization? If you wouldn't mind, Taylor, I'll go ahead and start with you.

Taylor Kim:
Sure. So I'll start with your last point there of what drives me to do this kind of work. And I think, to me, it really boils down to congruency. So I want to make sure that all our employees, our colleagues, spend a lot of blood, sweat, and tears, making sure that the user experience, like our client experience, our merchant experience is fantastic. And then we want to make sure there's congruency in what they experience in meaning of what they get back from the organization also has that same level of thought and care. So I think that's what drives me.

And then in terms of the different nuances, two really come to mind. So the first one is around recruitment. When I first started this work, because I used to do a lot of customer-facing research, and then I came into Shopify and I did a lot of employee-facing research, and then the recruitment phase definitely shrunk in my research plan. And at first I was like, oh, this is so awesome. This is so great. I'm saving so much time. I can just Slack anyone and be like, "Hey, want to be part of a study?"

And then I realized actually the dangers of that, having your users be so accessible and so close by. It means your bar for ethics has to be a lot higher because unlike, I've used screeners before when recruiting customer for a specific product and you only kind of know about the customer what you ask in the screener, like whatever your parameters are. But with employee-facing research, I am privy to lots of data sets that actually might not be relevant to my particular research study. And I have to be very mindful of that. Right? Because I do have sensitive information around somebody's tenure, like geographic location, their performance rating, compensation.

And it's like depending on what the product is that I'm studying, that stuff might not be relevant. So making sure that even though it's a lot easier to be speedy and cut corners during recruitment, making sure you have a high bar of ethics when you're doing it. And so recruitment's number one. And the second piece I want to touch on is basically the pressure around sharing back the insights.

So I think that is a lot more complex and there's a higher bar and higher responsibility to make sure you're closing the loop with your participants. So we can get more into this later about how we do this, but basically it boils down to the fact that whoever your participants are, you will see them again, right, in the virtual hallways or a real hallway. And so you can't not, not give somebody an outcome of what happened with their time, because you just took away so much of their time. They gave so much time to their research, you have a responsibility to loop back with them.

Katie Masciopinto:
Yeah, absolutely, Taylor. And Angeline, I know when we last talked, you spoke about how personal the data is that you're collecting, that in your work you were capturing selfie style videos of parents working with their children during their lunchtime routine. So I'm curious to hear how Taylor's experience, being really sensitive in the way that you're capturing that data, how that resonates with the work that you did.

Angeline Vu:
Yeah, I very much share that similar line of thought around having strong ethics in the way that we do recruits and having more upfront communication with participants about what that looks like. I think when we think about external participants, it's very minimal in the information that we give them with what we're doing with their data. We usually just say, "This is going to be used to improve the product experience," which is true. But with internal participants, the stakes are so much higher because the information is much more sensitive and much more confidential. And especially with this particular study where we were essentially asking them to invite us into their home, into their personal lives and share really candid snippets, that information is something that I really wanted to make sure they knew that I was protecting. It wasn't going to be shared out with anyone else but me in terms of the raw data.

And so upfront communication, even when we were doing the recruit and having clarity around, okay, what is our research goal? How do we plan to use this information? What can we anonymize? What can we possibly not anonymize? How do we go about sharing what information isn't going to be anonymized were all questions that I preemptively thought about so that I could share that more openly to kind of quell any concerns upfront. Those are things I would never do with external participants because I don't need to. And so there's just so much more legwork that goes into the beginning of the process.

Katie Masciopinto:
Yeah, absolutely. And I think that reminds me of, Lindsay, I know you talked about in your work how often when you're conducting one-to-one interviews with internal participants, that often you're navigating the very real human emotion of fear of people wondering, will I lose my job? How will this data be used? I know Taylor, you've discussed the power dynamic that exists between a researcher and a participant. So I'm curious if you would be willing to dive in a little bit more and think about what are the steps that you take during the research process to calm some of those nerves of the participants at the forefront of the research?

Linsey Feldman:
Well, in my experience working with clients, and oftentimes the projects I work on are digitalizing existing paper processes. So it's really important to understand the job and the tasks that the participants are doing. But I've found, in my experience, that often makes people very nervous that their job is going to go away with this new and improved system that we are helping build. So it's really important to help the participants know that by participating in this research, their job's not going to go away, but instead we're going to improve their job and help them spend more time on the parts of their job that give them joy and less time on the tedious paperwork aspects of it.

Katie Masciopinto:
Yeah, and I think that that also brings up the question of how do you then share back the insights? Because I think that's something here at dscout, we see oftentimes participants want to know how is this research being used, how is my voice and perspective being embedded into the products or the services that I'm sharing feedback on? And I know you all touched on that this is even more prevalent when you're working with those internal participants, people who are your colleagues, people who you may consider friends. So I'm curious how you all design your share-outs for both the stakeholders who are along for the journey, who want to learn from this feedback, as well as for the people who actively participated in the research itself.

Taylor Kim:
I can start and then maybe my colleagues here have something they can add on. But what I have learned as I matured as a researcher is the importance of looping back with the participants. And I think earlier on in my journey, this felt so daunting because I'm like, "Oh, I'm still in synthesis. I'm not ready to share out yet." But I realized they're not, what people are hungry for isn't like a long lengthy thing. They just want some signal that their time mattered and that you're doing something with the data. So what we've actually designed here at Shopify is at two points in the research cycle we share back. So right after the research wraps, it's literally an email template and we just talk. There's obviously a too long, didn't read couple sentences, like what we learned from the research, and it's literally what we learned from the research, couple bullet points, and then what we're doing next, a few bullet points.

And then that gets, obviously all the findings are anonymized and, what do you call it, so that any personal information's not there, but just high level findings of what we learned and what we're doing with it. And then at Shopify, we're lucky enough to have a pretty open culture where we can link out to specific people, "Here. Here are the people working on the project. Follow this link and you'll get updates on the project." So stuff like that really helps, but it's literally, takes five minutes to write this really simple email. So right after the research wraps, that's number one.

And then number two, at the second point when we share back, is whenever we call them shipping emails. So whenever we develop a new product or a program or service improvement, whatever it is, when something's changing and we have to notify our users, it's literally like, this is what's changing. And then we have why we built this. And then the research insights, again, are referenced over there. So then as our users or employees, they can all look at it and be like, "Oh, okay, so this was a data-informed decision." And if they were like, "Hey, I participated in that research, I feel like I was really able to shape this product offering," That goes a long way.

Linsey Feldman:
In my experience working with a variety of different clients, not every company has the same level of transparency. So while I would love to share a 10 bulletin, 10 bullets of we learned this, this, and this in our recent research study, I'm not able to do that with every client I'd like to, which is unfortunate.

Angeline Vu:
Yeah, the last time we chatted, I made notes to myself about everything Taylor said because I was like, "Oh, I need to do that the next time I do internal research." Because this was my first time, I think that's actually not something that was super top of mind for me because I was so focused on just the upfront communication and making sure that I did this work right. And so definitely something to keep in mind for the next time.

In terms of how we shared out these insights with our senior leadership team, which is who this work was really intended for, was we partnered with our people analytics team and really took a mixed methods approach to creating that deck. I think many tech organizations lead with data, not that qualitative data isn't data, but I should say they lead with quant data. And so it was really important for us to marry those together and make sure that the story was first rooted in data. And then I always like to think of qualitative data is bringing the more quant data to life, and giving it more color, and building more empathy into that story. And in terms of how we shared that out with participants, I haven't yet. So I think I'm taking notes and we'll definitely figure out a way to prioritize that after I get off this call.

Katie Masciopinto:
So thank you, Angeline. I love that. We're doing research in real time. We're learning from each other. I think one thread I want to pull on is this idea of empathy, because I think that you each mentioned in our last discussion how, because these are people who you're working with every day, you know them very well, that it's really easy to build and feel empathy for these end users. I was also struck by how you all talked about how it also translated to the research that you may run with external participants. So I'd love to ask you, how does internal research power empathy within organizations, and what is the risk that organizations run by not running internal research?

Taylor Kim:
I think so.

Angeline Vu:
Go for it, Taylor.

Taylor Kim:
Okay. Really quickly, I [inaudible 00:15:09] on this engine thing, but I think just the risk that the organizations run by not doing internal research is that you might design poor products or products that don't have a specific need. So really getting that product market fit is something where research helps. It's just in the external research world, I think similarly here. And I think what we're seeing is with, especially in remote-first organizations where everyone's virtual, it might be just, it's so easy to think your experience is everyone else's experience just because it's a lot easier to be in your kind of own lane. And I think what research does is it excavates like, oh, someone's having this experience in that corner and someone's having this other experience over here. So you're able to see more of a fuller picture. And I think that's inherently really valuable. And Angeline.

Angeline Vu:
Yeah, I was going to make the same point around how the risk of not writing internal research is the same as the risk of not writing external, but it's never interpreted that way. Internal is always kind of treated like more optional. But when we think about the reason for external research, it's really costly. If you don't do external research, if you're making assumptions, you're shipping products, and then you're realizing that it doesn't meet user needs. The same thing is true for internal research.

If you're kind of just making internal decisions without really understanding how your internal folks feel, what they actually need, that can be costly in terms of turnover, employee dissatisfaction. And I think when we talk about how does internal research develop empathy, so it's almost a hard abstract question to answer. And so I'll give an anecdote from the study. So one of the entries in our mission was around taking a picture of whatever you did for lunch that day.

And I remember one gal posted a picture every single day of her computer desk. She never took a single lunch because she was in meetings every single day. And I just, my heart just went out for her when I saw that. And I was like, what a strong visual, something that would never be captured in the same way that it would be captured here. And I think even having that visual to show where it's like, hey, how do we think about making sure that people have healthy work-life balances? How do you not have empathy when you see that? And so it doesn't take a lot. It kind of just takes the right amount of information communicated in the right way.

Linsey Feldman:
And also to answer the question about risks, I think many times companies feel they don't need to do research with their internal, their own employees because they're like, "They're our employees. We know who they are." And they feel they don't need to do that. But with my many years of experience as a researcher, I've had the pleasure of doing internal research for different companies within the same industry, interviewing people of the same role across different companies. And it's shocking to find out how different they can be.

And also, people, companies think they know what their employees are doing, how they do a task or really how they should do a task, but by taking the time to actually interview and talk to their employees, you find out that people have these weird quirks and workarounds to find efficiencies in their own jobs. And the companies are shocked to find that out. They're shocked to find out their employees are making convoluted Excel spreadsheets to make their jobs easier.

Katie Masciopinto:
This curiosity is bubbling up here, and I'm going off script for just a moment, but Lindsay, hearing you talk about that, I am curious, what is the impetus for running internal research? So when somebody brings you in in a consulting capacity, I'm curious what are they sharing as the reason for running that internal research or the catalyst that makes companies realize that, "Hey, this is really something that we should be doing?"

Linsey Feldman:
Well, typically they'll have either a paper process or an existing tool that they're looking to upgrade. So they will consult with World Wide Technology in a software capacity, but we need to understand what the users of that software are going to be doing if we're going to build the right thing.

Katie Masciopinto:
Yeah, absolutely. And Taylor and Angeline, I'm curious from your perspective, what are you finding? What are the ways that people are thinking about as the kind of signs or the indicators of, "Hey, this is an internal research project that we should run?"

Taylor Kim:
For me, it always comes down to they already have a certain objective in mind, and usually it's like, we need to change this user's behavior. They need to adopt this tool or start doing a certain set of behaviors or stop doing another set of behaviors. So they have this outcome that they're very tied to, and then they're like, "Okay, but we need some research so that we can make sure the change management communications is really in support of that." Or maybe even the fact that, "Hey, maybe we need to question why do we want that certain outcome?" Because something about it, they tried it without research and it's not working, essentially. Right? So that's what comes up at my desk.

Angeline Vu:
Yeah, I think for this particular project, it's just such a nebulous space, like future of work, working in a virtual first environment. This is something that happened because of the pandemic, and because it's so nebulous, because it's so new, it was kind of more of a no-brainer for research where it was like, "We need you guys to help unpack what this actually means." Otherwise we're kind of just wading through uncharted territory.

Katie Masciopinto:
And I think that that also brings up the good point that each of you has run research in this remote hybrid environment. And I know, I think that I was really struck Taylor, by the way that you talked about the tension between people wanting autonomy and independence, but they also want belonging and they want connection. And I know Lindsay, you talked a little bit about how people want their teammates to be available and reachable, but maybe they themselves don't want to be as available or as reachable. And so I think that it would be really helpful if you all could pull the curtain back a little bit and share one learning that you took away from these projects, or one light bulb moment that you all could share with the folks at home.

Linsey Feldman:
Yeah, I mean, going on what you said, people have different personalities and it really drives how they work from home. I mean, obviously we've all heard the terms introvert and extrovert, but that takes a whole new level when you're thinking about working from home where people who are traditionally more introverted, they want time away from their cameras, time away from Zoom meetings to actually get to work, versus the more extroverted people. They miss being in the office. They miss being face to face with people and they can't wait to get back.

Taylor Kim:
Yeah. Definitely certain personality markers or traits, such as extroversion, introversion, show up in our work. And I would also say it's more nuanced than that. I think it depends on the type of work, and also your team norms, is something that we've really found. So you're right, there's this funny tension or paradox going on. People love the flexibility and the autonomy, like doing laundry whenever they want, being at home to receive parcels. They love that. And then they also feel like, "Hey, I want someone to care. When I type a message, at the other end, I need to know that I matter, and that I belong to a community, and that we're here as a team together." And those seem to be at odds. But I also think, what we're finding is depending on the job, there's different balances of that that's important and depending on the person.

So one of the nuggets that I'll share is I think when we dig a layer deeper, it's definitely some personality traits that some people want more high touch responses or whatever because they identify being more extroverted. But I also think it's a little bit of, you can get better at documentation. And I think that is the key to making asynchronous work work well is when people start to feel like, "Oh, am I doing this? Am I doing wrong?" If you can go somewhere and it's just like, "Hey, this is how we do things on the team," or like, "This was the objective of what we're asking you to do," some kind of reassurance that provides goes such a long way. Because without that is when people started to get a little shaky and be like, "Oh, am I on the right path or not?" So the more you can create those moments for your people, that moments of reassurance I think would go a long way.

Angeline Vu:
Yeah. Plus one to everything that you guys said. I think there's a really interesting tension that kind of exists with working from home. It's kind of just, it's, the inverse of working in an office. And how do you solve for that touch in? So one call out that I was really struck by was just how different the responses were from parents who worked at Thumbtack versus non-parents. And this kind of came up in multiple ways, but one of the ways was going back to the lunch entry. A lot of them took pictures of them eating chicken nuggets with their kid during lunch or them playing guess who, or going to their kid's soccer game, being able to pick them up from school. And you could tell through their written responses and their selfie videos that it was such a blessing to them, and it was only a luxury that was afforded to them because they get to work from home, and they don't have to be in an office for nine hours a day.

And so you kind of think about that. And then the flip side of that, when we heard from non-parents, is because I'm not physically leaving an office, it's really hard to shut off. So there are no boundaries, and you really have to figure out those boundaries for yourself. And so when thinking through all these findings together, I always kind of come back to this idea of empowerment. You're empowered to spend your time in whatever way makes sense for you. So if you want to go grocery shopping in the middle of the day, instead of sitting at your desk to have lunch, go for it. As long as you're meeting your work goals and you're showing up to your meetings, but also you need to be empowered to set boundaries. So don't respond to that Slack at 7:00 PM or 8:00 PM. And then as a people team, we think about how do we empower our people? What do they actually need in order to feel comfortable doing that and really making sure that we enable that?

Katie Masciopinto:
Yeah, absolutely. Taylor, your work on the bursts comes to mind in thinking about that. How can people engage? How can people find that connection while still finding that autonomy? And I think one question I have for you all as researchers, but also humans in the workplace, how might we better adapt to better meet tomorrow's workplace environment?

Linsey Feldman:
I mean, I think Angeline and Taylor got to this pretty well in their last responses of people need to be empowered. They need to be empowered to do their work without unnecessary obstacles in their way.

Angeline Vu:
Yeah. I don't know if I have too much more to add on to that, but I think when we think about adapting to the workplace, a lot of it, it's something that has to be provided by from the top down, but also by the employee themselves. As an employee, you do have to figure out what your limits are and what boundaries you're going to set, and really honor that because no one's going to ultimately honor that for you, even if we set expectations. And so we think about what are the right tools, but also as an employee, how do we actually empower our employees to really draw those boundaries for themselves so that they feel good about showing up to work every single day?

Taylor Kim:
Yeah, I love that. And then I think definitely the onus is on the individual. And I also think there's a lot we can do at the team level, or a lead or a manager level, of understanding that everyone, every individual's going to be very different. And so what feels empowering to one person is different for another person. And so just getting really curious about each other, I think, will go a long way.

Katie Masciopinto:
Yeah. Absolutely. Absolutely. Well, thank you all for indulging me there and sharing a little bit of your perspective and what you've taken away from this research. I think I would like to zoom out a little bit with our last 10 minutes, and I want to recognize that some audience members may not work for organizations that are ready for this level of transparency yet. I know, Lindsay, you brought that up in your work. And so I would love to ask you each, what advice would you give to those individuals, and what is one thing that we all can do to bring the UX lens inward at our own organizations?

Taylor Kim:
I think this one is tricky because it really depends on the organizational context. One thing that I would say is when we're bringing the lens in, or we talked about kind of earlier, Lindsay, you touched on it too, of I think the default maybe, is that we don't want to bring the UX lens inwards because like, "Oh, we already know. We already know. They're our employees. Right? They're with us all the time." And so taking a moment to pause, and not adding weeks onto a project roadmap because nobody needs that, but saying, "Okay, so what are the assumptions?" And just even articulating the assumptions that you may have about the internal world or employees or whatever your environment is. And I think that's a great first step.

And then from there, because once those things are out, those assumptions that you have that you're building on are out there, then you can have another conversation about, "Okay, to what extent do we want? Do we have time or want to interrogate these assumptions, or do we think they're pretty solid and then we're better off shipping and iterating from there."

But I think quite often, as I see lots of organizations not being even willing to articulate the assumptions, that's step number one, I would say. Just put it out there and then put it somewhere in whatever, a digital whiteboard or something. It feels less scary than in your head.

Linsey Feldman:
I think for companies that are looking to do internal research, every company has ideas of where they could improve. And I think researching those opportunities is a great first step just because a company may think they need to improve in area A, doing just a little bit of exploratory research can actually confirm that and make sure they're going in the right direction. And they may find a huge gap that they weren't even aware of, that they have a problem that can be easily solved by making some basic changes.

Angeline Vu:
I think something that comes to mind for me is when people really know about UX research, that's when you start to see a real craving for UX research. And so step one is always demonstrating what is the unique value add of UX research and helping align everyone around that. Even with this particular project, I think they brought us on as UX research partners, not really knowing what we were going to add, but knowing that they wanted us to be part of the project, which I think is great. And so I remember getting the research questions, and a lot of them were tactical, which are research questions too, but they're like, is this the right tool stack? And for us, it's a lot about reframing where it's like, yes, and how do they feel when they're using these tools? And so finding different ways to help them understand what we can add to what they're already doing. And I think once you do that, it's almost like a light bulb moment where it's like, "Oh, that's what UX research can add." And like, "Yes. Absolutely. I want that."

Katie Masciopinto:
I love thinking about internal research as this light bulb moment and this idea of once you're exposed to user experience research, once you see what it can do, this hunger for it, both outside of and within organizations. Well, Taylor, Angeline, Lindsay, thank you so much for sharing your thoughts, your experiences, your stories with us. Before we break for this panel, I'm curious if there's anything else from your perspective that you all would like to share with the folks at home when maybe they're thinking about running internal research or advocating for internal research within their organization?

Taylor Kim:
My big tip would be that I think a lot of people wait until a team is set up for internal research or whatever, but this is an area where there's actually a lot of white space. So if you are curious about it, then go for it would be my advice. I think if you're waiting for a certain team to be stood up or have a certain mandate, I think it's going to take a little too long. So if you see the opportunity, jump on it.

Linsey Feldman:
And to that point, it doesn't have to be a huge research study. It can be a small set of interviews with five, seven people, and it doesn't need to take a lot of time.

Angeline Vu:
Yeah. I love the idea of lightweight research and being able to dabble in it without committing to a full-fledged project. And I think there's always a need for it. Obviously the future of work was kind of a very big thing that bubbled up, and when I think about internal research, I'm like, there's always something internally that needs to be cancered. There's an endless amount of questions that we have about how it works internally. And so it's always a great value add.

Katie Masciopinto:
Awesome. Well, thank you. I couldn't think of a better note to end on. Thank you all for being here. And for the folks listening at home, I hope that you found this as insightful as I did. Thanks all.