A 4-Step Framework for Maximizing your Generative Research Interviews

Transcript:

Ben Wiedmaier:

Hey, I'm Ben. Generative UX research is next level. It moves us beyond tactical inquiries and toward a more strategic user understanding. It helps us direct the way a company can innovate. Now, all of that means that it can feel challenging or even daunting to start. In this video, Nikki Anderson is going to join me to talk about a portable and flexible question framework that she uses to help answer generative research questions. By the end, you should feel more confident to seize the next opportunity that you have to inform your company's strategy with research. Let's take a look.

All right. Hello, and I'm joined by, as I previewed, Nikki. She is, depending on how you're looking, to my left or right. And we are talking about the TEDW framework and specifically how it can help you enrich and make more impactful your generative research. But before we get into the framework, Nikki, I was hoping you might explain what you mean by generative research, and then how this framework specifically helps you UXers do that kind of work specifically.

Nikki Anderson-Stanier:

Sure, sure. So generative research is really open-ended research. So what we aim to understand in generative research is who our customers, or even sometimes not customers, it's, let's completely get rid of that, who people are. How are they thinking? What are they thinking about? What are their processes? What are their routines? And this is agnostic of products. So it has really nothing to do with your app, nothing to do with your website, nothing to do with your product. It's more or less looking at concepts and how people think about these concepts. So for instance, very commonly I use this example, if I'm working at a travel company, and where a company that sells you tickets to places, to destinations in various ways, so planes, trains, automobiles, so if I'm working at that company, a generative research project would be about how people make decisions to travel, not how people make decisions to use our app.

So, that's where the distinction comes in. It's generating an understanding and generating knowledge about the humans and the associated concepts and how they think about those concepts. So you can, of course, in generative research, talk about your product. It's fine if you do. It's not like, "No, never mention it." It's okay. But I do want to start with, it's usually product agnostic and it's about concepts. And so TEDW, this amazing acronym, is really great because it helps us ask open-ended questions, so first, open-ended questions, questions that are just not leading, not biased in the slightest, questions that get us stories, which is what generative research is about.

So again, if we go back to this travel example of like, "Talk me through the last time that you decided to go on a trip," you're getting a story, and it's not about your product. You're getting a story about how somebody's thinking about things, which leads them to deliverables like mental model diagrams, customer journey maps, personas, service blueprints, even. So there are many different deliverables that come out of this that help, then, your team generate a better understanding for users. So, that's really like what generative research is, why it's so great, and then how TEDW fits into enriching that generative research.

Okay. So, TEDW. TEDW is an acronym. It's one of my favorite acronyms in the whole wide world. Not that I have a lot of favorites, but if I did, it would be my favorite. So TEDW is a really great way to phrase your questions in an open-ended sense. So what I'm going to do is I'm going to walk us through each of the letters, so the T, the E, the D, the W, what they mean, an example of how to use it, and some things to think about if you're going to use these phrases and this method of questioning. I will say that TEDW is also great to use in your personal life, or with networking, or if you're interviewing candidates for user research, for your user research team, let's say. My poor husband listens to me TEDW him all the time, and he has learned how to TEDW me back. So, that's equally as great and frustrating. And then again, using it for networking, using it for user research interviews with actual participants, and then using it for candidate interviews. So TEDW is a really great tool all around.

So let's dive into the first part of TEDW, which is the T. So the T stands for, "Talk me through..." So let me give you an example of a different way to ask this, or a way that I hear this that's a little bit more closed rather than open. "So when did you last use Uber?" That's a question that I hear a lot. "So when did you last use this? When did you last do that?" And a lot of researchers ask that question. And I'm not here to say that, "When did you do something?" or, "When did you use something last?" is a bad question, because actually it isn't. It's a pretty open-ended question. However, the problem with asking somebody when they last did something is you will often get what you ask for, which is either, "I don't know," so, "I don't know the last. I actually can't tell you, since I live on an island, the last time that I used Uber," or an approximation.

So let's say I'm living in New York City again and I used Uber yesterday. When we ask something like, "When did you last use Uber?" we're not really just asking people for the date and the timestamp of when they did this. We're asking them to kind of explain what happened, give us a rundown of their last experience. But when you ask a question like, "When did you last do something?" you will likely get a rather short and stunted answer. So what we instead are doing when we're using the TEDW framework, and we're going for this T, "Talk me through..." we're triggering a moment in somebody's head and we're having them explain to us what happened step-by-step in that experience. So, "Talk me through the last time that you used Uber, right from the start."

So you could also even open this up even more. "Talk me through your decision to take a car home, right from the beginning." So it even opens the question up even more, if you wanted to get at whatever trigger points that they were thinking of, like, "Oh, this is when I decided to take a car rather than the subway," or walking, or biking, or driving, or whatever method we want to talk about. And so this allows for people to tell you a story, and research is all about stories. And that's why TEDW is so great because it really gets these stories out of people. So what I do is I use T, "Talk me through..." at the beginning of an interview. So, that's usually my big opening question. So like, "Talk me through the last time you did this. Talk me through your last experience." And then what tends to happen is they will go on to a monologue, you're writing things down, and you're then going back and pulling apart those different threads that they're telling you.

If people are having a little bit of problems with understanding the depth at which you want to go to, I will often say like, "Imagine I'm trying to shoot a documentary of your life. I want it step-by-step. I want every single thought that went through your mind," so that they understand that level of depth. And so what this question can uncover, not only just those stories, but this is where you get into deliverables like mental model diagrams or customer journey maps. It could even lead to personas, but it's more routine-based, which is why I mentioned mental models and customer journey maps. So it can really lead to those more general and holistic deliverables for your team. So really, really great to use this, "Talk me through..."

One thing that I will say is it is a big question. It's an intimidating question. So build rapport with your participants beforehand. I always do a warm-up where I ask people things like, "What's your favorite hobby? What do you like to do in your free time? Have you read anything, or listened, or watched anything cool?" And when they respond to me, let's say I'm like, "What's your favorite hobby?" and they say knitting, I'm just not like, "Okay, cool. Have you read anything interesting?" I actually ask them like, "How did you get into knitting? What are you knitting? What is that like?" And that puts them in that mental model to answer questions in an open-ended way and also builds rapport. So if you are thinking about using this, just make sure that you're building that rapport and you're doing that warm-up before you go right into like, "Talk me through the last time you did something." But after you do that warm-up, you're in a great place to start with a question like this.

The next part of the acronym is E, and E stands for explain. And that can be broken down into several different phrases. It's not just explain. "Explain how you feel. Explain what you meant by... Explain what blank means to you." So instead of asking, and this is going to be not controversial, but it goes against some of the advice that I give... And I just want to stress, these are different levels of how to ask questions. Not every question that you ask needs to be a TEDW question. So just putting that out there. So if you ask the question, "How do you feel about our customer support?" that is a good question. I'm not going to discourage anybody from asking a how question, so, "How do you feel about customer support?" But similar to the T, the, "Talk me through..." when you ask somebody, "When did you last take an Uber?" and they say, "Yesterday," same thing. "How did you feel about our customer support?" "Fine," "Good," "Bad."

So what we want to try and avoid, and why I'm telling you to not ask, "How do you feel about something?" and to replace it with, "Explain how you feel about something," is because we're looking again for those stories rather those one-worded answers. That is not to say that if you ask somebody, "How do you feel about our customer support?" they won't go on and on and on. There's just a higher likelihood that they might respond with a one-worded answer. So, that's why we want to every so often pull it back from, "How do you feel?" to, "Explain how you feel."

Another thing that I will say is TEDW in general, and especially this E, "Explain how you feel," removes leading questions. So instead of asking something, a yes or no leading question like, "Are you have happy with our customer support?" you take away the subjectivity, you neutralize your question and you make it open-ended. So you're removing bias. You're removing the leading question. You're removing closed questions, so the yes or no questions, and you're opening it up. So, "Explain how you feel." It's one of my favorite questions, because then we don't get stuck in this biased question asking. So it's a great way to check yourself. And then, again, when you ask this, people are more likely to open up, give you a little bit more context, give you more of their stories so that you're not stuck with one-worded answers and then continuously asking, "Okay, but what do you mean by this? What do you mean by that? Why? Why? Why? Why? Why?" So it's almost a way to avoid the incessant whys.

And so what I would say is, as a great example of when I use this is when somebody brings up a subjective word. So we bring up subjective words all the time. We're happy, we're sad, we're tired, we're frustrated, we're hungry, we're angry. We are loads of things every single day. And these things come up in natural conversations. So of course they come up in research interviews where somebody will tell you, "Oh, well, I was frustrated by this experience," or, "Oh, this is good," or, "Oh, I like that better." So these are all areas of subjectivity. And as researchers, we want to remove as much subjectivity as possible. So we want to unveil that subjectivity. And explain and asking explain-based questions helps us reveal that subjectivity.

So if somebody says something to you, and this is actually a question that I get a lot from researchers is like, "When do I dig in? How do I know when to dig in to these generative research interviews? When do I ask follow-up questions? Because, yeah, 'Talk me through...' is great. But that's a loaded question. You get a lot from that. When and where do I dig in?" Subjectivity. If you hear a subjective word, you're digging into that to understand what that means, because good to me is different from good to Ben. It's different from good to everybody else in the world. So we want to understand what those mean. So, that's when this explain comes in. So, "Explain what you mean by good." This is especially great if somebody says a design is fine. It's like the most frustrating thing to hear when you're testing prototypes, "It's fine." "What do you mean by fine? Explain what you mean by fine." So again, this allows us to dig deeper on this subjectivity, which then gets us more stories, more valid data and more comprehensive data.

So if we say to our stakeholders, "People thought it was good," what does good mean? We can then explain that, no pun intended, if we use explain in this circumstance. And so what I would say is if you are going to use this, every so often, and this is going against my advice of not using closed questions, but put a can in front of it, because it just softens it a little bit. People don't usually notice the difference. So every so often I will be like, "Oh, explain what you mean by frustrated," or, "Explain what you mean by happy or confusing." And sometimes I will say, "Oh, can you explain?" And it just softens it a little bit.

You could also say like, "I would appreciate it if you could explain." That's a very British and American way of doing things. It's like the politeness. But just putting those words in there to soften it up a little bit will also help you with building rapport and asking these in a slightly more gentle way, but still getting you the same result. So I really highly encourage you to use explain when really digging into that subjectivity, because you're going to get really rich insights and really great data for your teams.

Next we move to D, which stands for describe. And again, similar to explain, this could be, "Describe how you feel. Describe what you meant by... Describe what you would do next." So again, this is another way to dig into people's subjectivity. So instead of asking somebody, so this comes up a lot in usability testing, this is where describe kind of shines, instead of asking somebody the age-old question of, "What would you do next?" you can use, "Describe what you would do next." And I know it sounds like such a small difference, and it really is, but it just gives that much more context to the participant of how they should be giving information to you and the level of depth that they should be going to.

So, that's why these small changes are really important, because they're context clues for your participants so that they understand that if they talk for five minutes, it's actually not going to annoy you, unless you only have three minutes left, but if they talk for five minutes, it's actually quite great because you have a level of depth that you wouldn't have gotten to otherwise. So by adding in describe, "Describe what you would do next," that is just a really great way to up-level your usability tests, actually.

And the other ways to use it is instead of asking, "What was it like the last time you ordered groceries online?" you can say something like, "Describe the last time you ordered groceries online." And then the difference in that is you have, "What was it like the last time you ordered groceries online?" "It was fine," "It was good," "Oh, it was frustrating." Again, we're getting these shorter, stunted answers and we want longer, more rich answers. So, "Describe what the experience was like. Describe what it was like."

And then similar, if you want to, and I do this a lot because I don't want to be like, "Explain what you mean by... Explain what you mean by... Explain what you mean by..." you can then cycle through. "Describe what you mean by frustrated." So another great way to dig into that subjectivity. So describe really shines in quite a few different ways. So we have the usability tests of, "Describe what you would do next." You have this process in emotions-based understanding by asking, "Describe what the experience was like." So this could also lead to things like customer journey maps and mental model diagrams. And then finally you have, "Describe what you mean by..." whatever subjective emotion or word that they brought up.

So that would then lead to you uncovering the subjectivity of your research and really being able to provide your team with a deep understanding of what the customer is meaning. This as well can lead to better personas. So instead of your personas being like, "Oh, so and so likes grocery shopping," it goes a little bit more in-depth because you can understand what it means to like something, and you can understand why somebody likes something. So again, that's where describe shines.

I oftentimes will, yeah, go between explain and describe so I'm not boring the poor participant. And actually, this is an okay question to just use. You don't really need to soften it because it's actually softer than explain. Because explain can come off sometimes as quite aggressive. So if you want to start with describe, actually, in your questioning of subjectivity, you can start with describe. The only thing that I would say is if you want to go a little bit deeper, you could ask, "Describe how you were feeling, starting with..." whatever, like starting with a particular trigger moment. Or, "Describe the process that you went through buying a mattress, starting with when you decided to buy a mattress."

One caveat actually with describe is that it can lead to slightly closed questions. So, "Describe how satisfied you are with something." We don't want to do that. We want to keep it open-ended. So just make sure that you're taking out that subjectivity and you're just inserting whatever word the participant said and asking them to describe what that means. So again, very similar to explain, but just a different way and definitely very usable in usability tests.

So now we will come to the last letter, which is W, which stands for, "Walk me through..." So, "Walk me through..." is a very process oriented kind of question. It's very similar to, "Talk me through..." However, "Walk me through..." is really great because it indicates to somebody that you want a walk through. And a lot of people know what a walk through is, which is kind of like a step-by-step understanding of what happened. So let's say that you're having some trouble with the, "Talk me through..." So the person isn't really understanding the level of depth. You can substitute, "Walk me through..." so maybe that they might get a better understanding of what it is that you're looking for from them.

But, "Walk me through..." also takes away the need to say, "What happened next? So what happened next? And then what did you do next? And then what was the next step?" Because we can get question fatigued a lot, both the researcher and the participant, which is why the Five Whys, it's a really great technique, but it can get very tiring for the participant. It's a very large cognitive load. So what, "Walk me through..." does is it allows them to go on this soliloquy where they're talking about this whole process. And then you're going through, similar to the, "Talk me through..." then you're pulling threads retrospectively. Starting with a warm-up before asking such a big question is really important, but I would really say give it a try. Try this, "Walk me through..." Especially if you have to try any of these, if you're going to be like, "I only want to try one of the four," try the, "Walk me through..." It's my absolute favorite question and it gets you really great results in this type of research.