Researching Hard-to-Reach Participants with The North Face

Busy, distracted, preoccupied.

Think your target participants are hard to research in context? Try reaching them on the side of a mountain.

From peaks to valleys, Senior Director of Insights at The North Face Vanessa Dillof collected hundreds of moments shared by elite athletes—providing deep insight into their product experiences and day-to-day lives.

Hear how researchers at The North Face conduct in-the-moment research, in extreme environments with busy participants, to drive company-wide innovation.

Vanessa and Ben share:

• How The North Face designed its Athlete Intelligence program
• What it’s like to align research with the lives of elite athletes (and challenging scouts)
• Why it’s crucial to understand your audience’s lives “off the mountain"
• How to streamline remote research design to maximize responses from busy participants

Transcript

Ben:

We are focusing on a use case with our remote qualitative platform. I have a very, very smart outdoor focused thinker, researcher, designer, leader Vanessa Dilloff. Before I fully introduce here though, just a couple of housekeeping notes. For those of you who are joining and interested in asking a question, Vanessa is going to have plenty of time to answer those. We're going to balance question and answering both during the course of the webinar and we have left plenty of time at the end of the webinar after Vanessa's presentation to answer your questions. Please use the Q&A button that you should see in your tool bar. In addition to posting a question there, be sure to up vote questions that you are most interested in hearing answers to. So again both, please post a question and up vote questions that you like. We will again, try to get to as many of them as we can.

Ben:

Secondly, this recording will be made available to anyone who registered. I know that's often a question we get with these. Yes, you will get a copy of this. You will be able to hear all of Vanessa's brilliance and my incoherent ramblings forever, for perpetuity. So don't you worry. Finally, shout out to anyone who is going to UXPA, the User Experience Professional Association conference next week in my beloved Arizona, in Scotsdale. In Chicago here it is still in the 50s, and cloudy if you were wondering. If you would like to get away for some sun and meet some very innovative thinkers in the UX space, we will have a both at UXPA International in Scotsdale in the next week. Drop us a line, we'd love to chat with you. Our VP of Success, Jamie Whalen has a panel with a few of our customers talking about how you can share qualitative ethnographic, mobile, the sticky rich thick data insights with your org.

Ben:

That's a great transition now to introduce our speaker for the day. Vanessa Dilloff leads Insights at the North Face specifically, but VF Outdoor brands broadly. Vanessa, welcome.

Vanessa:

Thank you so much Ben and hello everybody, happy Monday. Hope this is a good accompaniment to your lunch or whatever you have to be doing today. Looking forward to sharing this case study with you all. Ben should I just dive in?

Ben:

Yeah, I just wanted to say again, please use the Q&A. Vanessa will be doing the driving, I'll interject with your questions were relevant and let's get going Vanessa.

Vanessa:

Cool. All right, well let's go to the next slide here. Some of you my be familiar with this image. If you're not, this is Alex Honnold, one of the 150 or so athletes that we keep on our global athlete team. Arguably now the most famous on our roster. As some of you may now, Alex was featured in the documentary called Free Solo that won at the Oscars this past year. As you can see, Alex is a climber. There's a few things missing in this image, most notably a rope. Alex climbed El Capitan in Yosemite without the use of anything other than his hands and legs and body. He is just a remarkable figure and remarkable human being that all of us as researchers can appreciate. He's somebody that you would just want to talk to as an interesting person, but if you're in a job like mine where he is arguably the best climber in the world and we are the makers of outdoor apparel and gear, he is certainly somebody that, how could you not talk to and have a relationship with on a day to day basis.

Vanessa:

That was really the impetus behind the Athlete Intelligence Program that we decided to create. We had a lot of Alex's out there in the world, but not a great way to really connect with them. There were three main goals that started the program for us which we're now in about our third year of the program and it's just continued to grow over that time. The first goal was we needed a way to get honest feedback from these sources that we couldn't get this insight anywhere else. You could go out and talk to mainstream consumers and we do a lot of that as well, and it's super important to balance that, but these extreme leading edge users for us, we didn't have a great platform to interact with them. The previous way that we were engaging with the was a lot of bullet points and a lot of emails back and forth.

Vanessa:

As you can imagine, Alex has a lot of things to do in his day, but one of the things that he doesn't enjoy as much is spending a lot of time in front of a desktop computer interacting with people over here in corporate land. We needed a better way, a more useful tool to really reach our athletes where they wanted to be reached. Using a tool that we knew that they would have with them at all times. The cell phone was the one thing that would allow us to do that. I'll get into more of how that was the right way for us to interact, but it certainly changed the game for how we were interacting with them.

Vanessa:

Ben, if you could just go back one slide. The second reason here was really we had a process with our athlete team that was just cumbersome. They were getting asked a lot of the same questions over and over again because we had no centralized portal to really engage. We would have one designer reaching out one week and then another person the next week saying very similar type of requests. So just having a system and a way in which we could manage that feedback was super important.

Vanessa:

Then the third one here was really we needed to humanize these people. There was a lot of great wear testing feedback coming in and we were certainly incorporating that into the organization, but at the end of the day for designers, these were people and they had real stories. That part of it just wasn't coming through and in fact, many of our designers were getting frustrated that there was a lot of wear testing feedback coming through and a lot of changes being asked for in the product that we were creating, but they didn't necessarily know the why behind that and this platform really allowed us to start to change that so that they understood the broader context behind some of the feedback that they were getting.

Vanessa:

Going on to the next bit of it, yeah what is it? It's pretty simple. We basically submit, for those of you that are familiar with dScout language, we submit a mission to our athlete team. Those missions can be super targeted, so they literally can go to one individual if we want or more commonly within our organization, what we've done is built profiles of the different types of users that we want to get feedback from. So I made just send a mission to our climbers for example, or I may just send a mission to our skiers, or various forms. Generally for us it's a lot by activity, but we can also just send missions to our female athlete team if that's what we want to get feedback on, or it can go out to everybody depending on the ask.

Vanessa:

The athletes receive those missions, as I talked about, on their smart phones. The beauty for us is we literally can send these missions wherever the athletes are in the world, which for us takes us to the extreme ends of the earth. They can capture their feedback in video form, audio form, as well as quantitative feedback wherever they are. Then when they're back in cell phone range and cell phone land, that feedback will automatically upload to us. It's a fantastic way for us to get real time feedback, literally on the side of mountains and capture that context in a way that we otherwise would be completely missing prior to the platform.

Vanessa:

Then as those missions get completed, we have now that data base that I was talking about that really allows us to see all these feedback aggregated. We can sort by a certain athlete. We can go in and look at anything that Alex Honnold has ever said over the course of the three years, if we wanted to do that, and do a lot of cool filtering and just analysis of the feedback on the back end.

Vanessa:

So, the tool specifically that we use is called diary within dScout. It's a pretty cool platform that I'll walk through you some screen shots of. You can see here on the next slide, I think it's just coming up here, yeah the way that the athletes get invited into the mission. We can send an email, it's completely customized. We can use, actually in most cases, we use a lot of our athlete imagery so if they see themselves when they come into the mission, they see this both on their mobile device and in their email. You can see they get a notification that there's a mission ready for them. How long they have to complete it. Which we tend to give our athletes a bit of time because a lot of times they're out exploring the world and have better things to do than to complete our surveys. You can do this as short as a 24-hour turnaround in some cases to leave these missions more open ended.

Vanessa:

Then, once the athlete accepts that mission, which is the first step and we see which athletes have entered in to our mission and which have accepted, then we can ask them all different sorts of questions. From our end, some of the most, I think interesting, feedback that we get is straight from the open ended video format. One of the biggest criteria that I had, I think, in looking at vendors across this process is, I really wanted a tool that was idiot proof. Our athletes definitely don't, engaging with technology is not their preferred way to spend a day. They'd much rather engage with nature and other things.

Vanessa:

So here, literally they pushed a red button and they're recording. We just get really raw, honest feedback through that medium. It's not complicated. It doesn't feel like a survey in a lot of cases, it just feels like they're talking to us directly. I think our athletes really appreciate that and we do as well. The video functionality has been huge for us. There also is the ability though to ask a lot of different quantitative questions and ways to dig in deeper. That certainly plays a role for us as well. It's really a nice mix of [inaudible].

Vanessa:

This just gives you a little bit of the capture on the backend. If that's what the athlete sees, I thought it might be helpful for some of you just to see a little bit of what it looks like on our end of things. If you're the client and this is me logging in to the portal, what it looks like from my point of view. Here's what I was talking about earlier where we can go into a mission, we can see all of the individual athletes here coming up and what each of them had to day. We can also hopefully start to pick out here, we literally have people hanging off the side of massive cliffs, and ice climbing and showing us how this gear is used in context, which for us, just to be able to have that scenario on top of the feedback. It's one thing if you're getting bullet points in an email, but you lose completely is it a cloudy day? Is it a sunny day? What are they wearing underneath the jacket? All that we can capture with the visuals and the audio that we're getting from the teams.

Vanessa:

They can also get really close, tight in, and as you can see on a couple of these shots of where there might be experiencing wear or tear, some of those things that we really want to be able to address. We can provide a layer of richness that we just didn't have there before for our designers and our team. Hopefully you see some familiar faces here. Conrad Anchor, Jenny Chen. We got just a great athlete team giving us a ton of richness everyday.

Ben:

Vanessa, this is great. We've got few questions from attendees about the particular group that you used for your sample. Could you talk a little bit about who these people are? Is this anyone who's a member of the athlete team? Is it a subset? How did you select who would be a part of the AI program?

Vanessa:

Yeah, good question. For us, anybody that's a member of the athlete team is a part of the program in some capacity. We have different layers within our athlete team. As I was mentioning, in some cases we might send a mission to the full athlete team, in other cases it may just go to certain folks within in. But everybody who is part of the athlete team in some way is submitting feedback to some missions.

Ben:

What was the process like to onboard the athletes? Did you and your team have training sessions with them? Did you lean on dScout for this? How did you get, because like you said, these athletes are professionals in another area of their life, how did you get them to feel comfortable with and get comfortable using dScout?

Vanessa:

For sure. A couple of different ways. One was once a year we have an athlete summit where we do all get together and it's a big opportunity for the brand and corporate folks to engage with the athlete team in person. We launched the program at one of those summits where we were able to walk them through, okay we're all going to download dScout on our phones now and steps one, two, three on how to do that. Then, relied heavily on the dScout customer service team which has been phenomenal for this project, to engage with the athletes if they were having any trouble once a mission has actually started. A number our athletes would just reach out and say help me with X, Y and Z. There was always somebody there either on our team directly or on the dScout teams to support them to get up and running.

Ben:

Cool. That's awesome. We're getting a lot of other questions about some of the tactical stuff, what did you do with these data? How did you roll it up? I think you're going to get to it in a second, so why don't we keep going.

Vanessa:

Cool. All right. So let me just set up this video real quick here.

Ben:

Yeah, it auto plays [cross talk]

Vanessa:

Just to give you guys a sense. I thought it would be helpful to show you a little bit of sometimes what we get back from the athletes. A lot of different types of feedback, but in this case, this was actually a video and we just asked them to literally talk to us about their lives and again, getting at that human side of this. This was an introductory video, it was one of the first things we did when we launched the dScout platform and ultimately a video that we not just showed our employees here to give them a little bit of better understanding of who is on our athlete team, but this is actually a way where we went back and showed the athletes, look we're listening and we shared this video with them as well, which they really enjoyed. A super important part of this process is to make sure that the people on the other side that you're collecting the feedback from ultimately hear back from you and this is one of the ways in which we did that.

Athletes:

Hi North Facers. Hi, hello, hey everybody, hi, hey, hi, hey, hey there, hello, hello, hey good morning, hi everybody, hey, hey how's it going, hi everyone, hello, how's it going lads, hello, hi, how's it going, hello what's up, ciao, hey guys, bonjour.

Athletes:

Some funny thing about me that you might not know is that I'm good at DIY. I like restoring houses. Plastering is really not my forte. My forte is lifting heavy things because I find it's an unusual way of working out.

Athletes:

I like to clean the house because I fid it to be quite meditative.

Athletes:

Now we're about to walk into what is my favorite room of the house. This is where all of my sports stuff stay. You can see I got my skis there. They're getting ready for winter.

Athletes:

I chose to show you guys my woodshop.

Athletes:

My wife and I like to produce a lot of our own organic food on our property which is here in our greenhouse.

Athletes:

This is my gear room. This is where I spend a lot of time thinking about projects, thinking about training, thinking about life, thinking about new adventures.

Athletes:

Right now I'm in my home in Salt Lake, which I built.

Athletes:

This is my and John's rock shelf. These are the boots that we climbed Rainer in. These are the nail plates that we used for cramp ons, tied them to our feet. Some little memorabilia. All of these rocks up are all from the summits of different mountains around the west.

Athletes:

Something not a lot of people would know about me is that I'm scared of heights, kind of. It's something I'm constantly overcoming.

Athletes:

My number goal in life is to learn how to speak another language.

Athletes:

I volunteer at San Quinton State Prison where I help run a running club in the prison for the inmates.

Athletes:

Right now I'm in Chamonix, France. My spirit animal is a gazelle, except for the month of August when I'm in Chamonix, then I transform into a chamois which is a mountain goat, antelope mix. Much better for going down hill.

Athletes:

I once ran through the White House, literally.

Athletes:

When I was younger, I learned to climb before I could walk and that scared my parents a lot when I was little.

Athletes:

When I was a kid, I lived in a teepee. Now, I've got a couple teepees in my backyard, hey look, there they are.

Athletes:

This is my home village, this is my home mountain the Matterhorn, this is the [inaudible] and this is my home.

Athletes:

I'm recording this video in my new home, which is a mobile home, 23 foot RV, what it looks like for the most part. I'm mobile, ready to chase the snow.

Athletes:

I've never eaten at McDonalds.

Athletes:

I love french fries, yep, that's me.

Athletes:

I actually really dislike being cold and winter is my least favorite season. I think that's why I ski. It makes the whole year wonderful.

Athletes:

Ciao.

Athletes:

Peace.

Vanessa:

All right. So hopeful that gives you a little bit of a feel for how we bring some of this to life, but I'll start to get into some of the outputs. I know there's a number of questions as I was just going through on how do we actually bring actions to this within the organization. Hopefully I'll start to address some of that for you guys in the next few slides. What you're looking at here is one of the ways that we do that. We take all of this great, rich feedback that we're getting, the video being one platform, but another is we created a monthly magazine within the organization that we call the Explorer, where we bring some of this insight to life for our teams here internally as well as for the athletes themselves.

Vanessa:

You'll see here, this is an example of one of those magazines that we shared at the athlete summit a couple years ago now. But we've had an installment of this throughout the organization, just a fun way for the athletes to learn more about each other because they often only get to interact with each other a couple times a year through North Face events et cetera. So they really enjoy it, but then a way for our organization to really start to learn about them as well.

Vanessa:

You'll see here just a couple more examples. These are snippets that were featured in the magazine, but also that we built huge boards that lived in our cafeteria and other areas, common areas throughout the organization. Again, just trying to give everybody exposure to some of the insights that we were getting through the platform in fun and easy ways for everybody to start to engage. Here is an example of some of the passions that our athletes have outside of sport. That was really, again, just trying to bring these people to life as more than one dimension. We know that they're hugely passionate about their sports and their activities, but what else do they care about and those adjacencies would be really rich for our marketing team and for other groups within the organization to be thinking about.

Vanessa:

Again, just so more examples of that. What's inspirational for them? We know our athletes are hugely inspirational for a lot of different people, but I think this was one of the questions, I often will reach out the organization and say what do you want to learn from the team? And we submit, anybody can submit questions that they think are interesting to us to include in future studies and this was one that is like, okay they're so inspirational to us, but what inspires them? What would they be doing if they weren't an athlete? I think we got a lot of really interesting, a lot of creative and other forms of exploration. Our brand stands for exploration and we saw that our athlete team epitomizes in a lot of cases creativity and exploration in all forms. Art, photography and music being really big.

Ben:

Vanessa, what are some of the sources of inspiration for the questions? You mentioned that your marketing team had some things that they were interested in, I know from working with your team at North Face that they design team was very integral in creating some of the questions and the ultimate insights. Who were some of the other stakeholders? I know this served the organization in an empathy injecting way, but were there other direct stakeholders for these sorts of stories and data?

Vanessa:

Yeah, we would customize a mission to different sets of stakeholders. Sometimes it's very specific and it is around a specific product and we're getting wear testing feedback primarily to impact product management and design. We'll run missions around that. In other cases, it may be as broad as what does exploration mean to you? And that's really to help inspire a creative read for our marketing org. I would say that different leaders within the organization are certainly benefited from a uniform tool that can be used and applied in a lot of different ways. I think it's touched almost every function with our company at this point.

Ben:

That's great. We have a few outstanding questions about how the participants were, I guess, incentivized? I think it's important that we clarify that these were folks who were on the North Face team and so this was part of their sponsorship was that they had part obligation, part opportunity to review the gear that they used each and every day and to try and make it better. Could you talk a bit more about how you framed this ask to those very busy athletes?

Vanessa:

It's a good question. We're fortunate in this case that yes, it is literally written into their contract that they give us feedback, but I think where the athletes were motivated is, that had been in there for quite some time, that was part of their engagement as being part of the athlete team, but they weren't satisfied quite honestly with how that process was working and we weren't satisfied and I talked about some of those challenges, but from the athletes perspective, it felt like a lot of cases that feedback would go into a black hole or maybe a designer moves into another part of the organization and then they're telling the same thing to that new designer because maybe the athlete is on a team longer than our designers might be in that role. So how did we find a way to make sure that the athletes really felt like they were being heard?

Vanessa:

I think that's where I'm pretty confident now the athletes would participate in this even if it wasn't written into their contract because they feel like they do have a voice and they're being listened to and that we're responsive to what they're saying. That has made all the difference in the world. It doesn't hurt that they're contractually obligated to be my participants, but I think they would do it above and beyond.

Ben:

For me it's striking. I've worked with some customers who have either member lists or they're doing research, they want to do qualitative work with their own employees and I think, like you said at the outset, you could very easily just send these folks open ended questions or survey questions in an email, but when they are they in front of the desktop? When are they going to hammer out a four or five line response on this carabiner accessory that's built into their back, you can tell that I am not an outdoors person by training. But your design team is creating very specific actionable feedback that is rather challenging to coax out of someone via a text response. Or at least sitting on a keyboard. I think it struck me just as how you leveraged the videos to both get capture richer data for your product design, marketing teams and also to enable easier feedback offering from these athletes. I think that's a lesson that you can take to any sort of hard to reach, hard to engage participant or member group.

Vanessa:

Yeah, the other thing I would just build on that is we were very clear with the athletes up front of what this was going to be used for as well. That this really was a research tool and that their feedback would be used for those purposes. That it wasn't something that we were going to take some of their quotes and their feedback on product and all of a sudden it's going to be showing up in an ad campaign in the next couple weeks. I really did have to work at the beginning as well internally to make sure that people understood there was a church and state relationship and certainly we can reach back out to the athletes if there's something that we want to leverage external to the organization, but also being really respectful with the athletes of what this feedback is for and what it's not.

Ben:

That's great. I have a few more questions about the field work, but let's keep going because you have a few more great slides about the share outs and the deliverables that you produced for your stake holders.

Vanessa:

Great. Again, just a few more of those examples. We try to make it very visually based here because again, that's the richness of the platform. Yes, we're getting some text based feedback, but we're just getting a lot of great insight. Here you can just see, we just wanted to know what does home look like? We got a lot more bags and cars than we did actual homes, which was really interesting for us and super insightful because a lot of these folks on our athlete team, they're mobile, they're ready to chase the snow as you saw in that video. Home takes on a lot of different forms and that was inspirational for the team here.

Vanessa:

Again, they explained that, you can see a lot of the different ways that we saw. When we asked them to show us their home, it was also a lot of focus on the garage or the green spaces outside of the home of where the sun came through. So what is important to them and what does home mean? I think we saw, it was very apparent that it wasn't the traditional way that most people will show you their living room. We didn't see a lot of living rooms, we saw a lot of other things. I think that was particularly cool to just again start to humanize and really bring to life this personal for the organization.

Vanessa:

So I wanted to, it's not all qual and I thought that was an important point, we definitely look at this quantitatively as well and I thought this might be helpful for some of you out there that maybe are in more quantitative organizations. I was in one of those for about 10 years of my career so it's always trying to cater the insights to the organization. We do a bit of quan as well through the tool. One of the ways that we do that is we've set up, this is just an example of how we might do a wear test through the tool.

Vanessa:

We're looking at whatever the key metrics are for that particular product. In this case, we identified four key benefits that we want to evaluate the product on. This is completely dummy data by the way. Looks like this product, if it existed didn't do so well on the waterproofing, but just wanted to more articulate the point than the actual research. You can see, we'll send this out. This is something that we'll use with our leadership team, so we'll go through, we'll do an athlete study, we'll include a quote along with each of these features so they can get a little bit of the human side as well. But they've got overall a summery take away for the key measures of that product that we're looking at and then on the right hand side we're calling out some key points that we really want that executive audience to understand from the study.

Vanessa:

Then another way, so videos, the magazine, that quantitative summary, another way is also just engaging with the users in person. I think that is important. While dScout is predominantly a mobile tool, it can also come to life for us in in-person events with these features. I thought that was super important to mention as well is sometimes we'll take the feedback from the tool and we'll use these in-person, once a year events that we have with the team to really expand upon those ideas. You can see in that bottom middle image we'll literally bring some of the videos from the dScout platform to life. We have headphone stations set up at the summit where the athletes themselves can listen in and hear some of the feedback.

Vanessa:

One of the things that we also heard that may be applicable to some of you is, the athlete, sometimes they're contradicting each other and if your left as a designer on the other end of things it's like who do I listen to and why? It is helpful for the athletes to understand the other feedback that's coming into the organization so that they can sympathize and empathize with what other athletes are saying and ultimately, hopefully it doesn't change their perspective, but it does give them the context that look, maybe our job as designers is a lot harder when we're getting 40 different athlete's feedback back and sometimes they're contradicting each other. Some of these in-person events help us to bring that dimension to life.

Vanessa:

Finally, just a couple slides in closing just talking about the impact it's had within our organization. I've hit on this one a couple of times already, but to go from this bulleted point email feedback to these really deep, very raw stories and I think at the start of this we were concerned how honest would the team be? Would we get real information from this platform? I can tell you, from mission one that fear was set aside within the organization. In some cases, maybe too raw and too honest and really making, they didn't hold back which I think was our biggest concern. But we have definitely not seen that. We've seen quite the opposite that we've got this really great portal into their lives. The second piece of this is we'd much rather get this feedback early on in our design process and in our campaign development process. So we could address these issues.

Vanessa:

Before, sometime we could go to these athletes literally with a product launch and say, great, put this all over your Instagram and here's X, Y and Z ways that we want to promote this. But we didn't necessarily have the athletes buy in from the start of the process. Now they feel like they're definitely co-creators in this process and so when we're coming to them at the end, they're like of course. Yeah, I've been involved in this throughout, I know what it is and I support it. That was a really important piece in all of this.

Vanessa:

Then finally, we finally have this one place where all of this lives within our organization. Which is just super important that anybody within the organization has access to understand and learn about the athletes needs so if we onboard new employees, et cetera, they can all come in, see the feedback that we already have and be much more thoughtful and conscientious about the type of feedback that we did bring in, in addition to what we've already got.

Vanessa:

Then yeah, so finally, maybe just a few pieces of guidance or advice if you're thinking about doing missions similar to this one. One thing that we definitely learned with our athlete team that I don't think is unique to our athlete team, I think it's speaks to where humanity is at right now in our society, surveys are not something you want to spend 30 minutes engaging with. We started this out we'll have five parts and were very thoughtful about each part, we realized that's four parts too many. We've got to keep this really tight, really simple and our athletes, ultimately we have one part, they may expand upon that, but let that be their choice rather than forcing five parts on them and giving them an exhaustive questionnaire to fill out, that they just don't want to do. So we'd rather leave it up to them to give us more rather than less. Rather than demanding it from the start.

Vanessa:

I think the second thing here was really that this ability to interact, there's something still about all of us that loved kindergarten and show and tell and this platform has really allowed us to do that in a very visual and fun way so that it doesn't feel like it's taking a survey or it doesn't even feel like research. It just feels like they're talking to us and they're showing us things that they think are interesting or cool, which is great.

Vanessa:

Then the final piece for us of why I think this platform has worked so well for our needs, is the context. Particularly for those of you that might be in the design realm or just creatives of some kind, having this context is so valuable for us. You can talk to somebody through their computer or while they're on the couch or some of those other things or we all talk about the glass ear and those types of situations within research, but for us to be able to be in these real situations, to be able to see their homes, to be able to see what's going on, on the mountain, we can't be there always ourselves and it's always a mix. We do a lot of real in-person ethnographies and that type of work as well, but to get that at scale, to have 150 athletes at once giving us that context has been super rich and valuable for the team and just adds another layer to the feedback that we wouldn't get otherwise.

Ben:

That's awesome. As you can see Vanessa, we've got a ton of really good questions around a couple of different parts of the project. I'm hoping that we can go back to this slide, or this guy right here. So, for those of you out there who are new to dScout, this is the researcher dashboard. This is what Vanessa and her team see as the individual athletes answer those questions that make up the mission, which are called entries. A bunch of jargon, short enough to say these little data packets that she used to iterate and share feedback internally. Vanessa, I'm hoping we could talk about, at the top, who was the team pointing this effort? When I went out there with you, I met with the design team and I had a lot of great questions from them, but we have some questions around who were the day to day managers of these missions? Was it your team on Insights? Was it in collaboration with other teams? Could you speak a little bit about that?

Vanessa:

Yeah, it's a good question. At the outset definitely this was born out of the Insights team, which I head here at the brand, but it was always my long term goal that this didn't need to be something that is led by the Insights team long term. Anybody should be able to go in and interact with our athlete team. That's been the goal and that's the reality that we're now starting to see. As Ben mentioned, he was out here on site, he trained our design team or product management team, our sports marketing team does a lot of engagement with our athletes and were part of that training.

Vanessa:

Ultimately, I want everybody within the organization that is a key stakeholder to feel comfortable, to be able to at least start and establish a mission, reach out to the dScout team for help in guiding that. The Insights team we know, we have a very basic, if you want to start a mission let us know and so we get it up and running. There's a way very simply within the organization that we ask a few key questions about the type of mission and why and we're the, I hate to use the word, gate check, but we are involved in this whole thing on that mission, but we're not necessarily leading it. That's critical because we're a small team here, two Insights roles at the brand so if we were blocking every mission or being the filter for every mission, my goal of getting this feedback out there, there'd be a big backlog. Now we're in the field often, several times a quarter, engaging with the athlete team. I hope it even get to be more than that as we go on. But that's where we've gotten to at this point.

Ben:

That's great. To reiterate for those of you that might not have heard, there are two Insights in the entire brand. You're very much running an agency. I hear this from some of the research leaders with whom I speak that they're really hoping to democratize access to qualitative research, broadly defined. It sounds like you found a really nice way of doing it. Someone, maybe on the design team for a particular sport, has a question about a gear segment or a subset of the athletes, they submit a set of questions to you or have a meeting with you and then you and your team, the Insights team launch this mission and how often are you communicating with the athletes? I guess it very much depends on the kind of thing that you're asking the athletes to do, but what's the cadence of communicating with the athletes once they're in the mission? How often are you reminding them or checking in with them for any one of these missions?

Vanessa:

Some of the missions are just a one time deal, but one of the great things as well about the platform is it does allow us to do longitudinal stops. On a specific product, if we're looking, particularly we're looking for wear and tear, durability those things take time and they take number of uses. So one might ask them over the next six months, we want you documenting every week how did you use this product this week? What was your reactions to it? How was your, and then we'll do one final mission at the end of that process where we're asking them to summarize their feedback over that whole period.

Vanessa:

The week by week communication may be very, very simple. It's literally just come in, log in, tell us what you used the product for this week and how it performed. Then at the back end, the close of that three or six months, they might be reacting and filling out more detailed information for us. It runs the gamut depending on our needs, but we've done everything as short as, you've got 48 hours, give us some one time feedback to something that is more three, six months in nature. In that case we're definitely hounding them a little bit. I freely admit, we're not shy about reminding them of their role and making sure that they're submitting that feedback.

Vanessa:

One of the things that we have done that's really helped there is also just, we're not shy about calling out who has already submitted and doing fun things around that, sharing some of the early feedback that we get. It's a really nice way to get some of the other athletes involved. Saying, okay we've got X, Y and Z already coming into the platform, we've loved to hear from you.

Ben:

That's great. So you're reminding the athletes that they're all in this together and you at times might even share feedback that they provide with one another. So and so said this about this, what do you think? I love that way of engaging, to remind them that they're a community. This is a great segue to another question we're getting a little bit about and that is, a lot of the, I imagine the target market for the North Face are pro-sumers and consumers, how does what you learn with this group, this arguably very top tier group, inform the research that you do with consumers? What's the cadence between doing research on those two groups?

Vanessa:

That's a great one. I think we recognize that this group is super important for us and for certain product within our line, namely our series level and our most elite level product, our athletes really are the muse for what we design and what we create there so it's a direct one for one in terms of how that get fed into the organization, but we also fully recognize that their feedback could lead us completely astray for a lot of what we do within the organization and consumer feedback, mainstream feedback is much more appropriate. It's always walking that balance, making sure as well with the teams that are asking to do these missions, that they're doing missions for the appropriate scenario.

Vanessa:

If it's a product that is more likely to be sold through our maybe more mainstream traditional channels, I'll push back and say are the athletes really the right source of inspiration here? Or should we be talking more to somebody that's wearing this on their commute in Chicago and really making sure that we're getting the feedback from the right source. But, I do think in general what we're getting from the athlete team can set the tone in a lot of cases for where we want to go within the organization. Really making sure that we're staying true to our purpose and our goals because they're ultimately, if things can pass the athletes [inaudible] pass the consumers [inaudible] them. I'd say that's the biggest use case. Really making sure that we can get it past these 150 folks, we're probably going to be all right when we get out to mainstream America.

Ben:

Yeah, again, admittedly not an outdoors person, but my producer [inaudible] who sits next to me is a big outdoors living and she was so excited to see these various faces. I imagine for your consumers there's some, man I would love to be the kind of athlete like the people you're seeing here on screen. So I understand the logic behind passing mustard with these top 150, might very well pass mustard with folks who have hopes of becoming as good, as comfortable with the various hobbies that these folks are.

Ben:

I'm curious how you share, how do you get the, because I was really struck by how you weaved the design team, who again, not "researchers" by training, but you really weaved the designers into the day to day and the collection of the feedback. Can you talk about how you, because we've got some big questions about how it is you're socializing beyond deliverables. Are you adding stakeholders to their mission as viewers? Or asking people to take ownership of the missions and that's their first touch with the data? How do you get folks to be involved in and engage in what is arguably, very rich thick data? You've got videos, full transcriptions, open end responses, quant, you've got a lot for them to go through. How do you get them in that platform engaged?

Vanessa:

They're definitely not waiting for the fancy deliverables, by the core person on the team. They're very much in the platform along with us which is great. Those deliverables I showed before are more for the broader organization and for more of our leadership levels. If I'm partnered on a project with a couple designers, there in a platform side by side with me going through and looking at some of this real time. They're bringing it into their sketch reviews or creative kickoffs. Big thing within our organization is just having a story that accompanies your design. I think that's probably true for a lot of organizations. You have the design itself, but what was the impetus for it? What is ultimately the needs? If you just go into design thinking [inaudible] of their design, this platform really enables our deign teams to do that in a pretty unique way. They may just come in and pull a couple of these stories down and that becomes the impetus for them ultimately showing why they've designed something in the way that they have.

Ben:

That's great. You can message people from the platform, you can tag. Do you find that you're doing a lot of your synthesis and analysis in the platform or are you exporting images? Are you pulling the data out and using another platform to go and do tagging? How are you making sure of the data?

Vanessa:

I'd say it's a bit of a mix. I want people to use the platform more wholly and really understand some of that functionality because I think in the past we would have all these one off conversations that were really an email with the athlete team and again, if one designer has asked that question, I don't want the next designer to come in two weeks later and ask something very similar. I can't tell you the number of times people used to ask can I tour your van? Honestly, we got the van covered. It's only an eight foot square space. Everybody has seen the van at this point. It's little things like that hopefully Alex will never get another request for somebody to videotape the inside of his van.

Ben:

I love that you said that the athletes before were maybe hesitant or not as excited about sharing, but now, you've said now in the third year of the athlete intelligence program, your finding that this rather cagey hard to access group is more willing to and interested in providing feedback. Could you speak a little bit about how they're responding to this effort and initiative that you and the North Face have done?

Vanessa:

That's been the most rewarding part of it quite honestly for me. I set this up as much of course for our employees, but I definitely just recognized a pain point with our athlete team, just engaging with them and interacting at things like our athlete summit. They had a lot of feedback for me on, look we say we're athlete tested, expedition proven, that's been a line that's been around the North Face for more than 50 years since the company was founded. But, are we really living true to that? Are things really athlete tested? I think they wanted to be more involved within the organization on a day to day basis. I was fortunate enough to have a willing audience there, but how do we connect the dots?

Vanessa:

now, going back to those summits and having an athlete come up to me and say, look I finally feel like I'm really being heard and that you guys are taking to heart what I have to say. As a researcher, that's pretty much as good as it gets for me.

Ben:

Yeah, I want to dig a little bit on how you communicate to these athletes that they're being heard? I know you said that when you're doing some field work moderation, you might remind them of some of the feedback that their colleagues have given. I'm thinking about these in-person summits where the athletes are able to see the videos of other athletes in the missions, but do you ever share any versions of the deliverables with the athletes? Or provide briefs our round ups of what they learned in the mission? How are the athletes kept in the loop of how you and your team are using what they provided if at all?

Vanessa:

We share just about everything. Anything that goes in front of our leadership team around what were taking out of the platform, it also goes back to our athlete team. I think that's just something that I set up from the beginning is if they're going to take the time to give us all this feedback, the least we can do is make sure they're hearing from us on how we're using it and how we incorporated things. That quantitative summary that I sent you, the qualitative stuff, I'll send that to the team and I'll say look, if I've captured anything incorrectly based on what you had to say, please let us know. The last thing I want to do is use your quote and your name and all of this in a way that you wouldn't stand behind.

Vanessa:

That opportunity is always there with the team and I think you'll see as one of my closing slides here, we also just say thank you. We show their faces and that literally was stripped from the dScout platform so that was after one mission. So they literally could see, oh yeah, that's me and that was just an email that went out at the end of a mission. So that they saw, look, they pulled me down from this, they got my feedback. Sometimes they're also thinking that they're submitting things into a black hole. And it's like, no there I am. They heard me. It goes a long ways.

Ben:

Yeah, that back and forth has got to be so vital for a crew like this. I'm curious, looking ahead, now that you've been using a remote qual platform to do this kind of product focused feedback, human centered product feedback, are there other use cases that you think you would like to use dScout or other remote platforms for in the future? What other ways do you think you could use this sort of contextual candor and authenticity combination for, in any organization, I'm thinking about, we have a lot of folks on, who are attending, who might not be part of a hard product company but might have soft products are experiences or might be in services organizations. What other ways do you immediately see the impact by this platform?

Vanessa:

There's absolutely no reason that this only a tool that we happen to use with our athlete team. Right now, within the organization it was just where I saw the need and put the right tool to the right need, but we do a ton of consumer work, way more than we do athlete work. In those cases we do pre-screening for in-person ethnographies or as a replacement to in-person ethnographies in some cases, but almost always we will want to meet the people digitally before we spend three hours going on a hike or camping with them or other things that our organization does to get feedback. So if we can go through your closet virtually, that's a great way for us to get exposure to 200 closets immediately rather than going in 200 homes to try to see those closets. There's, in my mind, just endless different ways that we can apply it, it just happened to be through the athletes that I found the right example first.

Ben:

That's great. I love it. I know we've supported work on un-boxings I love that closet tour, steps of a journey, the path to something or a path to purchase, a journey map. It feels like when you break down whatever you want to learn into moments and you think about all the context you can get per moment, in aggregate what could you say, what could you do? Especially as you, I think importantly, reminded folks, you can ask quantitative measures, questions. You don't need a ton. We're not trying to replace your quantitative practice, but if you have a video, a few open ends and a few like or type or semantic differentials, you've got mostly the whole package. Then again it augments your fieldwork and augments your focus group work. It feels like you're really finding a nice balance between those things.

Vanessa:

Yeah, I think so. It's definitely like any research tool, no single one is right for everything, but it has been right for a lot, is all I can say.

Ben:

Well, I really do appreciate your sharing this story. For those of you who weren't able to join us at People Nerd San Fran, Vanessa was kind enough to share this story out there and so we had to drag her in, I guess to the interne and ask her to present again. It's such a cool use case. Again, you are going to get a copy of this. If there are any other questions, we have a few minutes, I want to make sure I get to those. We talked a little bit about honorarium, that is was a part of the contract of some of these folks. We talked a little bit about how it was Vanessa's centralized team working with other groups interested in accessing these athletes.

Ben:

I guess what happens, this is a good question, what happens if an athlete just didn't respond? You just didn't hear from someone at all? Do you have access to their schedules to know if they are, quite literally, on the side of a mountain?

Vanessa:

We do. We have a daily schedule where all of our athletes are in the world. So we know that some may be delayed in terms of the upload because they might be on an expedition or something of that sort. In general, if an athlete is consistently not responding to a mission that's something that I take up with our sports marketing team because it is part of the responsibilities of being an athlete. In those cases that's how we'll handle it. Ultimately, that person won't be part of the team going forward because we take it that seriously that that's a number one reason why we have you as part of the team is to be an ambassador and really help us get smarter and stronger.

Ben:

Yeah, and they're getting access to early data and alpha gear that should help them be safer and explore more wherever it is that they're doing that. It does feel like it's a great partnership between them and what is arguably such a great brand.

Vanessa:

Conversely, I would just say that those who are giving really rich feedback, I've shared that. The athletes go through a review process just like you and I might within our organization so I've certainly shared, so and so was just absolutely giving us way more feedback than they need to within the platform, but it's phenomenal. They're spending a ton of time and that goes, there's a financial impact to the athletes that are also just spending a lot of time in giving us the richness that we really appreciate that and we make sure they hear that as well.

Ben:

I've put Vanessa's info up here, if you'd like to get in contact with her directly. She has done this kind of work across a number of different business units and product units across VF, not just North Face, but the Smart Wool line as well. She's very sharp in not only using remote qual, but in using mixed methods broadly defined. She's really good at, if I may speak for you Vanessa, as your agent now, triangulating, you're really comfortable with and experienced in triangulating different research methods. Is there anything that you want to leave us with period or dot, dot, dot.

Vanessa:

I appreciate that Ben. For those of you that don't know, the North Face is actually moving on to Denver and so I'm also transitioning to a new adventure this summer. You can see there I'm actually starting up my own business. That's a new email for me, but will stay heavily involved with VF and North Face. I'm still super passionate about everything that I've done here over the past five years. It's been a great organization and the work is about as good as it gets as a researcher.

Ben:

I love your introduction when you said you hated being inside and so you needed to find a way to be a researcher who could be outside for the bulk of your work and it feels like you have found a pretty nice way of doing that.

Vanessa:

Yeah, that's right. It's all about outside is actually where I get most inspired. It is when we organizations and companies out an also just bring that outside in, which I think the dScout platform has been a fantastic tool to bring that outside in to organizations, which is ultimately what I want to do a lot more of in the future.

Ben:

That's great. Thank you again. All the best in your new adventures. Thank you all for joining us. Hopefully we'll see you both at UXPA and on the next call, we will share this recording with you. Thanks again. If we weren't able to answer your question, I'll be sure to follow up directly with you. Be well and we'll see you next time. Thanks Vanessa.

Vanessa:

Thanks everyone.