Finding the Golden Moments of the Winter Games Experience

Hockey, ice skating, curling… in Canada, the Olympic Winter Games are ingrained in culture.

During PyeongChang 2018, more than 85% of Canadians tuned-in to CBC/Radio Canada’s coverage.

As Canada’s national broadcaster and the rightsholder for the Olympic Games through 2024, CBC strives to create a robust, complete user experience for Canadians, and wanted to learn about how its audience consumes content amid a changing media landscape. It sees the experience as a metaphor for the consumption habits of today’s audience.

CBC came into the project with two core personas, a casual Millennial consumer and an avid middle-aged fan, but wanted to inform internal assumptions with empirical data and user feedback. It hired TWG, a company that helps organizations like CBC/Radio Canada to build the software they need innovate, grow, adapt and change. TWG has strong user research practice, and was hired to validate and update CBC’s core personas and provide context on how Canadians were following the Olympic Winter Games.

Using dscout, TWG researchers asked participants who aligned with these personas to regularly show and explain their interactions. This longitudinal diary study offered a lens into the Olympic Games experience as it was lived, through text, audio and video inputs.

dscout spoke with Irfan Pirbhai, User Research Team Lead at TWG, and Craig Saila, Director of Digital Products at CBC, to discuss the project and dscout’s role in it. (The conversations have been edited and condensed for clarity.)

Can you tell me a little bit about your goals for the project?

Craig Saila: We wanted to use this event as a metaphor for how Canadians experience media in general and what happens if we shift the norms of television watching. Part of the intent of the study was to talk to a number of Canadians across different parts of the country and ask, "If you aren't going to be watching the big event on television, how will you supplement that and how will you learn about these different massive cultural events that we have?"

Our hope was that we'd be able to get some insights that we could then use as a model to fast track some of the thinking around other experiences like, say, an election or a concert, or any big news story. We wanted to know how changing and shifting media habits affect how people watch or experience these events.

Let's talk about how the study was structured and set up. 

Craig: We knew there were a couple audience segments that our team at CBC had identified in the Canadian audience itself, so we picked segments tied to traditional sports fans or traditional Olympic viewers and the clichéd Millennial social media user.

Irfan Pirbhai: When we started the project, we had a week to audit and orient, so we took some time to audit the existing research that CBC had. We did a number of stakeholder interviews with folks across the organization, and then we did a service blueprinting workshop where we tried to understand all the different levels of CBC's broadcasting, operations, and social media apparatus and how they all fit into assumptions about how their audience would actually be engaging with the Olympic Winter Games.

Was there anything about the way you structured the study that made it particularly effective?

Irfan:  From our first touchpoint, we maintained a pretty friendly tone with the scouts, using emojis and humorous language to try to get them interested and excited about the screener survey and the research project. We also framed this as a way to impact the subsequent experiences of all Canadians who engage with this event. Approximately 90% of Canadians tune in to these games in some capacity when they occur. So, we talked about this as a civic opportunity to be able to impact your fellow citizens.

When it actually came time to launch the study, we did a pilot internally just to work out the kinks of how we structured the questions. We launched one day of pre-study briefs on group calls with half a dozen folks who joined in remotely. Those calls were a way to prime them, not only on the goals of the project, but also how we expected them to fill out their responses.

We invited them to treat the study as if we were filming a documentary. We wanted as much detail as possible about their triggers, the sequence of events that happened, and their feelings related to the event. I think this helped them think about the project in more open terms.  Our prompts suggested that we were actually interested in things beyond the more prosaic media consumption activities. We were interested in their emotional landscapes, their frustrations, their pleasure points.

We really tried to follow up with genuine curiosity to participants’ responses. If they were going through difficult times in their personal lives unrelated to the study, we tried to be supportive and step outside of the role of a traditional researcher and be an empathetic voice.

How did you know about their personal lives unrelated to the study?

Irfan: We had folks who were experiencing real-life stuff throughout the diary study, and they were pretty candid in sharing what was going on. In some cases, we found that participants seemed to be using the diary study as a diary. They were sharing stuff that was going on in their lives outside of the context of the sporting event itself.

We had folks who were going through a job change and job loss. We had somebody who had just attended a funeral. We had a couple of people who went on vacation and were still updating their dscout diaries, even though they weren't engaging with the games.

All of these things just brought a level of basic humanity to the project. The subjects weren’t just consumers of media and Olympic content, and possible new audiences. They were people going through all types of experiences. Having that as the bedrock context helped us understand not only how the event fit into their lives, but how important that was in relation to other aspects of their lives. It really made it more of a three-dimensional study.

What role do you think dscout played in this openness?

Irfan: We've done diary studies before, but I think the mobile experience of dscout, and having that constant fixture in people's eyes over the 18 days that we had them, increased their comfort and familiarity, and ultimately, I think their vulnerability and trust in the dscout platform, with the study, and with us.

Craig: The power of the video for our audience was pretty strong. Because dscout is on the person's device at their own prompt, there may be a bit more candor and intimacy in their responses.

Talk to me a little about your hypothesis before the study and how it compared with the results.

Craig: We had strong hypotheses that most Canadians are getting their insights and cultural moments from a mobile device through some kind of casual social media, whether a formal platform [e.g. Facebook, Twitter] or texting or emailing back and forth. But we wanted TWG to go in a bit more open-ended.

The result of the study confirmed a lot of what we knew around the Millennials' social media behavior, but was really was surprising for a lot of us was the experience of the avid, avowed fans, the kind of people who typically watch it on TV, and how much their usage pattern was disrupted with a different time zone and what they were using to fill it in. The study added a lot of real value to how we're thinking about the next one and future events.

Irfan: We actually wound up with four different types of sub-personas for each of the personas we started with, including a sub-persona that we didn’t expect. With dscout, we discovered the a  binge watcher! This was a person who treated the entire sporting event like a Netflix television series. They made a short list of the types of sports that they were most interested in, and then they watched the pre-recorded events from beginning to end. They would literally watch the entirety of it up until the finals, to the point that even if there was a live event for that thing happening, they would not watch the live event. They would still watch the prerecorded stuff at the point that they had left off because they didn't want spoilers.

How did you analyze and present the results of the study?

Irfan: After the study, we booked one-hour calls with subjects, one on one. These post-study interviews were an opportunity to go deep into the patterns that we were seeing for each individual and try to get into the nuts and bolts of why they were behaving in those ways, or what was really behind some of the frustrations or the pleasure that they experienced in these particular moments.

We took a lot of info from the daily responses and we mapped out every single day to the broadcasting schedule. Then we could see their engagement level exactly, and what they did in context with the live events that were streaming. That showed us who was tuning in live versus watching a replay, and what sorts of events were the really engaging ones.

After identifying the personas and packaging them up into slides, we took that spreadsheet and created two types of experience maps.

One map was a more longitudinal experience map over the 18 days. We were actually able to map out the importance of the event for these participants over an 18-day plotted map for each persona. For that, we were able to see when the self-reported meaning of the games peaked and when it dipped, and we could cross-reference that to certain events that were happening.

We were also able to do a “day in the life of.” We took a few of the participants from each of the persona groups and we plotted an experience map over a 24-hour period. We did one on the weekend and one during the weekday to highlight the differences. We actually included quotes and video in these high moments or low moments, and that's what we shared with CBC.

The format was like a keynote deck, but it was totally interactive. We could single-out a particular high and there would be a video snippet we would play, so that the folks at CBC would hear it from the participants’ voices and see their faces.

Craig: We found dscout incredibly effective because it’s just so deep and comprehensive. In a stakeholder presentation, we could capture video clips, show a customer journey, and illustrate how old and young people are the same in terms of their media behaviors.

How did you feel about dscout compared to other research methods you’ve used?

Craig: One of the big advantages we saw with dscout was the ability to capture the depth of data, including all the different points of customer response. So not only just the logging and tracking of what they were doing when but also the video responses. That consistency of that video response over that two-week time frame, along with that activity monitor was a very powerful illustrative example for stakeholders and a good way for us to consider how we wanted to pace out lot of the products. Looking at that customer journey timeline and the moments when people felt in-tune and not in-tune with the event provided a whole range of ways for us to analyze that set of information.

Our traditional audience research is more survey based. dscout represents a big shift for us in how we're thinking about approaching our audience, to be truly audience-first, understanding people using a CBC experience in that particular moment and creating an empathetic response within the product.

Irfan: In user research, video is currency. The mobile ethnography we did with dscout for this project allowed for a tremendous depth of understanding about our users, more than any other project I've worked on. We built relationships with our participants and got prolonged exposure into their mental models. It's that kind of understanding that truly great products and services are built on.

Ready to start building your own research project? Sign up for a free dscout account to see what's possible.