Changing the Game with UX Research

Just before we spoke on the phone, Veronica Zammitto spent a week in the Canadian backcountry mountain bike racing.

“It was seven days back to back,” she told us. “Multiple hours on the saddle each day. I was really pushing what I had been doing in terms of endurance.”

It’s a demanding sport—but that’s something Veronica was well equipped to handle. She’d spent the better part of her career leading research for some of tech’s most passionate and demanding users: gamers. And during her time with Electronic Arts (EA), she helped introduce, develop, and eventually helm their entire UX research process.

By the time she left the company in April 2019, she’d scaled her team from two to 35 people, across eight different international locations, and had managed EA’s robust portfolio of games including FIFA, Battlefield, Mass Effect, and NBA Live.

She took some time away from the trail to talk to us about growing a company-wide UX culture—and what she learned during her decade-long tenure at one of the world’s leading video gaming companies.

dscout: You were with EA for almost a decade before you left earlier this year. What brought you to the company in the first place?

Veronica: I came onto EA to start building UX research there. Even though things like play testing have been in the gaming industry for quite some time, it was done by producers and designers. These are people who have a lot of willingness to listen to users, but didn't really have the research skills to do so.

My background was in psychology and human-centered interaction. So when I got to EA, we began to ask: How can we really better understand users? And that was the start of the discipline there.

It sounds like there were a few roadblocks to getting the UX research process started. What were the biggest pain points when you first came on?

EA knew that, from a methodological point of view, they were struggling. They didn't outright say that it was methodological. They said things like, “We want to collect more information,” or “We want to be a little more systematic.”

Mostly they wanted to be more unbiased, and learn how to start scaling and aggregating data. The pain points were figuring out how to polish their own methods and aggregate data from multiple users in a way that really helps you to get better insights.

So how did you approach moving the company past that?

Because they’d done some early play testing, I brought in methods like eye-tracking and other psychophysiological measurements. I was really trying to push the envelope on those.

But when I looked under the hood of those initial play testing questionnaires, I realized that we still needed to establish solid foundations. So we worked to figure out how we were going to properly write questions for users. Or how we were going to set consistent usage scales, so we could compare apples to apples. Or how we’d execute usability studies, and moderate sessions without worrying that we were probing.

It was really establishing a good, strong foundation on multiple techniques and then setting basic processes for what “good” looks like.

Over the course of your time with EA, how did you see your role in UX research change? I would imagine that there were industry changes and changes in user expectations.

Absolutely. You're right. It was a long and amazing journey [with EA] with a lot of challenges. One of the biggest changes involved scaling, and there were two sides to that.

One was about scaling the research team itself. The other was scaling to do research with more players. It was tricky in the sense that I had people across multiple locations—Australia, Europe, and the East and West Coast of America.

So there was that tension between how we actually maintain the portfolio of the work that's being done while bringing in new, high-quality researchers. We were trying to establish basic practices with a sense of connection so everyone could truly, actually lean on each other while also being able to do their day-to-day work with their local development team.

So the second phase of that journey at EA looks like scaling the research team and partnering with different development studios. And while it’s been challenging, it was extremely exciting to develop other types of relationships across the company—working with senior executives and leaving them wanting to have UX research in their own house.

That was part of the reason why we were growing. They were starting to see the work that could be done, and seeing those results back. They saw how to apply those insights in their own products. They realized if they wanted more of that, they needed to have someone who could be on the floor with their development team to actually do it on a more regular basis.

And at any one time your team was working on multiple properties at once.

Absolutely. Research teams pair with development teams at different studios, but it's only one research team that covers the entire EA portfolio. So that could be everything from Battlefield to FIFA to Apex Legends.

Of course, some researchers would be more dedicated to certain properties and develop that expertise. But at the same time, everything is centralized, which is super helpful with creating those high level insights.

For instance, the sports games like FIFA, NHL, Madden and NBA were all unique—but they all shared aspects of sports gaming. They’re all a simulation. They all integrate real-life performance. They involve team play, and rely on different types of modes that can be applicable to all sports titles.

Overall, I would say that we were a centralized team, but with people embedded in certain groups. It’s more of a hybrid model. And I think that that has the sweet spot between that tension of being central and being distributed. It’s the best of both worlds. You can integrate your insights but also have people on the ground working next to the designers.

You’ve worked on a ton of different properties in your tenure with EA. So I have to ask: What’s your favorite game you worked on?

One of my favorites has to be Battlefield 1. It was a really good improvement to the franchise that I think actually gave a rebirth to Battlefield.

I think that the introduction of the new mechanics in the game were amazing, and the scale of it was more epic. We also redesigned what the multiplayer shooter looks like, and were pushing the envelope on what single-player shooters are and how stories can be told through them.

We wanted the stories to be short to make sure people could complete them while also giving the player enough to understand the dark side of war.

I actually got done playing Battlefield 1 today. It’s one of my favorites.

Oh, wow. I’m so glad you like it too!

It was a stark departure from other Battlefield titles—and the first game of any large first-person franchise to dive into World War I. Was that something users were asking for?

That's a great question. And it was actually the designers who wanted to bring that challenge of building a World War I game to themselves. Like you said, it’s a period of time that has not been explored in the shooter world. They had a really good idea of how they wanted to move in that direction.

And actually, the very early studies that marketing did were not very good in terms of people wanting a World War I game. Certain segments could not differentiate World War I from World War II. There was a little bit of concern at the beginning that people might not even get it.

So it wasn’t exactly an ask from the users. But it was a goal on the creative side—and user response informed us on the creative challenges. We learned from research that there was a gap in understanding—that we needed people to recognize where they were in time and how to actually play these games in this new environment. We needed to make sure to give users more cues. For example, adding certain text which clearly keeps you gauging that timeframe. Do you remember the opening scene?

Yes! It's the one where you’re in the company of Harlem Hellfighters and every time you die, you see the name of the soldier you were playing as appear on-screen along with the year of their birth and death. Then you immediately switch to another soldier and play as them until you die again. Rinse and repeat for the rest of the level.

Exactly, that one! So you see the years of birth and death. You can't miss seeing that. It's just eight digits and a random name. But it gives the user immediately a sense of, "You are in this time and space." And you also get this idea that you are not expected to survive.

That was our way of telling the user, "Wars are meaningless. You cannot win such an event. Look at the outcome.” There were those little cues that people could not miss that helped them to recognize and orient themselves. So by the end of the development, there were pretty much no concerns in terms of people really picking up where they were in time.

With each industry comes different users, and different insights on those users. What are some of the most surprising things you’ve learned about your users?

Gamers are easily one of the most passionate types of users, and that is pretty amazing. They really love the game and have an impressively high level of dedication. I mean, people get tattoos of their favorite game! Have you heard about that happening with other tech products? People will do it for games.

I can see that level of passion and dedication in your users being good and bad.

Yes, it’s a bit of a double-edged sword. Our users are very passionate—but they’re also very demanding. The standards are so, so high that you have to deliver a great quality product; otherwise, people won’t be satisfied.

The players are very earnest in their love and dedication. So if it’s not up to their standards, it’s going to go back to development. We had that type of experience with Mass Effect: Andromeda, where some users were very vocal about how unhappy they were about the ending.

The stakes are high—much higher than, I would say, many other tech products out there.

What's something you’ve learned from working with such a high stakes product?

I really think that UX research is about coming full circle. It's no longer just about understanding the users, how they interact, what their motivations are, and their behaviors. It all has to do with building better products.

Researchers have to be responsible for what happens with their insights. Are they being acted upon? Who on the team got assigned to it? If it wasn’t acted upon, why? We have to think about how to work next to the development team, and really help to translate findings into product improvements. We have to follow up. I like working closely with development and I would urge all researchers to do the same: to own your insights, and to really ensure they’re used to make our products better.