Change.org: Turning Limitations into Lemonade

Editor's note: Every human experiences time differently. Yet we’re all pretty fixated on getting as much of it as we can. We worry about finding it, losing it, running out of it, having none of it...Not long ago, Aruna Balakrishnan was one gear in Google’s impressive research machine. Today, she's the whole research team for Change.org. Time is often her most precious resource, but she strives not to let it be a limitation. Her users, after all, are aiming to change the world. Helping stakeholders understand those users is her contribution to those world-changing efforts. This year, she engaged dscout to help her do it.

There’s an old saying that “seeing is believing.” In research, the saying is, “hearing is believing.”

Let’s say I did some UX research, wrote up a report, and handed it to you, my stakeholder. If you haven’t witnessed the findings, then it’s just me, giving you my opinion. It doesn’t quite have the same impact as you hearing about user experience directly from...the users.

Video clips can make a big difference, but in my first year and a half at Change.org, I showed colleagues maybe 10 of them. Frankly, I just didn’t have time to do it. With traditional qualitative research, I get tons of audio data, tons of video data. When you have a 60-minute video session, it takes a lot of time to find just the right clip to illustrate a finding. Then I have to grab it, clean it, and find a dozen more that are similar or just as powerful.

Hello, mobile diaries

Recently, I was introduced to dscout in a qualitative research forum, and then again at an event where Fitbit and Troika researchers were explaining how they use it. After running two projects myself—including a diary study on Change.org’s new Android app—I would argue dscout delivers one of the more powerful formats of user data we have.

I hadn't the bandwidth to do much more for our new app than some basic usability. So this dscout UX research study became our in-depth research project for two things: ethnographic formative research around when people confront issues they care about, and an evaluative app beta test.

Using the dscout app, study participants answered open- and closed-ended questions, and they recorded certain answers with 30-second videos. This data was filled with so many insights because users focused on what was most important to them, and this seemingly small amount of data carried a lot of context.

Of course, if your study has hundreds of people—or even just 30 participants—you still end up with a lot of videos. When you have a lot of videos to review, having those clips pre-made is research gold. I was able to rapidly choose clips—without worrying about cutting them—and immediately share them. It made the data so much more accessible to our team.

Where limits serve a purpose

Even running a traditional diary study would have been impossible for me as a one-person research team. Recruiting, managing the participant communication, keeping track of who’s done what, and taking the time to analyze all the data—both the qualitative and quantitative coming in—is a huge undertaking to do manually.

The dscout platform simplified the process, eliminating much of the logistical coordination work and letting me focus only on the information I wanted to get and the information that I did get. At first, the restrictions around the number of questions you can ask and the length of video scouts could record were, admittedly, a bit irritating. Then I realized those limits served a purpose: focus.

When you have limited resources, you have to focus on exactly what you want to get out of the study. It contains the amount of data that you're getting. That makes analysis approachable and achievable quickly and easily.

That big moment

We learned so many exciting things from this project, such as how important local topics were to our users. With our new app, you can share your location, so we can showcase relevant local petitions. It's obvious that people want relevance, so we knew it was going to make a difference. But their reactions got us thinking about how to translate that user experience and journey onto our web experience.

We captured so much feedback like, "Wow, there are things happening right around me that I could get involved in!" That huge moment of joyful user experience inspired us to double down on our efforts to localize. We did that research—from set-up to analysis—in four weeks. We hadn't anticipated that it would be both so fast and so powerful.