Mobile user research and demand characteristics

In social research, self-consciousness is a concern to be reckoned with. Human beings are, after all, a self-conscious species: reflexively defensive and strategic about the way we appear to others, hoping for their acceptance.

We can’t help it. We’re inherently social, tribal creatures who thrive on communication and connection. We like people, and we want other people to like us.

Knowing others might be watching us, however, can arouse fierce existential anxiety. We have nightmares of being naked in public. We worry about giving a speech and remembering none of the words. We fear being judged.

In social research, self-consciousness manifests in a phenomenon known as “demand characteristics.” Sometimes a research participant, knowing they are under a magnifying glass, changes the way they act or the things they say. It can be subconscious or deliberate. It can be well-intentioned or malicious. In any case, that behavior modification affects the research at its most fundamental level: it degrades the authenticity of the data.

As a researcher, I’ve seen the damage inflicted on my data by demand characteristics. I once conducted interviews about online fan communities, a topic that I knew could become heated. One participant went off on a passionate rant, finishing with a slew of choice words, none of which are fit to print here. I was delighted. This was precisely the kind of answer I always hoped for, revealing all sorts of juicy ideologies about the way people should behave online. You can imagine my dismay when the participant promptly apologized for swearing and asked me to strike that answer from the record.

That tension between subject and researcher is the demand characteristic in action: the subject wants to do a “good job.” The researcher wants authentic data. Sometimes, these goals can be at odds.

Rising to this challenge is mobile research methodology, a perfect extension of our obsessive attachment to our smartphones. People can answer questions, take photos, submit video, and become data collectors in their own right, choosing when and where they want to deliver data. Participant observation, the most revered of social science methodologies, has been redefined and adapted to a digital age.

Mobile research often seems to escape the demand characteristics sandtrap. The moments and stories these participants have shared with me have ranged from the profound to the scandalous. There are heartfelt anecdotes about lost family members, trauma, and participants moved to tears by television. There are also participants who have shared stories about their sexual history and have filmed video in the bathroom.

Some of it is intensely moving: they are sharing intimate moments with us, things they might only tell good friends or a diary. Some of it is… not. Many participants have shown up shirtless and some of what I receive is, ironically, not safe for work. Yet, all of what I have seen convinces me that participants are, by and large, unconcerned with how I might judge them.

Mobile research is unique in both its intimacy and distance. This isn’t a paradox, but a potential resolution that digital technology brings to the problem of demand characteristics. I have two hypotheses about why and how mobile research circumvents that.

The first of these is what psychologist Rick Zubin described as “strangers on a train.” Feeling safe in anonymity, we sometimes disclose the deepest, most personal details of our lives to complete strangers. It’s an old folklore, reimagined: Whisper your deepest secrets into the hole of an old tree, and your secrets will stay in the tree forever. Tell your secrets to a stranger you’ll never see again, and you earn all the catharsis with none of the residual misgivings.

My second hypothesis is the well-documented phenomena that computer-mediated communication, or CMC, increases self-disclosure. The same anonymity that compels us to share with strangers on trains compels us to share with strangers via chat rooms, forums, blogs, and social media of all kinds.

When you break it down, a face-to-face conversation places substantial demands on the people talking: to keep a conversation going, we must craft appropriate statements within an appropriate timeframe and show our engagement and interest with an extensive repertoire of body language, from the subtle to the blatant. Compare this to CMC, where body language doesn’t exist and a timely response is not always necessary.

Mobile research functions in a similar way: to research participants, the researcher is a stranger, a voice who sends reminders and helpful feedback – never judgments about who they are. We don’t have a very strong presence; we’re embodied by very small screens, push notifications, and short interactions. The screen shields participants from the researcher, and our lack of presence makes it safe for participants to disclose to us. Participants can take the time to think about what they want to say and then respond on their own time.

These are only hypotheses, of course, and any good researcher has a variety of methods in their toolkit. Anyone who has so much as glanced at the Internet lately knows that anonymity can be abusive and dangerous just as much as it can be liberating. But in nearly three years of mobile research, the participants in my projects have been almost uniformly polite, responsive, and forthright. It’s not enough to merely receive excellent data -- we should also question why a method works. As mobile methodology and technology both move forward, we should also look inward: why we connect, how we connect, and how research has a place in the intersection.