The Selfie Matters

Digital and mobile technologies make screening for research participants easier.

And…harder.

Lower physical barriers mean more screener applications. Unfortunately, that doesn’t mean the most qualified participants rise to the top of your inbox. For that, you need some specific strategies.

Say you need 100 participants between the ages of 18 and 35 who own smartphones and receive multiple phone notifications every day. And say you’re shooting for 50:50 male:female ratio with a mix of people whose notification frequency ranges from a several times a day to just one or two per day.

All mobile research participants are obviously using a smartphone, so recruiting qualified people who get app notifications is pretty simple, right?

Sort of.

Take this respondent: a 26-year-old male who gets hourly notifications. He looks great on paper. Then you dig deeper, asking open-ended questions, and requiring answers in both text and video form. You find his written responses consist of short sentence fragments. He shot his video in his office cubicle, so he’s whispering the whole time.

Now what?

Just as you would avoid recruiting a potentially silent or overbearing focus group participant, recruiting “just right” mobile diary study participants is a worthwhile challenge. These users are critical to developing your insights, and it’s their voices that shine through in the quotes you present to stakeholders.

In analyzing dscout screener responses, we look for three “soft” criteria — we call it Read/Say/Write. Then, not only do we hit our recruiting numbers, but we have also confidence that each participant will be a valuable contributor to our project’s outcome.

Can you read this?

Sometimes applicants get too excited about the research project and will read only half of each question. That translates to a big loss in feedback given we have limited opportunities to ask each question. So on the screener, we check each applicant’s ability to carefully read and respond to instructions. It’s important to choose participants who can be both enthusiastic and attentive to detail.

Screening for this skill is surprisingly easy: Just look for consistency across answers. Here’s an example:

John D. indicates in Q5 that he receives multiple notifications throughout the day. Then in Q7, he says GrubHub is the only app that sends him notifications. Sure, it’s possible that John D. REALLY loves takeout for breakfast, lunch and dinner. But probably not.

It’s definitely worth a follow-up to find out. Maybe John D. is a great fit for your study. But odds are good that he did not carefully read one of those two questions. If he’s not closely reading the screener questions, he probably won’t be closely reading your study questions either. Keep searching.

Can you say this?

Screener questions that require video responses are extremely beneficial for determining if an applicant will be an enthusiastic, thoughtful, and informative participant — or none of the above.

Selfie videos give us a brief glimpse of someone with whom we could be working for many weeks. We’ll be asking for more videos during the study, of course, and we’ll be hoping to use them in our final presentation — so they need to be good!

When we watch screener videos, we look at applicants from two distinct perspectives — as an engaging respondent and as a competent amateur videographer:

Applicant as respondent

• Speaks knowledgeably about the topic at hand.
  *Make sure they weren’t trying to make it into the study by gaming the screener
• Composes visually functional videos.
  *Camera-shy respondents often try to circumvent the video requirement by aiming their camera at the floor, or by shutting off all the lights. What do you see in the frame?
• Presents themselves appropriately.
  *Read: Is fully clothed.

Here’s a really top-notch video we received in a past screener.

Can you write this?

We have yet to write the formula or AI code that auto-differentiates great responses from mediocre ones. But we’ve found that applicants who write responses in a few full sentences (not a few words or fragments) will typically follow up with thoughtful, articulate feedback in the real study.

When we have an easy screener — one with a less stringent set of recruiting parameters, and thus have fewer screener questions to ask — we take advantage of that latitude to ask questions that convey more about our participants’ lives.

Later, we can use these responses to draw colorful descriptions of our participants for stakeholders. Choose who you’d want in your study:

Q: We want to hear what your perfect day would be like. Where would you go? What would you do? Who would you see? Money’s no object here. Anything goes! There’s no right or wrong answer. Just do your best to be creative and think big.

Respondent A

Wake up and take a cycle class with my favorite DJ spinning live (currently Disclosure or MNEK), then shower and go to an amazing healthy brunch/lunch on a patio with my friends, followed by a few hours at the pool on a perfectly sunny day. Only 75 degrees. Then the temperature would suddenly drop to 60. I would pull on my boots and vest and go to an apple orchard with my boyfriend. We’d get hot cider and go on a hayride. Then, suddenly it would be thunderstorming, and I’d come home to snuggle up on the couch and watch a few Netflix documentaries. The end!

Respondent B

I could wake up whenever I wanted, no housework or schedule, go outdoors, maybe the beach all day, then watch TV all night.

When you have a huge pool of screener applicants who qualify with your hard criteria, it may be tempting to just pick the first 100 and start the study. It may even seem like an impractical burden to search for the best ones. But it’s not such a daunting task when you use the three soft criteria — read, say, write — as a starting point, especially when you consider the two most valuable rewards you’ll reap in the end: deeper engagement and easier insight development.

Engagement

Scraping and rescraping data are inevitable when you’re aiming for thorough, insightful research analysis. You know you’ll be watching, rewatching, coding, quoting and paraphrasing all of your data.

Challenging.

Now, imagine watching and rewatching, coding, quoting and paraphrasing 100 videos devoid of thoughtful content on the topics you asked about — and shot from the “up-the-nostril” angle.

Painfully challenging.

It’s significantly more enjoyable, not to mention useful, to work with text and video responses from 100 thoughtful, articulate participants. Working with engaged participants makes it easier for you to also be engaged. And your engagement translates to more meaningful findings for and engagement by your stakeholders.

Insights

Quality participants are good for your mental health, but they also make it naturally easier for you to get behind your own insights because what you’re sharing with your client is authentic and meaningful.

Consider your insight development process. When you’re using data from participants who not only fit the target demographics, but who are also smart, communicative people, then you can let your data do the heavy lifting.

Gleaning information to find the most interesting patterns and ideas is so much easier when you can get out of the way and let your participants run the show. And isn’t that what we’re all really after?