Use This Process to Source High-Quality Research Participants
Most of us have made mistakes in the recruiting stage at one point or another. Here's how to minimize those risks throughout the process.
When it comes to the importance of quality research participants, I’ve had my own fair share of mishaps and learnings. Take this scenario, for example:
I was early in my career as a qualitative researcher. I had tirelessly prepped a project for weeks—creating screeners and discussion guides, vetting recruiters, reviewing participant grids, etc.
We had traveled halfway around the world and finally arrived on the doorstep of our first in-home interview, clients in tow.
To my horror, not only did the participant not own the right product, but they were combative about our plans to video record the conversation. After 15 excruciating minutes trying to salvage the interview, I steered the group towards a graceful exit and onto the next interview with fingers crossed, my confidence and credibility both sunk.
Finding yourself in an apartment in Paris with a bad recruit is one end of the spectrum. But even with remote research, the impact of a recruiting mishap can feel just as detrimental. The pain of that moment inspires my work every day managing the dscout participant panel and designing both sides of the recruiting experience.
Jump to…
Setting the stage for qualified and engaged participants
Learning from the experiences of real people is at the heart of qualitative research. But who you talk to and the quality of their participation matters. Low-quality participation can leave you uninformed and uninspired, or worse—down the wrong path in your decision-making.
What makes a high-quality participant will depend in part on the goals and methodology of your research. Checking the boxes of a recruiting specification is only the start (see other recruiting tips here).
Below are some additional characteristics to look for in a quality participant.
Quality participant characteristics
✔ Authenticity
Is the person being truthful about their identity, experiences, and feelings? Masking your study requirements during screening can help you feel more confident about the responses since even the most well-intended participants can feel tempted to stretch the truth if they sense what you’re looking for.
Multiple-choice lists of varied items are more reliable than “Yes/No” questions or frequency scales, and open-ended prompts asking for details about a process or experience can help you suss out who truly knows something about it. But be watchful of AI-generated responses that say the right things, but sound overly formal or rehearsed.
✔ Communication and expressiveness
Can this person communicate well enough for me to learn from them? A high-quality participant is able and willing to share rich details and dive deeper on the topics you hope to discuss. One word answers won’t give you the nuance or stories that can make your research resonate with stakeholders.
But beware that rigid standards in what constitutes good communication skills can introduce bias and exclude participants unnecessarily. Perfect grammar probably isn’t necessary for someone to get their point across, but they should be able to coherently organize and present their thoughts. Open-ended written and video responses during screening can give you a crucial preview of what you can expect during the research.
✔ Effort and thoughtfulness
Do they even care enough to do a good job? Someone who isn’t willing to put in the effort to follow instructions or answer questions fully will waste your time and provide problematic, lackluster data. Ask questions during screening that require close attention to instructions or answer choices, or demonstrated effort responding to open-ended prompts.
Quality participation isn’t just all about the participant. Researchers can take steps to ensure quality participation and data as well.
Practices to ensure better data and participation
✔ Organize reasonable screening activities
Everything we’ve discussed so far could lead you to implement exhaustive screening steps. But go too far, and you’ll bias your sample towards professional respondents who have the time and motivation to do anything for the payout.
To get quality participants, be respectful of people’s time and data, and aim for the minimum information you need to make a decision. Consider your research goals and methodology. One-on-one interviews (IDIs) might require more rigor, but a quick unmoderated survey with a large sample might have more leeway.
✔ Set clear expectations
A good participant wants to do a good job for you—for their own sense of satisfaction, desire to please, or worry that their effort will not lead to the promised reward. It can be incredibly frustrating to them when expectations or instructions are unclear.
At every step of the research, from screening to fielding to payments, communicate what’s coming next and what the participant needs to do to be successful. It’s a win-win situation for both sides of the metaphorical glass.
✔ Coach and guide participants
Some people have the potential—but perhaps not the experience or instincts—to understand what would make them shine as a quality participant. This is especially true with fresh voices to research participation that we so desperately seek. Providing guidelines, coaching, and feedback can go a long way in training a well-intended participant to do better.
✔ Offer a positive research experience
Overall, research experiences that are respectful, fun, and engaging motivate timely, high-quality participation. Remember that you are working with real people, not data points. Model the effort and thoughtfulness you hope to get in return.
Wrapping it up
In the end, people are people, and not automatons who will do everything exactly as we hope. Participants and data quality will always have some variability. But keeping these measures in mind will help you—and your stakeholders—feel confident in your findings.
As for the past project I mentioned, thankfully we had over-recruited and had great conversations the rest of the trip. But it definitely dispelled some of my naivete and led me to find new and better strategies for recruiting—anything to avoid a repeat of the sinking feeling of an awkward encounter under watchful eyes in a Paris apartment.
How dscout can help
At dscout, the quality of our scout pool is of utmost importance to our business and is promoted through both behind-the-scenes and researcher-guided measures. Scouts are both vetted and trained during their onboarding process, and study responses are automatically evaluated for authenticity and expressiveness.
Researchers can ask for photos, videos, and written responses during screening, and are empowered to hand-select participants from a larger set of qualifying applicants. Researchers can also flag in the system any problematic participants that make it through these controls. We have a dedicated team to review and silently blocklist anyone who doesn’t meet our high-quality standards.
Amy is the Director of Scout Operations at dscout.
Jessica White (Director of Product Management at dscout) also collaborated with Amy to make this piece happen!
Subscribe To People Nerds
A weekly roundup of interviews, pro tips and original research designed for people who are interested in people
