Using Private Research Panels for Ongoing, Rapid Customer Feedback

After a successful research project, we may walk away feeling more connected with our users. Only a few days (or weeks) talking with a group of scouts and we develop a slightly better understanding of their lives and how they could (or do) interact with our product/service.

Instead of running your next project with an entirely new group of participants, imagine how much more you could learn about your core users by meeting with the same participants regularly.

Private panels can be an effective way to quickly tap qualified participants for multiple engagements to help answer your urgent, iterative research questions. They are often used to help build company-wide empathy for the user since you get to know them over a long period of time.

To outline the value of panels and showcase how to maintain one in dscout, CXR Research Advisor Lauren Duquette shares tips on how to navigate one successfully. Lauren has become dscout's internal private panels expert. She's iterated on ways to meet customer needs, refining the approach she'll outline below.

Her experience as part of dscout's training team (onboarding new hires onto the platform) and her combinations of SME and platform experience give her a unique view on building this type of study.

A note: we definitely think of remote-qual through a “dscout” lens—but many of these tips can help improve your remote project implementation on any platform. Some dscout specific vocab to be familiar with, in case you’re not a user: we call our research participants “scouts” and our studies “missions.”

PSA: With dscout Private Panels, you can leverage features like a filterable panelist database, one-click "add to study," and pre-screen, invite, and scheduling reminders.

Check out our other features and learn more about how we can help build your next private panel.

Jump to:

• Defining a private panel
• Benefits of a private panel
• Private panel use cases
• When to not use a private panel
• Eight strategies for successful panel building
• Private panel templates
  

What is a private panel? What makes it "private?"

A private panel is a group of screened participants (or scouts) hosted on dscout for a designated period of time, with the expectation of being tapped for multiple engagements.

These participants are housed in a specific location (in our case, a diary mission), separate from a larger pool of scouts or participants. They are a subset of the larger population and can include dscout scouts, folks you bring yourself, or a mix of the two.

What are the benefits of a private panel?

Some customers enjoy the community-building a private panel offers. Participants feel part of something larger than any one individual study or activity. They get to see their feedback used in the product or experience, which is very motivating for them.

This often leads to stronger participation and better quality data—it moves the relationship from transactional to cooperative. They have a better sense of how the product has grown, which sharpens their ability to provide feedback; the loop feeds itself in a very innovative way.

This community-building effect works both ways, too. The companies I've helped launch private panel programs report heightened awareness of the user. Seeing the same set of faces helps build the user to product connection.

It's easy for a product person or engineer to hop in and listen (or even speak!) to a specific user and hear the reaction to a development or improvement. The community nature allows for stakeholders beyond UXRs to passively or actively build relationships with users over time.

It's fun to hear an engineering or product person ask about a specific user's reaction to a development or improvement. In this way, the company feels closer to the customers they serve just as the customers feel more valued and included.

Another benefit is that once you've created the recruitment criteria—and settled on those criteria—you don't need to go back again and again to find folks...you already have them! It's a time-saver and confidence-booster. The quality check was done with the first screening.

This all means that you and your team can get moving more quickly on research and spend less time wondering, "are these folks a fit for this project?" This is the most common benefit I hear from clients leveraging private panels.

Recently, as research and insights have become more ubiquitous throughout orgs, many clients have begun cross-checking and supplementing participants’ reported behavioral data on dscout with usage data collected on their platform/app’s backend.

A BI or data science team may have intel on truly how frequently someone is encountering, say, a certain popup message or whether or not a user is experiencing a bug, allowing user researchers to confirm, supplement, or deny what a participant is saying.

This becomes even easier and more meaningful with private panels, my clients have found, since these panel participants are more familiar with your feature/product than the typical user and you have much more info and context about each participant compared to a scout in a typical one-off mission.

Private panel use cases

Here are some ways that you might use a private panel:

✔ Ongoing refinement

As you're developing a new app or feature, recruit a target market to gut-check and refine as you, your product, design, and eng teams start spinning the experience up.

✔ Market research

Maybe you aren't building anything yet, but you'd like to keep your finger on the pulse of a subset of folks. Use a panel to dig into how this group feels about a current event or specific topic relevant to your weekly team meetings. You could pair this with your marketing or PR efforts to check how "brand-aligned" they are.

✔ Needs-based building

This is really a combination of the first two. Recruit your target group, then concurrently ask for feedback on how they currently leverage your experience and show them new versions of it, refining and evolving as needed. See how our friends at Headspace used needs-based building.

When should you NOT use a private panel?

The hard work of building a private panel is mostly up-front (although maintaining community is equally important—we'll cover that soon), so if your user base is still being defined or your recruitment criteria change project-by-project, a private panel might be a stretch.

Having to re-screen and potentially remove folks from your panel is not only a lot of rework, but it's a bad experience for the folks comprising the panel. If you don't have firm recruitment criteria, I'd work on that first.

The other big consideration is time. Setting one of these up can take some of it—especially up front. A careful screener to get the right mix of folks, ongoing studies to keep them engaged, regularly messaging for updates, and even reviews to check that the panel mix matches your needs is a much different undertaking than a series of one-off studies. I would argue that effort is worth it, but it's still effort, so if you're regularly time-strapped, this might need to wait.

Now that we've covered a bit of what a private panel is and why you might want to create one, let's explore my best practices for doing this in dscout.

8 strategies for successful panel building in dscout

Below are my pressure-tested suggestions for how to run a private panel on dscout. There are a lot of creative ways someone could hack together a panel on dscout—these steps have produced impactful outcomes more times than not.

Do keep in mind that building a panel on dscout is a scrappier way to use the platform and does require some flexibility in terms of experimentation and risk-tolerance. I can't guarantee there won't be surprises along the way, but the method below has been tested and has the fewest gray areas to work through.

When in doubt, contact your dscout account team for additional advice.

1. If screening via dscout, work with your Research Advisor to create a screener

It’s important to set strong expectations in the screener, including how long scouts can expect to remain in the panel and the minimum reward they can expect to earn if selected.

2. If recruiting your own participants, set strong and clear expectations while screening

Let participants know during screening that dscout is a trusted research partner of your organization and that they’ll need to be comfortable downloading the dscout app and submitting selfie-style videos and photos.

Be sure to also detail how long they can expect to remain in the panel and the minimum reward they can expect. Learn more about how to onboard new participants to dscout in this guide. Participants brought directly into the engagement mission via mission webpage will not be added to the dscout participant pool.

If you’d like to screen your own participants via dscout via screener webpage, please note that they will then be added to the dscout participant pool.

3. Select and invite your participants

If recruiting through dscout, select your first wave of participants by following the steps in this guide. If you’d like to keep the screener open so you can invite additional waves of participants as necessary, please work with your Research Advisor to increase the application limit.

Then, launch your engagement mission by following the steps here (or here, if you’re bringing your own participants).

4. Create your engagement mission (panel house)

It’s important to house your panel in a long-term Diary mission, to keep scouts engaged and keep the panel top-of-mind (especially if you’re bringing your own participants). This’ll also allow you to message scouts within dscout.

We recommend drafting at least one or two parts for this mission before your initial launch and then opening a new part/activity each week based on your research/feedback needs for that week. Your engagement mission can have up to 10 parts. If your panel will be running for more than 10 weeks, or you need more than 10 parts, you can create additional engagement missions as necessary.

5. Closely manage your engagement mission

It’s important to send frequent (and if possible, personalized) reminders to your scouts in your engagement mission. Be sure to keep scouts updated, provide frequent feedback via entry comments and messages, and overall make them feel they’re a part of a larger community.

We recommend checking in via message, comment, or with a paid activity to complete at least once a week. Without close management, there will be much higher drop-off.

6. Develop a compensation cadence with your Research Advisor

To keep scouts engaged and encourage them to provide high-quality data, be sure to develop a clear payment cadence. We recommend compensating active members of the panel after completion of each part in the engagement mission as well as after each additional activity (such as a multi-part Diary mission or Live mission).

7. Develop and invite participants to one-off missions

Select a subset or all of the scouts in your panel to take part in one-off Live interviews and/or Diary missions to quickly answer your urgent research questions. Be sure to give the scouts you select a heads up in the engagement mission about when and how they’ll receive their invitation to these extra activities.

I also recommend keeping a consistent mission naming convention, such as “[Engagement Mission Name]––Paid Activity #X!” so scouts can connect the dots between the engagement mission and other paid opportunities in their apps.

8. Thank your participants for their time

Once you’ve gotten all the feedback that you need (or you’ve reached the end of your 10 part engagement mission), let your scouts know that the panel has come to a close and thank them for all of the feedback they’ve provided.

If possible, it’s always a nice touch to let them know how their feedback has been/will be used and where, if anywhere, they can keep an eye out for the feature, product, or service they’ve helped build. Then, close your mission!

A few templates to get you started

Here are some copy templates to help you start communicating and setting expectations with your scouts:

✔ Check-in message example copy

Send a message like this on an off-week where you might not have a paid activity for your panel.

Hello, [Mission Name] Community! Thanks again for all of your helpful feedback last week. Since your last paid activity, we’ve been hard at work taking your thoughts and opinions into consideration while building our app/feature. Stay tuned for your next paid activity (which should become available early next week). It’ll be all about [topic]. Bonus! You may see some of your ideas from Week X baked into the design we’re showing you!

✔ Positive feedback comment example copy

While perusing your panelists’ entries as they come in, leave feedback as entry comments to encourage quality responses.

Hi Name!

Great work on this entry. What you mentioned in QX reminds me of something you said during Paid Activity Y. Would you mind speaking more to your point here?

✔ Engagement mission part open message example copy

Send a message such as the following when opening up a paid activity within your engagement mission.

Happy Friday, everyone! It’s the end of Week 5 of this community, which means your next paid activity is open for you to complete. Please submit all 3 required entries to part 5 by *11PM on X/XX* to earn your $20 reward.

As always, you’re welcome to opt out of participating in any paid activity by sending me a message, however you will not receive compensation if your entries are submitted past the due date.

Let me know if you have any questions! Keep up the great work!

✔ Additional paid activity opportunity example copy

Send this type of message in your engagement mission when inviting panelists to a separate related Diary or Live mission .

Hi there! Thanks for being a member of this feedback community. I wanted to let you know that we’ll be sending out invitations for our next paid activity, [Live Mission Name], *tomorrow morning*. We’ll be showing you a few designs that we’d like your honest feedback on.

Time slots for these interviews are on a first come, first-served basis, so be sure to accept the mission, pass your hardware and network pretest on your computer, and schedule your time slot *ASAP* once you receive your invitation.

Looking forward to chatting soon!

✔ Screener teaser example copy

A teaser is the first contact you'll have with potential applicants from our scout pool. It acts as an ad to entice them to apply to your study. It should give away just enough so that scouts get the gist of what they're applying for.

For more context on what exactly a screener teaser is, explore our 5 Major Lessons from 1000+ Participant Screeners piece.

Calling all social media users! We’re interested in the ways you use social media in your day-to-day life. Apply now by answering a few questions and recording a short video.

Selected participants will be invited to participate in a series of Diary missions, Express missions, and/or Live interviews over the course of 10 weeks and earn at least $25 total (depending on the number and types of missions you take part in).

✔ Screener opt-in example copy

Include this question in your screener (via dscout or outside of dscout) to set expectations with your participants!

If selected to participate in this opportunity, you will be invited to a series of Diary missions and/or Live interviews. You will be compensated for each mission/interview that you fully complete. You will be expected to be available for these missions for 10 weeks. We expect the time commitment to be less than one hour per week. Is this something you’re willing and able to be a part of?

• Yes
• No

✔ Mission overview example copy

The mission overview officially introduces your scouts to the project and is a great space to get them excited to participate.

Welcome to your mission! We’re looking forward to getting to know you better and hearing your thoughts and opinions about social media. Your insights will be used to help improve the experience for users like you!

Over the next 10 weeks, this mission will be your home base. You will have a new part unlocked each week, which may require one or more entries to be submitted. You will be compensated after each activity or part in this mission that you complete.

You may also be invited to participate in additional Diaries and Live interviews, for additional incentive. You’ll be notified here when you’ll be invited to participate in such an activity.

We’re excited to hear your feedback! Please reach out if you have any questions about your participation in this community.

✔ Engagement mission example part use cases

• Provide survey-style, one-off feedback on a prototype or concept
• Write a love or breakup letter to a product or brand
• Track a behavior over the course of a week (multi-entry part)
• Provide insight into who scouts are (likes, dislikes, day-to-day)
• Ask their thoughts on a current event and the impact that has on their life or their perception of your brand

See how dscout's Private Panels add-on makes it easy to build and manage your private research panels, right in the dscout platform.

Book time with our team to learn more.