Podcast Episode 4: Data Decisions (w/ Dr. Peter K. Enns)

Don’t forget to subscribe to the People Nerds Podcast on Apple Podcasts or Spotify to stay up-to-date with the latest episodes!

Let's Talk Sample Sizes

In this episode of the People Nerds Podcast, we sat down for a conversation with Dr. Peter K. Enns, a Professor of Government and Public Policy at Cornell University, and the Robert S. Harrison Director of the Cornell Center for Social Sciences. He is also the Co-founder and Chief Data Scientist at Verasight.

In our conversation, we talk all things big data—including the relative merits of qualitative and quantitative data, how to achieve rigor and quality in your surveys, and the importance of representation in your samples.

Many user researchers focus mostly on small sample qualitative research. So what can we learn from a survey scientist with n sizes that routinely number in the thousands? Here are some of the takeaways we valued most.

For full episode transcription, click here.

Bigger samples? Sure, but there’s more to it

Many user research stakeholders value quantitative readouts and beefy sample sizes. We have talked in the past about how to push back on stakeholders when it comes to sample size and preach the value of qualitative data.

Peter shares on the flip side that even while sample size is important to his research, it’s not the only thing that makes research rigorous. He preaches the importance of methods and measurement—that a thorough understanding of why a method or analytical process is used is as important as what method is used.

Data quality is everything

One thing that came up over and over when talking to Peter: no matter what research you’re trying to run, data quality—and data sourcing—is one of the most important elements you can pay attention to as a researcher.

Where is your data coming from? In-house or third party? How are those participants getting paid? What are the possible biases in your project design, and how are they being accounted for? And how are you sure that your sample is representative of the groups you are attempting to speak for?

If these questions can’t be answered, then it’s possible that you’re working with bad data—and bad data makes for bad decisions.

Representation is important, no matter how niche you get

Every researcher tries to make sure their recruit is balanced and representative. But Peter observes that he sometimes sees researchers forget to prioritize flexibility once the recruit zeroes in on a sub-group, without which might erase the potential nuance and heterogeneity present in those groups.

In other words, demographics cannot be destiny, especially when it comes to material outcomes, whether that's policy-based or experience-based ones.

Whether it’s “young voters”, “New Yorkers,” or “new customers,” researchers should be careful to consider what a representative spread within those smaller groups looks like and aim to reproduce it in the sample. Just because you’ve hit your target audience, doesn’t mean they’ll all act the same.

Ethical research is a win-win

We talked to Dr. Enns a lot about data quality and bias. But how do we combat it?

One important way we can work to combat low-quality data is to move toward a reciprocal, ethical relationship with our participants. If participants believe that we are only after their data—and don’t care about them as people—then they will adopt a similar stance, and only care about money (and not about giving quality answers).

But Peter says that building long-standing trust and partnership with his participants gives him a business edge. When participants feel invested in, they will invest back, with honest and complete answers. This leads to higher-quality data, which leads to better business decisions. Having well-paid, respected participants is a win across the board.

Interested in checking out other People Nerds podcast episodes? Read more of our breakdowns here.