I don't keep track, but I do take virtually every survey that comes my way. Recently, these have included:
Commenting on whether my university should allow guns on campus;
A master's student's survey regarding attitudes and barriers to public engagement;
Someone studying the portrayal of invertebrates in the media via survey;
Feedback after a professional conference I attended;
A feedback survey about the scicomm events/trainings/resources newsletter I run
A survey I'm running to characterize and support scicomm newsletters fun by other folks (blogs and Substacks count!)
I have a really simple reason for taking all these surveys. And for why I developed a habit of saying yes to (nearly) every survey I encounter.
Surveys are a tool people use to understand important things about how and why people do things (and how/why we might need to help/change policy, etc.).
I have a lot of opinions about policy, how things get run, etc. I’m guessing you do, too.
But, hardly anyone actually responds to surveys!
Just consider how low voting rates are (and how high the stakes are!)! Horrendously low: not even 50% in 2022, and only 66% in 2020 [1]. And those were two of the top three voter turn-outs in decades (at least in the U.S.).
Consider, then, just how rare it probably is for people to respond to surveys where the stakes are even lower. Indeed, survey-based researchers can be lucky to get ~60% response rates. This is a big deal, because researchers, policymakers, community organizations, local governments, and the like regularly use survey results to inform policy, curriculum development/adjustments. But, concerns about the perspectives that aren't accounted for can rightfully weigh heavily on these decisionmakers (although see [2]).
So, where does that leave those of us who are trying to inform efforts to make the world a bit better?
Worrying a fair bit about response rates.
For example, I've used survey research to help us understand what it takes to:
reduce institutional barriers to transdisciplinary research [3];
overhaul the academic prestige paradigm to foster, reward, and value scicomm teaching, research, and practice;
teach scicomm effectively [4];
help instructors productively integrate drawing in field and lab courses;
identify aspects of how an academic department can foster well-being and success of all its members;
help graduate students be less afraid of writing [5].
In every case, response rates helped us gauge how confidently we could interpret the data. In other words, response rates have bearing on how much could we rely on the information we collected through the survey. That has a direct, downstream influence when we make recommendations about what did or didn't work, what needed to be improved, and/or how, specifically, things needed to change.
These factors are at play for everyone doing survey research or distributing surveys, whether the survey is collecting info about who does scicomm or soliciting public comment on policy changes.
I feel like I have distinct perspectives on a lot of things -- perspectives that should be considered. I'm often an odd-ball: an artsy person in science spaces, a scientific person in arts corners, a first-gen student and academic and a non-trad grad student (which means I don't always fit in academia or back home). I was a working immigrant in a second-to-me-language for several years. I've changed careers a lot of times. I want those perspectives considered. I want my datapoints to provide nuance to these datasets. I also want my datapoints to help make the case for decisions, programs, and investments that help make the world a better place.
So, I take every survey (and every opportunity to vote!).
Of course, more is not always more -- not even in survey research. But, to the extent that more is helpful, I take surveys to help in my own small way.
How about you?
What's a small, consistent thing that you do to help inform good management, planning, and options in your workplace, community, or beyond? (And of course, feel free to chime in about how or why you do/don't take surveys!)
NOTES
[1] See this Pew report for details: https://www.pewresearch.org/politics/2023/07/12/voter-turnout-2018-2022/.
[2] This paper is a good starting point if you're curious about how all this works. Hendra R, Hill A. Rethinking response rates: new evidence of little relationship between survey response rates and nonresponse bias. Evaluation review. 2019 Oct;43(5):307–30. https://doi.org/10.1177/0193841X18807719
And, if you're wondering how response rates affect publication, see this paper: Carley-Baxter, Lisa R, Craig A Hill, David J Roe, Susan E Twiddy, Rodney K Baxter, and Jill Ruppenkamp. 2009. “Does Response Rate Matter? Journal Editors Use of Survey Quality Measures in Manuscript Publication Decisions.” Survey Practice 2 (7). https://doi.org/10.29115/SP-2009-0033.
If you're considering using surveys as a research method, this paper is helpful for factoring response rate into other important measures of validity: Holtom, B., Baruch, Y., Aguinis, H., & A Ballinger, G. (2022). Survey response rates: Trends and a validity assessment framework. Human Relations, 75(8), 1560-1584. https://doi.org/10.1177/00187267211070769
[3-5] These are all unpublished data. I'm happy to chat more about what we found and how we conducted these studies, though! If you're curious about progress/stage of the work: [3] Manuscript is in prep; [4] research is on-going; [5] manuscript is in review.
As always, if you're having trouble accessing any of the papers linked to on this blog, don't pay for access! Just contact me. I'm happy to provide you a PDF. Better yet, contact the authors - most researchers are delighted to hear someone is interested in their work.
This is the end of this post. If you see a prompt to subscribe, you're welcome to do so! Your paid subscription helps me allocate time to the resources I share here.
Comments