Because both scholars and policymakers draw from military personnel survey results, survey participants need to be representative of the population. This research examined response rates for several major online U.S. Department of Defense military personnel surveys and found that younger service members, particularly younger enlisted personnel, tend to have very low response rates, even when surveys that are only 50 percent complete are defined as completed. The authors identify possible explanations, including military- and nonmilitary-specific situational and technological barriers and motivational factors. Low response rates do not necessarily yield results with bias. No minimum response rate has ever been established as a scientific threshold for minimizing nonresponse bias. Strategies to increase response rates can be costly, and previous research shows that they might not necessarily change the results in any perceptible or practically significant way. Thus, the authors propose ways to first understand how well surveys are capturing a representative sample of service members. Weighting the data along demographic characteristics might correct for some biases, but significant gaps in attitudes and experiences could remain. If nonresponse biases are present, the authors recommend reporting the limitations along with the results and identifying the factors that contribute to the bias (e.g., lack of access, trust) so that the survey researchers and sponsors invest only in recruitment strategies that would actually target the source of the problem and not just exacerbate it.
"Approved for public release; distribution unlimited."
Bibliography (note)
Includes bibliographical references (pages 53-59).
Processing Action (note)
committed to retain
Contents
Chapter one. Response rates on the 2012 RAND survey of airmen on information and communication technologies and well-being -- Chapter two. Response patterns by age or rank group in other large recent surveys of U.S. military personnel -- Chapter three. Low response rates in survey research and their implications -- Chapter four. Conclusion and recommendations for future air force and other military surveys -- Appendix A. Strategies to promote participation in the 2012 RAND survey of airmen on information and communication technologies and well-being -- Appendix B. Survey invitation for the 2012 RAND survey of airmen on information and communication technologies and well-being.