Measuring Nonresponse Bias in a Cross-Country Enterprise Survey
Nonresponse is a common issue affecting the vast majority of surveys. Efforts to convince those unwilling to participate in a survey might not necessary result in a better picture of the target population and can lead to higher, not lower, nonresponse bias.
We investigate the impact of non-response in the European Commission & European Central Bank Survey on the Access to Finance of Enterprises (SAFE), which collects evidence on the financing conditions faced by European SMEs compared with those of large firms. This survey, conducted by telephone bi-annually since 2009 by the ECB and the European Commission, provides a valuable means to search for this kind of bias, given the high heterogeneity of response propensities across countries.
The study relies on so-called “Representativity Indicators” developed within the Representativity Indicators of Survey Quality (RISQ) project, which measure the distance to a fully representative response. On this basis, we examine the quality of the SAFE Survey at different stages of the fieldwork as well as across different survey waves and countries. The RISQ methodology relies on rich sampling frame information, which is however partly limited in the case of the SAFE. We also assess the representativeness of the SAFE particular subsample created by linking the survey responses with the companies’ financial information from a business register; this sub-sampling is another potential source of bias which we also attempt to quantify. Finally, we suggest possible ways how to improve monitoring of the possible nonresponse bias in the future rounds of the survey.
The American Association for Public Opinion Research (2008), Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 5th edition, Lenexa, Kansas: AAPOR.
Bańkowska K., Osiewicz M., Pérez-Duarte S., Linking qualitative survey responses with quantitative data. Methodology, quality and data analysis from the matching of the ECB/EC Survey on Access to Finance of Enterprises and the Amadeus database, forthcoming.
Groves, R.M. (2006), Nonresponse Rates and Nonresponse Bias in Household Surveys, Public Opinion Quarterly, Vol. 70, 646-675.
Heij, V. de., Schouten, B., Shlomo, N. (2010), RISQ manual, Tools in SAS and R for the computation of R-indicators and partial R-indicators, http://www.risq-project.eu/publications.html
Kruskal W. and F. Mosteller 1979. Representative sampling, III: The current statistical literature. International Statistical Review 47:245-265.
Montaquila, J. M., Olson, K. M. (2012), Practical Tools for Nonresponse Bias Studies, SRMS/AAPOR Webinar.
National Research Council (2013), Nonresponse in Social Science Surveys: A Research Agenda, Washington, DC: The National Academies Press.
Schouten, B., Bethlehem, J.G., Beullens, K., Kleven, O, Loosveldt, G., Luiten, A., Rutar, K., Shlomo, N. & Skinner, C. (2012), Evaluating, Comparing, Monitoring, and Improving Representativeness of Survey Response Through R-Indicators and Partial R-Indicators. International Statistical Review, Vol. 80, pp. 382-399.
Schouten, B., Cobben, F. & Bethlehem, J. (2009), Indicators of Representativeness of Survey Response. Survey Methodology Vol. 35, pp. 101-113.
Schouten, B., Morren, M., Bethlehem, J., Shlomo, N. & Skinner, C. (2009), How to use of R-indicators?, http://www.risq-project.eu/publications.html
Schouten, B., Shlomo, N. & Skinner, C. (2011), Indicators for Monitoring and Improving Representativeness of Response. Journal of Official Statistics, Vol. 27, pp. 1-24.
Shlomo, N. & Schouten, B. (2013), Theoretical Properties of Partial Indicators for Representative Response. Technical Report, University of Southampton
Stoop, I. (2005), The Hunt for the Last Respondent: Nonresponse in Sample Surveys. The Hague: Social and Cultural Planning Office of the Netherlands.
Sudman, S. (1966), Probability Sampling with Quotas, Journal of the American Statistical Association, Vol. 61, No. 315 (Sep., 1966), pp. 749-771.
How to Cite
The Austrian Journal of Statistics publish open access articles under the terms of the Creative Commons Attribution (CC BY) License.
The Creative Commons Attribution License (CC-BY) allows users to copy, distribute and transmit an article, adapt the article and make commercial use of the article. The CC BY license permits commercial and non-commercial re-use of an open access article, as long as the author is properly attributed.
Copyright on any research article published by the Austrian Journal of Statistics is retained by the author(s). Authors grant the Austrian Journal of Statistics a license to publish the article and identify itself as the original publisher. Authors also grant any third party the right to use the article freely as long as its original authors, citation details and publisher are identified.
Manuscripts should be unpublished and not be under consideration for publication elsewhere. By submitting an article, the author(s) certify that the article is their original work, that they have the right to submit the article for publication, and that they can grant the above license.