Measuring Nonresponse Bias in a Cross-Country Enterprise Survey

Authors

  • Katarzyna Bańkowska
  • Malgorzata Osiewicz ECB
  • Sébastien Pérez-Duarte

DOI:

https://doi.org/10.17713/ajs.v44i2.60

Abstract

Nonresponse is a common issue affecting the vast majority of surveys. Efforts to convince those unwilling to participate in a survey might not necessary result in a better picture of the target population and can lead to higher, not lower, nonresponse bias.

We investigate the impact of non-response in the European Commission & European Central Bank Survey on the Access to Finance of Enterprises (SAFE), which collects evidence on the financing conditions faced by European SMEs compared with those of large firms. This survey, conducted by telephone bi-annually since 2009 by the ECB and the European Commission, provides a valuable means to search for this kind of bias, given the high heterogeneity of response propensities across countries.

The study relies on so-called “Representativity Indicators” developed within the Representativity Indicators of Survey Quality (RISQ) project, which measure the distance to a fully representative response. On this basis, we examine the quality of the SAFE Survey at different stages of the fieldwork as well as across different survey waves and countries. The RISQ methodology relies on rich sampling frame information, which is however partly limited in the case of the SAFE. We also assess the representativeness of the SAFE particular subsample created by linking the survey responses with the companies’ financial information from a business register; this sub-sampling is another potential source of bias which we also attempt to quantify. Finally, we suggest possible ways how to improve monitoring of the possible nonresponse bias in the future rounds of the survey.

References

The American Association for Public Opinion Research (2008), Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 5th edition, Lenexa, Kansas: AAPOR.

Bańkowska K., Osiewicz M., Pérez-Duarte S., Linking qualitative survey responses with quantitative data. Methodology, quality and data analysis from the matching of the ECB/EC Survey on Access to Finance of Enterprises and the Amadeus database, forthcoming.

Groves, R.M. (2006), Nonresponse Rates and Nonresponse Bias in Household Surveys, Public Opinion Quarterly, Vol. 70, 646-675.

Heij, V. de., Schouten, B., Shlomo, N. (2010), RISQ manual, Tools in SAS and R for the computation of R-indicators and partial R-indicators, http://www.risq-project.eu/publications.html

Kruskal W. and F. Mosteller 1979. Representative sampling, III: The current statistical literature. International Statistical Review 47:245-265.

Montaquila, J. M., Olson, K. M. (2012), Practical Tools for Nonresponse Bias Studies, SRMS/AAPOR Webinar.

National Research Council (2013), Nonresponse in Social Science Surveys: A Research Agenda, Washington, DC: The National Academies Press.

Schouten, B., Bethlehem, J.G., Beullens, K., Kleven, O, Loosveldt, G., Luiten, A., Rutar, K., Shlomo, N. & Skinner, C. (2012), Evaluating, Comparing, Monitoring, and Improving Representativeness of Survey Response Through R-Indicators and Partial R-Indicators. International Statistical Review, Vol. 80, pp. 382-399.

Schouten, B., Cobben, F. & Bethlehem, J. (2009), Indicators of Representativeness of Survey Response. Survey Methodology Vol. 35, pp. 101-113.

Schouten, B., Morren, M., Bethlehem, J., Shlomo, N. & Skinner, C. (2009), How to use of R-indicators?, http://www.risq-project.eu/publications.html

Schouten, B., Shlomo, N. & Skinner, C. (2011), Indicators for Monitoring and Improving Representativeness of Response. Journal of Official Statistics, Vol. 27, pp. 1-24.

Shlomo, N. & Schouten, B. (2013), Theoretical Properties of Partial Indicators for Representative Response. Technical Report, University of Southampton

Stoop, I. (2005), The Hunt for the Last Respondent: Nonresponse in Sample Surveys. The Hague: Social and Cultural Planning Office of the Netherlands.

Sudman, S. (1966), Probability Sampling with Quotas, Journal of the American Statistical Association, Vol. 61, No. 315 (Sep., 1966), pp. 749-771.

Published

2015-04-30

How to Cite

Bańkowska, K., Osiewicz, M., & Pérez-Duarte, S. (2015). Measuring Nonresponse Bias in a Cross-Country Enterprise Survey. Austrian Journal of Statistics, 44(2), 13-30. https://doi.org/10.17713/ajs.v44i2.60

Issue

Section

Q2014