Survey Mode Effects in the European Social Survey Round 12

Verdoes is presenting a paper on survey mode effects in the European Social Survey. His co-authors are Sveinung Arnesen (UiB/NORCE) and Ruben Mathisen (UiB).

Light lunch will be served, as first come, first served.

The event is hybrid, if you can not join us in the Corner room at Sofie Lindstrøms hus, you can join us digitally (external link).

Abstract:

Political scientists rely extensively on survey data to compare political attitudes across countries and track how they change over time. Yet as major cross-national surveys shift from face-to-face interviewing to self-completion modes, observed changes in political attitudes may reflect changes in how surveys are administered rather than genuine political
change. Despite this concern, comparative experimental evidence on the magnitude and scope of survey mode effects across political attitudes remains limited.

We exploit the allocative mixed-mode design of Round 12 of the European Social Survey (ESS), in which respondents are randomly assigned to either face-to-face interviewing or self-completion (web or paper). Our preliminary analysis includes almost 10,000 respondents from Norway, Finland, and the United Kingdom and over 390,000 responses across 41 survey items covering 11 themes central to political science.

We find systematic differences between responses obtained through face-to-face interviewing and self-completion, although the magnitude of these mode effects varies substantially across variable themes and countries. We further show that these differences extend beyond aggregate levels to substantive inference: regression analyses using face-to-face data produce significantly larger effect sizes and explain more variance than equivalent analyses using self-completion data. These findings demonstrate that survey mode influences both measured levels of political attitudes and conclusions about relationships between political variables.