User-Centered Survey Design: Considering Group Membership Effects on Survey Responses

Abstract

Attending to diversity is an essential factor in interface design [22]. Today, there is a need for understanding the complex factors shaping the design practice of interfaces, instruments, and research tools, including surveys. With growing use of online surveys for data collection, designing surveys that address user diversity is becoming increasingly important [15]. However, there is also an overall trend of declining survey response rate across all modes [1], disrupting the accuracy of responses. While many studies on social identities exist, such as in-group/out-group status of the researcher compared to the respondent and their effect on face-to-face survey responses (e.g., [8, 13], we know very little about whether participants’ identities (i.e., university affiliation) affect responses to online survey designs. This study attempts to fill this gap by exploring if college students would agree to continue participating in a survey that uses different university affiliations across design conditions. While findings from this study did not show differential responses based on whether participants saw an in-group member of their own university or an out-group member of a rival university, we believe that it is most likely due to participants who did not agree to informed consent not being analyzed. Whether university affiliation affects participants’ willingness to participate in a survey in the first place has yet to be investigated. Practical and methodological implications are discussed in relation to how user-centered survey design could help increase survey response rates and improve diverse representation in samples, advocating for inclusive rather than exclusive participation.