“Garbage in, garbage out” is a deliciously vivid truism in the world of data collection. Surveys that are poorly written or conceived will collect unsound data, which will produce inaccurate insights, which in turn will lead to impotent – or actively harmful – decisions and actions. This is no small concern for an XM program. Surveys are often the sole means by which a program captures experience data (X-data) about people’s perceptions of an organization, and if the data they collect is wrong or misleading, it will cripple their downstream XM efforts.
To help XM professionals write surveys that generate valid and reliable insights, XM Institute has identified five categories of survey methodology practices they should consider as they design and deploy surveys:
- Survey Strategy. This category concerns the overall approach for designing and deploying the survey, which guides all decisions throughout the process. Activities here include defining the objectives of the survey, choosing a target audience, and identifying which data to collect and how to analyze it.
- Question Design. This category covers the text and format of the questions and answer choices provided in the survey. Activities here include writing question text, providing appropriate response options, and selecting question type.
- Pace and Flow. This category is about the overarching structure of the survey. Activities here include determining question order and survey duration, creating smooth transitions between questions, and creating an engaging, logical structure.
- Look and Feel. This category encompasses all the visual and tactical elements of the survey. Activities here include incorporating branding elements into the survey, formatting the questions and visual components, and ensuring the tone of the survey and accompanying communications match the organization’s personality.
- Survey Deployment. This category covers the distribution of the survey to potential respondents. Activities here include selecting distribution mechanisms (such as digital intercepts, SMS, or email), determining timing of response collection, and identifying which – if any – incentives to offer respondents.
In the XM Institute report Best Practices for Designing Survey Questions, we explored the Question Design category, examining how XM professionals can write surveys that are both informative for the business and engaging for respondents. Based on interviews with survey methodology experts across Qualtrics, we identified 16 best practices across three key areas: question-wording, response options, and question selection.
Here’s a short video that highlights key findings and examples from this report:
The bottom line: Follow best practices to ensure your surveys produce valid, reliable insights.
Isabelle Zdatny, XMP, CCXP, is an XM Catalyst with the Qualtrics XM Institute