Creating a Good Survey #02: Defining Target Respondents and Deriving Key Questions

Once the survey goals are clear, the next step is to specify 'who' to ask and 'what' to ask. This process is directly linked to the quality of the survey results, so a careful approach is needed.

1. Defining Target Respondents: Whose Voice Should We Hear?

To achieve survey goals most effectively, defining the correct target respondent group is paramount.2 The type and depth of information you can obtain vary depending on whom you survey.

• Goal Alignment: You must target individuals who best align with the survey goals. For example, if the goal is 'collecting user feedback on new feature A,' it's more advantageous to specify the target as 'users who have used new feature A at least once in the last 3 months' rather than simply 'all users of our service' to obtain more accurate data.
• Specific Characteristic Definition: When defining the target respondent group, consider specific demographic information (age, gender, region, etc.), service usage behavior (usage period, frequency, frequently used features, etc.), and the presence or absence of specific experiences.
  • Bad Example: "Our product users"
  • Good Example: "Female users in their 20-30s who have used our product at least 3 times a week for the past 6 months and have recently churned to competitor product B."
• Question Adjustment Based on Target: The content, tone, and terminology of questions should be adjusted considering the characteristics of the target respondents. For example, surveys targeting Gen Z and those targeting seniors should differ in the terminology used and question style to enhance respondent understanding and elicit honest answers. Similarly, the level of technical jargon used in surveys for IT experts and IT non-experts should differ.

If the target respondent definition is unclear, problems may arise with the representativeness or generalizability of the survey results. For example, if you receive many responses from people who rarely use a specific feature in a satisfaction survey about that feature, the results will not accurately reflect the actual user experience.

2. Deriving Key Questions: What Must We Ask to Achieve Our Goals?

Once the target respondent group is defined, you now need to define what information is crucial to identify (Learning Points) based on the survey goals and specific objectives, and then concretize this into actual question formats.2

• Goal-Based Question Design: Questions should not merely list what you are curious about but should directly contribute to achieving the set survey goals and each specific objective. Each question should be able to answer, "How does the answer obtained through this question help achieve our goal?"
• Defining Key Information (Learning Points): Clearly define what information is essential to know to achieve the goal.
  • Example (Goal: New product concept acceptance survey):
    • What is the first impression of the new product concept?
    • Which feature of the new product is most attractive?
    • What do you think is the differentiation point of the new product compared to existing products?
    • What is the purchase intention for the new product?
    • What is the appropriate price range for the new product?
• Brainstorming and Question Selection: Brainstorm various question ideas based on the key information. Then, select and refine questions based on how directly each question contributes to achieving the goal, whether respondents can clearly understand and answer it, and whether there are any redundant or unnecessary questions.
  • Selection Criteria Example:
    • Is this question absolutely necessary? (Goal relevance)
    • Does the respondent have the information to answer this question?
    • Is the question clear and free of ambiguity?
    • Is the question asking only one thing?
    • Is the question too burdensome or sensitive for the respondent?

In particular, it is crucial to boldly exclude questions that are not directly related to the survey goal. While there might be a temptation to ask many questions, an increasing number of questions can lead to respondent fatigue, resulting in lower quality answers or higher survey dropout rates. Constantly ask yourself how each question contributes to achieving the survey goal, and maintain a concise survey by removing questions unrelated to the goal; this is key to obtaining high-quality data. You should always keep in mind, 'Is this question absolutely necessary?'

The process of deriving key questions is a crucial step in converting survey goals into actual measurable items. In this process, distinguishing between "what you want to know" and "what you need to know," and focusing on the latter, is the core of efficient survey design.