lecture 7 adjusted notes
Motivation
The White House proposed setting aside half of this year’s budget surplus.
Discussion on the allocation of the remaining surplus:
Version 1 Options:
(a) Offer a tax cut
(b) Fund new government programs
Version 2 Options:
(a) Offer a tax cut
(b) Fund new government programs specifically focused on education, environment, healthcare, and national defense
Source: New York Times (Jan 30, 2000)
Public Preference Data:
60% of respondents preferred a tax cut.
22% preferred funding new government programs.
Today's Topics
Focus will be on designing an effective questionnaire.
Designing the Questions
Initial steps include:
Plan what to measure:
Formulate questions that will obtain necessary information and exclude unnecessary data.
Design structure and wording of each question:
Ensure clarity and precision in questions.
Determine order and layout:
Organize the questionnaire layout effectively.
Pretest the questionnaire:
Use a small sample to identify any omissions or ambiguities.
Goals of Questionnaire Design
Two primary goals:
Align the questions used in the research with how respondents understand them.
Align the answers as stated by respondents with how researchers comprehend these answers.
Questionnaire Design Process
Adapted from Essentials of Marketing Research, Malhotra, p.218
Specify the Information Needed:
Define the research questions and hypotheses.
Select the Mode of Administration:
Choose how the questionnaire will be distributed.
Determine Content of Individual Questions:
Decide the specific questions to include.
Overcome Respondent’s Inability/Unwillingness to Answer:
Develop strategies to ensure respondents can and will answer questions.
Choose the Question Structure:
Decide whether questions will be open-ended, closed-ended, or scaled.
Determine the Question Wording:
Craft clear, unambiguous language for questions.
Arrange the Questions in Proper Order:
Organize questions logically to maintain flow.
Choose the Form and Layout:
Design physical appearance and formatting of the questionnaire.
Reproduce and Pretest the Questionnaire:
Finalize and test the questionnaire for effectiveness.
Specifying the Information Needed
Marketing Research Process Steps:
Define the research questions.
State the hypotheses connected to these questions.
Identify the information needed to answer each hypothesis.
Research questions include:
RQ 1
RQ 2
RQ 3
Corresponding hypotheses might look like this:
Hypothesis for RQ 1
Hypothesis for RQ 2
Hypothesis for RQ 3
Required information examples:
Info needed for H1
Info needed for H2
Info needed for H3
Mode of Administration
Different questionnaire assembly methods and their strengths and weaknesses:
Online:
Cost: Low
Speed: High
Quantity: High
Response Rate: Low
Flexibility: Moderate
Interviewer Bias: Low
Telephone:
Cost: Moderate
Speed: High
Quantity: Moderate
Response Rate: Moderate
Flexibility: Low
Interviewer Bias: Moderate
In-Person:
Cost: High
Speed: Moderate
Quantity: Low
Response Rate: High
Flexibility: High
Interviewer Bias: High
Mail:
Cost: Low
Speed: Low
Quantity: High
Response Rate: Low
Flexibility: Low
Interviewer Bias: Low
Question Content
Guidelines for question formulation:
Avoid unnecessary questions.
Ensure each question contributes to needed information.
Avoid double-barreled questions:
Example: “How satisfied are you with the design and price of the iPhone?” should be split into two questions:
“How satisfied are you with the design of the iPhone?”
“How satisfied are you with the price of the iPhone?”
Ensure responses are Mutually Exclusive and Collectively Exhaustive (MECE):
Avoid overlapping answers and ensure all possible responses are included.
Example of a question lacking MECE:
“How many times have you visited McDonald’s this month?”
Answer options:
___ 1-3 times
___ 4-6 times
___ 7-9 times
Example of a demographic question that is not exhaustive or mutually exclusive:
“What is your age group?” with options not covering all ages.
Overcoming Inability to Answer
Strategies for addressing potential respondent unawareness:
Screen for familiarity before asking key questions on new topics.
Allow an “I don’t know” option.
Provide background information before posing questions.
Avoid overly specific questions when generalities may suffice.
Example:
Instead of asking specific gallons of gasoline purchased, ask how many times gas stations were visited.
Overcoming Unwillingness to Answer
Techniques for addressing sensitive information requests:
Use range responses for personal data (e.g., income).
Third-person technique for embarrassing topics.
Preface sensitive questions with statements that normalize the inquiries (e.g., “Recent studies show that most Americans are in debt”).
Position sensitive questions toward the end of the questionnaire.
Question Structure
Types of question structures to consider:
Open-ended questions
Closed-ended questions (multiple choice, scaled)
Balancing information quantity with ease of completion is crucial.
Open-Ended vs. Closed-Ended Questions
Open-Ended Questions:
Advantages:
Yield more information, allow for nuanced responses.
Disadvantages:
Subjectivity, time-consuming, complex coding/analysis.
Closed-Ended Questions:
Types include multiple-choice questions and scaled responses.
Example:
“What is the most important attribute in a smartphone?”
“How important is screen size on a scale of 1-7?”
Wording of Questions
Proper wording is vital for clarity:
Questions should be clear and focused.
Avoid ambiguous terms such as "usually" and "often."
Use simple language and terminology that respondents can easily understand.
Avoid leading questions that may bias responses.
Example:
Instead of “Were you pleased with the great service you received at Hilton last week?” use neutral phrasing.
Flow of the Questionnaire
Suggested flow mirrors a logical progression:
Qualifying/Screening Questions
Warm-Up/Introductory Questions
Main Questions (Easy to Difficult)
Demographics (Sensitive Information Last)
Provide transition statements to help move between different sections of questions.
Avoid order bias by randomizing questions or utilizing a funnel technique.
Layout of the Questionnaire
Overall layout should be professional and user-friendly:
Ensure plenty of space for open-ended answers.
Use clear, bolded instructions when necessary.
For online surveys, include a progress bar to provide respondents with feedback on their progress.
Characteristics of a Good Questionnaire
Essential characteristics include:
Achievement of research objectives and testing hypotheses
Logical sequencing and clear phrasing
Attention to respondent understanding and willingness to answer questions
Comprehensive pre-testing and evaluation
Online / Mobile Questionnaire Considerations
Online questionnaire length should be kept short (10-15 minutes max).
Limit the number of open-ended questions to one or two.
Pay attention to wording for clarity within a few seconds of reading, ideally under 140 characters.
Conclusion and Next Steps
Transition to the next topic: Sampling Framework
Key considerations before conducting a survey include:
Select a survey mode of administration (How)
Design and implement a sampling scheme (To whom)
Design the survey questionnaire (What)
Include measurement scales, wording, sequence, and layout denominations.