Chapter 4: Research Design – Key Points
Research Design Overview
A research design is a master plan outlining methods for collecting and analyzing information for a project.
After defining the problem and objectives, determine the research design.
Three basic designs: exploratory, descriptive, and causal.
Each design serves a different purpose and has distinct methods, advantages, and drawbacks.
Knowledge of design enables advance planning, time savings, and potential cost efficiencies; research can be iterative and may involve multiple designs.
Exploratory Research
Definition: unstructured, informal research to gain background information about the general nature of the research problem.
No predetermined procedures; small, nonrepresentative samples may be used.
Used to gain understanding, define terms, clarify problems/hypotheses, and set priorities.
Uses:
Gain background information
Define terms
Clarify problems and hypotheses
Establish research priorities
Common methods:
Secondary data analysis
Experience surveys (key-informant/lead-user methods)
Case analysis
Focus groups (often discussed in later chapters; used to guide descriptive/causal work)
Notes:
Often the first step in a research project due to efficiency and low cost of secondary data.
Helps shape subsequent descriptive or causal designs.
Descriptive Research
Definition: undertaken to describe who, what, where, when, and how; used to describe a population and, if the sample is representative, to project findings to a larger population.
Two main types:
Cross-sectional: single point in time; like a snapshot. Large, representative samples; often called sample surveys.
Longitudinal: multiple measurements over time; uses panels.
Cross-sectional details:
Measures demographics, attitudes, behaviors at one time point; can be used to estimate market characteristics and potentially predict outcomes.
Samples are designed to be representative with a margin of error; results reported with margin of error (e.g., ±3%).
Longitudinal details:
Repeatedly measures the same units over time (movies of the population).
Panels: respondents who agree to provide information at regular intervals.
Panel types: continuous (same questions each wave) and discontinuous/omnibus (vary questions across waves).
Panels and market-tracking:
Continuous panels track changes in attitudes/behaviors and can study brand-switching over time.
Discontinuous panels provide access to a broad group for various research purposes.
Market-tracking studies monitor the same variables (e.g., market share, sales) over time.
Brand-switching studies:
Compare how consumers switch between brands over time to avoid misinterpreting cross-sectional changes.
Longitudinal data provide clearer insights into loyalty and switching dynamics.
Causal Research
Purpose: measure causality in relationships (if X, then Y).
Key concepts:
Independent variables: what the researcher controls/manipulates (e.g., advertising spend, price, packaging).
Dependent variables: what is measured (e.g., sales, market share, satisfaction).
Extraneous variables: other factors that may affect the dependent variable.
Experimental design basics:
Change in dependent variable should be attributable to the change in the independent variable, control for extraneous factors.
Notation example for a typical before-after design with randomization:
Experimental group: O₁ X O₂
Control group: O₃ O₄
Pretests: O₁ and O₃; posttests: O₂ and O₄; X denotes the manipulation of the independent variable.
Experimental effect: E = (O2 - O1) - (O4 - O3)
Two common experimental designs:
Before-after testing: random assignment to control and experimental groups; measure pretest, apply manipulation to experimental group, measure posttest; compare changes to estimate E.
A/B testing: compare two or more independent variables/variants simultaneously to see which performs better.
Validity considerations:
Internal validity: the degree to which a change in the dependent variable is caused by the manipulation of the independent variable.
External validity: the extent to which results generalize to real-world settings.
Threats include non-equivalent groups, uncontrolled extraneous variables, artificial lab settings, and limited generalizability.
Lab vs Field experiments:
Laboratory: high internal validity, controlled environment; may lack generalizability to real-world settings.
Field: conducted in natural settings; higher external validity but more extraneous variables and costs.
Modern tools:
VR and other technologies can simulate real shopping environments for more realistic testing while maintaining control.
Test Marketing (Field-Setting Experiments)
Definition: field tests to evaluate a new product or variations in the marketing mix in real-world conditions.
Types of test markets:
Standard test market: product/mix tested through company’s normal distribution channels; realistic but exposes product to competitors.
Controlled test market: conducted by outside firms that guarantee distribution through predefined distributors; faster access but may not reflect actual distribution.
Simulated test market (STM): staged environment mimicking real conditions to gauge consumer response; faster and cheaper, but may be less accurate.
Uses:
Test sales potential for new products
Test marketing-mix variations (price, promotion, placement, etc.)
Selecting test-market regions: representativeness, isolation, and control over distribution and promotion.
Pros and cons:
Pros: most accurate real-world forecast before full launch; can pretest marketing-mix elements.
Cons: expensive and time-consuming; risk of competitor sabotage; exposure to competitors; ethical considerations.
Practical notes:
Test marketing provides external validity but does not guarantee perfect results; use to inform decisions and potentially save large-scale costs.
Key Relationships and Quick References
Research design linkage:
Exploratory → Descriptive or Causal as next steps; used to define terms and hypotheses and guide data collection.
Descriptive → can be extended to causal analysis if hypotheses about relationships are tested.
Experimental notation essentials:
Independent variable manipulation: X
Dependent variable measurement: O (pre/post tests as O₁, O₂, etc.)
Experimental effect: E = (O2 - O1) - (O4 - O3)$$
Basic validity concepts:
Internal validity: are results due to the manipulation?
External validity: do results generalize beyond the study?
Practical takeaway:
Start with exploratory research to gather background and define terms, then use descriptive to characterize the population, and employ causal research with well-designed experiments to establish causality. Test marketing provides field validation of new products and marketing mixes, despite higher costs.