BMGT351: Marketing Research Methods

1.2: Overview and Process

What is marketing research?

Planning, collection, and analysis of data relevant to marketing decision making, and communication of results to management.

Descriptive Marketing Research

Describe what is happening now

- Explore new opportunities

- Provide insights regarding segmentation opportunities

Diagnostic Marketing Research

Explain why it is happening

- Evaluate the effectiveness of a current marketing strategy

- Investigate the reasons for decreasing market share

Predictive Marketing Research

Predict what will happen

- Evaluate the market share for a new product line

Platform entry (Reshef, 2023)

In August 2017, Yelp partnered with Grubhub. Yelp users could order from restaurants listed on Grubhub i.e., Grubhub restaurants "entered" Yelp's market

Is this good or bad for restaurants?

- Bad: seller competition

- Good: demand expansions

Entry is good for high-quality Yelp restaurants and bad for low-quality counterparts.

Customer Groups

Consumers, employees, shareholders, suppliers

Controllable Marketing Variables

Product, pricing, promotion, distribution

Uncontrollable Environmental Factors

Economy, technology, competition, laws and regulations, social and cultural factors, political factors

Marketing Managers

Market segmentation, target market selection, marketing programs, performance and control

Problem-Identification Research

Market potential research, market share research, image research, market characteristics research, sales analysis research, forecasting research, business trends research

Problem-Solving Research

Segmentation research, product research, pricing research, promotion research, distribution research

Marketing Research Process

1. Problem Definition

2. Development of Approach

3. Research Design Formulations

4. Fieldwork Data Collection

5. Data Preparation and Analysis

6. Report Preparation Presentation

Step 1: Identify problem(s):

May be stated as symptom or opportunity

Example: advertising spend decreasing on Facebook, young coffee-drinkers care about sustainability

Tasks involved:

Discussions with decision makers

Secondary data analysis: data collected for purposes other than problem hand

Qualitative research: Unstructured, exploratory, based on small samples, case studies, interviews.

Formulate research objectives

Are we looking for information to describe customer segments?

Are we testing some hypotheses (statements)?

- Based on objectives, determine research design

Exploratory

Clarify problems and gather evidence

Descriptive

Answer who, what, where, when

Casual

Identify cause-and-effect relation

Secondary Data

Previously collected, might have been collected for a different purpose, internal/external databases are both good sources

Primary Data

New data, collected for problem at hand, experiments, surveys, observations, focus groups

TV advertising and online search (Joo et al., 2014)

Problem: How should managers allocate ad spend across TV and Google? Are they complements or substitutes?

- e.g., more TV ad -> more or less Google search?

Approach

- Identify casual impact of TV advertising on online search

Data

- Keywords searched at Google by hour in each state "etf" , "fidelity" , "investments" , etc.

- Kantar Media's "Strategy" database: record of advertised brand, product category, estimated ad price, etc.

10% increase in a brand's TV ad spend increases online brand search by 1.7% over 96 hours.

How smartphone use changes UGC (Melumad et al., 2019)

Problem: Whether and how medium (PC vs. smartphone) impacts UGC

Approach: Identify causal effect of medium on UGC

Data: Randomized controlled experiment, Task: Write a review of undergrad dining hall experience, Smartphone (treatment) vs. PC (control)

1.3: Qualitative Research

Qualitative

Focus Group, Interview, Observation

Quantitative

Observation, Experiment, Survey

Pros of Qualitative Research

More probing: unstructured & open-ended

Deeper understanding of consumer behavior/psychology

No need for hypotheses -> suitable for exploratory research

Cons of Qualitative Research

Subjective interpretation of data

Interview bias

Often small sample size (why?)

Observer interference/social pressure

Focus Groups

A group 6-12 people led by moderator in in-depth discussion

- Moderator

- Participants

What vs. why

- Large-scale polling to understand sales

- Probing into consumer motivations and reasons

Recruiting Participants

All participants should be comfortable with each other

- Group dynamics are important

Homogeneity is key to foster open discussion among participants

- Expert and novice in same group?

- Trade-off: homogeneity vs. population representativeness?

Selection criteria for participants should depend on research objective:

- gender, age, power (experience level)

Online focus groups

Tends to be smaller than traditional focus group (4-6 participants)

- No geographic barrier

- Access to hard-to-reach population

- Lower cost

- Increased openness

- Lack face-to-face intimacy

- Group dynamics harder to manage online

In-depth interviews

One-on-one conversation

More attention to interviewee

Unstructured

Useful for uncovering motivations, beliefs, attitudes, and feelings

Pros of in-depth interviews

No group pressure, no within-group influence

Focus of attention: longer time per respondent

Nonverbal feedback

Cons of in-depth interviews

Takes longer to collect more data

Interviewer skills are critical

Observational Studies

Systematic process of recording behaviors without communicating with people involved

Most effective when observed behavior occurs frequently

No data on motivations, attitudes, intentions, or feelings

Record observable behaviors: repetitive, frequent, predictable

Behavior observation alone can generate actionable insight for marketers

Big data analysis

Analyze millions of data points

Shapes, colors, functionality, celebrity endorsements, brands, etc.

Discovering consumer insights

Big data analysis excellent for prediction

Qualitative analysis better for discovering insights

- Consumer insights often counter-intuitive

The missing ingredient: Cake mix

Cake mix consumers value convenience and simplicity

Therefore, make mix as simple as possible?

Pros of observational studies

Detailed behavioral data

Doesn't rely on how subjects answer questions

Less bias due to questioning or recall errors

Cons of observational studies

No data on motivation, attitudes, intention, or feeling

No projection into future

1.4: Experiment

What is the effect of changing X on Y?

X = Nike's spending on sponsored marketing with Ronaldo

Y = Nike sales

Observation:

- Increase influencer marketing spending by 12%

- 5% increase in sales!

Measuring Marketing Campaigns

Suppose Nike has exposure data from Facebook and matches with purchase data

- Group E sales: 10.2K units

- Group NE sales: 10K units

Without sponsored marketing, sales would have only been 10K (counterfactual)

With sponsored marketing, sales turns out to be 10.2K (actual outcome)

- So Ronaldo boosted sales by 2%!

Now can we renew the contract with Ronaldo?

E and NE groups not comparable!

Imagine Pfizer comparing vaccine effectiveness between athletes and hospitalized patients

Facebook's targeting algorithm confounds counterfactual

How can we achieve apple-to-apple comparison?

Treatment and control

To measure true effects, E and NE groups should be similar

1. Facebook's ad targeting implies E customers more likely Nike-fans than NE customers

2. True lift due to Ronaldo = Sales (E with ad) - Sales (E without ad)

3. Instead, we have measured Sales (E with ad) - Sales(NE without ad)

Key ideas

To measure true effectiveness of sponsored marketing, need to compare outcomes with counterfactuals

- Treatment and control

Need to estimate counterfactuals for same group for whom actual outcomes are measured

- Randomized assignment to treatment and control

Using experiments to measure campaigns

Nike asks Facebook to randomly show sponsored marketing to Group T and show something else to Group C

Two groups are now likely very similar due to random assignment

Group T sales: 10.1K units

Group C sales: 10.05K units

Sales lift caused by Ronaldo = (10.1-10.5)/10.05

What is an experiment?

Manipulate treatment variable to observe changes in dependent variable

X -> Y

X = treatment variable

Variables that researchers can control directly aka explanatory, independent

E.g., price, customer service, packaging, distribution, products features

Y = dependent variable

E.g., sales, customer satisfaction

Did X cause Y?

Z = Extraneous variable(s)

Factors that one does not control but may also affect the dependent variable aka: confounding variables

E.g., Nike loyalty, health, weather, pandemic, competitors' actions

How can we design experiments to rule out extraneous variables?

WFH decreases productivity?

"A study of data-entry workers in India found those toiling from home to be 18% less productive than office-frequenting peers." - The Economist (2023)

Without random assignment, selection bias cannot be ruled out.

Same outcome could be obtained from selection:

- Productive workers choose to work in the office

Less productive workers choose to work from home

I.e., not so much about WFH as selection effect

Demonstrating Causality

1. Concomitant Variations

- Y and X must have co-movement (positive or negative)

2. Appropriate time order of occurrence

- X happens prior to Y

3. Elimination of other potential causal factors

- No other Z that causes Y

Common extraneous variables

Selection bias

Systematic differences between treatment and control groups due to biased selection process

In observational studies, bias often caused by participants self selecting into treatment vs. control groups (cf. Ronaldo)

Randomization

By randomly assigning participants into treatment and control groups, ensure two groups are comparable

- Any differences in outcomes can be attributed solely to treatment effect - no other Z

- Self-selection bias eliminated by randomization

Alternative approaches when randomization infeasible

- Matching

- Consider only subsets of T and C groups that are comparable across, e.g., age, income, zip-code, etc.

- Other statistical designs

True experiment

The only research strategy that can determine that something causes something else; involves randomly assigning people to different treatments and then looking at the outcome.

Quasi-experiment

Similar to true experiments but missing randomization

- Useful when randomization too costly or impossible

A/B testing

Randomized experiment with two variants, A and B

- e.g., in A, price font-size is 1 and in B, price font-size is 4

Can generalize to more than two

- e.g., different shades of blue for sponsored links

Often used to identify whether change in website design/features can increase outcome of interest (e.g., click-through rate, conversion rate)

Easy to implement; quick to gather data

Validity

Degree to which an experiment actually measures what it is trying to measure.

Internal validity

Competing explanations for experimental results ruled out (correctness)

- Small sample size, failed randomization, not allowing enough time for effect to take place

External validity

Degree of realism: how can experimental findings can be generalized to other settings?

- Participants not representing targeted population, experiment market not representing national market

Wendy's menu tasting

Recruit 5 loyal Wendy's customers

Invite subjects to lab to taste new menu items

Subjects asked to rate menu items

2.1: Measurements and Scale

Measurement

Assigning numerical values to phenomena according to specific rules for rep quantities/qualities

- Assigning numbers 1 through 5 to fast food restaurants

Nominal Scale

Values assigned as labels

Rule: Classifies subjects into mutually exclusive and collectively exhaustive groups

E.g., classify subjects by gender, geographic area

Values have no true meaning but simply represent labels to help classify

Statistics: Frequencies/Percentages, Mode

Ordinal Scale

Rule: Determines order but not magnitude of difference/distance

Show positive relative to other options.

E.g., if Gap is assigned 1 and Old Navy 2, you know GAP is preferred over Old Navy (order), but you don't know "by how much" (by small or large margin?)

Typical application: Rankings

Interval Scale

Rules:

Measures order and relative distance

Intervals between adjacent ranks assumed equal

But ratio between values not meaningful

Examples:

Time: 9:00am is later than 8:00am, and time interval between 8:00am and 9:00am is same as interval between 9:00am and 10:00am

But ratio between two time points not meaningful

Temperature is another example

Ratio scale

Rule: Measures absolute order and absolute distance

Differences and ratios both meaningful

Typical applications: Market share, price, absolute income levels (not ranges), absolute age

Example:

If consumer is willing to pay $800 for Apple's iPhone 13, but only $400 for Samsung's Galaxy 21, this implies she is willing to pay twice as much (absolute measurement of both order and distance)

Comparative Scales

Rank Order

Constant Sum

Paired Comparison

Rank order scale

"Please rank following brands in terms of quality"

Constant sum scale

"Please allocate 10 points among following brands"

Paired comparison scale

"For each pair of drinks listed, check preferred option"

Non-comparative Scales

Itemized Rating Scales

- Likert

- Purchase Intent Scale

Itemized rating scale

Measure awareness, familiarity, interest, attitude, purchase intent, usage frequency, satisfaction (e.g., on 1 to 5 scale, how do you rate your satisfaction?)

Likert scale (Typically 5-7 responses; why odd-numbered?)

Semantic differential scale: bi-polar word pairs

Purchase intent scale