Marketing Research 6-12

Categorical measurement

Measure with values that can be counted as part of distinct groups, nominal or ordinal

Coding

Categorizing open-ended response data into themes

Complex question

Question with confusing language, multiple stipulations or generally taxing to the respondent

Continuous measurement

Scales with values that can be expressed numerically and places in a logical order, interval or ratio

Funnel approach

Survey ordering technique use to gather top of mind responses before more specific questions are asked

Interval

Continuous measurement with a clear order, distance between values is the same

Likert scale

Interval scale of measurement that provides a range of options from one extreme to another (looks like a table)

Loaded questions

Questions that suggest respondents give a socially desirable answer

Multi-channel surveys

Multiple surveys focusing on a singular research objective distributed across platforms that target various consumer segments

Nominal

Categorical measurement with no natural order to categories

Ordinal

Categorical measurement that can be placed in a clear order

Pathway approach

Survey ordering technique in which the survey is organized in the order in which experiences occur

Pseudo research

• Studies designed to produce a particular outcome

• Results are intentionally biased

• Not ethical

Ratio

Continuous measurement with a clear order, distance between values is the same, 0 point is treated as origin

Screener questions

Placed before core survey in order to determine if a potential respondent qualifies to take the survey

Stackable surveys

Surveys broken up into a series of smaller surveys dispersed over time

Visual weight

Scale design where stronger focal points on certain scale options can lead to skewed results

Action indicators

Set of questions that attempt to quantify the degree to which a respondent might take action as a result of an advertisment

Message testing

Gauge effectiveness of singular ads and campaigns prior to full development and production

Aided awareness

Questions that use a predefined list of brands or products in order to obtain a definitive read on awareness for brands in the category

Conjoint analysis

Statistical analysis used to understand how customers value different combinations of features and price points

Dial testing

Research method used in advertising where participants use a dial that the adjust to indicate their feelings

Net promoter score (NPS)

A metric used to gauge customer loyalty and satisfaction by measuring the likelihood of customers recommending

Owned brand attribute

A key attribute a company is well recognized for in the industry

Reliability

Indicator of an instruments consistency in its ability to measure

Unaided awareness

Respondents asked to list all of the brands, products or services they can recall in a particular category, seeming if company is top of mind or not

Validity

Notion that an instrument is measuring what it is suppose to measure

Voice of the customer (VoC) research

• Customer satisfaction tool

• Gathers and analyzes feedback with experiences and expectations of company

Confidence level

Indicator of how confident researchers can be that if a study were replicated it would yield the same result

CPI/CPS

Cost per interview/cost per survey

Expert sampling

Sampling from experts on a specific field

Incidence rate

Percentage of those in the population who possess the characteristics that make them eligible for a research study

Margin of error

Indicator expresses as a percent range that estimates how much a stat can fluctuate when using it to describe the true population (aka confidence interval or sample error)

Non-probability sampling

• Not random sampling

• Researchers subjective judgement

• Or how to access population

Parameter

Data resulting from a study gathered from a census

Probability sampling

Sampling that uses random selection

Quota sampling

Gathering sample in proportional relation to predefined groups (cells) in the target population

Research panels

Collected group of potential respondents who opt in to participate in future research studies

Stratified random sampling

Sampling technique where subjects are split into mutually exclusive groups proportionate to the population (strata) before simple random sampling is employed

Systematic sampling

Sampling method where researchers choose every “nth” record on a list

Extremity bias

Tendency of some respondents to use extremes when taking surveys

Fielding

When primary research studies in their data collection phase

Interviewer bias

Respondents answer being influenced by the presence of an interviewer

Non-contacts

Individuals who do not receive the request to participate in a survey

Non-response error bias

Occurs when there is a difference between results of those who completed a survey and those who didn’t

Piping

A survey software that automatically inserts text into survey base of prior question/data

Response bias

When respondent answers a question in a way that misrepresents the truth

Self-selection bias

Individuals that feel strongly about a topic are more likely to participate

Validation question

Question designed to check the consistency or accuracy of a respondents answers

Cross-tabulations

Data tables that cross results by a segment of the dataset

Data cleaning

Process of identifying and correcting errors and inconsistencies prior to analysis

Descriptive statistics

• Statistics that describe basic features of the data

• Measures of central tendency (mode, median and mean)

• Measures of spread (range and standard deviation)

Measures of central tendency

• Center of distribution

• Mean

• Mode

• Median

Measures of spread

• Describes how similar or varied distribution is

• Range

• Standard deviation

Top 2-box scores

Combine percentages of those who answered the highest and second-highest value in a rating scale

Analysis of variance (anova)

Inferential test statistic used to test differences of mean scores among 3 or more groups

Chi-square

Inferential test statistic used for testing relationship between categorical variables

Choice-based conjoint analysis

Has respondents place values on combos of features (need to make trade off for preferences)

Full-profile conjoint analysis

Asks repondent with a series of holistic descriptions and asks them to select which one they’d most likely buy

Linear equation

Formula of a line that can be used to predict future outcomes

Multiple linear regression

Examines group of factors at the same time in order to help determine which combination of factors best predict a dependent factor

Paired t-test

Inferential test statistic used to test mean scores (continuous variables) between 2 groups when the data can be represented in pairs

Post hoc tests

Statistical tests run to determine where significant differences are occuring between groups more than 2

r square

Statistical measure that explains to what extent the variance of one variable explains the variance of another variable (closer to 1 the stronger the relationship)

Type I error

When null hypothesis is incorrectly rejected (null hypothesis is true)

Type II error

When null hypothesis is accepted when should be rejected (null is false)

Z-test

Inferential test statistic used to test differences between 2 percentages (measures of spread)

Conditional formatting

Color coding cells within table based on certain parameters

Dashboards

Visual tool used to display many data results at a glance

Detailed findings

Summarizes and interprets all data collected in a research study

Executive summary

Overview that includes results and recommendations

Methodology

Details how data was collected and includes limitations

Purpose

Details research objective and how results will be used

Respondent profile

Provides details about collected sample and demonstrates how representative it is of the target market population

Open-ended questions considerations

• Avoid yes/no questions

• Keep questions simple & clear

• Prioritize close ended Qs

• Consider survey real estate

• Requires coding

• Respondents may self select

Bad survey questions

• Ambiguous (unclear)

• Complex (many parts)

• Double-barreled (2 or more questions in one)

• Leading (wants respondent to answer a certain way)

• Loaded (socially desirable response)

New product development

• Brand read/comparison

• Optimal features

• Purchase intent

• Pricing

• Brand acceptance

Online surveys

• Best:

  • Brief to average length

  • Some level of complexity (skip & piping)

  • Reaction to images or video clips

  • Anonymous so more comfort for responds (more truthful)

• Worst:

  • Need clarification

  • Populations without internet

• Notes:

  • Very affordable

  • Results come in fast

Snail mail survey

• Best:

  • Good for some level of complexity, or questions that require additional thought

  • Targeting specific geographies

  • For sensitive topics where anonymity is important

• Worst:

  • Long surveys requiring a lot of writing

  • Complex surveys that require skipping questions or multiple versions

  • Quick timeline

  • Requiring clarification

• Notes:

  • Cost is moderate to high

  • Very slow

  • Response rate is low

Telephone survey

• Best:

  • Moderate length (less than 10m)

  • Surveys that need to be screened

  • Surveys that need a guided approach

  • Requires getting past gatekeepers for appropriate respondent

• Worst:

  • Random studies of general population

  • Sensitive studies

• Notes

  • Cost is moderate to high

  • Moderate speed

  • Once respondents are reached response rates are high

Survey invitation components

• Researchers credentials

• Purpose of survey

• Reason respondent is being asked

• How responses will be used

• If responses will be kept anonymous

• Survey due date

• If there is an incentive

Factors impacting response rates

• Distribution method (mail, phone, email)

• Survey length

• Survey complexity

• Potential respondents interest in topic

• Type of survey

• Blind study or sponsered

• Type of audience

• Accuracy of contact information

• Incentives

• Reminders

Softing the list

Telling potential respondents ahead of time that they will be invited to participate in the survey (often employee or customer satisfaction survey)

Survey implementation biases

• Non-response bias

• Non-contacts

• Refusals

• Self-selection bias

• Response bias

• Deliberate falsification

• Lazy respondents

• Extremity bias

• Interviewer

• Social desirability