Research Methods Exam 2

What is sampling?

Selecting a relatively small number of elements from a larger defined group of elements

Why do we collect a sample?

Impossible/Unreasonable to conduct a census; Less time consuming and less costly; Influence the type of research design, the survey instrument, and actual questionaire

Population

an identifiable group of elements (people, products, organizations)

Sampling Units

Target population elements actually available to be used during the sampling process

Ex: Study the brand's advertising strategy; A brands recent ad copy.

Sampling Frame

The list of all eligible sampling units

Sampling Errors

A statistical error; It's any bias that results from mistakes in either 1.) the selection process or prospective sampling units or 2.) determining the sample size

Nonsampling Errors

Is an error that occurs during data collection, causing the data to differ from the true values. Ex: data entry errors, biased survey questions, biased processing/decision making; Hard to detect/define.

Probability sampling

Knowing the probability of being selected for the sample

Nonprobability Sampling

Not knowing the probability of selecting each sampling unit (don't know the sampling error) based on the judgment of the researcher and may or may not be representative of the target population.

Simple Sampling

A probability sampling procedure in which every sampling unit has a known and equal chance of being selected

Systematic Sampling

defined target population is ordered in some way, usually a customer list, taxpayer roll, or membership roster and selected systematically

Stratified Random Sampling

Separation of the target population into groups, called strata, and the selection of samples from each stratum.

1. Divide target audience population into homogeneous subgroups/strata

2. Combine the samples into a single sample of the target population

Convenience Sampling

Draws samples at the convenience of the researcher; Commonly used in the early stages of research and pretesting questionnaires

Snowball Sampling

A set of respondents is chosen, and they help the researcher identify additional respondents; Typically used when the defined target population is small and unique or compiling a complete list of sampling units is difficult; Referral Sampling

Determining the appropriate sampling design & what to consider

Research Objective: calls for qualitative or quantitative
Degree of Accuracy: are inferences or just insights needed?
Resources: is there a tight budget?
Time Frame: how quickly does the research need to be completed?
How Much Prior Knowledge: are there complete lists? how easy is it too generate sampling frame of respondents?
Scope of Research: international, national, regional, or local?
Statistical Analysis Needed: is statistical projection or hypothesis testing required?

What is a measurement?

The process of developing methods to systematically characterize or quantify information about persons, events, ideas, or objects of interest.

Abstract

An abstract idea or concept formed in a persons mind; A combination of a number of similar characteristics of the construct

Example: consumers intention to support the organization

Variables

Consists of those characteristics that collectively define the concept

Construct Development

is the process in which researchers identify characteristics that define the concept being studied by the researcher

Construct

An unobservable concept measured indirectly by a group of related variables

Consumer Concrete Properties

age, marital status, sex, income, brand last purchased, dollar amount last purchased, types of products purchased, color of eyes and hair

Consumer Abstract Properties

Attitudes toward brand, brand loyalty, high-involvement purchases, emotions (love, fear, anxiety), intelligence, personality

Organization Concrete Properties

name of company, number of employees, number of locations, total assets, fortune 500 rating, computer capacity, types and numbers of products and service offerings

Organization Abstract Properties

competence of employees, quality control, channel power, competitive advantages, company image, consumer orientated practices

Scale Measurement

Assigning a set of scale descriptors to represent the range of possible responses to a question about a particular object or construct

Nominal Scale

Requires respondent to provide only some type of descriptor as the raw response

Example(s):
Please indicate your marital status:
___ Married ____ Single ____ Divorced
Do you like or dislike chocolate ice cream?
___ Like ____ Dislike

Ordinal Scale

Allows a respondent to express relative magnitude between the answers to a question

Example:
For each pair of retail discount stores, circle the one store at which you would be more likely to shop:
_____ Costco or Target
_____ Target or Walmart
_____ Walmart or Costco

Interval Scale

Demonstrates absolute differences between each scale point

Example:
How likely are you to recommend the Santa Fe Grill to a friend?

Definitely Will Not Rec. Definitely Will Rec.
1 2 3 4 5 6 7

Ratio Scale

allows the researcher not only to identify the absolute differences between each scale but also to make comparisons between responses

Example(s):
Please circle the number of children under the age of 18 currently living in your household
__1 __2 __3 __4 __5 __6 __7 If more than 7 please specify __

In the past seven days, how many times did you go online to shop at Amazon?
______ # of times

Reliablity

Results are consistent

Validity

Results satisfy objectives

Scale Reliability

extent to which a scale can reproduce the same or similar measurement results in repeated trials; measures consistency

Scale Validity

assesses whether a scale measures what it is supposed to measure

Likert Scale

An ORDINAL scale format asking respondents to indicate the extent to which they agree or disagree with a series of mental or behavioral belief statements about a given subject

Example: DA - Disagree SDA- Somewhat DA AG- Agree

DA SDA SAG AG
I buy many things with a credit Card ___ ___ ____ ____
I wish we had a lot more money ___ ____ ____ ____

Semantic Scale

A unique bipolar ORDINAL scale format that captures a persons attitudes or feelings on a given object; Semantic Differential Scale; Use even number of anchors.

Example:
1 2 3 4 5 6 7 8
Expert ___ ___ ___ ___ ___ ___ ___ ___ Non Expert
Honest ___ ___ ___ ___ ___ ___ ___ ___ Dishonest

Bipolar Scale

The inappropriate narrative expressions of the scale descriptors; A well-designed scale has truly bipolar anchors; Use even number of points.

GOOD Example:
(Like) 1 2 3 4 5 6 7 8 (Dislike)

BAD Example:
(Like) 1 2 3 4 5 6 7 8 (Interesting)

Balanced Scale

has an equal number of positive and negative response alternatives

Example:

• Very Satisfied

• Satisfied

• Dissatisfied

• Very dissatisfied

Unbalanced Scale

Has more options on one side

Example:

• Very Likely

• Somewhat Likely

• Likely

• Unlikely

• Very Unlikely

Forced Choice

does NOT have a neutral descriptor; Even amount of options

Comparative Scale

A respondent expresses their attitudes, feelings, or behaviors about an object on the basis of some other object

Example:
Please rank your top three preferences of types of music you enjoy listening to by writing your first, second, and third choices below.

First Choice ______________________
Second Choice____________________
Third Choice______________________

Non-Comparative Scale

Used when the objective is to have respondent express their attitudes, behavior, or intentions about a specific object WITHOUT making references to another

Example:
Smiley Face pain descriptor

Measurement Scale Issues

When phrasing the question element of the scale, use clear wording; Make sure scale point descriptors are relevant and adequate

Double Barreled Questions - BAD

Includes two or more attributes in the same question, but responses allow comment on a single issue

Example:
How well do you get along with your managers and coworkers?

Leading Question - BAD

Influences the respondents answers

Example:
Our customer service team is the most responsive in the industry. How responsive --or unresponsive-- do you think they are?

Double Negative Question - BAD

Contains two negative thoughts in the same question

Example:
Do you oppose to not allowing policy to enforce all employees to wear a mask while flying?

Loaded Question - BAD

Suggests a socially desirable answer or involves emotionally charged issue

Example:
Should Americans buy imported automobiles that take away American jobs?

Ambiguous Questions - BAD

involve possible responses that can be interpreted a number of ways

Complex Questions - BAD

Are worded in a way making the respondent unsure of how to respond

Designing Measurement Scales: Steps & Questions

1. Understanding the Research Problem

2. Identifying and Developing Constructs

3. Establishing Detailed Data Requirements

4. Understanding the Scaling Properties

5. Selecting the Appropriate Measurement Scale

Developing Questionnaire Design: Steps

1. Confirm research objectives and information requirements

2. Select appropriate data collection method

3. Develop questions and scaling

4. Determine layout and evaluate questionnaire

5. Obtain initial client approval

6. Pretest, revise, and finalize questions

7. Implement survey

Step 1: Confirm research objectives and information requirements

Understanding whether the current advertising strategy is effective; To identify information you need to collect consumers opinion about the current advertising strategy

Step 2: Select appropriate data collection method

Determine the most appropriate and effective communication method for interacting with the targeted respondents

Step 3: Develop questions and scaling

Question Format

• Unstructured: open-ended questions formatted to allow respondents to reply in their own words - allow unlimited response - often skipped**

• Structured: close-ended questions that require the respondent to chose from a predetermined set of responses or scale points

  • reduces respondents effort

  • easy to answer

Step 4: Layout and questionnaire assesment

Give overview of research; Screening Question(s); Arranging questions; Include demographic questions; End with thank you statement

Step 5: Obtain initial client approval

Copies of questionnaire should be given to all parties involved in the project; Researchers MUST obtain final approval BEFORE pretest

Institutional Review Board (IRB)

reviews and approved, prior to initiation, all research that involves use of human participants as the source of data, to make sure their rights are protected

Step 6: Pretest, revise, and finalize questionnaire

• Pretest: survey sent to 10-30 subjects

• Pilot Study: 100-200 respondents of target population

Step 7: Implementing survey

Collect the data using the agreed upon questionnaire

Data Analyzation Process

1. Data collection

2. Data reduction

3. Data display

4. Conclusion drawing/verifying

Data Reduction

Researchers make decisions about how to categorize and represent the data; The most systematic method is to read transcripts and develop categories to represent the data.

Coding

Unstructured data (open ended answers) requires some type of "coding" prior to analysis

Code Sheet

has all the codes on it; the codes can be words or numbers that refer to categories on the coding sheet.

Comparison - Analyzing

Developing and refining theory and constructs by analyzing the differences and similarities in passages, themes or types of participants

Iteration - Analyzing

Working through the data several times in order to modify early ideas

Memoing

writing down thoughts after interviews, focus groups, or site visits

Negative Case Analysis

Used during the ITERATIVE process; Researchers look for cases that contradict the developing theories; Establish boundaries and conditions for theory; Researchers should be skeptical

Data Display

Reduce and summarize data and convey major ideas

Credibility - Qualitative Research

Describes the rigor and believability in qualitative analysis

Emic Validity

Affirms that key members of a culture/subculture agree with findings

Cross-Researcher Reliability

Degree of similarity in coding by different researchers

Triangulation

Requires that research to be addressed from multiple perspectives

Data validation

Determines if surveys, interviews, and observations were conducted correctly and free of errors

Data Editing

Checking the data for mistakes by the interviewer, the respondent, or in the process of transferring information from scanner databases or other sources to the company data warehouse

Data Entry

Procedure used to enter the data into a computer file for subsequent data analysis

Error Detection

Identifies errors from data entry or other sources

Tabulation

Counting the number of responses in categories

Cross-Tabulation

Categorize responses to two or more questions

Missing Data

Situation where respondents do not provide an answer to a question

Mean

Average of data; Most commonly used measure of central tendency

Median

The middle value of an ordered set of values

Mode

The most occurring value in a distribution of values

Range

The spread of data; Distance between the smallest and largest values

Short Answer Q1: Write a question that measures people's response toward an advertisement using 1.) Likert Scale and 2.) Semantic Scale

Likert: This advertisement made me more likely to purchase a product from this company

1. Completely Disagree

2. Somewhat Disagree

3. Disagree

4. Agree

5. Somewhat Agree

6. Completely Agree

Semantic Scale: How was your last experience purchasing a product from company? (Pleasant) 1 2 3 4 5 6 7 8 (Not Pleasant)

Short Answer Q2: Write advantages and disadvantages of using UNSTRUCTURED questions in a survey

Advantage: Respondents get to put in their personal input/thoughts
Disadvantage: Likelihood for the question to be skipped entirely or will get no response