UCLA COMM 150 (based on midterm info)

What type of sampling strategy did Esther Duflo use in her study about immunizations? What about in her study of bed nets? Briefly explain

experimental sampling for both because they utilized a manipulated variable.

for immunizations it was whether the camp had incentives or not, and for bed nets it was whether they paid or not for the bed net, and how that would alter if they came back or not.

what are the four sampling strategies?

experiments, surveys, field research, and existing data

(3) What are two benefits and two drawbacks of conducting qualitative field research? Briefly explain.

benefits-->

1. allows for an insider view

2. exploration of complex causal mechanisms (more than just x causes y)

1. difficult to generalize (one story about one group in one area cannot account for every group like it etc)

2. difficult to replicate (hard to do the study again to collect more data)

3. is the researcher completely objective?

describe the circle graph of reliable but not valid (include level of systematic or random error)

all the dots meet closely together in the upper right hand corner - reliably shot but not accurately. (1. High systematic, low random)

describe the circle graph of valid but not reliable (include level of systematic or random error)

on average the measure is close to the bullseye but not reliably shooting anywhere (2. High random, low systematic)

describe the circle graph of neither reliable nor valid (include level of systematic or random error)

the dots are all at the top, spread out with neither getting close and neither close together

(3. High random and moderate systematic)

describe the circle graph of both reliable and valid (include level of systematic or random error)

all dots in the center (Low systematic and low random)

What is social desirability bias? Identify two ways a researcher may be able to design a survey question to mitigate this bias? In your answer provide an example

Social desirability bias is a phenomenon that occurs when people will answer whatever they believe is the most "socially correct," especially when being interviewed. Some people with more radical or unpopular opinions may be less likely to share their actual thoughts if they feel they will suffer some kind of backlash.

ways to mitigate this:

Anonymity

Avoid leading questions

Neutral wording

what is the level of measurement for: race

AND WHY

Nominal - categorical without intrinsic order (eye color, gender, etc.)

what is the level of measurement for: income

AND WHY

ratio - numeric - continuous with a true zero point (you can make zero dollars/no income)

what is the level of measurement for: number of children in a family

AND WHY

ratio - numeric continuous with a true zero point (you can have no kids)

what is the level of measurement for: temperature of a pool

AND WHY

interval - numeric - intervals between values are meaningful with no true zero point (meaning zero is just a measurement point, not a signifier of nothing)

what is the level of measurement for: whether you win contest or not

AND WHY

binary- can only have two categories (winning or not)

what is the level of measurement for: A softball player's shoe size (5, 5.5, 6, 6.5, ... 11.5, 12)

AND WHY

interval - numerical without meaningful zero

what is the unit of analysis in this example?

Are older people more afraid of crime than younger people

individuals / people

what is the unit of analysis in this example?

Longer engagement periods lead to longer marriages

couples (couples/marriages (dyads))

what are target populations and sampling frames?

Target populations - the population to which researcher would like to generalize their findings

Sampling frame - set of all cases from which the sample actually selection, usually a list

what is cluster sampling and why do we do it?

break the population into cases called clusters (internally hetero externally homo) select at random from these clusters

we do this to save on resources/money where it may be too difficult/cumbersome to get the entire population (esp. for big populations)

what is stratified sampling and why do we do it?

start with breaking down the population into groups that we call strata - within each strata idea is that we use a variable to move people into strata

- Each stratum should be composed of very similar individuals in terms of the variable(s) used in stratification

-Each individual should belong to one and only one group overall

we do this when we want to properly represent the population as closely as possible

is this cluster or stratified?

: we want to know about drinking behavior among men and women for the population of Comm 150 students over the past three years. Look only at 3 of 12 classes

cluster

is this cluster or stratified?

three towns with 100k, 200k, and 300k population. sample 10, 20, and 30 respectively

stratified

as sample ______ increases, sample _____ decreases

size, error (becomes more representative)

what is the basis of simple random sampling?

each possible combo of cases has an equal probability of being sampled.

why use systematic sampling?

1. easy to do (e.g., exit poll)

2. if you believe the population is randomly distributed along the scale

3.if you are concerned about the clustering effect of other sample (sometimes not the best to just get the first round of people (not representative))

what is systematic sampling

sample every kth case (k = natural counting number)

what were esther duflo's findings?

She learned that

1. people were more likely to come if there were incentives involved along with conveniently placed camps (like lentils),

2. that people were willing to pay for abed net even if they had gotten it discounted/free before,

3. that offering deworming was more successful than what people might have thought would increase educational ratings.

can esther duflo infer causality? if so, why?

Yes -- she was able to have a natural and random process of measuring outcomes, with each independent variable playing a role in the output of whatever it was they were looking for, utilizing a totally random sampling system to minimize error. In the immunizations example, they had three camps, those with incentives, those without, and no camps at all, which were then sectioned and selected randomly in an effort to eliminate possible exogenous variables.

what are exogenous variables? why is its elimination important?

variables whose measurement is outside of the model (not accounted for)

elimination can allow for strong causal inferences

difference between internal validity and external validity

internal = was study conducted well?

external = does this study properly represent the population it is meant to?

name the challenges to internal validity (6)

1. history

2. maturation and testing

3. regression to the mean (statistical regression)

4. selection

5. instrumentation

6. any poorly conducted study

what is history in regards to internal validity?

Other events in the environment that might affect the outcome.

Example: a television program that airs during the course of an experiment

----If the program talked about the debate and a lot of students knew about it, it would change their overall evaluation

what is maturation and testing in regards to internal validity?

Changes taking place within subjects over time during the course of the experiment

Example: subject getting tired of the course of the experiment

Example: subjects perform better on a test the second time they take it

Example: subjects start to understand the socially desirable response over time or figures out what the study is about

what is instrumentation in regards to internal validity?

The way in which the measurement is taken changes over time

Example: a person employed to count event gets bored and less accurate as time passes

what is statistical regression in regards to internal validity?

Extreme measurements tend to move closer to the mean on second observations

Especially due to random measurement error

Example: people who score low on a pre test are likely to score closer to the average on a post test

----Even without treatment or anything (sometimes people jsut do poorly on a quiz)

---- REMEMBER - GETTING BETTER FROM THE WORST

what is selection in regards to internal validity?

Selection

Systematic differences in the composition of control and treatment groups. Huge issue in non-experimental, observational research

Example: people who sign up for the treatment

Vaccine - if people just sign up via volunteer, you will get a certain group of people

how can you avoid selection issues of internal validity?

proper random assignment

difference between complete observer and observer participant

Complete observer: when subjects have no knowledge of being observed (e.g., hidden camera (ethical consideration) looking on from outside)

Observer participant: limited interaction, but subjects know they are being observed (e.g. cameras; notetaker)

difference between complete participants and participant as observer

Complete participant: subjects do not know they are being observed

Participant as observer: subjects know they are being observed by fully engaged participant

what is random error and what are some examples of it?

unpredictable and unbiased errors that cannot be accounted for but usually do not skew the data too intensely - usually averages out the bigger the population gets

examples: misreading/misunderstanding questions/instructions, forgetting information, etc.

what is systematic error and what are some examples of it?

biased errors that suggest there is a systemic or instrumental impact that influences all the answers gathered from the population sample. (it keeps it from being equally accessed or unbiased)

examples: the instruments used to collect data are incorrect, social stigmas/social desirability biases/ incentivization for a certain answer

T/F the unit of analysis will sometimes have the same unit of measurement as the independent variable

false - they will never be the same measurement

correlation v. causation

Correlation refers to a relationship or association between two variables, where changes in one variable are related to changes in another.

Causation means that one variable directly affects or causes a change in another variable.

CORRELATION DOES NOT IMPLY CAUSATION

what is an ecological fallacy?

Incorrectly assuming that an association on a population level reflects an association on an individual level

what is a spurious relationship?

the appearance of causal relationship where in fact the relationship is either a coincidence or due to a confounding variable

why is it important to have a hypothesis and theory before conducting research?

- if we come up with it afterwards, we may start to see connections/relationships and see that as the result

what does acquiescence mean in regards to systematic errors in survey questions?

To "acquiesce" means to be more likely to agree than disagree, which can be largely impacted in the way we phrase our questions.

If the researcher says "Do you agree that Beyonce is the greatest performer?"

The participant is more likely to agree, as there is an established bias within the question.

what does anchoring mean in regards to systematic errors in survey questions?

"Anchoring" refers to an effect that impacts a secondary question, which focuses on the importance of how we order our questions.

If the researcher tells a student to tell them the first three numbers of their UID, and then asks how much they would pay for a bottle of wine, there is a likelihood that the number will be impacted by the first number shared.

what does ordering mean in regards to systematic errors in survey questions?

"Ordering" is a lot like anchoring in the sense that the order of questions asks can impact how people perceive the overall question.

If the researcher asks "how often are you dating?" and then asks "how happy are you?" Many may misunderstand the question as being directed to happiness in relationships.

what is informed consent?

A participant's agreement to take part in a study after being told what to expect.

what is coercion?

another challenge in informed consent - do the participants have autonomy to decide whether they want to participate or not?

if a prof said to their students "you dont have to participate in this study, but if you want a good grade you should"

what is deception?

Deliberately misleading or misinforming subjects about some aspect of the study

- Violates subject's right to informed consent

- Violates trust in the investigator-subject relationship

the good science ethic

- Scientists also have an ethical obligation to be as honest and accurate in their research as possible

---Fabrication, falsification, and plagiarism are unethical

what are the four problem areas of human subject research?

1. potential harm

2. lack of informed consent

3. deception

4. privacy

what does potential harm entail when discussing the four problem areas of human subject research?

Physical harm

-- also mental harm

Humiliation or embarrassment

Loss of self esteem

Loss of trust in others

what are the stages of field research (6)

1. topic/setting selection

2. gain access

3. present oneself/establish roles

4. take notes/begin analysis

5. develop analysis

6. leave the field/report findings

what are the three ways of looking in qualitative research?

1. experiencing

2. enquiring

3. examining

identify the Uoa, IV, and DV:

Increasing study time will improve test scores

UOA: test takers

IV: level of study time

DV: test score grade

identify the Uoa, IV, and DV:

Drinking water before a workout increases running performance

UOA: runners

IV: drinking water before a workout or not

DV: running performance level (weak, average, strong)

identify the Uoa, IV, and DV:

Using study apps will reduce procrastination among students.

UOA: students

IV: using study apps or not

DV: procrastination levels

non-probability sampling (what are the 4 types?)

1. convenience

2. purposive

3. quota

4. referral/snowball

what is convenience sampling?

Researcher creates their own sample

E.g. studying my students, standing on the sidewalk and asking passerby (not everybody is going to be walking down the same corner (not equal probability)

Easy but not representative if you're looking for info on all students

what is purposive sampling?

Select specific (typically few) cases interested in

E.g., using "bell weather states: state that seems to be pretty representative of the population (ohio)

what is quota sampling?

Sample until you get x people with specific trait

E..g, sample until you get 25 light drinkers and 25 heavy drinkers

what is referral/snowball sampling?

Referral /snowball

Existing subjects recruit more subjects

E.g., gang members

Ok if you don’t care about random because it’s easy

Works with students if we think they are homogenous on the aspects we’re looking for

when should you use non probability sampling?

When probability sampling isn't feasible

ex: If you're in a gang, you're not going to be easy to approach

When uninterested in generalizing to population (e.g., forming hypothesis

homogeneity versus heterogeneity in variation

Heterogeneity - variation ( a group that has this has a lot of variation on a certain attribute)

Homogeneity - people are very similar with respect to a certain trait

as we increase the heterogeneity, we increase our what?

sampling error

ex: Drinks - if we have wide variation - average measure is going to be more likely different from the population if there was lower variation

why would 1,000 people be sufficient for a population sample of the US?

Function is concave - each additional person we sample while they decrease out expectation for error, they do so at a lower level of the person before them

----As were increasing our accuracy, it starts to level off

----Often for the population, it levels off pretty quickly

Soup analogy - you do not need to have half of the soup to know if it tastes good, just make sure its stirred well (*population needs to be representative)

what are the two types of variation?

Over time: over years, etc. (utilized a lot for economics/recession)

Cross-sectional: same point in time, and across sections

Individual or group focus, mutually exhaustive, exclusive – education impact on business economic news consumption → HS, College degree, etc.

an example of a spurious relationship given in lecture?

9/11 conspiracies -- coincidence in numbers on the planes, passengers, etc.

four examples of confounding variables given in lecture

1. ice cream shark --> confounding heat, but ppl assume ice cream sale increase = increase shark risk

2. Teens & sleep & study → ABC news → infer test score = sleep; but instead study

3. Sex & money: higher sex = higher paycheck, no b/c maybe someone more confident + attractive is likely to be more successful

4. Sanctuary cities: larger cities → have higher crime, not b/c racial diversity

what kind of error was the butterfly ballot? (and what happened?

systematic

Ex. 2000 election between Bush and Gore. Butterfly Ballot, Floridian Palm Beach County using new ballot— punching out a hole (hanging chat: not completely out). Gore on the 2nd right and someone punches 2nd hole but that’s Pat Buchanan— lot of people voted for Buchanan. 3,000 as opposed to probably 700 on average.

what is an example of a selection issue of representativeness? (and some material facts)

Syrian Assad reading - using a sample not representative of the population. (Ethiopia phone study)

Only surveyed 98 people living in Syria. (small population, especially not relative to people who don't even live there) according to article 1,000 being the minimum.

Only 18% of people in Syria can access the Internet.

what is standard deviation?

how much variation there is in the population

what is standard error versus sampling error?

Standard error: estimate of standard deviation

Sampling error: difference between measured and population

within subject versus between subject

Within: show ppl hundreds of faces and see which ones they rates as familiar, how does faces effect familiarity → changes in a single person

Ex. social anxiety – measure overtime in a single person, on a day high or not

Between: show different ppl different faces and ask them to rate which ones are most familiar → across individuals

Ex. social anxiety – comparing a lot of different ppl, and their levels of social anxiety

three different survey designs

Cross sectional: respondents surveyed at only point in time (most surveys)

Longitudinal: same questions asked multiple times periods to different ppl

Panel studies (aka time series cross sectional or TSCS): when cross sectional subjects surveyed w/ same questions over time (look @ both across different individuals & over time)

what were the sampling designs of Esther duflo's studies of bed nets and immunization? (hint: not the same)

bet net - convenience sampling

immunization - cluster

Immunization: three conditions, for each condition → cluster sampling, cluster is the village

Bed nets: convenience sampling → b/c just walking to the street and just handing out coupons.

Name three experiments discussed in the readings or class lectures (in person or online), and

identify whether each was a field, lab, or survey experiment. You do not need to explain why.

Field - Esher Duflo immunization

Lab - Nixon Kennedy debate

Survey - Parents keep out