Social theory exam 2

Sampling

The process of deciding what or whom to observe particularly when we cannot observe anything or everyone

• Heterogeneous opinions in any population

• Limited resources

Probability sampling

Type of sampling where the sample is selected on probability

• Typically involves some random selection mechanism

• Large representative samples

• Key characteristics

  • Selection through random choice

  • Everyone has an equal probability of selection

Benefits of probability sampling

• Representativeness→ a sample has the same distribution of characteristics as the population from which it was selected

• Generalizable→ degree to which you can apply the results of your study to a broader context

• Cost efficient

• Avoids bias→ can be conscious or unconscious, those selected may not be typical or representative of the larger population

• Margin of error→ the degree to which a sample average differs from the population average

Size of margin

Larger sample → smaller margin of error → closer approximation to true population

Identifying population

Group we are interested in generalizing

Establishing sampling frame

The list of units composing your population

Use random selection to sample your elements

Individual units comprising your sample

Simple random sampling

• Pulling names out of a hat

• equal probability of selection

• Random number generator chooses people

Systematic sampling

Pick every k^th observation, fixed interval as to who is being chosen (based of scramble data)

Cluster Sampling

Randomly select individuals from clusters

• Sampling frame does not exist

Stratified Sampling

Divide people into different groups and then sample them based on belonging

• In proportion to population portions or oversample certain groups necessary

  • Randomly select people in these groups

Non-probability sampling

A sample that is not drawn using a method of random selecion

• Not generalizable (all have their own forms of bias, but typically have a reason to do so)

Convenience sampling

Selected due to convince

• happen to be in the area and they are easy to reach

• Not suggested unless the people passing are the actual target sample or you are doing exploratory work (and need something cheap and easy)

• doesn’t have a sampling frame

Purposive sampling

Units selected based on which will be the most useful or insightful

• Think about the quality of the data the give will give you and then select the individuals based on what you want

• specific groups with unique characteristics

  • Late diagnosed autistic women (would not be able to do this from random sampling

  • A thing like picking big brother contestants would be random sampling

• Could be used to access small populations

• Key consideration: Can we access that people that we need

What type of cases should be purposively examined?

typical:

• Wanted to study the comeback of being a rustbelt city

  • Would want to look at pgh

Extreme:

• Anything but extreme

  • Trad wife—> Resemble the rise of tradition gender roles in america

    • study trad wives and see why they want to be this way

Important:

• Choose cases about what is most important foe your group

  • Like harvard, yale, Princeton when studying elite universities

Devient

• Unusual, unexpected, hard to explane

• Anomalies or outliers

  • Like characters in zombie movies that are immune to virus

Contracting type

• Two things that relate to study, but have different outcomes

  • Two people grow up in the same way, but go down two different paths

Sampling for range

Maximizing respondents’ range of experience with the phenomena under the study

Quota sampling

Units are selected on the basis of pre- specifies characteristics, so the sample will have the same distribution of characteristics assumed to be in the population being studied

• Hand selection

Snowball sampling

Interviewees are suggest additional people for interviewing

• useful when we don’t know a lot about the population or when the population is difficult to locate

• Word of mouth

• Networks

Key Informants

The first point of contact a researcher has with their population

• We want someone who is high up and can give strong connections

Saturation

When additional data fail to yield new insights and simply reinforce what the researcher already knows

What are the benefits of non representative samples?

• Rich information

  • Function of the sample size

• When to stop?

  • When you have reached saturation

• Causal Mechanisms

  • Rich data helps because we can literally ask why

  • X causes Y

Survey

A social research method in which researchers ask a sample of individuals to answer a series of questions

• Pre-written questions

  • Typically closed ended, so there are fixed responses

• Highly structured

• Not much room for conversation

• Often examine thoughts, opinions, and behaviors

What are the advantages of surveys?

• Good for describing large populations (when used with probability sampling

• Large samples are feasible

• Breadth of topics

• Reliability and comparison

• Very reliable

• High external validity→ results can be generalized

What are the disadvantages of surveys

• Assessing causality is difficult

• Measurement validity→ The question asked may suffer from validity problems

  • Hard to get complex ideas from surveys

• Lack of context→ Does not give a feel of the way that the situations we are asking about are playing out

Sampling error in surveys

if we want to generalize, there will be a little bit of error

• Can have error between sampling frame and population→ Results are not representative

Nonresponse error

Hard to get people to respond to surveys

• Not generalizable of the population that we are targeting

• Data are only as good as the questions asked

Closed ended questions

Subjects can respond only in present ways

• Present options given

Open ended questions

Subjects can respond in their own words

• hard to analyze

Dichotomous questions

• Closed ended

• Only 2 options are given (eg. yes or no)

Lickert scale

• Closed ended

• Ranking

• Measures things like satisfaction, importance, quality, etc.

Survey Block

Group of questions within the survey

• Typically related to the same thing

• Streamline the process for respondents

Bipolar scale

Takes 2 opposing statement and put them on opposite ends of a scale

• Closed ended

• Measuring preferences along a spectrum and the subject are asked to answer on the scale

• They are forced on either end of the scale even if you don’t believe that they are opposing each other. Researcher creates the scale

Nominal

• Closed ended

• Set of choices

• Thinking like a select all that apply from a list of races

Ranking

Respondents rank their priorities or preferences

• Problem: What if you feel equally about thing or what if you don’t care about any of them

• Vey hard to analyze

Mutually exclusive

Categories that do not overlap one another

• are the options distinct

Example of mutual exclusivity

eg. What best describes your age?

• if given 21-25 and 25-35, it is not mutually exclusive, confusion is caused for survey takers

** How can we make this mutually exclusive?

• give the options of 21-25 and 36-35

Exhaustive

All potential responses are available

• are any options missing?

To make a question exhaustive…

• brainstorm all of the categories

  • If you run out of all brainstorming capacity, can add the “other” category

  • “other can also be used if you have a primary focus, but you do not want to force subjects into an answer

What is pilot testing when thinking about exhaustive questions?

Can give the question to a small group and ask them to think of anything that is missing

Forced choice questioning

Removing some options from the menu

• Responses cannot be neutral, subjects are being forced to answer

• Good to have a “prefer not to say” and “don’t know”, but too much “prefer not to say” can cause response bias

Acquiescence Bias

The tendency to agree no matter what

How to construct an open ended question

• no pre established responses

• Require elaboration

• Brief and non-cumulative

• Some questions can be embedded with in closed ended questions

  • Like the “other” category

• Some are purely open ended

Benefits of open ended questions

• Deeper understanding

• Encourages respondent engagement

  • Respondents want to feel like their voices are being heard

  • Improves validity

Face to face interviews

When you interview a person face to face

• In public space, home, research office

  • Location can change respondents answer

What are the advantages of a face to face interview?

• High completion rates

  • Unlikely that people will not finish the interview

  • People don’t like to say no

  • People like to talk about themselves

What are the downsides of a face to face interview

Interviewer effect

• People may be unwilling to say something that will hurt your feeling (tells us what we want to hear given the traits that they can infer)(who are you)

• May disclose thing to interviewers, who have the same race, for example, that they otherwise would not

Social desirability

• Idea that respondents will answer in ways that is expected of them by society (what does society want)

Cost

• Face to face is the mist expensive by both money and time

Telephone surveys

Interview given over the phone

Advantages of phone interviews

Higher completion rates and data quality

• Similar to face to face, but better than other options

• Random digit dialing

More cost efficient than face to face

Fewer interviewer effects

• More distance

• Harder to discern traits over the phone

Disadvantages of phone surveys

Response rate

• People don’t answer numbers that they don’t know

• Less rapport

• Response fatigue

  • People don’t like long calls

• Sampling bias

  • Older and lonely people are more likely to answer the phone

Self administered surveys

Take survey on their own (often sent through the mail)

• typically very structured

• Paper Pencil

Advantages of self administered surveys

Low cost

Low bias

• there is no interviewer, so no contact

Convenient for the respondent

Disadvantages of self administered surveys

Easy to just ignore the survey

Easy to throw out survey that looks like junk

Lower response and completion rate

Internet surveys

Respondents answer online

• most rapidly evolving

• Typically sent through email

Advantages of online surveys

Cheap

• Don’t have to pay staff

• Don’t need materials

• Data is automatic

• Easy access and administration

• Fast

Disadvantages of online surveys

sampling bias

• Coverage error

  • Come people on the internet are not representative of everyone on the internet

Response rate

• May not get email or might just delete it

Questionnaire ordering

Which questions you chose to place where impacts survey results

• establish rapport

  • start easy and end with demographics

• Avoid monotony

  • alternate topics

  • Vary response options

    • If people are bored, they may answer the same questions with the same response (straightener)

• Quality controls

  • Speeders

  • Straighteners

  • Attention

    • A question that makes sure they are paying attention

      • Freebie

      • Simple math

• Prevent order effects and priming

Order effects

When the order in which questions appear biases the responses

Priming effects

When exposure to a particular wood, image, or feeling shapes how respondent thinks and feels

Formating

Goal is for respondent to have a streamline, confusion free experience

• consistent

• Uncluttered

• Intuitive

• Clear

When surveys aren’t well formatted, respondent is encouraged

Survey blocks

Composite variables

Index

scale

Contingency / filter question

Split ballot design

Composite variables

Combines multiple survey items to create a single value that captures a multifaceted concept

Index

Sum

Scale

Average

Filter questions

a question of a series of questions associated with a conditional response to a prior question

Split ballot design

One half of the sample receives one type of question and the other half gets another group of questions

• random assignment

Survey design

The process of creating, formatting, and sterilizing your survey, it is critical for ensuring reliable and valid research

Does question wording matter?

Yes!

When in doubt, KISS (keep it simple stupid)

KISS

Short

• Don’t add fluff words, if people don’t understand, they won’t respond

Consistent

• Make sure that a question will mean the same thing to everyone

Easy

• Keep it to a middle school reading level, unless targeting higher education

Double Barreled question

Ask about two or more ideas or concepts in a single question

Problems with double barreled questions

Can be difficult and long

• eg. Do you support increasing taxes and improving public service?

Fix to the problems with double barreled questions

Make them two separate questions

• eg. Do you support increasing taxes?... Do you support improving public service

Problem with questions requiring expertise

People may not know what you are talking about, so they might guess or not answer

• reduces validity… not getting the truth

  • eg. What do you think about proposed tax plan?

How do we fix the problem with questions requiring expertise

• Provide definitions, examples, or other necessary background information

• Avoid jargon

• Regardless of expertise, a person should be able to answer the question

Involved negation

contains words “not” or “no” or “don’t” or “without”, or other negative words

Double negative

• decreases validity

  • eg. Do you not agree that the customer service was not helpful?

Double negative

This is a con, respondents do not know if they are answering the question

Fixes for negation

Remove negation

• Eg. do you agree or disagree with the following statement; customer service was helpful

Unclear wording

Ambiguous, vague, multiple meanings

Unclear wording problems

• People do not understand the questions in the same way

• Not reliable

  • eg. What do you think of DOGE?

    • Could be money, dog, or department of government efficiency

What are the fixes for unclear wording problems

• be precise

• Offer response categories

Leading questions

Influences or guides respondents toward a particular answer, often implying a preferred response

Problems with leading questions

All questions are made to get a response, but if it is designed that way it is a leading question

• stating the way other feel

• Use a word that brings up emotion and/or moral standing

• “do you agree”

Fixes for leading questions

• removing bias

  • give both options “whether or not” “do or don’t”

• Make neutral

Loading Wording

Welfare vs helping the poor

Requires excess time

Recall Bias

Recall Bias

When respondents do not accurately remember a past event or experience or leave out details when reporting about them

• Likely to happen when you ask about something that happened long ago

• People will skip, Guess, require excess time to think about it

Fix of excess time

• be specific

• Shorten time frame

• Offer ranges

• Offer the “don’t know” / “don’t remember option

Asking about sensitive topic problems

• Don’t always have to avoid, but be conscious

• Discomfort, shame

  • People may drop out or skip skip answer

  • Social desirability of nonresponse

    • Likely religion, politics, sexual encounters

Fix for asking sensitive questions

• emphasize anonymity

• Remove loaded questions

• Normalize behaviors to make people comfortable

Statistical Literacy

The ability to understand, interpret, and critically evaluate statistical information and data based claims in everyday life, research, and the media

• Ability to:

  • identify how data are collected, summarized, and presented

  • Recognize potential biases, limitations, and manipulations in statistical reports

  • Asses the methodological rigor of public reports, headlines, and media claims based on data

  • Be critical consumers of info rather than passive recipients

Bias

Systematic errors or distortions in the research process that lead to inaccurate, unfair, or misleading conclusions

• Threat to validity, reliability, and thus objectivity

• Recognizing bias is crucial to ethical, accurate, and worthy social research

Sampling bias

How is our sampling process bias?

• Like a nonrandom sample (picking people that will give you the answer that you want)

• People dropping out (attrition)

• People who die mid study (survivorship bias)

• Non-response

Response Bias

How are the responses to our questions biased?

• social desirability

• Question wording

  • Acquiescence bias (saying yes no matter what)

• Question order

Researcher bias

How is the researcher bias?

• Funding

• Want to support their these (changing data to make response significant)

• Politics and beliefs

• Implicit bias

• Measurement Bias

• Misleading conclusions

Implicit bias

Trying to find patterns that back their beliefs

Media Bias

• Switching up words to change what is said

• Selective reporting

• Oversimplification

• Lack of content

• Sensationalization or misleading headlines

Use the following statement as info for the next 3 cards

1. Black men with criminal records receive fewer job opportunities than white men with a criminal record 

2. Having a criminal record harms job opportunities for black men more than it does for white men 

   1. Race of criminal affects job prospects

Which statement has a causal relationship

The second statement

Causality

One variable causes the other

• X and Y have to be related (correlation)

• Time order

• Non-spuriousness (no third variable)

  • Time spent in prison

  • Age

  • Education Level

  • Social Supports

  • Class

  • Are you a good worker?

Statement 1 has no causality, it is just a link

Experiment

The researcher manipulates one or more independent variable(s) to determine the effect(s) on the dependent variable

IV= Cause

DV= effect

Does X cause Y

Can eliminate all other explanations, which are held constant

• isolating effect of X on Y

Experimental subjects

Your sample

• people participating in the experiment

• Do not need to be representative (can just throw out into the world)