Studied by 4 people
Join or Die :/ (In a Ben Franklin way not a Republican way)
Money ball essential question
Quantification question: how do you measure success?
Billy Bean’s “Moneyball” method
Push through the random losses and allow the statistics to work (increase the sample size → wins eventually)
Focus on “on base” percentage instead of traditional baseball stats
Consider success of the whole team
Moneyball: Status quo method
Focus on individual “rockstar” status on the field and in personal life (attractiveness, age, girlfriend, etc)
Billy Bean’s daughter
Represents bean’s unresolved relationship with her and with baseball
Why does Billy Bean not go to the Red Sox?
If you go to a team with a lot of money, it takes away the fight and helps a team that doesn’t need it
Essential question of the film Courage Under Fire
What are the limits and strengths of primary sources and direct observation?
What research methods does Nat Serling use? What methods should he be using?
It’s not an ethnography (not long enough); maybe a RAT (2-6 weeks)?
Rashomon effect
Effect of subjectivity of perception on recollection, by which observers of an event are able to produce substantially different but equally plausible accounts of it.
Quantitative research methods
Surveys, polls, content analysis, field experiments, lab experiments, focus groups (hybrid)
Qualitative research methods
Ethnography, R.A.T., Participant observation, historiography, legal studies, cultural studies, materials analysis, focus groups (hybrid)
Non-random sample methods
Convenience, judgment or purposive, quota, snowball
Random sample methods
SRS, Stratified random sample, cluster sample, systematic sample
Blank foreign elements
Elements in a sample that should not be there
Are case studies a type of research?
No -- but they are good for meta-analysis
Convenience sample
You take what you can get (often: colleges, military, prisons)
Judgement/purposive sample
Really specific population, dependent on research question (useful if you want to understand, but don’t necessarily want to generalize broadly)
Quota sample
Selection of a sample that is as similar as possible to the sampling population on the criterion variable
How to build a Quota sample
Divide up big population into categories
Sample according to quotas for each category you have set
How to build an SRS sample
Use a random number generator
SRS
Simple random sample; relies on the law of large #s and central tendencies theory
How to build a Stratified random sample
Divide population into meaningful subcategories
Elements in sample are chosen randomly
How to build a Cluster Sample
Divide population into clusters
Randomly select the clusters
What is the problem with cluster samples?
You will leave out clusters
Systematic sample
Survey every x person (skip interval) -- starting point is random
Can be used in content analysis (ex: “consider every x op-ed”)
How long do ethnographies take?
1-2 years
How long do RATs take?
2-6 weeks
How long do focus groups take?
½ day or 1 day, with several sessions and a break.
Goals of institutions (related to Moneyball?)
Do what they set out to do
Keep itself alive
Avoid debate over credibility
Focus Groups: pros
PROS - appeals to business world (fits the culture), can get detailed understanding
Focus Groups: cons
expensive, complicated, requires exceptional moderators, no generalizability, not very strong predictability (though often used)
Survey
Quantitative method that relies on self-reported information; but must be standardized to the best of the researcher’s ability
Poll
Must be standardized to the best of the researcher’s ability
Sample building: potential problems
Sample building: potential problems
Hard to find satisfactory sampling frame
Positionality - we might approach different people differently, even if the rest of the method is standardized
Blank foreign elements - items marked incorrectly or atypically
Language barriers could be a limit
GREATEST DANGER = researcher will over-shape the results because of what and how they think to ask questions
CONS of open-ended questions
Too open ended → long responses → difficult to standardize and compare
If too long → can turn off respondents, or collect poor data because people stop trying
Close-ended questions
Researcher can over-shape the results because of what and how they think to ask questions -- the respondent has little input
4 levels of quantification (in order, basic → sophisticated)
Nominal → ordinal → interval → ratio
Ratio
Adds the possibility of a true zero
Interval
Adds equidistant factor between ranked items
Ordinal
Adds ranking (AKA: distancing, sequencing), but not equidistant guarantee
Ex: the Richter scale (as in, used for earthquakes)
Nominal
Sorts phenomena into categories through the naming processes (IMPORTANT: categories must be exhaustive and mutually exclusive)
True zero
Absence of the variable or phenomena you are trying to understand
Which level of quantification does NOT include sequencing or true zero?
Nominal
Ordinal Scale
In play when the difference in difficulty of choice between items is not equal
How to write a survey question: DO
ensure questions are answerable; If asking contingency questions, make sure it is clear the two questions are related; repeat important questions with different language
How to write a survey question: AVOID
double barrelled questions; negative questions (i.e. “to what extent do you agree -- this should not apply to…”; slang or biased language
Double barrelled question
A question that asks people to provide 1 answer to something that really asks more than one question within the same utterance (in order to answer, you’d have to agree with both parts → invalid and/or combined results). AVOID THIS!
How to organize your survey (placement)
Prioritize: easy to get through (looks clean with blank space between questions, concrete, uncluttered, no abbreviations)
Keep it short to avoid respondent fatigue, avoid set responses
Set responses
Type of response on a survey when people don’t read the question, they just select blindly. Bad. Avoid by keeping surveys short and clear.
Should you ask the most sensitive question first?
No, never. Not in surveys, not in focus groups.
On a survey, where is the best place for the most sensitive and substantial questions?
In the middle
On a survey, when do you ask demographic questions?
Either first or last, if not in person. If in person, asking dull questions first is best.
What is an adequate survey response rate?
~50%, ~60% is good, 70% is very good (for random samples)
What is the ideal size for surveys?
Trick question, there is none. The more the better to allow for accurate generalizations to broader populations.
What is the ideal size for focus groups? What should their qualities be?
6-12 participants; AVOID: people who know each other,
Should you do covert participant observation?
No. Dr Kerric Harvey will be ashamed.
Co-variance
When one variable impacts another on dimensions of direction and magnitude (changes in one are systematically related to changes in another)
Common types of covariance
Directional
Magnitudinal
Directional
Positive relation, inverse relation
Magnitudinal
Perfect relation, zero relation
Positive relation
As value of 1 variable goes up, the value of the other also goes up (DIRECTION)
Inverse relation
As the value of 1 variable goes up, the value of the other goes down (DIRECTION)
Perfect relation
The value of 1 variable determines exactly the value of the other variable (MAGNITUDE)
Zero relation
No systematic covariation found between the variables (MAGNITUDE)
How long does it take to plan a focus group?
6-8 weeks (participant selection takes time = intentional audience)
How long are focus group sessions?
1-2 hours (90-120 minutes)
How many questions should you prepare for a focus group?
3-5 “real” questions per session (1-2 hours, 2-3 sessions during whole day)
2-3 warm up questions
3-4 standardized follow up questions (DO NOT DEVIATE FROM THE SCRIPT)
How many people should you invite to a focus group?
20% more than you will actually use, in case people drop out
Cross-talk
Referred to in a focus group, when participants interact with each other and not just with you. Good thing, should be promoted.
Who are the problematic people in a focus group?
Either won’t talk at all, or won’t shut up
4 phases of a focus group
Preparation, Day of, Findings, Presenting suggestions for action based on these findings
How big is a focus group team
At least 3 people: moderator, scribe, bird dog, also maybe a gopher
3 phases of DAY OF in a focus group
Welcome, ask questions and lead discussion, wrap up
Purpose Statement
Should always be written at the beginning of FOCUS GROUP. Articulate goals and objectives in terms as concrete and specific as possible.
Do you want to become part of the group in ethnography?
No, you want to be a fly on the wall
Do you want to become part of the group in participant observation?
Yes, but you do not want to be an influence.
How do you wrap up a focus group?
Gather all of the assistants, have them review and tidy up their notes, as a group: summarize events & general impressions for reliability
What is notable in the findings and analysis of a focus group?
Repeat comments OR exceptions to the pattern
What goes in a focus group interim report?
Mission restated, executive summary (logistics), who was in the focus group (by category, not by name), rundown of the days events, graphics/infographics/bullets, conclusions, recommendations for action
What is the general goal of ethnography?
NOT to prove anything, but to deeply understand the culture. But, you need to have an intellectual framework (theories) going in to make sense of it.
Does ethnography have generalizability?
No ¯\(ツ)/¯
Basic research
Science for the pure sake of research, aka “pure science”
Applied research
Sponsored research, for institutions, policy, businesses, etc. Much more common than basic research these days.
Ethnographic fieldwork - steps and percentages
Lay of the land (first 25-30%), In the Thick of It (middle 40-45%), Wrap up (last 30-35%)
Possibly list of “observation points” in Ethnography (name 5 or so?)
Kinship systems; power - how it’s defined and who has it; wealth; institutions of birth, death, marriage, sex, and coming of age; economic systems; child rearing; governance; belief structures (secular & theological); geneology; incest; identity-building; criminality; mental health
3 levels of taboo
Idiosyncratic, culture, universal
Idiosyncratic
supernatural element and/or an individual’s personal taboo
culture
traits that are considered unacceptable
universal
incest and in-group murder, as contextually defined
How do you take notes during ethnography?
On paper only, with a field notebook or notecards, extensive labels. Have BOTH a personal diary for yourself + a field journal, which will become public later. Write reflections every day.
Should you build relationships during ethnography?
No, you should keep a critical distance. In general, break no laws, disrupt no ethics, and steer clear of religion, politics, relationships, and sex.
What is produced at the end of an ethnography?
Several reports and compiled notes, maybe a book, maybe a documentary
How to properly video while still maintaining anonymity of the subject?
Distortion lens, saran wrap, vaseline, mud (LOL)
What to know before doing field research?
History, politics, lingo, cultural norms, etc
When do you leave the field, in research?
When you run out of money, time, are threatened, or hit diminishing returns.
Which is generalizable? Qualitative or quantitative?
Quantitative
Dangers of overt participant observation
People who offer to help are self-selected surveys; you should be wary.
Steps of ethnography preparation
Where are you going and why?
Free association brain dump → leave this with someone else (will be compared at the end of positionality). Decide if you want to do a solo or team research.
Big read → research, literature, etc. Develop observation points and find potential specific questions you are interested in.
What is a RAT?
Value/valence enhanced participant observation. 4-6 weeks.
Likert Scale
Scale on surveys where you indicate “degree to which you agree” 1-5 or 0-5, etc.
Likert Scale Issues
Essential question: do you include a true zero (ratio level) or do you not (interval level)?
Classic ex of this debate: is it possible to have zero self esteem? Is there a difference between 0 and 1? If there is not, then you can omit the true zero.
Note
Flashcard