Critical Thinking Midterm 2

Chapter 6

Surveys and Observations

What is a survey/poll?

Method of posing questions to people online, in personal interviews or in written questionnaires.

What claim are surveys used for?

frequency claim

What are the 4 question formats?

Open-ended

forced-choice

likert scale

Semantic differential format

What is a open ended question?

answer how they like, but difficult to categorize and code.

What is a forced-choice question?

Pick 2 or more options

Likert Scale

5 or 7 point scale. From Strongly agree to strongly disagree. Likert-type scales are similar but different format.

Semantic Differential format

rate target using numeric scale anchored in adjective. Yelp or rate my professor. Star system

What validity to look for when interrogating Survey results?

construct validity

* how well was variable measured

What not to do when creating a survey?

Have double barreled questions

Negatively worded questions

Have an order in which question order can effect response

What is a double barreled question?

Ask two questions in one survey. Might agree to one and not the other.

Negatively worded question?

A question containing negative phrasing that can cause confusion

Self reports can be?

the best or only way to get answers

What to watch out for when making self reports?

Response sets. : shortcuts people take

What are he 3 kinds of response sets?

Acquiescence

fence-sitting

Trying to Look Good

What is Acquiescence? Solution?

yea-saying. when people say yes to everything instead of thinking carefully.

solution: same question but asked in opposite manner

What is fence-sitting? solution?

answering middle of scale.

solution: Take away neutral option. choose forced choice questions

What is trying to look good? solution?

answers to make them look better than they really are. Decreases constuct validity.

solution:

tell them survey anonymous

add special questions to catch them.

implicit association tests

ask their friends to rate them

What is hard for people to self report?

why they chose something.

memories of events

quality of stuff 4 online ratings

What is observational research?

researcher watches ppl or animals and systematically records how they behave or what they are doing.

What claims does observational research support?

frequency claims

association claims (operation. var)

causal claims (operation. var)

What 3 things can construct valid. of observations be threatened by?

observer bias

observer effects

reactivity

What is observer bias?

When observers’ expectations influence their interpretation of particular behavior or outcome of study.

What is observer effects?

Participants behavior changes to match observer expectations. ex. the horse!

How to prevent these things?

training observers

codebooks

multiple observers

masked study design

What is a masked study design (blind design)?

observers are unaware of purpose of study and conditions to which participants have been assigned.

What is reactivity?

Change in behavior when study participants know another person is watching.

Solutions to reactivity?

Blend in: make unobtrusive observ. make ur self less noticeable

wait it out: let them get used to you

measure behavioral results: onubtrusive data measure. trace behavior left behind

Chapter 7

Sampling

Why do we need generalizability? (validity and claims)

used when interrogating external validity

important for frequency claims

what are the 2 external valid concerns?

samples and settings

what is a population?

entire set of people or products in which you are interested in

sample?

smaller set, taken from that population

census?

if you test every set of people or product

When is there good external valid?

if sample can generalize population aka is representative of pop of interest

What is a population of interest?

specific people they want to generalize

ex. lab mice, women, undergrads

For generalizability samples are either what one of two things?

biased or representative

What is a biased sample (unrepresentative)?

some members of pop of int have increased probability than others of being included in sample

what is an unbiased sample (representative sample)?

all members of pop of interest have equal chance of being included in sample

When is a sample biased?

a researcher’s sample may contain most unusual people

What are 2 ways a sample may be biased?

1. researchers may study only those they can contact conveniently
2. study only those who volunteer

what is convenience sampling?

using a sample of people who are easy to contact and readily available to participate

what is self selection?

when a sample is known to contain only people who volunteer to participate

what is an externally valid approach to internet surveys?

when members are invited via random selection

How to obtain a representative sample?

probability (random) sampling is best option

What is probability (random) sampling?

every member of pop of interest has an equal and known chance of being selected

which validity does it meet?

external validity

what do probability samples have?

external val and generalizability

what is a non-probability sampling?

non-random sampling and can result in biased sample

What are the 2 main types of probability samplings?

simple random sample

systematic sampling

simple random sampling?

most basic.

example: assign a # to each individual in pop and then select certain ones using a table of random #s. Many way to do this.

systematic sampling?

select 2 random #’s like 4 and 7 and choose 4th and 7th person until sample # is desired

What are 3 variations of basic technique?

cluster sampling

multistage sampling

stratified random sampling

cluster sampling?

clusters of participants w in pop of int randomly selected, then all indiv. used.

ex. 900 HS choose 100 HS use all students from 100 HS

multistage sampling

2 random samples selected: a random sampling of clusters and random sampling of ppl within those clusters

900 HS choose 100 HS take random sample from each 100 HS and use all

Stratified random sampling?

researchers purposefully selects particular demographic categories (strata) and then randomly choose ppl from them proportionate to their membership in pop of int.

What is the difference between strata and clusters?

strata are meaningful (race,age) while clusters are arbitrary (any random set will do).

Final sample size of strata reflect proportion of pop.

clusters NOT selected w this in mind

what is oversampling?

variation of stratified random sampling

researchers intentionally overrepresent 1 or more groups.

Random sampling vs random assignment?

random sampling increases external validity while random assign. is used in experimental designs

What is random assignment?

when researchers want to place participants in two diff groups they assign them at random.

ex. treatment vs control group

Which validity does random assign meet?

internal validity

Why settle for unrepresentative sample?

random samples are difficult to get

if external validity is not vital researchers may choose nonprobability sampling technique

What are 2 non-probability sampling techniques?

convenience sampling

purposive sampling

what is purposive sampling?

when researchers want to study only certain kinds of people and recruit only them.

think: purposefully not random

What are two variations of purposive sampling?

snowball sampling and quota sampling

snowball sampling?

can help find rare individuals. participants are asked to recommend acquaintances.

quota sampling?

similar to stratified random. Researchers identify subsets of pop of int. and sets target # but quota is hit nonrandomly

What matters most in external validity?

crucial for frequency claims but with association or causal might not matter.

samples are either what (EV)?

externally valid or unknown EV

Chapter 8

Bivariate Correlational Research

What is an association claim?

describes relationship found between two measured variables.

What is a bivariate correlation?

an association that involves exactly two variables

How do we analyze bivariate correlation?

look at 2 variables at a time only even if there are more.

describe relationship by using scatter plot and correlational coeff. r

What are corr coef r’s 2 qualities?

direction and strength

How to describe assoc with categorical data

bar graph can be used especially when comparing categories.

When is study correlational?

when method of study involved measuring both variables.

When is it not?

if one variable is manipulated thru experiment it becomes a causal claim.

what are 2 important validities when interrogating association claims?

construct validity and statistical validity

What are the 6 questions to ask?

1. how strong is relationship
2. how precise is estimate
3. how has it been replicated
4. could outliers be affecting assoc.
5. is there restriction of range
6. is the association linear

Answering Q1

how strong is relationship

What is effect size?

strength of relationship between 2 or more variables

when all else is equal…

a larger effect size is considered more important than a small one. r=.26 Vs r=.06

when are small effect sized important?

in athletes bc small differences can lead to a batter in baseball to make more runs.

when small effect size is common for that person or thing

Q2

how precise is estimate?

What kind of CI do small samples have?

wider (less precise) confidence intervals

when is association statistically significant?

When 95% confidence interval does not include zero.

meaning it is unlikely to have come from pop in which assoc is zero

when is assoc NOT statistically significant?

CI’s that do contain zero

cant rule out that the true assoc is zero

Q3

has it been replicated

why replicate?

estimating population association

Q4

Could outliers be affecting assoc?

what is an outlier?

an extreme score; a single or few cases that stands out from pack.

whats wrong with outliers?

can make medium sized correlation appear stronger or strong one seem weaker

in bivariate correlation when are outliers a problem?

when they involve extreme score on both variables

Q5

is there restriction of range?

what is restriction of range?

in correlational studies, if there is not a full range of scores on one of the variables in the assoc. it can make correlation appear smaller than it is.

ex. not including all GPA’s and only the top

What to do if this happens?

statistical technique: correction for restriction of age

When do we ask about ROA?

when correlation appears weaker than expected

Q6

is the association linear

what is a curvilinear association?

relationship between 2 variables is NOT a straight line

ex. bell curve

when looking at r for curvilinear association?

keep in mind that r is for straight lines