1/74
Looks like no tags are added yet.
Name | Mastery | Learn | Test | Matching | Spaced |
---|
No study sessions yet.
observational research
when a researcher watches or monitors participants (people or animals) and systematically records their behavior
naturalistic
observation “in the wild”
controlled
observation in controlled setting such as lab
observer bias
refers to when observers’ expectations influence their interpretation of the participants’ behaviors or outcomes in the study
minimize observer bias by
having very specific coding criteria that observers are trained to use, multiple raters, masked design
multiple raters
ensures a check on rating reliability
inter-rater reliability
computed from pairs of ratings, typically based on a subset (15-20%) of the data
typical guidelines for assessing reliability:
0-0.2 = poor agreement
0.3-0.4 = fair agreement
0.5-0.6 = moderate agreement
0.7-0.8 = strong agreement
>0.8-1 = almost perfect agreement
>.7 or above is preferable for any publishable study
setting differences between raters
quantitative approach- average ratings
raters discuss where disagreements were made and decide what score to use
an independent third raters that would moderate between raters and decide what score to use
whatever is done, this should be reported in the methods
blinded design
the observer is unaware of predictions and/or conditions in which the participants are in
observer effects
refer to when observers’ expectations influence how they behave towards participants
reactivity
refers to the tendency of participants to act differently if they know they are being observed
not the same as observer effects- the observer isn’t cuing a specific behavior
the mere presence of observation alters behavior of the participant
naturalistic observation is intended to decrease reactivity
minimize reactivity by
blending in by unobtrusively observing, wait until participants are used to being observed, measure behavior outcomes (measures that behavior occurred rather than directly observing behavior)
pros of survey research
best if the perspective of the self is of interest, it’s possible to accurately self-report a behavior, using open-ended responses to get to know the phenomenon
pros of observation research
allows measurement of behavior that otherwise would be hard to access from self-perspective
cons of observation research
difficult and likely much more time consuming
Dr. Ewell, a developmental psychologist, is planning on conducting a study that involves watching children play together to determine how sharing behavior occurs in same-sex friend pairs compared to opposite-sex friend pairs. Dr. Ewell is concerned that the children will behave differently because of the presence of research assistants. He is concerned about:
reactivity
imagine that Dr. Ewell calculates a correlation (e.g., ICC) for his two raters. Which of the following would be the best value for Dr. Ewell to find?
0.89
surveys (or polls)
are a method of gathering information from participants via self report
online surveys
always available
in response to a choice (e.g., purchase, attend event)
study participation
federal agency data for population
in-person questionnaire
study participation
in response to a choice or event (e.g., purchase, attendance at event)
assessment for services
interview survey
study participation
in response to a choice or event (e.g., stressful life event, natural disaster)
assessment for services
construct validity of a survey depends on a good match of
the type of information needed, with the feasibility of self-report in your target population
quality of the questions
self-reports are ideal for
what people think they are doing
what people think they remember
what people think is influencing their behavior
experiences only accessible to the person (e.g., the content of a dream or memory)
attitudes and judgments
exploring a question because the method is low cost
accessing a large and representative sample that spans multiple geographic locations
self-reports are not good for
precise analysis of behavior
precise details or confidence for memories of events
what may actually be influencing their behavior
when self-report not possible for a population (e.g., infants)
open-ended questions
allow the participant to fill in the response in any way they like
they provide rich data, but they can be hard to code
open-ended questions should be used when
sensitive or socially disapproved behaviors
research questions on the explicit content of a self-generated response or information implicit in the response
preliminary research
leading questions
elicit bias by using non-neutral words in framing question
double-barreled questions
ask two+ questions in one
negatively worded questions
often difficult to interpret, especially with likert scales
participant’s perspective
consider to improve validity of self-report data
shortcuts and biases, social desirability
response sets
when participants give consistent responses across questions to save time, rather than accurately answering each question
social desirability
refers to when participants may respond in a way they think is socially desirable
biased vs. representative samples
biased samples: unrepresentative, some members of the target population have a higher probability of being included in the sample compared to other members
representative samples: based on probability sampling, every member of the population of interest has an equal chance of being selected for the sample
convenience sampling
researcher requests volunteers from a group who meet general criteria but are recruited in a variety of “non-random” ways
purposive
researchers target or hand-pick participants that meet specific criteria based on knowledge of the research question
snowball
variant of purposive, but now participants are asked to recommend others they know that would also meet criteria
self-selected samples
sampling only those who invite themselves
simple random sampling
random draw from sample frame
systematic sampling
pick first participant at random, then call contact every nth person
cluster sampling
sample from pre-existing sub-groups of your target population
sub-groups sampled from are chosen randomly from all possible sub-groups
stratified random sampling
researcher selects particular demographic categories on purpose and then randomly selects participants within the categories
allows you to force a match of your sample demographic to the population by randomly drawing within population “strata”
oversampling
researcher intentionally over-represents one or more groups in order to get an accurate estimate from them
correlations tell us that
two variables are linearly related and how strong this relationship is
effect size allows us to compare
correlations across measures of different scales
bivariate
doesn’t account for third variables
the least squares regression equation
errors in prediction based on our regression line are minimized using a
least squares method
regression equation
describes the numeric relationship between the variables
predicts a dependent variable (DV) with one independent variable (IV)
multivariable regression
each predictor variable (X) gets a beta weight (b) that reflects in relationship with Y
root mean squared error
average amount of predictive error for regression in the sample, expressed in units of the Y variable
indication of good model fit: lower, because this is amount of error in predicted Y
RMSE answers
after we use X to predict Y, how much variability is left in Y?
R2
the proportion of the total variability in one variable that is predictable from its relationship with the other variable
indication of good model fit: higher, because higher values reflect more variability of Y accounted for by predictor variables
R2 answers
how much variability in Y did the regression explain with X?
when R2 = 0
variability of Y around the mean of Y (total variability in Y)
variability of Y around the regression of predicting Y from X
when R2 = 1
variability of Y around the mean of Y (total variability in Y)
variability of Y around the regression of predicting Y from X
when you have one predictor variable
squaring the correlation coefficient gives you R2
sum of squares of Y given X
sum of squares for predictive errors
indication of food model fit: lower, because this is unexplained variability in Y after accounting for X
interpretation template
For every 1 unit increase in X, there there is a change in Y by beta, even even after statistically controlling for (i.e., holding constant) ___. Overall, the model accounts for R2 % of variability in the exam performance.
interpreting the strength of evidence for
causality from correlational studies
necessary to interpret the beta
estimated difference in outcome for 1 level difference of predictor variable
external validity
refers to the degree to which a claim generalizes to a larger population or to other situations
third variables
“nuisance” variables, outside variables
moderators
grouping, answers “for whom”
modifies the strength of the association between the independent (X) variable and the dependent (Y) variable
informs external validity
to which people and situations does the relationship apply to
mediators
mechanisms, answers “why”
help explain the association between two variables that have an existing correlation, because the mediator variable arises from or is internal to the independent (X) variable
informs internal validity
by answers how variable X might cause a change in variable Y
decisions that help for maximize internal validity
often hurt external validity because they limit the population and situations studied
decisions that help you maximize external validity
often require large sample sizes and heterogeneity in types of people and situations examined
it is more difficult to conduct highly controlled experiments with large samples and in different situations
steps in testing mediation
is beta coefficient significant for path c?
is beta coefficient significant for path a?
is beta coefficient significant for path b?
regression
direct replication
same concepts
same operational definitions
conceptual replication
same concepts
different operational definitions
replication + extension
same concepts
plus some new concepts
goals of theory-testing mode
internal validity
goals of generalization mode
external validity