Studied by 10 people
what type of research can be conducted using observations?
finding phenomena
what type of research can be conducted using correlations and quasi-experiments?
finding relationships
what type of research can be conducted using experiments?
finding causal relationships
what is meant by the precision of a theory?
accuracy of classification of which new data can be explained by the theory
e.g. “A will score higher than B in X, but lower than C, and B will score higher than A in Y".”
principle of parsimony
choosing the most straightforward theory from among theories fitting the data equally well
why are falsifiability and testability important?
to distinguish science from pseudoscience and compare studies based on their degree of falsifiability
internal validity
extent to which the observed results represent the truth in the population we are studying
→ observed mean difference
external validity
extent to which the results of the study are generalizable to other situations, populations, …
construct validity
the extent to which your test or measure accurately assesses what it's supposed to
statistical validity
the extent to which drawn conclusions of a research study can be considered accurate and reliable from a statistical test
how can correlations be interpreted?
direction and size
how can regression be interpreted
prediction
advantages of within-subject designs
requires fewer participants
increases chance of discovering a true difference among conditions
more statistical power (individual variation is reduced)
advantages of between-subject designs
minimizes learning effects across conditions
shorter sessions
easier to set up and analyze
what is the difference between a quasi-experiment and a true experiment?
no randomization in a quasi-experiment
alpha error
type 1 error → false positive
beta error
type 2 error → false negative
effect size
how large the difference/correlation/relationship is
true effect
effect size in the population, cannot be observed only estimated
(should be made before the study)
observed effect
calculated after the study, also an estimation of the true effect
statistical power
probability that the effect is statistically significant and correctly rejects the null hypothesis
what does low/high statistical power mean?
low power → small chance of detecting a true effect, results likely to be distorted
high power → large chance of detecting a true effect
cohen’s d formula
d = (m1-m2) / SD
what are the conventional values for small/medium/large effect sizes?
0,8 → large
<0,2 → small
in between → medium
which factors affect the statistical power of a study
effect size
alpha
sample size
how can the power of a study be increased?
increasing sample size
increasing the measured effect size
increasing the alpha error (because more results are accepted as significant)
disadvantages of small sample studies?
more fluctuation and inflated effect size
→ publication bias
which results should you not trust?
results of studies with small samples cannot be trusted
cross-sectional design
all measures are collected in a single assessment
longitudinal design
measures collected in repeated assessments
advantages of experimental research
manipulation isolates the effect of interest, so alternative explanations are minimized
disadvantages of experimental research
difficult to conduct (time & money)
biases (volunteer & selection bias)
can be ethically problematic
limited generalizability
advantages of observational research
more generalizability
easier to obtain larger samples
disadvantages of observational research
longitudinal studies are expensive (time & money)
greater risk of biases and confounds
data more likely to be incomplete and of poorer quality
cohort studies
assess prospective changes → looking forward in time
case-control studies
assess retrospective predictors → looking backwards in time
selection biases
biases introduced in the selection process, so that proper randomization is not achieved
e.g. sampling bias, allocation bias, non-response bias, publication bias, volunteer bias
information biases
biases introduced by systematic differences in the collection and handling of information in a study
e.g. misclassification bias, observer bias, interviewer bias, social desirability bias, recall bias, performance bias, detection bias
moderator
a variable that alters the strength of the linear relationship between a predictor (X) and an outcome (Y)
which statistical analyses are commonly used to test moderation
if X and Z are categorical and Y is continuous → ANOVA
if X, Z, and Y are continuous OR if X and/or Z are categorical and Y is continuous → Multiple linear regression
if X and Z are continuous and Y is categorical OR if X is continuous and Z and Y are categorical → Logistic regression
How to interpret a moderation effect from a linear regression?
Center X and Z scores (→ individual score-M)
Calculate interaction with centered scores (X*Z)
Perform analysis with centered scores and new interaction effect as predictors
If significant → plot simple slopes
When should a moderator variable be measured?
at the same time as predictors
mediator
The IV influences the mediator variable, which in turn influences the DV
total effect (c)
effect X has on Y including the effect of the mediator
direct effect (c’)
effect X has on Y without taking the mediator into account
indirect effect
total effect - direct effect
absolute mediation
indirect effect explains the complete effect
How is mediation commonly tested?
using regression…
Show that X predicts Y
Show that X predicts the mediator
Show that the mediator predicts Y
Show the mediator produces an effect of X on Y
bootstrapping
testing of mediation effects using a resampling technique to adjust the standard errors of the coefficitents
3 things to consider before conducting a mediation analysis
have a directionality assumption (avoid reversal causal effect)
consider when to measure the mediator
choose reliable measurements
When should the mediator be assessed?
after the predictor and before the outcome
What is the difference between mediation and moderation?
Mediation describes indirect effects, moderation describes conditional effects
signal detection theory (SDT)
measures the ability to differentiate between stimuli (information-bearing patterns) and noise (random patterns)
→ measures how humans make decisions under circumstances of uncertainty
e.g. witness tries to identify a criminal, trying to remember whether you know someone, looking for spelling mistakes, discovering a spider on your wall
According to SDT, what are the 2 factors affecting human discrimination decisions?
sensitivity
decision/response criterion
sensitivity (in SDT)
strength of the signal
ability of the observer
decision/response criterion (in SDT)
consequences of decisions (pay-off matrix)
frequency of signal
According to SDT what are the possible responses (matrix)
theoretical assumptions that explain differences in sensitivity
in reality, dichotomous events (there is a signal or not) but subjective experience varies and is usually distributed around the mean
what does it mean when someone responds liberally/neutrally/conservatively?
liberal → lot of false alarms but few/no misses
neutral → responds with yes and no equally
conservative → many misses but no few/no false alarms
d’ (d prime in SDT)
standardized difference between the means of the Signal Present and Signal Absent distributions (strength of the signal relative to noise)
→ d’ = z(Hits) - z(False Alarms)
β (in SDT)
observers ability to correctly identify a stimulus (willingness to give Yes responses)
→ β = y(Hits) / y(False Alarms)
(β = 1 means neutral responses)
what does a low/high d’ mean?
low = 0 (personal cannot discriminate at all, guessing)
high = 4.66 (almost perfect at discriminating, 99% accuracy)
what problems are caused by missing data?
Response rate bias
low statistical power
invalid conclusions
missingness mechanisms
reasons why data is missing
ignorable missingness mechanisms
MCAR (missing completely at random) → independent of observed or missing values
MAR (missing at random) → partly depends on observed values but not missing ones
non-ignorable missingness mechanisms
MNAR (missing not at random) → depends on missing values themselves
proactive strategies for minimizing missingness
advanced warnings
personalized surveys
follow-up reminders
monetary incentives
listwise deletion
deleting all cases with any missing values
(violates a fundamental principle of missing data analysis)
pairwise deletion
still including cases with missing values into the analysis
(attempts to minimize the loss that occurs in listwise deletion)
imputation
replacing missing data with substituted values
(SPSS: → Analyze → Multiple Imputation → Analyze Patterns)
mean item imputation
mean of the observed values for each variable is computed and the missing values for that variable are imputed by this mean
(can lead to severely biased estimates)
advantage of using multiple imputation over single imputation
does not provide a deterministic idea of what the missing value should be, but allows it to have a range of different scores
what is the best way of handling missing values?
Item(s) missing → mean item imputation
Scale(s) missing → multiple imputation (if not MCAR)
disadvantage of comparing an experimental treatment with a waiting-list/treatment-as-usual group
easy to show that experimental condition is effective but overall less power
formula: percentage of improvement
(pre-post)/pre*100
Lasagna’s Law
overestimation (with a factor ten to one) of the number of patients available for inclusion into your study in a certain period
when do you use Bonferroni correction?
→ if you test multiple times (to reduce inflated error probability)
ways to reduce unnecessary within-group variance in treatment outcome measurements
specific hypotheses
specific instruments
inclusion- and exclusion criteria
treatment manual, trained therapists, trained assessors
inspection for outliers
treatment dropout/´refusal in psychoptherapy
~25%
What are ERPs and what brain activity do they reflect?
= voltage fluctuations in the ongoing EEG that are time-locked to an event (e.g. stimulus onset or response execution)
→ reflect the sensory, cognitive, affective, and motor processes elicited by the event; usually labeled by their polarity (N/P)
Which properties make ERPs useful?
covert monitoring of processing when overt behavior is difficult to measure
can measure processes not evident in behavior
→ e.g. in infants, animals, coma patients, …
major challenge of the ERP technique and how to deal with it
many different processes happen in the brain at the same time
→ using a zero measurement (starting point assessment)
cognitive artifact
something physical or digital that has aided a mental process
→ leftover remnants indicative of the efforts it takes to unravel mental processes
challenges of recording/studying ERPs in clinical populations and their solutions
The effect of medication on cognitive processing → compare to unmedicated patients
opposing effects of comorbid disorders → investigate individuals of the same disorder with and without comorbidity
individuals with disorder show more artifacts → adjust recording procedures (less conditions, less electrodes, rest breaks, etc.)
how are ERP components usually quantified?
latency = amplitude and time between stimulus and peak
conclusions that can be drawn from ERP studies
presence, size, or timing of a specific mental operation, and the effect of manipulations or individual differences on these factors
Big Hypothesis (Rossignol et al., 2012)
people with high levels of social anxiety have a greater P1 (encoding of faces) and P2 (attentional resources) effect than people with low levels of social anxiety.
why did they use ERPs? (Rossignol et al., 2012)
attentional bias can already be detected 100ms after the first stimulus is presented
Conclusions (Rossignol et al., 2012)
perceptual processing of social cues is extra strong in people with social anxiety (P1 component), but linking attention is generic for all anxious states
what differences do you look at when comparing ERPs across conditions/individuals?
amplitude differences
latency differences
How can ERPs be used to study disturbances in clinical populations?
between-group comparisons (diagnosed/no diagnosis)
correlational approach (relate ERPs to symptoms/traits)
longitudinal studies (assess risk for psychopathology)
examples for direct vs. indirect measures
direct measures: questionnaires (directly infer attitudes)
indirect measures: reaction time (infer attitudes from results)
advantages of indirect measures
measure implicit attitudes, which predict automatic behavior
projective tests and their disadvantages
tests that commonly use ambiguous stimuli
e.g. Rorschach, TAT
→ low reliability and validity
advantages of modern indirect measures
more objective than projective tests
reliability and validity can be determined
how does the Implicit Association Test (IAT) measure associations?
measures mean reaction time for compatible and incompatible blocks of stimuli → short RTs indicate stronger associations
which 4 types of stimuli are needed for IAT?
2 target stimuli and 2 attribute categories
formula IAT effect
mean RT(incomp. block)-mean RT(comp. block)
problems with the interpretation of IAT effects
two types of targets → effect can be caused by either target being more associated with one stimulus or both (usually both)
problems with most indirect measures
lack of convergence
reliability
validity
general vs individual stimuli
Note 
Flashcard (22)