302 notes
Stat rev
/
Descriptive Stat - describe
Inferencial: explain and theorize
Measures of central tendency
Mean
Median
Mode
Measure of variability/ dispersion
Variance: how much points vary from mean
Standard deviation: variance squared
Percentiles:
Range
Minimum
Maximum
Frequency Distribution
Normal
Skew: where does the tail point? (low end = neg skew, higher end= pos skew)
Kurtosis: narrow graph , flat (short) graph
Correlation
Strength
Direction
Charts
Bar charts convey frequency
Pie chart conveys percent
Research Methods for Stat Assumption
Observations are independent from each other
No individuals duplicated
Data are normally distributed
Large samples and random sampling
Relationships are linear
Special stats to test for non linear
Samples are representative of pop
Random sampling
Inferential Stats
Parametric Stat Procedure (normal dist.
Correlations
T-test
ANOVA Regression
Nonparametric STat Procedure (non-normal dist)
Chi-square
Mapping research methods to stat tests
Tests for differences btw means of groups with an experimental or quasi-exp design
T-test
ANOVA
Determine the association or relationship btw variables w a non exp design
Correlation
Regression
Correlations
The degree of linear relationships (association) btw 2 variables
r
-1 to 1
Correlation does not = cause
Stat vs. pract signif
r and d are effect sizes
Science
EMpirical - based on exp & observations
Objective - Everyone perceives the same way
Tests for superiority of theories
Other characteristics of science and the scientific approach
Control:
Tentative
Self-correcting/replication
Progressive, always accumulating more
Theory- driven
Parsimonious, simplest explanation
Working assumptions (philosophy) of science
Realism
Belief the object exist outside of our mind and sight
Rationality
We need reasoning or logic, not just intuition of faith
regularity
Phenomena exists in similar patterns, consistency
determinism/Causality
All event happen because of some preceding event or cause
Discoverability
We don’t always know, but with enough scientific data we could find out eventually
Ky terms
Law:
Theory: abstract (set of) statement describes general principles about how variables related to each other
Characteristics of a good theory
Falsifiable
Parsimonious: simple explanation
Supported by data
Research Question: testable statement
Hypothesis: prediction researcher expects from the results of their research; if-then statements.
Research terms and distinctions
Producer v consumer of research
Basic v applied v translational research
Basic: like lab setting
Applied: applying basic science to real life
Translational: building interventions and trying to improve real world problems
Peer review:
Journal-journalism cycle: journalists take journals and publish them in layman’s terms for everyone else to enjoy
Lit review sources
Empirical studies
Conceptual article (theories propositions)
Reviews, meta analysis
Research sequence
Observations
Develop hypothesis
Choose a research strat/design the study: experiment or non-experiment?
Choose population & sample
Conduct study
Analyze the data
Report results
Open science
Preregistering a study
Power analysis to determine sample size
Publishing data and syntax (analysis commands): publish data so others can have access to it
Chap 2: sources of info/ ways of knowing
Methods of acquiring knowledge
Empirical but not objective
Common sense: shared practical knowledge
Intuition: gut feelings
Tenacity: acquire knowledge from mere exposure
Mysticism: insight provided by private experience
Superstitions
Prob w intuition:
easily swayed by good story
availability heuristic (pop-up principle)
present-present bias
confirmantion bias (cherry-picking, “HARKing”)
Methods of aquring knowledge
Nonempirical (and nonscientific)
Authority
Logic: drawing conclusion thru assumptions
Science
Meaningful
Control confounds (rule out alt exp)
Rule out chance
Chap 7: sampling
Before sampling decisions
WHat is DV?
Goals of science: describe, explain, predict, intervene
Textbook claims: freq, association, causal
Study design: exp, quasi-exp, nonexp
Research question/hypothesis
Likely biased/ me-search: personally biased
Samp/pop
POpulation: the entire set of pp to whom the research intend to generalize to a bigger pop
Sample: subset of po targeted for inclusion in the study (always smaller than pop)
Census: sample = pop
External validity
The extent to which we can confidently generalize result from our samp to pop, other pop, etc
How rep is the samp?
Who can we gen the reults to?
Sampling
Probability samp: random samp from pop
Pop frame: list of all the people in the study pop targerted for inclusion in the study
Simple random: everyone has a chance of being selected
Strat random: oversampling may occur if minority pop is present
Cluster: random sample of an entire group, multistage
Non Probability samp: nonrandom samp which results in biased sample; don’t start w full list of population
Convenience
Quota
Uncontrolled
Purposive
Snowball
haphazard/intercept
Biased samp
Def: some pp are more likely to be included/overrepresented in our samp
Approachable
Available
eager/willing to respond/participate (self selection)
Product/Service “raters”
Stronger opinions
More willing to share ideas w others
More engaged w internet
Motivated by incentives offered
Most common research participant
WEIRD
Western
Educated
Industrialized (countries)
Rich
Democratic
Who are non-participants?
Contemplate how rep our samps are
Who is left out?
Ways to find out?
Chapter 5: variables and measurement
Measurement = Creativity
Conceptual vs. Operational Definitions
Contsrauct: label of theorateical dimension on which ppl are thought to differ
Conceptual definition: abstract or gen meaning of construct
OPerational definition: how construct is measured or observed
More objective and specific the better
Measuring anxiety
Conceptual def: uneasy and distress ab future uncertainties
Operational def:
Self report (5 item, likert scale)
Parent report (open-ended)
Behavioral observation (5 min condition anxious behavior)
Physiological measure (pulse for 30 secs)
Variable
Symbol that can assume a range of numerical values
Some property of an organism/event that has been measured
An attribute that varies having at least 2 levels or values
An aspect of the environment (e.g. testing environment) that can take on different characteristics with different conditions
Types of variables
IV and DV
participant/subject
Continuous vs discrete /categorical
Quantitative and quantitative
Confounded (design confound): changing multiple things at once
Measurement
The assignment of #s to events/objects according to rules that permit properties of the events /objects to be represented by the number system
Conceptual def → operational def
Scales: measuring device to assess a person’s score or status on a variable
Label: using numbers to keep track of individuals
Nominal: using #s to group objects or people, #’s don’t have any sort of other meaning
Ordinal: #s to order ppl or objects from most to least, ranked
Interval: #s represent if a variable or attribute is present
Ratio: has absolute zero, (var1/var2)
Implicit measure
Def: indirect assessments
Example
Implicit association test
Word fragment completion test
Feet, prose, built, fertile, berry, exists
Computer eye/face tracking/reading
Physiological measures
Func Mag resonance Imaging (fMRI): measures brain activity
Blood flow
Blood oxygenated level
Neuronal activity..
Saliva (cortisol-stress)
Heart rate (anxiety)
Skin temp (stress-emotion)
Wearable sensor to measure auditory analysis (network analyses)
Eval of measurement methods/instruments
Reliability and validity
Interpreted to what they assess
Valid < reliable
Neg rel uninterpretable; 0-1
Magnitude of valid more important than direction; stronger better
Relationship:
You can have reliability without validity
You can have validity without reliability
Sq root of a test’s reliability is the upper limit of its validity
Ex. reliability = .81, validity caps at .90
Types of reliability
Test - retest (time; coefficient of stability): retaking same test w same ppl
Alternate forms (form, coefficient of equivalence): two separate forms of same test, mult versions
Internal consistency (item): divide items, calc score and compare
a) split half, odd-even
b) coefficient alpha
Kuder-richardson (KR-20)
Inter-rater reliability (rater): extent to which two or more raters are consistent
Projective tests
Used mostly w kids
Test validity
Construct related
convergent : demonstrates dif measures of same construct are pos correlated to one another
Discriminant: dif measures of diff contrasts are not pos correlated, not or neg correlation
Content- related: the adequacy with which the specified content is sampled
Criterion related: dependent variables
Predictive: can we predict the dependent variable
Concurrent: using current study to
Postdictive: we have criterion (dependent first) then look at IV
Face: whether the test measures what its supposed to, from test taker POV
Exam 2
Chapter 10: fundamentals of experimental control
3 primary study designs
True experiments - gold standard
Manipulation of IV
Random assign
Research has control over: selection of participants, assignment to conditions, presentation of conditions
Quasi experiments
Manipulation of IV
No random assignment
Preexisting groups
Non experimental/ correlational
No manipulation of IV
No random assignment
Important study design characteristics
Experiment vs. non-experimental
Manipulation vs. no manipulation (most fundamental design characteristic)
Identify variables as between subjects or within subjects
Operationalization of the IVs and DVs
Control variables (known predictors)
Sampling and location of study
Theory for standard of comparison
Method of difference: if two groups differ on only one variable, then that variable may be considered to be the cause of the difference between groups
Hence, we design studies with a comparison group, often a control condition
Randomized control trial – gold standard
Researcher and participant blind to conditions and hypotheses
Random assignment to conditions
Comparison (control) group
Random sampling
Random sampling
The process of choosing a representative group from an entire population of interest such that every member of the population has an equal and independent chance of being selected into the sample/study
Enhances external validity (degree to which we can generalize)
Sample size related to statistical power which influences or ability to detect effects
Random assignment
An unbiased assignment process that ensures every participant has an equal chance of being placed into each experimental condition
A control technique that equates groups of participants
Defining feature of an experiments
Use random table
Ethical considerations
Reduce selection threats
Why?
Allows to assume equivalence of groups and rule out their differences
Avaiods any bias in the assignment of participants to experimental conditions
Facilitates our ability to make causal inferences by ruling out selection threats (enhances internal validity)
Controls known and unknown threats
BAsic exp designs
Between subjects: each participants receives only one variable of the IV
Within subjects/repeated meausre: each part receives all levels of IV, DV measured multiple times (over time)
Mixed factorial: combination of between and within subjects variables
2 Single factor between subjects designs
1 experimental group and 1 control group
2 experimental groups
Exp control
Gen def: any means used to rule out threats to the internal validity of research
Two approaches
Study design
Statistical
Example: known gender differences
Study design
Choose from narrowly defined population to rule out differences between participants (e.g. males only)
Randomly sample from the population
Include a standard against which to compare the effect of a particular variable (control group or condition)
Control within 2 study design
Statistical control
Mathematical means of making everyone equal on a measured variables
Measure nuisance variables (variable we are not interested in)
Control: placebo
Traditional control: no level of the IV (none)
Placebo: neutral level of the IV; experience administration of the IV
Sham: bogus transcranial magnetic stimulation (non-invasive pulses)
Be aware of Placebo effects – improvements in the control group simply because they believe the received a treatment
Pilot study
A small sample study usually completed before the focal study, to begin to test the effectiveness or characteristics (e.g. strength) of the manipulation
Practice; ensure study protocol is smooth
Establish the amount of time needed to administer the protocol/study
Manipulation
Assignment or presentation of one or more levels of the IV
Crucial for construct validity
Confederate: an actor who is directed by the researcher to play a specific role in a research study
Manipulation check
Additional dependent variable added to the experiment to confirm the manipulation was perceived (noticed/interpreted) the way the researcher intended to
Rating scales
Structured observation during experiment or debriefing
Pilot study data
Not a test of wether the IV had a significant effect on the DV
Pretest
Advance measure of DV
Advantages
Test equivalence of groups on DV
Provides a baseline measure to determine change
Disadvantages
Effects of testing or practice effects can be problematic
Pretest differences
Before collecting data
Match participants based on pretest data and assign to different conditions
After collecting Data
Calc difference/change scores
Statistically control/covary out pretest scores
Matching
Control procedure to ensure that experimental groups are equated o one or more known (but not unknown) variables before the experiment
Conducted before rando assignment to groups
Typically match on subject variables, pretest, DV, or individual differences
Reduces selection threat
subjects have unique differences and it may influence our data
Requirements
Match variable and DV must be significantly correlated
Feasible to pretest and get the data on the matching variable
Disadvantages
Time consuming
May waste some participants
Potential for testing threat to internal validity (bc of the pretest)
Doesn’t rule out unknown differences in GRP
Solomon design
Designs to avoid
One-group posttest only: no preemptive data to determine change/effect of treatment, no comparison group
One group pretest-posttest design: no comparison group
Within subjects design
3 conditions admin over time
Repeated measures of response (DV)
Advantages
Need less participants
Equivalence of participants across conditions is certain = more iterpretable / rigorous than btw-sub designs
Disadvantages
Limited research topics
Testing/ practice effects
Order (remembrance of certain words based on the order they are said) and sequence (two things being related back to back are more likely to be remembered) effects
Issues
Order effects: changes/ unique reactions in parti performance resulting from the ordinal temporal position in which the stimuli occurs
Sequence effects: changes/ unique reactions in part performance resulting from interactions among the stimuli
Counterbalancing
Necessary when presenting mult stimuli and order might matter
ABC = levels of IV
Arranging conditions (or stimuli) so that each condition occurs in each ordinal position
Incomplete CB: some combos but not all possible; controls order but not all sequence effects
Complete: controls order and sequence effects
Reverse CB: conditions presented in order first time, and then in the reverse order
ABC, CBA
Block randomization
Order of condition is randomized but each condition is presented once before any condition is repeated
Controls order and sequence effects
Latin Square design
WS study design in with every condition appears in each ordinal position at least once
Controls order but not all sequence effects
GEN control strategies for any design
Validated stim/ existing paradigms
Conduct study in the lab as opposed to the field
Use previously validated measure of the DV
Reliable and valid
Objective and standardized
Sensitive
Standardize the protocol
…
Chapter 3: research validity
…?
Claims
Frequency
Association
Causal
Basic characteristics of an experiment
IV manipulated
Common design
Internal validity
Extent to which we can infer a relationship btw 2 variables is causal or absence of relationship/difference implies absence of cause/difference
Determinants
Internal consistency
IV, DV, NO CONFOUND
Quality of research design: controlling third and confounding vari
Threats
History:
event btw pre and post test impacting all/most participants but not of interest
Maturation:
naturally occurring change
Testing:
familiarity/practice within test
Morality/Attrition:
unique ppl drop out
Selection:
random assignment failure
Regression to the mean
Extreme score migrate to middle
Instrumentation
Failure to use the same pre and post test
Noncompliance
Lack of standardization
Diffusion of treatment
Exposure to a level of the IV the were not supposed to get
Threats to external validity
Population validity
Interaction btw subjects and treatment
Ecological validity
Interaction btw setting and treatment
Temporal validity
Interaction btw history and treatment
Statistical conclusion validity
Appropriateness of inferences made from data as a function of conclusions drawn from stat analysis
Is the difference btw groups (or IV and DV)stat sig?
Is the difference / relationship simply a function of chance
Threats to stay in conclusion validity
Low stat power, small sample sizes, low response rate
Violations of assumptions of stat tests
Measures w low level of reliability
Construct Validity
Alignment of theory with construct measures
Extent to which labels placed on what is being observed are theoretically relevant
Do the results support the theory underlying the research
Threats to construct validity
Loose connection btw theory and exp
Misalignment btw conceptual and operational def of a construct
Diffusion of treatment
Role demand
Role demands: participants’ expectations of what the experiment requires them to do
Con lead participants to engage …
Subject roles
“Good subject”: want to be a good subject and conforms to hypothesis, to the extent that they may not be genuine
“Negativistic subject”: uncooperative/indifference; they undermine the research by acting inconsistent or not genuine
“Apprehensive subject”: tentative; concerned about researcher expectations
(Evaluation apprehension)
Solutions to subject roles
Deception (confederates)
Separate the IV and DV in time
Use unobtrusive methods/measures
Chapter 12: factorial designs
Have at least 2 independent variables
Moderation
Third variables that alter the relationship between the IV and the DV
The “depends on variable”
They explain when and under what conditions the IV and DV are more strongly related to one another
Mediation
Also uses third variable
Explains “why” the IV and DV are related to one another
Designs
All within subjects variable
All between subjects variables
Mixed factorial design
Combination of within and between subjects design
Study designs
# of IVs and levels
2 IV ex
2x2: 2(high v low) x 2(high v low)
3 IV
2x2x2: 2(male vs female) x 2(high v low) x 2 (attractive v unattractive)
Factorial designs
Experiments that examine 2 or more IVs at a time
Involves all possible combos of at least 2 values (levels) of 2 or more IVs
Allow us to test interactions
Test multiple hypotheses
Require more time and participants
T-test
A statistical procedure used to compare 2 means simulatationuly
These 2 means represent 2 levels of 1 IV
Does not allow for the testing of interactions or an examination of the joint effects of 2 IVs
ANOVA
A stat procedure used to compare 2 or more means simultaneously
Used to study joints effects of 2 or more IV
Calc an F-stat which indicate whether or not the means are stat signif
t^2=F
One factor/one way ANOVA = t-test
Use post hoc
Post - hoc
Determine where the differences lie in ANOVA
Main (overall) effect
The effect of 1 IV avg over all levels of another IV
10 pt difference: if the averages of an IV are ten points or more different from each other, they are signif
Main effect A(x): avg of the two extremes the middle of all the variables on each end
Main effect B (m): avg of each variable, then the difference between the two
Interaction
The effect of 1 IV depends on the levl of another IV
2 variables acting upon eachother
The joint effect of 2 or more IV upon the DV; cant be predicted by the main effects of each IV
Main effect are interpreted in context of interactions
Determine cell means
If lines cross on line graph, interaction IS stat sig
special
Interaction w no main effects: Antagonistic Interaction:
The 2 ivs tend to reverse each others; no main effects
Synergistic interaction
2 iVs reinforce each others effects
Ceiling effects
1 variable has a smaller effect when paired with a higher level of 2nd variable
Floor effect
Slope limited by the range of measurement; data cannot take on a lower value
Review
The condition that a presumed cause must be present whenever the effect is as described as: construct conjunction
Advantage of field exp: artificiality is minimized
4 out of 5 tried this gum: frequency claim
The ability to gen research findings: external validity
Testing effect: react different to post-test due to exposure to the pre-test
Threat to internal validity
History: other than the IV that occur btw 1st and 2nd measurement of the DV
By increasing internal validity, we decrease external validity
Threat to construct validity: loose connection btw theory and subject
If neither variables have an effect. You can still have an antagonistic interaction
Exam 3
…
Chaper 13: single subject and quasi exp
Contrasting designs
Non Exp — quasi — true
Control continuum: degree of researcher control over confounding variable
Realism of setting
internal/external validity
Methodological rigor
Best design: the one that best answers your research question
unethical/Impractical to manipulate
Unethical
Gang membership
Religious affiliation
Hazards like second-hand smoke
Parental involvement/ treatment
Impractical
Best retained in grade
Membership on the “deans list”
Going to college
Studying abroad
Unethical and impractical
Experience natural disaster
Undoable
Vaccine
Trauma (e.g. injury)
4 types of quasi -exp
Single factor design without manipulation
One quasi exp IV = group membership
Levels of IV are preexisting
Single factor design with manipulation
All members on one group selected/designed to be in one condition
All members of another group selected/assigned to other condition
IV confounded w group membership (so single factor)
Mixed factorial design with manipulation
One btw-sub quasi-exp IV that is not manipulated
One within-sub exp IV that is manipulated
Single group over time
Quasi or wtv
Nonequivalent-control group design
Research design having both exp and control group…
Delayed control group / waitlist design
Pre and post testing of control group is not simultaneous with the exp group; instead delayed
Design without control group
Interrupted time-series design
Research design that allows the same group to be compared over time by considering the trend of the data before and after …
Multiple time-series design
Repeated treatment design
Multiple treatments administers, so we can see multiple before and afters over time
Withdrawal of treatment
Implement treatment, then take it away and continue to monitor DV
Desired results for equivalent – control group design
…
Single subject exp
Performed inpsych for as long as psych has existed
Psychologists often studied themselves
Introspection: careful reporting of one’s own experiences
Particularly popular before 1930
Advantages of single-subject experiments
Focuses only on individual behavior
Focuses on big effects
Avoids ethical and practical problems
Flexibility in design; accommodate individual differences
Disadvantages of single subject exp
Lack of power: the probability that astat test will find a signif difference when there actually is a difference in pop where data are drawn
Stat vs. practical/clinical signif
Harder to detect small effects
Less stat procedures available
Limited designs & topics – can’t test btw subjects effects
Difficult to control extraneous variables
Limited external validity
Order and sequence effects
Control strats for single subject
Gather mult baseline assessment and ensure stable baseline (pretests) (AAA)
Use comparison design (AB)
Use withdrawal of treatment (ABA)
Repeat treatments (ABAB)
Change only one variable ata time or employ an alternating treatments design
Gather data on more than 1 DV
Use successively more stringent data
…
Chapter 6: survey/ questionnaire
Questionnaires
Self-report measurement of attitudes, opinions, etc,
usually by means of open and close ended questions and sampling methods
Usually completed by the individual about his/herself,
Sometimes can be about someone else
Some limitations in how insightful we are
Survey uses
Gather attitudes, preferences, opinions, concerns, needs, ideas
Look for relationships, initial evidence of contiguity
Effect of satisfaction w some event
Dispel myths
Educate
Designing surveys
1. Determine the purpose of the survey and appropriate sample
2. Decide if the identity of the responder will be collected (necessary of linkages)
3. Consult research literature for existing, validated measures of psychological constructs of interest
4. Determine how the data will be analyzed given the research questions and hypotheses
5. Types of question
Open ended
Close ended (likert, forced choice, semantic differential)
Scenario/ vignette based
6. Determine order of the items (skip logic/branching) demos first or last
7. write/ compile items
8. Pilot test the items
Practical question
Anonymous vs. confidential?
Length? (# of questions/time to complete)
Survey fatigue
Frequency? (pulse surveys?)
Advantages of open-ended questions
More complete answers, reasons sometimes revealed
Useful for preliminary/exploratory studies
Best for small samples
Survey uses
Gather attitudes, preferences, opinions, concerns, needs, ideas
Look for relationships, initial evidence of contiguity
Effect of/satisfaction with some event
Dispel myths
Educate
Practical questions
Anonymous vs confidential
Length? (# of question/time to complete)
Survey fatigue
Frequency? (pulse surveys)
Advantages of Qualitative data
Richness of data (lived experiences
Transcribe audio
“Coding” (categorize/interpreting) requiring
Qualitative data software packages
Advantages of close ended data
Easier to analyze
Respondents do not have to think as hard
They do not have to articulate their answers
Can test for consistency in responding
Useful in larger samples
Scenaria/ vignette-based q’s
Short description of persons or a social situation which contain precise references to what they are thought to be most important factors in judgment
Writing items (construct validity concerns)
Avoid bias/ leading q’s
Sequence the items; use branching/skip logic
Address a single issue per item (avoid double-barreled items)
Avoid double negative items
Make alternatives clear for close ended questions
Define anchors
Mutually exclusive
Exhaustive (other __)
Use reverse-scored items cautiously
Response tendencies
Problem and solutions
Willingness to answer (=missing data)
Forced responding but ethical concerns
Positions preference/default
Change up the meaning of the anchors across items
Acquiescence (yea or Nay Saying) (extreme responding)
Reverse-scored items
Central tendency (fence-sitting)
Even number of response option
Social desirability (self-deception, impression management)
Anonymous and measure social desirability
Careless responding
Insufficient effort responding; on-serious responding
Insert “check” items
“For this item, please select strongly disagree” (directed response)
Bogus items
“All my friends say i would make a good poodle”
“I have never used a computer”
ENhancing response rates
Response rate = # of ppl who respond/the number of ppl invited
# of requests (including reminders)
pre-contacts/heads-up/warning
Personal appeal
Salient topic
incentive / compensation
Mode of administration and response rates
Overall (30%)
Face-to-face (90%)
Lacking anonymity to the researcher
Very likely to respond in a way that better portrays the subject/ lie
Written response
Mag (1-2%)
Mail (10-50%)
Online (40%)
Tech issues
Organizational research (36%)
Chapter 6: observational research
Observational research
Observe: to record behavior
Research in which the researcher simply observes, records, and codes/rates ongoing behavior
Describes the manner in which the data are collected
Implies non-experimental, but not exclusively
Often reveals frequency of behavior
Takes place in the field or the lab
Why observe
Lacking verbal skills
infants/toddlers
Animals
Dementia patients
Natural behavior in the real world
characters=istics of observational research
Randomized sampling in terms of days/times of observations in natural environment[
Observer is primary measurement tool
Carefully train and select observers
Record observer and analyze observer/observer characteristics
Protocol followed
Careful record keeping, diaries
Relies on an inductive approach
Coding conducted
Pointers for observational research
Define behavior objectively & completely in a detailed manual to ensure construct validity
Be selective
Frequency; duration, rating
Train observers
Systematic note taking
Use recording devices
Calculate interrater reliability
Observation coding examples
Coach
Positive vs. neg statements
Teacher
Who they call on (e.g. boys vs girls)
Student- teacher interactions
Verbal
Nonverbal
Content
4 forms/levels of observational research
Naturalistic
Conducted in a way that a participant’s behavior is disturbed as little as possible
Participants are not aware of observation
Other names:
Unobtrusive, non reactive, non participant research
Common in animal research
Advantages
People aren’t altering their behavior
May catch spontaneous behavior, stuff happening on a whim
Disadvantages
Lots on environmental factors
Physical trace: unobtrusive measure of results of behavior that make use of physical evidence
Animal feces, wearing paths into the ground, etc.
Observer-participant
Observations are made such that there are no interactions, but participants are aware of the observer’s presence
Effects of being observed
Hawthorne effect
Observer characteristics could have an effect
Physical appearance,
Participant-observer
Investigator participate in a naturally occurring group and record their observations without participant awareness
Researchers usually disguised
View behavior from an insider’s perspective
Best for small group activities that are not available to the public
Careful records/diaries to reduce subjectivities
Complete participant
Observations made within observer’s own groups
Participants aware
Most intrusive-reactive
Problems
Intrusiveness by observer
Observer bias
Reactivity by participants threatens construct validity
Issues of privacy
Spectrum, like: least intrusive/least reactive —------most intrusive/most reactive
3 dimensions of observational research
Participant awareness of researcher observation
Research interaction w participant
Researcher membership in participant group
Chapter 8: Correlational/Non Exp research
Correlational research
Nonexperimental research
The measures 2 or more variables to determine the relationship between them or
Determine the relationship between 2 groups
3 primary characteristics
No manip of IV
Low control
No causal inferences
NOT simply calc of a correlation
Result in association claims
3 correlational designs
Vary in timing of the measurement of the predictor/IV and the outcome/DV as determined by theory
1) predictive
2) concurrent
3) postdictive
Temporal precedence necessary for casual references
Other nonexp designs
Case study: researcher investigates a particular existing situation, event, person, or animal that has come to their attention
Archival data: correlational research using archival data; using existing records to test hypotheses about the causes of behavior
Archival research
Archival data: pre existing records or archives
Advantages:
Availability and conveniences
Low cost
Extensive
Suited for certain research questions/topics
Often longitudinal nonreactive (assuming not a survey)
Disadvantages
Collected for another purpose
Quality may not be ideal
May not be readily analyzable
Susceptible to researcher bias
Hard to rule out alternative hypotheses
Data sources and for content analysis
Annual reports; mission statements
News articles
Open-ended responses
Open-ended responses to survey questions
Transcribed interviews/focus groups
Letters and emails
Qualitative archival data content analysis
Manifest content: the content of a text or paragraph as indicated by measuring frequency of some objective word, phrase, or action
Latent content: that content of a text or paragraph as measured by the appearance of themes as interpreted by the researcher
Focus groups
Carefully planned series of discussion with 6-10 people designed to obtain perceptions on a defined area of interest
Researcher’s interest/focal concern provides focus
Group interaction generates qualitative data
Sampling a and group composition influences data collected and discussion
Moderator/facilitator asks questions, makes sure, everyone contributes, listens closely
When to use focus groups
Look for range of ideas or feelings have ab a topic
Try to understand differences in perspective between people
Uncovering factors that influence opinions, behaviors, and motivation
Want ideas to emerge from groups
Pilot test ideas, plans, materials, or policies
Need info to design a large-scale quantitative study
Need info to help shed light on quantitative data already collected
Chap 14: meta analysis
Replication crisis
Direct replication
Conceptual replication
replication -plus-extension
questionable/unethical research practices
Underreporting of null findings
HARKing
P-hacking
Pre Registered studies
Need for meta analysis
Studies often report conflicting results
What is a true relationship? Or is it effective?
Individual study results interpreted based on stat significance which is influenced by sampling error
Aggregating results
Box-score method
All studies given eual value = problem
Need To quantify the quality of each study
1st meta-analysis examined the efficacy of psychotherapy
Meta-analysis
Secondary research design; results from other ppls studies
“Primary studies” are coded
Quantitative technique; calculate effect size
k= number of studies
N= total number of people in those studies
Studies weighed by sample size
Effect size corrected for unreliability
Focuses on the magnitude of the effect size, NOT stat signif
Coding
Calc avg. across mult studies correlations/effect sizes
Meta-analysis steps
Topic selection/research cues/ hypotheses
Specify inclusion criteria
Locate empirical studies
Select final set of studies with effect sizes
Extract data and code study characteristics
Deal with independence of data
Calculate effect sizes and correct for artifacts
Test for outliers and moderators
Interpret and draw conclusions
Effect size (d)
d=difference
D represents the strength of treatment or intervention
The standardized difference between the 2 means
Or
D = mean of the exp group, mean of control, divided buy pooled std. Dev
.2 is small, .5 is medium, and .8 is large
Convey practical sign
Advantaged
summarize/aggregate large volumes of literature & focuses on the magnitude o f the effect
Gives more weight to larger samples
Corrects low reliability
Resolve conflicts in literature
Test study- level moderators
Find subtle/weak relationships/trends
More standardized in objective method
General pop parameters
Criticisms of meta-analysis
Too many judgment calls
File-drawer problem
Publication bias
Fail-safe N
Combining apples and oranges
“Garbage in – garbage out”
Exam 4 additional topic
Chap. 4: ethics in psych research
Overview
Historical events
Belmont report
Apa guidelines
3 forms for research misconduct
Informed, debrief, conception
Hist events (problematic studies)
Nuremberg trials: nazi scientists experimented on jewish people super harshly and murdered many
Milgram studies: 65% of part cont to comply with authority (role demands), led to believe they were shocking people
Unethical because undue stress to the participant
Stanford prison exp: dr.philip zimbardo; rapidly adopted roles, emotionally traumatized prisoners
Belmont report (1979)
Ethical principles and guidelines for the protection of human subjects of research
1. Respect for persons (autonomy)
Also protect individuals who can’t make good choices for themselves (children, elderly, disabled)
2. Beneficence
Maximize benefits and minimize harm to individuals
3. Justice
Are individuals offered/given something equal in exchange, think about give and take
2002 APA Ethical principles
Respect for persons (autonomy)
Beneficence
Justice
Integrity
Fidelity & Responsibility
A. authorship
B. avoid fraud with replication
Research misconduct
Behavior formed intentionally, knowingly, or recklessly
Does not include honest error or differences in opinion
Fabrication: faking data to fit a hypothesis
Falsifications: inappropriately manipulating results
Plagiarism: using others words/ideas without credit to the og author
Questionable research practices (QRPs); variation of falsification
Failing to disclose negative outcomes/ null findings
Stopping research early if a preferred outcome seems to have been achieved
Inappropriate research design (claiming causality)
Flawed statistical approach
P-hacking to find sig results
TAMU Institutional Review board
3 types of review: expedited, exempt, full board
Full board may include clergy, philosophy, …
Collaborative institutional Training initiative (CITI)
application /proposal contents including recruitment materials
Amendments for changes
Renewal of protocol every 1-3 yrs
Completion report
Report adverse events