Advanced Research Methods Exam 1

tenacity (tradition)

“birds of a feather flock together” and walking on the R side of hallway

Intuition

it feels right

authority

relies on info from an expert

good starting point and quickest way to obtain answers

rationalism

logic

all 3 year old children are afraid of the dark

Amy is a 3 year old girl

Amy is afraid of the dark

empiricism

direct observation/sensory experience

includes personal experience

limits to tenacity

can lead to opposite conclusions

potential inaccuracies

difficult to correct

limits to authority

authorities can be wrong or conflict

can be biased

not all authorities are experts

limits to rationalism/logic

assumes premise statements are correct

we aren’t always good at logic

empiricism

observations can be misinterpreted

limited personal experience

social cognition biases influence it (confirmation bias, availability heuristics)

not everything can be directly observed

variable

characteristics/conditions that change different values for different individuals

hypothesis

statement that describes/explains relationship between variables (not complete theory yet)

inductive reasoning

small set of observations forms a general set of statements

deductive reasoning

general statement forms specific examples

elements of scientific method

empirical

public

objective

tentative conclusions

parsimony/Ockham’s Razor

quantitative vs qualitative research

most is quantitative (has stats)

laboratory vs field research

duh

basic vs applied research

basic is foundational and knowledge for the sake of knowledge

Belmont Report principles

1. Respect for persons

2. Beneficence

3. Justice

APA Code of Ethics

developed 1953, revised most recently 2002

5 general ethical principles

89 specific ethical standards

not obligations, just suggested guidelines

10 different categories

specific rules to provide bases for charges of unethical conduct

5 general principles of APA code of ethics

1. Beneficence/Non-Malfeasance

2. Fidelity/Responsibility

3. Integrity

4. Justice

5. Respect for People’s Rights and Dignity

APA ethics issues: No harm

minimize harm, all risk must be justified, halt study if needed

clinical equipoise

only providing the best possible treatment to participants…eliminates possibility of a control group in case studies

APA ethics issues: informed consent

subjects must be given sufficient information to make decision on whether to participate in study

easy to understand language (8th grade level)

in some circumstances no consent is needed (ie observational research)

elements of informed consent

description of study

time it will take for participants

disclose possible risk (may be minimal)

signatures

ways to contact PI/IRB

confidentiality and anonymity

ways to review final results of the study

participants may leave at any time

special populations of informed consent

children, prisoners, disabled people

parent/guardian gives consent, child gives assent

take care for coercion, undue harm, potential risk vs benefit

APA ethics issues: deception

should be avoided is possible, if used must justify it and consider alternatives

cannot conceal possible physical pain or emotional distress

reveal purpose of deception during debriefing

types of deception

passive deception: omission of information

active deception: actively giving false or misleading information

confederate

pretending to be in study but is really a researcher

APA ethics issues: confidentiality

all info is kept secure and private

ensured through anonymity

information cannot be linked to a specific individual

strategies to maintain confidentiality

only aggravated data reported

use coding system to link identifiable info to people

deidentify data records

IRB criteria

at least 5 people, one nonscientist, one from outside of community

IRB review guidelines

no risk: exempt

minimal risk: expedited (less than everyday)

at risk: full (more than everyday)

IACUC criteria

at least 5 member, one vet, one scientist experienced in animal research, nonscientist, somebody separate from institution

IACUC goals

minimize harm and discomfort (including habitat, food, sanitation, etc.)

must be conducted by qualified individuals

research must be justified (including species and # used)

theory

set of underlying statements about mechanisms that underlie a particular behavior

organizes/unifies observations of the behavior/relationship with other variables

generates predictions about the behavior

constructs

hypothetical entities, cannot be directly measured but are assumed to exist

can observe behavior associated with certain constructs

operational definitions

how a construct is defined/measured

limitations of operational definitions

operational definition is not the same as the construct itself

can leave out important components of a construct

can include extra components not part of the construct

validity (general)

measurement procedure must accurately capture the variable that it is supposed to measure

face validity

whether a measure superficially appears to measure what it claims to measure

simplest and least scientific

concurrent validity

how well scores from a new measure compare to scores from a more established measure of the same variable

important when developing new measures and procedures

assumes “well established” procedure is valid

predictive validity

how accurately scores predict future behavior

speaks to usefulness of measure

construct validity

how well a procedure measures the underlying construct of interest

depends on operational definition

requires several studies that examine the procedure in a wide variety of situations

also needs convergent and divergent validity

convergent validity

strong relationship between scores of 2+ different measures measuring the same construct

divergent validity

little/no relationship between scores of measurements of different constructs

ie personality tests should not be related to intelligence tests

reliability

consistency of measurements produced by a specific measurement procedure

reliable=repeatable

concept of reliability

assumes variable being measured is stable

any inconsistency is due to error

measured score=true score+error/noise

low error, high reliability

observer error

individual who makes measurements can introduce human error

environmental changes…difficult to obtain ideal circumstances with every participant

participant changes…focus, mood, etc.

test-retest reliability

compares scores of 2 successive measurements of the same individuals, correlates scores

parallel forms reliability

when different versions of the instrument are used for the test and the retest

split half reliability

correlates half of the items on a homogenous test with the other half

high score on second half should mean high score on first half

inter-rater reliability

DWTS

agreement between multiple observers who simultaneously record measurements of the same behavior

requires operational definition beforehand

important that judges do not confer with one another

nominal scale

categorical data reported as frequencies and percentages

ordinal data

represents differences in a series of ranks, relative standing is the ONLY thing you get from this scale

interval scale

rank order plus equal intervals

score of 0 is just a point on the continuum, not absolute zero

ratio scale

like interval scale but with a true zero point (0 is the absence of variable being measured)

self-report measures

ask participants direct measures

may be biased

physiological measures

based on physical manifestation of underlying construct

may require expensive equipment, may not provide valid measure of construct

behavioral measures

based on behaviors that can be observed/measured

many options, possible to select behaviors that seem best to define and measure the construct

behaviors may be only a temporary or situational indicator of the underlying construct

multiple measures

provide more confidence in validity of measurements

can have more complex stats, desynchrony (lack of agreement btwn 2 measures)

can limit problems by combining measures into a single score for each individual

range effect

measurement not sensitive enough to detect a different

floor/ceiling effect (task is too easy or too hard)

artifact

a nonnatural feature accidentally introduced into something being observed

experimenter bias (limit by standardizing.double blind study)

demand characteristics

potential cues/features of a study

roles subjects may adopt

good subject role

negative subject role

apprehensive subject role (social desirability bias)

faithful subject rule (ideal participant)

descriptive research strategy

describes individual variables as they exist naturally

data in percentages/averages

limitations: not concerned with relationships between variables

correlational research strategy

measures 2 variables for each individual, consistent patterns are easier seen in a scatterplot

describes a relationship, does not explain it

experimental research strategy

answers cause and effect questions about the relationship between two variables

“true” experiments

requires rigorous control

quasi-experimental research strategy

almost, but not quite, true experiments

cannot determine cause and effect

ie: no random assignment (sometimes you can’t, like smoker and nonsmoker)

non experimental research strategy

demonstrates a relationship between variables but does not attempt to explain it

very clear alternate explanation (participant variable)

statistics used for different research strategies

correlational: correlational stats (r value)

experimental, quasi, non: t test, anova, chi square

descriptive: mean, SD, frequencies/proportions (non numerical data)

external validity

extent to which results of a research study can be generalized

sample to population, one study to another, research to real world

threats: any characteristic that limits ability to generalize

selection bias

THREAT TO EV

when sampling procedure favors the selection of some individuals over others, sample may not accurately reflect population

volunteer bias

THREAT TO EV

are those likely to volunteer for a study different from those who would not volunteer for a study?

participant characteristics

some samples may not generalize well to the population

cross species generalization

THREAT TO EV

must consider which species are best for which research questions

novelty effect

THREAT TO EV

subjects may perceive and respond differently than normal because of the novelty of the research experience

can help this by planning extra measurements to that participants can adjust

multiple treatment interference

THREAT TO EV

participation in one condition may influence participation in another condition

ie fatigue or practice

experimenter characteristics

THREAT TO EV

demographics or personality characteristics (research assistants)

assessment sensitization

simply being measured can alter participant response, regardless of treatment

assessment causes them to self reflect

generalizing across features of the measures

generality across response measures (physiological, behavioral, self report)

time of measurement (immediately after treatment vs 3 month after)

internal validity

concerned with factors in the study that raise questions about interpretation of results

threats: any factor that allows for alternative explanations for results

extraneous variables/confounds

THREAT TO IV

extraneous variables are not always bad, but confounds are

change systematically along with variable being studied (confound)

provide an alternate explanation

environmental variables

THREAT TO IV

general threat for all designs (size of room, time of day, gender of experimenter, etc.)

individual differences

THREAT TO IV

personal characteristics that differ from one person to another (height, weight, age, personality)

assignment bias

THREAT TO IV

when assignment has caused consistent individual differences between groups

uneven distribution

time related variables

THREAT TO IV

threats for within subjects designs (practice, fatigue)

fixed with counterbalancing order of tasks

threats to internal and external validity

experimenter bias, demand characteristics, participant reactivity (artifacts)

exaggerated variables (huge range)