Research Methods 344

The Theory-Data Cycle

Theories lead researchers to pose research questions, then creating a research designed around a hypothesis

Data gives feedback on the theory, leading to support/revision.

Characteristics of good sci theories

• Supported by data

• Falsifiable

• Have parsimony

Do studies prove theories?

No - can support them, but studies do not prove theories

Parsimony definition

The degree to which a scientific theory provides the simplest explanation of some phenomenon

Merton’s Scientific Norms (standards of practice)

• Universalism

• Disinterestedness

• Communality

• Organized Skepticism

Basic research

“Fundamental”/”pure” research aims to gain new knowledge about observable phenomena and facts

• Does not aim to extrapolate or apply the data

Translational research

“bench to bedside” - discoveries in the lab aim to produce practical applications, treatments, etc.

Applied research

Problem-focused, specific - often uses prior knowledge and experimentation to evaluate/solve/improve existing problems

Experience as a Source of Information

Bad - Has no comparison group, is confounded

Example demonstrating the need for a comparison group

Bushman 2002 study?

Examined effect of catharsis on aggression - Good research that controlled for potential confounds, systematic comparison

• i.e. sitting, punching bag, punching bag w/ face

How is research probabilistic?

Findings are NOT expected to explain all cases - there may always be exceptions

What is intuition?

Hunches, gut feelings, sixth senses

NOT reliable evidence because of biases humans are susceptible to

Comfort affirmation bias

Tendency to be swayed by good stories

Availability heuristic

Being persuaded by what easily comes to mind

eg. believing higher probability of shark attacks because of movies

Present/present bias

Failing to think about what we cannot see

eg. tendency to consider real-time rewards more than larger rewards in the future, i.e. instant gratification

Confirmation bias

Focusing on evidence supporting our existing beliefs; ignoring that which doesn’t

Bias blind sport

Believing that oneself is not subject to being biased

What questions to ask when encountering news in Popular Media

• Could this be disinformation?

• Can this be cross-checked with legitimate sources?

• What is the context of this story?

• Is the source politically biased?

Types of untrustworthy vs trustworthy Sources of Info (image)

What are variables? What are their characteristics?

• They change/vary from person to person - eg. factor, condition, trait

  • at least two levels

• Constant: eg. 60% of highschoolers pursue college - doesn’t change

• They can be measured and/or manipulated

Measured variables are…

Observed and Recorded

Manipulated variables are…

Controlled, Changed

How do researchers measure variables?

The conceptual variable (eg. expression of gratitude, depression) is measured through an Operational Definition (can be directly measured, eg. surveys, score on diagnostic tests)

What are the Three Types of Claims

• Frequency

• Association

• Causal

Frequency claims

Describes a particular rate/degree of a single variable

Involves only one measured variable

eg. “1 in 4 working students skip class due to job, college”

Association claims

Argues that one level of a variable is associated with a particular level of another variable

Involved at least two measured variables - which are said to correlate

eg. “People with higher incomes spend less time socializing”

Types of measured associations

Positive, Negative, Zero

How can association be used?

We can make predictions based on them - extrapolating from evidence

• Stronger association —> More accurate prediction

• Positive and negative associations can predict, Zero associations cannot

Causal claims

Argues that change of one variable is responsible for changing the other

• Causal claims are supported by experiments

• eg. “Social media overuse leads to anxiety”

What else can claims be based on, if not Research?

• Experiences

• Intuition

• Authority

What are the Four Big Validities?

• Construct validity

• External validity

• Statistical validity

• Internal validity

Construct validity

How well the operational variable of a study approximates (measures) the conceptual variable

External validity

How well the study’s results generalize to some larger population

• eg. random, inclusive, and representative samples; other situations/contexts

Statistical validity

How well the data supports the conclusions made

• eg. strength of an association, statistical significance

• Margins of Error

Internal validity

How likely A, rather than some other variable, is responsible for changes in B

• i.e. Third Variable Problem

What is special about Interrogating Causal Claims?

the Three Criteria for Causation:

• Covariance: an association under controlled experimentation

• Temporal precedence: study method has A come first, before B

• Internal validity: study method ensures no plausible alternative explanations for the change in B

Significance of the Tuskegee Study (1930s)

Illustrated 3 major violations of ethics

• Participants were not treated respectfully (false pretenses, withholding treatment)

• Participants were harmed (syphilis, later generations)

• Participants were a targeted, disadvantaged social group

Significance of the Milgram Obedience Studies

Illustrated a difficult ethical balance

• Ethical questions about debriefing

• How to balance risk between gain for society vs. participant wellbeing/mental

What is the Belmont Report?

1979 document outlining core ethical principles + guidelines to Protect Human Subjects of Research

Belmont Report Principle #1

Respect for Persons

• People treated as autonomous agents, those w/ diminished autonomy entitled to protection

• Application: Informed Consent Process

  • Voluntary participation free of coercion, undue influence

  • Providing information & Ensuring its comprehension

Belmont Report Principle #2

Beneficence

• Maximize possible benefits and minimize risks

  • Emotional/psych distress/trauma, embarrassment/shame/guilt, invasion or breach of privacy/confidentiality

• How to mitigate risks:

  • Confidentiality, privacy, anonymity, debriefing, emphasizing voluntary participation

Belmont Report Principle #3

Justice

• Equitable selection and assigning of participants

What oversees/ensures the principles of the Belmont Report?

Institutional Review Board (IRB)

• Committee responsible for enforcing ethical research involving human participants

Ethical Decision-making Balances (3)

• Research risks vs. Benefits

• Rights of individual participant vs. Societal gain

• Free participation vs. Coercion

What are the equivalent counterparts of the Belmont Report and IRB for Animal Resarch?

• Animal Welfare Act (AWA)

• Institutional Animal Care & Use Committee (IACUC)

The Four Rs of Animal Care

• Replacement: Avoiding using animals if possible

• Refinement: Tuning procedures and care to reduce distress

• Reduction: Using the fewest # of animals as possible - minimize harm

• Rehabilitation: Post-experiment welfare - recovery

Reasons for discrepancies exist in research data?

• Unclear definitions of the operational definition

• Between-evaluator biases, differences

Common Types of Measures?

• Self-report measure

• Observational/behavioral measure

• Physiological measure

No superior one - appropriate method depends on the context

Two main scales of measurement

Categorical: i.e. nominal

Quantitative: includes ordinal, interval, and ratio scales

Reliabilty

Are scores consistent?

Three types of Reliability

• Test-retest reliability

• Interrater reliability

• Internal reliability (Does the test measure what its supposed to?)

Can reliability be quantified?

Yes - by the correlation coefficient r

Validity

How accurately the method/test measures what it claims to measure

Types of Construct Validity

• Face validity

• Content validity

• Criterion validity

• Convergent validity

• Discriminant validity

Face validity

The construct is obvious - looks like what is being measured

eg. low FV: “Your Favorite Meals Reveal If You’re An Introvert or Extrovert”

Content validity

The measure contains all parts that your theory says it should contain

eg. Comprehension as a measure of all dimensions

Criterion validity

Also called “Predictive validity”

• How well findings correlate with concrete outcomes

eg. The left has better criterion validity

What does using a known-groups paradigm help?

How well the experimental test measures a trait as compared to an established assessment strategy

• eg. how well the experiment’s criteria for depression measure depression, as compared to what the BDI categorizes subjects as

Convergent vs. Discriminant validity

Convergent: scores on the measure are related to other measures that are theoretically similar (eg. Job stress vs. Workplace burnout)

Discriminant: scores on the measure are unrelated to other measures that are theoretically different (eg. Typing speed vs. Hair color)

Relationship between Reliability and Validity

• Reliability is necessary, but not sufficient for validity

• They are not the same

Internal reliability vs. Internal validity

Internal reliability: Test measures what it claims to

Internal validity: Method used ensures no Third Variable Problem