Research Methods Exam 1

how to recognize a claim when you see it

how to distinguish one type of claim from another

how to interrogate those claims

What questions should you be asking

how to determine whether those claims are valid (dig deeper)

What are the answers?

how to make appropriate claims of your own

Empirical Approach

Drawing conclusions based on unbiased/objective, publicly verifiable data!

Making systematic observations and collecting data using a tangible metric

Systematic observations vs. non-systematic observations

Tangible metric vs. non-tangible metric

Theory

Hypothesis

Data cycle

heory → direct predictions → collect data → evaluation

Evidence supporting theory

Evidence against theory

Characteristics of Good Theories

Supported by data

Falsifiable

Parsimonious

Proof vs disproof

Reducing error

Weight of the evidence

Types of Research

Basic

Applied

Translational

Basic research

Applied research

Translational research

Publication process

Benefits vs. Risks

Scientific Journalism

Benefits vs. Risks

Research

Research is probabilistic

Experience

Experience has no comparison group

Experience is confounded

Researc vs. Intuition

Biased by two main things:

Faulty thinking – a good story; availability heuristic; present/present bias

Motivation – only see what we want to see; asking biased questions; bias blind-spot

Trusting an Authority

What is the source of their claim?

Do they cite evidence/research to back their claim?

Is their evidence appropriate and accurate for this particular claim?

Variable

a “thing” that varies; contains multiple levels

Constant

a “thing” that does not vary (or we are not interested in how it varies)

Measured variables

manipulated variables

What is a hypothetical construct?

Conceptual

operational definitions

Three Types of Research Claims

Frequency, association, causal

Frequency research claims

Association research claims

Causal research claims

How do you know your claim is VALID?

Ask questions about validity of study...

4 fronts to challenge the validity of a claim on

Construct Validity, External Validity, Statistical Validity, and Internal Validity

Construct Validity

are the researchers measuring what they think they are measuring?

External Validity

can the researchers generalize their results to the whole population and to other “real-world” situations. Ask questions about the sample, way sample was collected

Statistical Validity

what is the strength of the claim based on the statistical results? Did the researchers use the appropriate statistical tests?

Frequency claims

margin of error

Association claims

correlation coefficient

(r closer to 1 is stronger); significance (low p value is more significant)

Causal claims

effect size

(higher is stronger); significance (low p-value is more significant)

Internal Validity

a study’s ability to rule out alternative causal explanations only have to worry about this type of validity when making a causal claim.

Accomplished by controlling for extraneous variables that might affect outcome.

Three Criteria for Cause:

1. Covariance

2. Temporal Precedence

3. Internal Validity

Covariance

A and B are correlated

Temporal Precedence

A precedes B

Internal Validity

A is causing B, not something else

Two obligations of researchers

Sources of Ethical Directives

The Belmont Report

APA ethical principles

APA ethical standards

The Belmont Principles

Principle of respect for persons

Principle of beneficence

Principle of justice

Principle of respect for persons

Individuals should be treated as autonomous agents.

Those who are not autonomous are provided special protection

Principle of beneficence

Protect participants from harm and ensure their well-being (cost/benefit analysis).

Potential benefits:

Knowledge, Improving research methodology, Practical outcomes, Benefits for researchers, Benefits for participants

Potential costs:

Time/effort for participants, Participants’ mental/physical health, Money, Impact on the field of research, cannot withhold beneficial treatments

Principle of justice

Fair balance between the people who participate in research and the people who benefit from the findings

Institutional review Board

Seven concerns of the IRB

Lack of informed consent

Invasion of Privacy

Coercion to participate

Potential physical or mental harm

Deception

Violation of confidentiality

Lack of debriefing

Informed consent

Potential problems of informed consent

Compromising the validity of the study

Participants unable to give consent

Impractical to get consent

Times when consent can be waived

Minimal risk to participants

No adverse effects to participants

Research cannot be done with consent

Contents of informed consent form

Invasion of Privacy

in private areas

Coercion to participate

Participant feels like they would be penalized if they did not participate.

Given excessive rewards for participating

Physical and Mental stress

Definition of minimal risk

Cost-benefit analysis

Deception

Why use it?

What constitutes deception?

Objections to deception

When is it OK to use deception?

Debriefing

When and why is it used?

Goals of debriefing

Confidentiality in Research

Ways to ensure confidentiality

Anonymous responding

Coded data

Changing details of a particular case