Chapters 3, 4, 5 - Research Methods

Chapter 3: Three Claims, Four Validities, Interrogation Tools For Consumers of Research Chp. 4 - Ethical Guidelines Review

MEASURED variable

Observed and Recorded. Variable that is being measured without interference of scientist

ex. Asking participants how happy they currently are

MANIPULATED Variable

The scientist is changing or “manipulating” the IV

ex. Randomly assign people to either watch a funny or neutral video. Scientist is trying to MANIPULATE mood

Some variables can ONLY be MEASURED

CAN NOT be MANIPULATED

ex. demographic variables, economic status, gender, race

Frequency Claim

Involves only ONE MEASURED variable

Gives a specific number of occurrence or frequency

ex. 1 in 4 working students skip class due to job, College Employment Survey Shows

Casual Claims

Argues that one variable CAUSES the change for another variable

Involves at least ONE MANIPULATED variable and at least ONE MEASURED variable

ex. Always on Facebook, Insta, & Twitter? Social Media overuse leads to anxiety, says new study 2 out of 5 Americans say they worry every day

Association Claims

Argues that one level of a variable is associated with a level of another variable

Involves at least TWO MEASURED variables

ex. People with higher incomes spend less time socializing

ex. Research suggests wisdom is linked to happiness and mental health

Statistical Validity

the extent to which the NUMBERS support the conclusions.

Strength and significance - Type I (False Positive)/ Type II (False Negative) error

External Validity

how GENERALIZABLE the findings are outside of a study

WHO did they study, and HOW were those people selected?

Internal Validity

Is the variable that you are focusing on the true variable influencing your DV?

Ruling out alternative explanations/confounds

ex. Social Media use → Anxiety (Could something else explain the relationship?) Stress!

Face Validity

Does this measure look like it measures what it’s supposed too?

ex. If you want to measure anxiety, ask “How often do you feel nervous or worried?” → Good Option

But if you ask “How many siblings do you have? → Poor option

Construct Validity

measures whether a test accurately represents the intended theoretical concept

ex. Measuring anxiety include many things. If your questionnaire only asks about worry → Poor option (missing other important parts)

ex. If my questionnaire covers ALL the important components of anxiety → Good option

Tuskegee Syphilis Experiment

demonstrated 3 ETHICAL violations

1. Lack of respect for participants (Info abt penicillin was withheld)

2. Participants were harmed

3. Participants were a targeted, disadvantaged social group

Milgram Obedience Study

brings up ethical QUESTIONS

Is causing great stress in participants ethical?

Debriefing: participants only told that the “learner” had no lasting damage, were never told that the learner wasn’t shocked

Belmont Report

outlines 3 basic ETHICAL PRINCIPLES

1. Respect for persons

2. Beneficence

3. Justice

Respect for persons - Belmont Principle

People should have the FREEDOM to decide whether they want to participate

INFORMED CONSENT - states that their participation is voluntary/ They can drop the study at any time

COMPREHENSION - clear language that can be easily understood by mostly everyone

Beneficence - Belmont Principles

MAXIMIZE BENEFITS AND MINIMIZE RISKS

ex. Emotional distress, Psychological Trauma, Embarrassment, Breach of confidentiality, Social Stigmatization

Institutional Review Board (IRB)

• Committee responsible for ensuring that research using human participants is conducted ethically

• uses Federal Guidelines for reviewing research studies

• Federal Policy for the Protection of Human Participants (Office of Human Research Protections)

Justice - Belmont Principles

Researchers shouldn’t choose a disadvantaged group simply because they’re easier to exploit or access

Deception

Sometimes researchers can’t tell participants everything beforehand because knowing the true purpose could change their behavior

Need an appropriate debriefing afterwards

Debriefing

Researchers explain the deception and important info about the study afterwards.

Data Fabrication

MAKING UP data

A researcher never collected certain results but invents them

Data Falsification

MANIPULATING research materials/processes or changing/omitting data

A researcher changes or leaves out observations in a way that makes the research record misleading.

Animal Research

Animal Welfare Act

IACUC

Replacement (The Four R’s)

FINDING alternatives in animals,

if they can use computer simulations, they should consider doing that instead

Refinement (The Four R’s)

ALTERING Research procedures to reduce distress to the animals

Reduction (The Four R’s)

using the FEWEST number of animals possible in research

statistical way to determine how many participants/animals should be used instead of randomly picking a number

Rehabilitation (The Four R’s)

AFTERCARE and REHABILITATION of animals

After the study ends, researchers need to care for the animal so they aren’t experiencing any distress or long term effects

Ethically Balancing Animal Welfare, Animal Rights, and Animal Research

Researchers must follow the guidelines

Researchers must be able to defend their use of animals in research

Content validity

Does it include ALL the important parts?

EX. Being a good reader might involve:

Vocabulary + comprehension + fluency + decoding

If your test only measures how fast someone reads, you're missing important components. → Low validity

If the test adequately covers all the important components: → High validity

Validity

The appropriateness of a conclusion or decision.

Criterion Validity

measures how well a test or tool relates to a real-world outcome

You create an aptitude test for salespeople. People with higher aptitude scores later have higher sales numbers. → High test score → High sales

Known-groups method (Criterion Validity)

Suppose you develop a depression questionnaire.

Give it to:

Group A: People known to have clinical depression
Group B: People not known to have depression

If your questionnaire gives appropriately different scores between the groups, that's evidence the measure can distinguish groups it theoretically should distinguish.

Convergent Validity

Similar things should be RELATED

ex. YOU create a new self-esteem questionnaire.

You compare your new self-esteem test with an already-established self-esteem test.

Your self-esteem test ↔ Established self-esteem test are strongly related

Discriminant Validity

Different things should NOT be strongly related

compare your self-esteem questionnaire to a measure of something theoretically distinct. → SHOULD NOT be related

Conceptual Variable

WHAT am i studying????

ex. Academic Achievement, Intelligence

Operational Variable

HOW am I measuring/manipulating it??

ex. Using GPA, IQ test score, Score on a questionnaire

3 WAYS to MEASURE variables

Self-Report → Ask the person

Observational/Behavioral → WATCHING

Physiological → use biological measurements

Nominal Scale

Just NAMING. No MEANINGFUL NUMERICAL ORDER

Numbers are just labels ex. Pizza = 1 Pasta = 2 Pear = 3

Ordinal Scale

HAS AN ORDER, there is a meaningful ranking but the distance between ranking isn’t equal

1st place, 2nd place, 3rd place

Interval Scale

EQUAL Intervals, NO TRUE ZERO

distance between values is meaningful and equal

ex. 500 → 600 is the same score difference as 600 → 700

Ratio Scale

EQUAL intervals & TRUE ZERO

ex. Dorm A recieves 0 applications = zero applications

Test-Retest Reliability

If I give you the test again, do I get similar results?

Monday: IQ = 110
Later: IQ ≈ 110

Interrater Reliability

Do the RATERS agree?

Ex. Researcher A says: “The child showed aggression 6 times.”

Researcher B says: “The child showed aggression 7 times.”

Pretty similar → good interrater reliability.

Internal Reliability

Do the ITEMS agree? ITEMS measuring the same underlying thing

Suppose your self-esteem questionnaire contains:

Question 1: “I feel good about myself.”

Question 2: “I feel that I have many good qualities.”

All three are intended to measure self-esteem

If one question instead asks: “How often do you eat pizza?”