Psyc 2017 Exam 3

Experiments Establish Covariance

Manipulation of the independent variable introduces observable variance. Can then observe how the dependent variable varies along with the IV

What are the three ways experiments support causal claims

Experiments establish covariance

Experiments establish temporal precedence

Well-designed experiments establish internal validity

What are the three potential threats to internal validity in an experiment

Design confounds, selection effects, and order effects

Design confounds

Another variable that coincidentally varies along with the IV and provides an alternative explanation for the results

Selection Effects

A type of design confound. Occurs when participants in one group of the IV are systematically different than participants in the other level(s) of the IV

Order Effects

How the sequence in which stimuli or information are presented can influence individuals’ perceptions, memory, judgements, and decision-making

What are the four validites

Construct validity, content validity, face validity, and criterion validity

Comparison group/conditions

Groups of individuals with different levels of the independent variable or conditions representative of different levels of the independent variable

Control group

Neutral level or non-level of the IV

Placebo Group

Type of control group exposed to an inert treatment

Treatment Group

Nonneutral level(s) of the independent variable

Experiments establish temporal precedence

Manipulation of the IV occurs before the DV is measured. Ensures causal direction: IV —> DV

Well-designed Experiments Establish Internal Validity

A well-designed experiment rules out “third-variable explanations” or confounds.

Unsystematic variability

Random effects that influence both or several comparison groups (NOT confounds)

Random Assignment vs Random Sampling

Population —→ Random Sample ——> Random Assignment

Random assignment is an interval validity/experiment concept

Random sampling is an external validity concept

Matched Groups

Participants in different conditions are matched based on some individual characteristic, then randomly assigned to separate conditions/groups/IV levels

Between-Groups Design

Also known as independent-groups or between-subjects design

Posttest-only

Pretest/Posttest (also called pre-post)

Within-groups Design

Also known as within-subjects design.

Repeated-measures (which doesn’t include pre-post design)

Concurrent-measures

Posttest-only Design

Participants are assigned/exposed to one level of IV and measured/tested on the DV once. Ex: Two groups, one takes laptop notes, other takes hand-writing notes. Both groups tested on comprehension

Pre-post Design

The DV is measured/tested before and after assignment/exposure to IV level. Ex: Two groups take a verbal GRE score. One group takes a mindfulness class, other takes a nutrition class, then get another verbal GRE score

Advantages of the Pre-post Design

Measures direction of change and enhances statistical power, disadvantages being more procedurally complicated

Repeated-measures Design

Participants are exposed or assigned to each level of the IV and the DV is repeatedly measured at each level. Ex: One group taste chocolate with confederate then rates it, then tastes chocolate alone and rates ita gain

ANOVA is at fault

Despite the DV sort of being measured repeatedly in a pre-post design, a pre-post design is NOT a kind of repeated-measures design. In pre-post, participants are NOT assigned to each level of the IV, making it NOT a repeated-measures design

Concurrent-measures Design

Participants are exposed to all levels of the IV at roughly the same time, and a single preference is the DV

Advantages of Within-groups Designs

Ideally, we want participants in different comparison groups to be representative of each other (interchangeable). Within-groups designs expose participants to multiple IV levels, making them very interchangeable.

Disadvantages of Within-groups Designs

The effects of the IV on the DV are confounded with the order of IV level exposure. Includes practice (or fatigue) effect, carryover effect, and demand characteristic

Practice (or fatigue) Effect

Participants get better (worse) at a task from practice (fatigue). Biases or interferes with determine the impact of an IV level because it happened later in the order

Carryover Effect

Effect carrying over from one condition (Ex: IV level) to the next

Demand Characteristic

Something about the study that leads participants to guess the study goals and change their behavior accordingly

Counterbalancing

In a repeated-measures design, participants can be randomly assigned to different orders of IV exposure

Full being all possible order combinations are available, partial being some possible order combinations are available.

Construct Validity

The extent to which the operational variables in a study are a good approximation of the conceptual variable. Discussed with respect to reliability and validity. Need to consider manipulation checks

External Validity

The extent to which a study generalizes to the population or other contexts. Discussed with respect to sampling method and building a representative sample. Need to also consider similar contexts or situations

Describe how researchers can design studies to minimize possible obscuring factors

By controlling variables, randomly assigning participants, and using control groups

Confounds

A third (or additional) variable that could explain empirical findings. The presence of a confound threatens the internal validity of the study

Threats to Internal Validity

Maturation, History, Regression (to the mean), Attrition, Testing, and Instrumentation.

Maturation Threat

Changes in the variable that emerge spontaneously over time

History Threat

When some external or real-world event affects members of one of the experimental groups

Maturation vs History Threats

Maturation: Personal development, people mature

History: Contextual development, history happens around people

Countermeasure to both: Include a control group

Regression (to the mean)

A statistical concept, when an extreme level in an observed variable is likely to return to the mean level in the future

Attrition Threat

Attrition - reduction in participation from pre to post test

Only a problem if attrition is systematic

Countermeasures: Remove extreme data points from attritting participants, investigate why participants drop-out and adapt incentives, use statistical methods that can handle missing data

Testing Threat

A change in a participant’s response on the DV as a result of experiencing the DV more than once (practice/fatigue effects)

Countermeasures: Include a control group, use different tests/instruments

Instrumentation Threats

When an instrument or measurement tool changes over time. Ex: A rater changing their criteria over time, or using non-equivalent instruments to measure the same concept.

Countermeasures: Rigorous training of raters, use post-test only design

Null effects

0 correlation, 0 difference. Can be result of two things: Not enough between variance, too much within variance

Interrogating Null Effects: Not Enough Between Variability

Insensitive measures, ceiling and floor effects, weak manipulations, and insufficient power

Interrogating Null Effects: Too much within variability

Measurement error, individual differences, situation nose, insufficient power

Insensitive Measures

Measure hasn’t been operationalized in a way to distinguish differences in the conceptual variable. Ex: A 0 or 1 scale vs a 0 to 1 scale including all decimals and numbers in between. Pass or fail vs a A+ to F grading scale

Ceiling and Floor Effects

Questions are “too easy” or “too hard”. Items are “too agreeable” or “too disagreeable”. Insufficient variability

Weak Manipulation

Manipulation of the IV did not have an effect or change the thing that wanted to change. Insufficient variability in IV.

Measurement Error

Measurement error of psychological instruments introduces noise to the dependent variable. Use reliable and valid measures

Individual Differences

Participants have their personal differences and idiosyncrasies. Can account for individual differences with a within-groups or pre-post design. Or restrict your sample

Situation Noise

Any kind of external distraction that could cause variability within-groups that obscure between-groups differences. It can be minimized by controlling the surroundings of an experiment (sacrifice external validity)

Insufficient Power

Statistical power is the likelihood that a study will yield a statistically significant effect when there is a true effect. Probability of not making a Type II error (retaining a null hypothesis when that hypothesis is false)

Ways to Maximize Power

Have a large sample size, use strong experimental manipulations, study theoretically plausible phenomenon, reduce erroneous causes of variability.

Factorial Design

A research design with at least two fully crossed factors, the levels of which are either manipulated or specifically selected. A factorial experiment requires at least one IV

Reasons to Use a Factorial Design

Can test theories and can test limits (external validity). Testing limits- does the effect of an IV apply to some individuals but not others?

Interpret interaction effects with words, tables, and graphs

Describe the variations of factorial designs

Given a factorial notation, identify the number of independent variables, the number of levels of each variable, the number of cells in the design, and the number of main effects and interactions that will be relevant

Interaction Effects

Degree to which an IV effect depends on another IV. Mathematically identical to “moderation effects”. Not unique to experiments (degree to which the relationship between a predictor and outcome depends on another predictor)

Main Effects

The effect of the IV if there is no interaction with other variables. Based on “marginal means” - means each level of a factor averaged over all levels of another factor (as if the other factors weren’t there)

Cell

One of the possible combinations of factor levels in a factorial design

Factorial Variations

Between-groups factorial designs, within-groups factorial designs, mixed factorial designs, increasing the number of IV levels, and increasing the number of IVs

Number of factors: 2

k₁ x k₂ factorial (two-way factorial)

Number of factors: 3

k₁ x k₂ x k₃ (three-way factorial)

2 × 2, 3 × 2, 8 × 4, and a two-way factorial design all have what in common

1 possible two-way interaction

2 possible main effects

2 × 2 × 2, 2 × 4 × 2, 3 × 2 × 4, and a three-way factorial design all have what in common?

1 possible three-way interaction

3 possible two-way interactions

3 possible main effects

Increasing the Number of Levels

Instead of a 2 × 2 design, maybe we have a 2 × 3 design

Instead of a 2 × 3 design, maybe we have a 4 × 3 design

Tuskegee Sylphilis Study

Unknowingly giving black people syphilis. Violations or unethical choices include: participants not treated respectfully, harmed, and targeted and a disadvantaged social group

Milgram Obedience Studies

Debriefing- information the participants about a study’s true nature, details and hypothesis. This study did not do that, and tricked people

Belmont Report

Published in 1976, it outlines three main principles for ethical research: Principle of respect for persons, principle of beneficence, and principle of justice

Principle of Respect for Persons

Informed consent- consent based on being fully informed about the risks and benefits of participating in a research study. Consideration of (lack of) autonomy: Groups such as children, prisoners, people with disabilities. Cannot take advantage of people via some power dynamic

Principle of Beneficence

Do good. There should be a balance between the costs and benefits to participants, which includes protecting one’s anonymity and confidentiality

Principle of Justice

Who bears the risks? Who bears the benefits? There should be a balance between those who participate in a study and those who benefit from it

APA Ethical Standards

Institutional Review Boards, Informed consent, deception, debriefing, data fabrication, plagiarism, and animal research

Institutional Review Board (IRB)

A committee responsible for interpreting ethical principles and ensuring that research using human participants is conducted ethically. Composed of 5+ people, needing a: scientist, individual with academic interests outside the sciences, community member with no ties to the institution, and a prisoner advocate when necessary

What is the job of the IRB

Review proposal for research projects and determine the degree it conforms with ethical standards. Doesn’t typically meet in-person. Procedures and quality vary across institutions

Informed Consent

Potential participants must be informed of the risks and benefits of participating in research. Consent may not be required in cases of naturalistic observation in low-risk public settings and self-report of non-intrusive questions

Deception

Deception might be necessary depending on the study. Two kinds: deception by omission, and deception by commission (lying)

Debriefing

Description of the deception and explanation of why deception was used in the study. (Goal is to restore an honest relationship with the participant)

Data fabrication

Researcher manipulates data to fit a hypothesis (Ex: Altering, deleting, creating, etc)

Plagiarism

Representing another’s ideas as one’s own.

Self-plagiarism

“Potentially unethical” practice of reusing one’s writings verbatim

Animal Research

Use the Guide for the Care and Use of Laboratory Animals. Three R’s: Replacement, Refinement, and Reduction

Dr. Kline, an environmental psychologist, conducts a study to examine whether visiting zoos causes people to have more positive attitudes toward environmental conservation. He asks a group of 45 people attending the zoo on a Saturday morning about their attitudes. He finds that 69% of the people report having a positive attitude after their visit. Which of the following is true?

His study does not quality as an experiment because participants were not randomly assigned by the researcher

Dr. West wants to know what kind of exercise makes people happiest. She randomly assigns participants to either a weight-lifting or cardio group. Each group does the assigned exercise for 30 minutes. They then report on their mood on a scale of 1-10. What could Dr. West do to eliminate potential confounds in her experiment?

Use the same room and exercise instructor for both groups (This would add a degree of control and limit the influence of confounding factors)

Random selection enhances _____ validity, and random assignment enhances _____ validity

External; internal

Using a matched-group design is especially important in which of the following cases?

When you only have a few people in your study

When a participant arrives for her experiment, Dr. Dajani flips a coin to determine which of two experimental conditions the participant will receive. Using this strategy increases the ______ of her study

Internal validity (Helps ensure groups are “interchangeable”)

Dr. Georgiou wants to help undergraduates overcome homesickness. She gives a survey to 500 undergraduate and picks the 50 who scored the highest on her measure of homesickness to complete her treatment. After three weeks, she tests them again and finds their homesickness scores are significantly lower. Which of the following is a threat to her study?

Regression to the mean (they started at one extreme, only one direction to go)

Dr. Kebede wants to determine whether energy drinks improve the performance of the school’s soccer team. He gives half the team the energy drinks and half the team water. He has trained observers watch the players and rate them on a scale of 1-10 for overall soccer performance before and after they consume the drinks. Which of the following is an example of observer bias in his study?

Because they expect the energy drink group to perform better, observers notice better plays from that group

Observer bias can threaten which of the following big validites?

Internal validity and Construct validity

Regression to the mean is especially problematic in which of the following situations?

When one group has an extremely high score at pre-test

A doctor is curious as to whether drink additives will help people consumer fewer calories during a meal. She collects a random sample of 63 overweight students on campus and measures the calories they eat during lunch using a calorimeter. She then gives a drink additive to the same 63 participants to use at dinner and measure how many calories they eat (again using the calorimeter).

The addition of a control group that does not use the drink additive would help her address which of the following threats to internal validity?

History

Testing threats involve _____, whereas instrumentation threats involve ______.

Participants; measurements

A “difference in the difference between the differences” would indicate which of the following?

A three-way interaction

Dr. Green is interested in conducting a 2 × 2 × 3 mixed factorial design, with 20 participants in each cell. Which of the following would NOT be a possible number of participants for this study?

20

Lakshmi wants to know whether a new TV show helps children learn their ABCs. She discovers there is an interaction with whether or not parents watched the show alongside the child. Which of the following statements best describes this finding?

The show improved ABC skills only for kids whose parents watched along with them

Dr. Price conducted a story on how toddlers learn their colors. She thinks that the place where toddlers are asked about colors and the objects they play with while learning will affect their outcome. 76 children were in the final group. They participated in one of four between-subjects conditions (19 children each) differing in location (kitchen vs living room) and objects (blocks vs rings vs puzzle).

What would make this a mixed factorial design?

If location was a within-subjects variable

Dr. Price conducted a story on how toddlers learn their colors. She thinks that the place where toddlers are asked about colors and the objects they play with while learning will affect their outcome. 76 children were in the final group. They participated in one of four between-subjects conditions (19 children each) differing in location (kitchen vs living room) and objects (blocks vs rings vs puzzle).

What statement is true based on the excerpt?

This is a 2 × 3 design

Why are factorial designs useful in testing theories?

They allow researchers to understand the nuances of how variables interact (Relationships can be examined at specific levels of another variable)