Exam 2 PSY450 Advanced Research Methods

Goals of the Experimental Research Strategy

To establish the existence of a cause-and-effect relationship between two variables.

Four Basic Elements of an Experimental Study

Manipulation, Measurement, Comparison, and Control

Manipulation

Researcher manipulates a variable by changing its value to create a set of two or more levels. The Independent Variable (IV).

Measurement

Results in a set of scores for each condition. The Dependent Variables (DV).

Comparison

Scores in one condition are compared with scores of another condition.

Control

All other variables are controlled to be sure that they do not influence the variables being examined.

Statistical Significance

The difference is large enough and consistent enough to rule out chance as plausible explanation.

Treatment Condition

The specific situation or environment a group of participants experience in the research study.

Levels

Different values of the IV. Selected to create and define the treatment conditions.

Extraneous Variables

All other variables in the study other than the IVs and DVs.

Directionality Problem

The existence of a relationship does not always explain the direction of the relationship.

Third-variable Problem

The existence of a relationship is not necessarily direct (i.e. causal). A third unidentified variable mat be responsible for producing the observed relationship.

2 Important Characteristics of Confounds

1.) An extraneous variable becomes a potential confound only if it may influence the DV.

2.) A variable that changes randomly, with no relation to the IV, is not a threat.

Techniques for controlling extraneous variables

1.) Holding variables constant

2.) Matching values across conditions

3.) Control by randomization

Outcome Research

Investigates the effectiveness of a treatment.

Process Research

Attempts to identify the active components of the treatment.

Within-subjects design

One "group" of subjects generates two "groups" of scores of comparison.

Simulation

Conditions in an experiment that closely duplicate the natural environment.

Mundane Realism

Superficial, usually physical characteristics, of the environment.

Experimental Realism

Psychological aspects of the simulation or the extent to which participants become immersed in the experiment and behave normally.

Field Study

Research conducted in a place that the participant perceives as a natural environment.

Between-subject Design

Each set of scores is obtained from diff groups of participants.

Between-Subject Design Advantages

Each individual score is independent from all other scores. Individual scores are not influenced by Practice or fatigue effects, and Contrast effects.

Contrast effects

The result from comparing one condition to another. The perception of a condition is influenced by its contrast to the previous condition

Between-Subjects Design Disadvantage

Require a relatively large number of participants. Each score is from a unique individual who has unique personal characteristics.

Assignment Bias

Only possible for between-subjects designs. Groups with different characteristics (that are not the variables of interest) threaten the internal validity of study.

Two major sources of confounds in between-subjects designs

Individual differences, and Environmental variables

To protect against assignment bias, researchers create equivalent groups and they must be:

Created equally, Treated equally, and Composed of equivalent individuals.

Three primary techniques to limiting Confounding by individual differences

Random assignment (Randomization), Matching groups (matched assignment), and Holding variables constant.

Strategies to minimize within group variance

Standardize procedures and treatment settings, Limit individual differences, and Greatly increase sample size.

Attrition

participant drop out

Diffusion

Treatment spreads to the control group

Compensatory Equalization

Subject demands treatment received by the other group

Compensatory Rivalry

Control group changes their normal behavior (e.g., works extra hard)

Resentful demoralization

Participants in control group become less productive and less motivated.

single-factor two-group design

Simplest version of between-subjects design. Researcher manipulates one IV with only two levels. Analyzed with an independent-measures t-test

Advantage and disadvantage of single-factor two-group design.

Advantage: simplicity, Disadvantage: Provides little information.

Single-factor multiple-group design

Analyzed with a single-factor analysis of variance (ANOVA) for independent measures. Provides stronger evidence for a cause-and-effect relationship than a two-group design.

A single factor ANOVA

Only indicates if you have significant main effect. If that factor only has two levels then you know there is a statistically significant difference between those two levels.

Threats to Internal Validity for Within-Subjects Design/Repeated-Measures Design

Environmental variables, and Time-related variables

Five time-related threats

1.) History

2.) Maturation

3.) Instrumentation

4.) Regression to the mean

5.) Order Effect

History

An event outside of the experiment that affects participants' scores in one condition differently from another condition.

Maturation

A systematic change in participants' physiology or psychology that occurs during the research study and affects participants' scores.

Instrumentation

Changes in the measuring instrument over time.

Regression to the mean

The tendency for extreme scores on a measurement to move towards the mean (regress) when the measurement is repeated.

Order Effect

Participating in one condition may have an effect on scores in the next condition.

Carry-over effects

When participants in a specific condition influences the following conditions

Dealing with Time-Related Threats, Shortening the time between conditions would:

Decreases the risk of time-related threats, Increases the likelihood of order effects.

Dealing with Time-Related Threats, Increasing the time between conditions would:

Decrease the likelihood of order effect, Increases the risk of time-related threats

Counterbalancing

Changing the order in which conditions are applied form one participant to another

Asymmetrical order effects

When different sequences have different order effects. e.g., A-B has a different effect than B-A.

Complete counterbalancing

Requires using every possible sequence. Easy with only two condition. Becomes more complex as the number of conditions increases.

Partial Counterbalancing

Does not use every possible sequence in conditions. e.g., (2!=1x2=2, 3!=1x2x3=6, 4!=1x2x3x4=24)

Advantages of with-subject design

Requires relatively few participants, Eliminates problems based on individual differences, Reduces variance.

Disadvantages of within-subjects design

Potential time-related factors, Participant attrition,

Three factors that differentiate within-subjects or between-subject designs

Individual differences, Time-related factors and order effects, Number of participants.

Matches-subjects Designs

Uses a separate group for each condition. Each individual in one group is matched one-to-one with an individual in every other group.

Advantages of multiple-group designs

Data are more likely to reveal the functional relationship between the two variables.

Produces a more convincing demonstration of a cause-and-effect relationship than a two-groups design.

Disadvantages of multiple-group designs

If too many treatments, distinction between treatments may be too small to see significant differences.

May increase attrition if more time is required of participants.

Counterbalancing is more difficult as number of treatments increases.

Factor

An independent variable (IV) in an experiment.

Factorial Design

A research design that includes two or more factors (IVs)

Factorial Matrix

the levels of one factor are listed in columns and the levels of the second factor are listed in rows.

Main Effect

Describes how one factor, independent of all other factors, affect behavior. A statistically significant mean difference among the levels of one factor, regardless of the other factor(s).

Interaction

Describes how combinations of factors jointly affect behavior. When one factor has a direct influence on the effect of a second factor.

How to Identifying Interactions

Graphing the results of a study As a rule of thumb, non-parallel lines indicate an interaction between the factors.

Compare the mean differences in any individual row with the mean differences in other rows. If the size and direction of the differences in one row are the same as the corresponding differences in other rows -> no interaction. If differences change from one row to another -> evidence of an interaction.

Mixed Factorial Design

Includes at least one IV that is between-subjects and at least one IV that is within-subjects.

What Statistical test do you use for a single factor design

Use a one-way ANOVA to evaluate the statistical significance of the mean difference.

What Statistical test do you use for a factorial design?

Use a factorial ANOVA. The type of factorial ANOVA depends on whether the factors are: Between-subjects, Within-subjects, Mixed designs.

How many subjects are needed for 2x2 independent groups (between-subjects) factorial?

5 unique subjects per cell x 4 cells = 20 subjects needed

How many subjects are needed for 2x2 independent repeated measures (within-subjects) factorial?

5 subjects/cell, same 5 in all 4 cells - 5 subjects needed

How many subjects are needed for 2x2 mixed factorial (one between and one within IV)?

5/cell, same 5 in two cells, different 5 in two other cells = 10 subjects needed. [ Columns = Within, Rows = Between ]

Descriptive Statistics

Help organize, summarize, and simplify results. Turns a large pile of numbers into a smaller more manageable amount of numbers.

Inferential statistics

Help make generalizations from your sample to a broader population. Requires hypothesis testing. p-value is inferential statistics.

Statistic

A summary value that describes a sample. e.g., the average IQ score for a sample

Parameter

A summary value that describes a population. e.g., the average IQ scores for a population.

Central Tendency

A single score that defines the center of a distribution.

Mean

Mathematical average of a set of scores

Median

Score that divides a distribution in half

Mode

Most commonly occurring score

Range

The difference between the highest and lowest scores.

Standard Deviation (SD)

Square root of the variance. Describes the average distance from the mean.

Variance

The average squared distance from the mean

Line graphs

Preferred for continuous variables. Unlimited intermediate values exist. Ratio or interval data. An exception is when looking for interactions.

Bar graphs

Required for discrete variables. No intermediate values. Nominal or ordinal data.

Hypothesis Testing

A statistical procedure by distinguishing whether patters in the data representing systematic relationships among variables in the population or patterns in the data produced by random variation.

Null Hypothesis (H0)

There is no statistical difference between populations. e.g., M1=M2

Alternative hypothesis (H1)

here is a significant statistical difference between populations. e.g., M1 not qual to M2 OR M1 < M2 OR M1 > M2

Only two possible outcomes for Hypothesis testing.

Fail to reject the null hypothesis, Reject null hypothesis (with some probability).

Alpha

the level of significance chosen by the researcher to evaluate the null hypothesis. Traditionally, alpha = 0.05 (or 5%) is the cut off for significance

Type I Error

Reject null hypothesis, but be wrong

a false report

A researcher finds evidence for a significant result when there is no relationship in the population

Type II Error

Fail to reject null hypothesis, but be wrong

2 Factors That Influence the Hypothesis Test

The number of scores in the sample, and The size of the variance.

How do studies using the experimental research strategy differ from other types of research?

a. Only experiments can demonstrate a cause-and-effect relationship between variables.

b. Only experiments involve comparing two or more groups of scores.

c. Only experiments can demonstrate that relationships exist between variables and provide a description of the relationship.

d. Only experiments can demonstrate a bidirectional relationship between variables.

a. Only experiments can demonstrate a cause-and-effect relationship between variables.

Dr. Jones is interested in studying how indoor lighting can influence people’s moods during the winter. A sample of 100 households is selected. Fifty of the homes are randomly assigned to the bright-light condition where Dr. Jones replaces all the lights with 100-watt bulbs. In the other 50 houses, all the lights are changed to 60-watt bulbs. After two months, Dr. Jones measures the level of depression for the people living in the houses. In this example, how many dependent variables are there?

a. 100

b. 50

c. 2

d. 1

d. 1

Research indicates the people who suffer from depression also tend to experience insomnia. However, it is unclear whether the depression causes insomnia or the lack of sleep causes depression. What problem is demonstrated by this example?

a. the directionality problem

b. the third-variable problem

c. the extraneous variable problem

d. the manipulation-check problem

a. the directionality problem

In an experiment, what is the purpose for manipulating the independent variable?

a. It helps establish the direction of the relationship by showing that the dependent variable changes when you manipulate the independent variable.

b. It helps eliminate the third-variable problem because you decide when to manipulate rather than waiting for the variable to change.

c. It helps establish the direction of the relationship and it helps eliminate the third-variable problem.

d. Manipulation does not establish the direction of the relationship or eliminate the third-vari able problem.

c. It helps establish the direction of the relationship and it helps eliminate the third-variable problem.

In order to establish an unambiguous relationship between two variables, it is necessary to eliminate the possible influence of which of the following variables?

a. Extraneous variables

b. Confounding variables

c. Independent variables

d. Dependent variables

b. Confounding variables

Which of the following characteristics are necessary for an extraneous variable to become a confounding variable?

a. It must change systematically from one participant to the next.

b. It must change systematically when the independent variable is changed.

c. It must have no systematic relationship with the dependent variable.

d. It must have no systematic relationship to either the independent or the dependent variables.

b. It must change systematically when the independent variable is changed.

In an experiment comparing two treatments, the researcher assigns participants to treatment conditions so that each condition has fifteen 7-year-old children and ten 8-year-old children. For this study, what method is being used to control participant age?

a. Randomization

b. Matching

c. Holding constant

d. Limiting the range

b. Matching