PSY 3001W Exam 3

Land of Zero Variability:

Every time someone performs a task, they do it exactly the same way
• Everyone behaves like everyone else


Example of land of zero variability:

• Experiment: Will our new "pep pill" make people run faster?

Inferential Statistics:

• We use inferential statistics to try to infer from the sample data what the population might think.
• We use inferential statistics to make judgments of the probability that an observed difference between groups is a dependable one or one that might have happened by chance in this study.

Results:

• Because there is no variability, we can simply compare the sample mean to the population mean
• Now the sample is no longer like the population of undrugged people. There is now a population of drugged people who behave differently than undrugged people.

Do experiment again: First attempt

Pick a guy, give him the pep pill, have him run a mile.
• He runs it in 5:45. What can we conclude about the pep pill?
Nothing.

Do experiment again: Second attempt

Pick a guy, have him run a mile BEFORE he takes the pep pill. It takes him 6:00
• To double-check, have him run it again. Now it takes him only 5:50.

Do experiment again: Third attempt

• Measure how long it takes everyone to run a mile

What is the problem of the third attempt?

We can't measure everyone's running time, so we don't know the distribution of running times for the population (mean and SD)


Problem: the population of undrugged runners might be identical to the population of undrugged runners. (the drug might have no effect)

How can we estimate the population distribution?

By measuring running times for a sample

What is the problem of this?

The population of drugged runners might be identical to the population of undrugged runners (the drug might have no effect)

What is the null hypothesis?

Assume that there is no difference between the populations from which the samples were drawn (drug had no effect)

Ho: Ue \= Uc

Alternative hypothesis or research hypothesis:

There is a difference between the populations (the drug had an effect)

Ha: Ue does not equal Uc

Testing the null hypothesis

Are the sample means "significantly different?"

If the probability of obtaining our result when the null hypothesis is true (p-value) is less than .05, reject the null hypothesis.
Test Ho at the .05 level of significance

T-test:

Tests the significance of the difference between sample means

Compute t based on:

Sample means, standard deviations, sample sizes

Compute SD's on:

Sample sizes (t is very large for large differences)

What is the probability of obtaining that value of t by chance if the samples were drawn from identical populations?

Make a decision based on p-value

For any statistic we compute (t, F, etc.) can we compute its p-value?

Yes

P-value:

Probability of obtaining that value of the statistic or a more extreme value of the statistic if the null hypothesis is true

When doing a t-test in SPSS:

• Select alpha (level of significance)

• Enter your data in SPSS, click on t-test

• SPSS computes t(obs) and tells you the probability of obtaining t(obs) by chance (p-value)

-Determine if the p-value (probability of obtaining Tobs by chance) is less than alpha
If you are doing a one tailed (directional) test, divide p by 2.

Determine if the p-value (probability of obtaining t(obs) by chance) is less than alpha

If you are doing a one-tailed (directional) test: p\=0.12 (divide p by 2)

Test the significance at the .05 level of significance:

Determine p-values

If p < .05, conclude that the sample means are:

significantly different (IV had an effect)

If p \> .05, conclude that the sample means are:

NOT significantly different (IV had no effect)

There is always a chance that:

Our conclusion is wrong

In SPSS, there are different t-tests for independent groups and repeated measures:

Interpret the p-value the same way

Testing the null hypothesis is a...

a decision based on probability

Type 1 error:

H0 is true but you reject the Ho (IV had no effect but you conclude IV had an effect) FLASE ALARM

Type 2 error:

H0 is false but you do not reject H0 (IV had an effect but you conclude that IC had no effect) MISS

Probability of Type 1 error:

Alpha \= level of significance

Probability of Type 2 error:

Beta

Probability of correctly deciding H0 is false:

\= 1- B\=power

Type one and type two error example:

Researcher believes new teaching method can improve math ability in grade school children more than current teaching method can improve math ability in grade school children more than the current teaching method

Ho: new teaching method is no better than old one

H1: new teaching method is better than old one

Type 1 error: researcher concludes that new teaching method is no better than old one
In reality, there is no difference between the old and new teaching methods (false alarm)


Type 2 error: researcher concludes that new teaching method is no better than old one
In reality, the new teaching method is more effective than the old method (miss)

Which type of error is worse?

If you make a type 1 error and conclude that new teaching method is better but its really the same, there is no cost.

If you make a type II error and conclude that new teaching method is no better, you lose out on a beneficial teaching method.

Type II error seems worse than type I error

BUT, what if the new teaching method is inexpensive? A type 1 error is more costly, maybe worse than a type II error.

T-tests:

• Independent-groups t-test
• Related-samples t-test

Independent-groups t-test:

Compares the means of two independent groups in order to determine whether there is statistical evidence that the associated population means are significantly different.

Related-samples t-test:

The Paired Samples t Test compares two means that are from the same individual, object, or related units. The two means typically represent two different times (e.g., pre-test and post-test with an intervention between the two time points) or two different but related conditions or units (e.g., left and right ears, twins). The purpose of the test is to determine whether there is statistical evidence that the mean difference between paired observations on a particular outcome is significantly different from zero.

Type one and type two error example \#2

Test of general feelings of hostility: those who score high have a tendency to exhibit violent behavior in the future.

Ho: student is just like non-aggressive students

H1: student is dangerously aggressive

Type 1 error: conclude that the student is overly aggressive
In reality, student is not aggressive (false alarm)

Type II error: conclude that the student is not aggressive
In reality, student is in danger of becoming violent (miss)

Type II error is worse

Types of ANOVA tests

Independent groups ANOVA (IV manipulated between groups)


related-samples/repeated measures ANOVA (IV manipulated within subjects)

Independent groups ANOVA:

IV manipulated between groups

Related-samples/repeated-measures ANOVA:

IV manipulated within subjects

Chi-square:

Compare counts for nominal data; Measurement scale for DV is nominal; \= chi square

Correlation:

Describe relationship between two continuous variables

Choosing a statistical test:

Degrees of freedom for paired-samples t-test:

df\= N-1

Degrees of freedom for independent samples:

df\= n1 + n2-2

Degrees of freedom (Df)

Related to sample size

Effect size (r)

strength of association

Effect size increases as...

T increases

0 < r < 1

Small effect size:

r\= .15

Medium effect size:

r\= .30

Large effect size:

r\= .40

Power of statistical test:

Probability of correctly detecting an effect
• Power\= 1-B

-power analysis

B\=

Probability of a Type 2 error

Power Analysis:

Given effect size and level of significance, determine N needed to detect effect

Effect size table:

Primary way researcher controls power is to:

Change the sample size


To increase power, increase sample size

To detect smaller effects, increase sample size

To increase power...

Increase the sample size

To detect smaller effects...

Increase the sample size

Interpreting non-significant results:

We "fail to reject H0" rather than "accept H0"

Reasons results may be non-significant even though H0 is false (Type 2 error):

• Level of significance (a, probability of a Type 1 error) is very low...Increases probability of a Type 2 error


• Sample size is too small for effect size

Compute and report effect size for non-significant results

A statistically significant result has little practical significance when:

• The study has poor external validity
• The effect size is very small
• The treatment is too costly to implement
• The effect size is comparable to that for an existing treatment

Multiple tests:

Probability of rolling 3 in at least one of two rolls: 11/36 \=.31

Probability of committing a Type 1 error in AT LEAST ONE OF TWO TESTS, each at the .05 level: .0975

If you perform multiple statistical test, there is a high probability that, by chance, at least one result will be statistically significant

Hypothesis testing:

Probability of committing a type I error (false alarm) when the level of significance is .05: .05

Replication example: dice

Probability of tolling two 3's in two rolls: 1/6 x 1/6 \= 1/36 \= .03

Hypothesis testing
Probability of committing a type 1 error (false alarm) when the level of significance is .05: .05


Probability of committing a Type 1 error in BOTH TESTS when you perform two tests, each at the .05 level: .05 x .05 \= .0025

If you repeatedly obtain significant results (you reject H0) in replications of a study, it is VERY unlikely that you are committing a Type 1 error

Hart and Albarracin (2011)

Examined how verb form affects interpretation of a written passage

Description of one main shooting another after gambling disagreement
"Imperfective" form: smith was firing gun shots
"Perfective" form: smith fired gun shots

"When violent, unlawful actions were described in the imperfective (1) rather than perfective (2) aspect, the perpetrator of the actions was viewed as engaging in them with greater harmful intent. "

Results could have implications for how information is presented to juries

Replication study: Eerland et al (2016)

12 independent research teams performed replications

A meta-analysis of the results did not detect an effect of verb form on "subjects" judgements of criminal intentionality or on the level of detail imagined by subjects while reading the passage.

"Taken together, our studies did not provide evidence that describing actions in imperfect aspect resulted in greater perceived intentionality... or more detailed processing of those actions. This overall pattern of results was consistent across studies."

Quasi-experiments:

Lack some of the features of true experiments
• Involve manipulation of an IV or introduction of a treatment
• Often lack randomization
• One- group posttest-only design
• One-group pretest-posttest design

F tests

Aka Analysis of variance
a more general statistical procedure than the t test

Systematic variance

the deviation of the group means from the grand mean, or the mean score of all individuals in all groups.

between groups

error variance

the deviation of the individual scores in each group from their respective group means.

within groups

Cohens d

Effect size estimate
-expresses effect size in terms of standard deviation units.

Quasi experiments

Lack some of the features of true experiments
Involve manipulation of an IV or introduction of a treatment
Often lack randomization

types of quasi-experiments

one-group posttest-only design
-one-group pretest-posttest design

One-group posttest-only design

called a "one-shot case study" by Campbell and Stanley (1966)

lacks a crucial element of a true experiment: a control or comparison group.


Ex: employees in a company might participate in a 4-hour information session on emergency procedures. At the conclusion of the program, they complete a knowledge test on which their average score is 90%. This result is then used to conclude that the program is successfully educating employees.

One-group pretest-posttest design

way to obtain a comparison is to measure participants before the manipulation (a pretest) and again afterward (a posttest). An index of change from the pretest to the posttest could then be computed

-Include threats to internal validity

History:

Events that occur during participation and effect behavior

Maturation:

Changes due to the passage of time that affect behavior

Testing:

Taking a test can affect subsequent testing

Instrumentation:

Changes in measurement instruments (including observers) over time

Threats to internal validity controlled by true experiments:

1. History
2. maturation
3.testing
4. instrumentation
5. Regression toward the mean
6. Subject attrition (mortality)
7. Selection

Regression toward the mean:

Extreme scores are likely to be followed by more moderate scores

Subject attrition (morality):

Participants selectively drop out of experiment

Selection:

When control and experimental groups are chosen in such a way that they are not equivalent

Nonequivalent control group design:

• Uses an experimental group and control group, but they are not equivalent (e.g. natural groups)
• Groups are "self-selecting"

Nonequivalent control group pretest-posttest design:

Assignment to groups not random;

Ex: Evaluate effects of regular writing exercises on writing quality

• One lab does exercises, other lab does not

• Nonequivalent groups (use pretest to show equivalence; or use pretest to show differential change for the two groups)

• Possible additional problems: Different section leaders (a confound), observer bias, contamination

Selection differences/selection bias

usually occurs when participants who form the two groups in the experiment are chosen from existing natural groups.


The differences become a confounding variable that provides an alternative explanation for the results.

Control series design:

Interrupted time series design with a control group

Interrupted time series design:

Examine a series of observations before and after a treatment and look for a change in behavior

EX; Do daily quizzes increase attendance?

Increase in attendance from one day to the next could be random variability

Instead, record attendance many days before and after introducing quizzes.

Single case experimental designs

NOT case studies
aka single-subject designs

• Traditionally used in studies of reinforcement and behavior modification
• Researcher manipulates an IV (unlike case study)
• Behavior recorded during baseline period-description of behavior as it exists and as it would be in the future without introduction of a treatment
• Behavior is recorded after treatment is introduced

Reversal design (or ABA, or ABAB,...):

A\= baseline period (no treatment)
B\= treatment period


his basic reversal design is called an ABA design; it requires observation of behavior during the baseline control (A) period, again during the treatment (B) period, and also during a second baseline (A) period after the experimental treatment has been removed.

Example of reversal design

For example, the effect of a reinforcement procedure on a child's academic performance could be assessed with an ABA design. The number of correct homework problems could be measured each day during the baseline. A reinforcement treatment procedure would then be introduced in which the child received stars for correct problems; the stars could be accumulated and exchanged for toys or candies. Later, this treatment would be discontinued during the second baseline (A) period.

Horlon (1987)

EX OF ABA (REVERSAL) DESIGN; effects of facial screening on spoon-banging behavior in mentally-impaired 8-year-old girl. Treatment involved placement of soft cloth over the face of the girl; which caused her to spoon banging

Multiple baseline design:

Measure baseline in several situations (e.g. aggressive behavior at home, school, and daycare) \= multiple baseline across situations

Introduce treatment at different times in the different situations

Evidence for treatment effectiveness is that behavior changes only when the treatment is introduced.; To demonstrate the effectiveness of the treatment, such a change must be observed under multiple circumstances to rule out the possibility that other events were responsible.

Types of Multiple Baseline Designs

across subjects, across behaviors, across situations

Multiple baseline across situations

Measure baseline in several situations (e.g. aggressive behavior at home, school, and daycare) \= multiple baseline across situations

Introduce treatment at different times in the different situations

Multiple baseline across subjects:

Measure behavior of several subjects over time; introduce treatment at different times for different subjects

Multiple baseline across behaviors:

Measure different behaviors of a single subject over time; introduce treatment (e.g., rewards) at different times for different behaviors

Problems that even true experiments may not control:

1. Contamination
2. Experimenter expectancy effects/Observer bias
3. Reactivity

experimenter expectancy effects/observer bias

actual change in the behavior of the people or nonhuman animals being observed that is due to the expectations of the observer

tendency of observers to see what they expect to see