Stats Ch. 8 - Hypothesis Testing: significance, effect size, and power

population mean

in inferential stats, we use sample means to estimate the what?

equal the value

we expect the value of the sample mean to what with the value of the population mean?

the less probable it is that the population mean is correct

what happens when there is a larger difference b/w the sample mean and the population mean?

hypothesis

a statement or proposed explanation for an observation, phenomenon, or a scientific problem that can be tested using the research method

hypothesis testing or significance testing

a method for testing a claim or hypothesis about a parameter in a population, using data measured in a sample

to determine the probability that a sample statistic would be selected from a population if the hypothesis regarding the population mean were true

what do we use hypothesis testing for?

state the hypothesis - null and alternative hypothesis

step 1 of hypothesis testing

null hypothesis (H0)

a statement about the population parameter that is assumed to be true
A statement of "no difference."

null hypothesis (H0)

in hypothesis testing, we test whether this value should be retained or rejected

alternative hypothesis (H1)

a statement that contradicts the null hypothesis by stating that the value of the parameter is less than, greater, or not equal to the value stated in H0

set the criteria for a decision

step 2 of hypothesis testing

we first set criteria for making a decision about the null hypothesis

to determine whether the null hypothesis should be retained or rejected, what do we do?

we state the level of significance for a test, which is typically set at .05 for behavioral research

how do you set criteria for making a decision about the null hypothesis

when the p-value is less than or equal to .05/5%
when the probability of obtaining a sample mean would be less than 5% if the null hypothesis were true - sample data favors the alternative hypothesis and the data provide strong enough evidence to conclude that it is likely incorrect

when do we reject the null hypothesis?

the null must go!

when the p-value is low, ___

compute the test statistic

step 3 of hypothesis testing

to evaluate the likelihood of obtaining a sample mean of a certain value if the null hypothesis is correct

we use a test statistic to what?

the number of SDs by which the sample mean deviates from the population mean stated in the null hypothesis

what does the test statistic tell us?

make a decision

step 4 of hypothesis testing

test statistic

we use what to make a decision regarding the null hypothesis?

on the probability of obtaining the sample mean, given that the null hypothesis is true

we base the decision on what?

if the probability of obtaining the sample mean would be less than or equal to 5% if the null hypothesis were true.
-we conclude the value stated in the null hypothesis is wrong

reject the null hypothesis when?

if the probability of obtaining the sample mean would be greater than 5% if the null hypothesis were true.
-we conclude there is insufficient evidence to reject the null hypothesis (does not mean it is correct)

retain the null hypothesis when?

prove

it is not possible to _____ the null hypothesis

p value, varies b/w 0 and 1

the probability of obtaining a particular sample mean is stated as a what?

the criteria set in step 2 (level of sig)

we compare the p value to what?

less than or equal to .05 and the test statistic reached statistical significance.

if the null hypothesis is rejected, then the criterion or p value was what?

1. retain the H0 correctly: null finding
2. retain the H0 incorrectly: "false negative"
3. reject the H0 correctly: power
4. reject the H0 incorrectly: "false positive"

The decision regarding the null hypothesis can be correct or incorrect. There are 4 possible decision alternatives regarding the truth or falsity of the decision made. What are they?

null finding

H0 is correct and we correctly decide to retain the H0
(no difference, not related)

type II error (beta)

"false negative"; H0 is false, but we incorrectly retained the H0

The power of the decision-making process is

significant result: H0 is false and we correctly reject the H0

Type I error (alpha)

"false positive"; H0 is true, but we incorrectly rejected the H0

stating an alpha level

how do reserachers control for type I error?

alpha level

the level of significance or criterion for a hypothesis test

one-sample z test

a statistical procedure used to test hypotheses concerning the mean in a single population with a known variance

-evaluate 1 of 3 alternative hypotheses described in step 1
-follow 4 steps of hypothesis testing

what do we need to do to conduct a z test?

step 1 for one-sample z test

state hypotheses
-state H0 & H1
-nondirectional tests - H1 is stated not = the value in H0
-directional tests - H1 is stated greater or less than value in H0

step 2 for one-sample z test

set criteria for a decision
-level of sig is .05, then alpha = .05
-rejection region in tail of z distribution
-nondirectional - split alpha (.05) into two tails
-directional - rejection region in upper or lower tail

step 3 for one-sample z test

compute the test statistic
-test statistic = z statistic or obtained value
-compare obtained values to critical values

step 4 for one-sample z test

make a decision
-reject H0 if obtained value exceeds either critical value; otherwise retain H0

type III errors

for one-tailed tests, this can occur when we retain the H0, not because the effect did not exist, but because we place the rejection region in the wrong tail

the size of the effect
different measures are used to determine effect size

hypothesis testing is used to determine whether an effect exists in the population, but it does not tell you what?

effect size

a statistical measure of the size of an effect in a population

cohen's d

It takes the difference between two means and expresses it in standard deviation units. It tells you how many standard deviations lie between the two means.

Small, medium, and large effect sizes

Cohen's d effect size conventions are standardized rules for identifying what?

that an effect is greater than or less than the hypothesized population mean

the sign of d indicates what?

small effect size

d = 0.2

medium effect size

d = 0.5

large effect size

d = 0.8

effect size increases, power increase

relationship b/w effect size and power

reject the H0

effect size increases, power increase:
the larger the difference b/w the sample mean and the hypothesized population mean stated in the H0, the more likely we are to what?

increasing sample size will increase power

the relationship b/w sample size and power

1. increase effect size
2. increase sample size
3. increase alpha level
4. decrease population SD
5. decrease population standard error

what 5 things increase power?

increase power

likelihood of rejecting the H0

decreasing the beta (likelihood of committing a Type II error)

decreasing what will increase alpha and thus increase power?

1. test statistic
2. p value (report p value as > or < .05)
3. effect size
4. include figure or table to illustrate significant effect and its effect size

what do we include when reporting the result of a one-sample z test?