Module 7 - Statistical Significance and Hypothesis Testing

Inferential Statistics

Make conclusions about the population based on the data from the sample

Descriptive Statistics

Organizes, summarizes, and describe the data without interpretation

Significant

This means that it was probably due and not due to chance

Highly significant

In research, the result means it is very probably true. it doesn’t necessarily mean that it is highly important or that the clinicians should definitely do this

Statistical significance

This means that we are saying that there is a difference and the difference is true and didn’t occur by chance

Practical significance / effect size

This is telling you how big the difference is or the magnitude

Validity of an experiment

Statistical significance is the mathematical measure of what?

The people in your study performed similarly

If the results are narrow, what statistical significance is it?

Statistical significance

The measurement of the difference between two sets of data

Why we think that there is the difference and for us, we want to be able to say that the difference is due to the IV

When we are looking at statistical significance, what are we looking at explaining why?

Systematic variance

You see everyone in the experimental side of the study is performing similarly

• Sampling error

• Measurement error

• Individual differences

What are the 3 biggest unsystematic variances that we try to control for?

Sampling error

There was some sort of problem with getting people in our sample

Measurement error

We know that no test instrument or method of assessment that is measuring the DV is perfect

Individual differences

We can’t control for every single individual difference

Hypothesis

This is developed before the study begins and is a statement that describes the relationship between the 2 variables

• By looking at previous studies

• Theories that are out there

How do we predict the relationship between 2 variables?

Known extraneous variables

What are systematic variances due to?

Systematic variances

These are due to known extraneous variables

Unsystematic variables

What is due to the measurement / sampling error as well as individual differences?

• Measurement / sampling error

• Individual differences

What are unsystematic variances due to?

Amount of errors that we have in the study

What is our whole focus in research of trying to minimize?

• State the hypothesis

• Set the level of risk

• Choose the sample size

• Determine the critical value

• Compute the test statistic

• Reject or accept the hypothesis

What is the hypothesis testing process?

Null

What means zero?

Null Hypothesis

This is a statement that is the opposite of what the researcher expects. In other words, it states that there is no relationship that exists between variables or no difference will be found between experimental treatments

Researcher’s expectations

What does the null hypothesis not reflect?

Statistical non significance

Is the null hypothesis statistical significant or nonsignificant?

Alternative hypothesis

The statement of the researchers expected results that there was a relationship between the two variables

• One-tailed (directional)

• Two-tailed (nondirectional)

What are 2 types of alternative hypotheses?

One-tailed (directional)

• Predicts the direction of the results (i.e., treatment A will improve skills when compared to treatment B)

• May say if there is a bigger increase and there will be a positive change

• This is considered a more rigorous alternative hypothesis as you are more certain of what the direction will be

One-tailed (directional)

Between one tailed and two-tailed alternative hypothesis, which is more rigorous?

Two-tailed (nondirectional)

• Does not predict the direction of the results (i.e., students will respond differently to treatment A when compared to treatment B)

• Won’t predict which treatment is better

Stressful situations and stuttering are not related (i.e., when people who stutter are put in stressful situations, their stuttering does not increase)

What is an example of null hypothesis?

There is a cause-and-effect relationship between stressful situations and stuttering (i.e., stressful situations cause people to be more dysfluent)

What is an example of an alternative hypothesis?

Consequences of making an error

How do you determine the level of risk?

• The null hypothesis is accepted when it is true (correct decision)

• The null hypothesis is rejected when it is false (correct decision)

• The null hypothesis is rejected when it is true (Type I Error)

• The null hypothesis is accepted when it is false (Type 2 Error)

What are 4 outcomes when you test the research hypothesis?

The null hypothesis is rejected when it is true

Type 1 Error

Type 1 Error

The null hypothesis is rejected when it is true

Type 2 Error

The null hypothesis is accepted when it is false

The null hypothesis is accepted when it is false

Type 2 Error

The null hypothesis is rejected when it is true (type 1 error)

What outcome when testing the research hypothesis is the more grievous of the mistakes to make?

False Positive

Is a type 1 error a false positive or false negative?

False Negative

Is a type 2 error a false positive or false negative?

A Priori

Before the study

Before the study

When is the sample size suppose to be chosen?

More powerful the test

What does a larger sample size mean in regards to the power of the test?

Higher

Does a smaller sample size have a higher or smaller risk for errors or unsystematic variances?

Type 2 Errors

Are small sample sizes at risk for type 1 or 2 errors?

Critical values

This is going to be different based on your alpha or confidence level of the trueness of the study

Alpha or confidence level that you have the trueness in the study

What is the critical value based off of?

0.05

What p value do you want as your cut-off number? In other words, what decimal do you want that relates to being 95% sure / confident?

• Hypothesis

• Distribution of the sample

• Sample size

You must choose a test statistic based on these things.

Normally distributed

Are parametric tests considered for normally distributed or distributed free?

Distribution Free

Are non parametric tests considered for normally distributed or distributed free?

Non-parametric tests

Can be used in experimental research and does not depend on the population being normally distributed

Parametric

Based on the normal distribution of a variable and is used in experimental research

• Interval

• Ratio

What 2 levels of data are used for parametric tests?

Ordinal

What level of data is used for non parametric tests?

• Hypothesis

• Distribution of the population

• Sample size

• Other sample characteristics

What are some characteristics that the choice of the statistic test depends on?

You cannot reject the null

What happens if the test statistic does not exceed the critical value?

Effect size

What can clinical significance be determined by?

Reject the null

What can you do to the null if the test statistic exceeds the critical value?

When the test statistic exceeds the critical value

When can you reject the null and support your alternative hypothesis?

If the test statistic does not exceed the critical value

When can you NOT reject the null?