Hypothesis Testing and Statistical Inference

Alternate Hypothesis (Ha or H1)

- AKA experimental hypothesis
- Statement that the population parameter has a value that differs from the null hypothesis

Null Hypothesis (H0)

the statistical hypothesis tested by the statistical procedure; usually a hypothesis of no difference or no relationship

- Accept null
- Accept Alt
- Reject Alt

For hypotheses we never ..

- Reject null
- Retain (or fail to reject) null

For hypotheses we ONLY ...

1. State the hypothesis
2. Set the significance level (alpha)
3. Collect the sample data
4. Calculate the test statistic (p-value)
5. Make a decision based on p-value & alpha
6. Draw the conclusion about the population

What are the steps of hypothesis testing?

0.05 or 0.01

-Typically, alpha is set at __________________, but remember, your rejection region changes depending on if your hypothesis is directional or non-directional

Non-directional (2-tailed)

- H0: There will be no change in pain with physical therapy treatment

- HA: There will be a change in pain with physical therapy treatment

Directional (1-tailed)

- H0: There will be no change in pain with physical therapy treatment

- HA: There will be a decrease in pain with physical therapy treatment

Population

The complete collection of elements to be studied. The group to which the results of research are intended to be generalized.

Sample

A subset of elements drawn from a population to draw conclusions or make estimates about the larger population.

Sampling error

The chance difference between the statistic calculated from a sample and the true value of the parameter in the population.

Sampling with replacement

A sampling method in which each unit selected for the sample is put back into the population after selection before the next unit is drawn.

Sampling without replacement

A sampling method in which each unit selected for the sample is not put back into the population after selection before the next unit is drawn.

- Stratified
- Cluster
- Systematic
- Simple Random Sampling

What are the types of probability sampling procedures?

Stratified sampling procedure

Individuals are selected from pre-identified subgroups based on some characteristic

Cluster sampling procedure

The population is divided into clusters or areas (usually along geographic boundaries) and a random sample of the clusters is selected. Then, all of the units in the selected samples are measured.

Systematic sampling procedure

Individuals are selected at random intervals (every 4th person on the list)

Simple Random Sampling

All individuals in the population have an equal chance of selection

- Convenience sampling
- Purposive sampling
- Quota sampling
- Snowball sampling

What are types of non-probability sampling procedures?

Convenience Sampling

Sample is selected from subjects who are convenient or readily available to the researcher

Purposive Sampling

Subjects deliberately selected based on predefined criteria chosen by the investigators

Quota Sampling

After identifying subgroups or strata of interest, convenience sampling is used to select the required number of subjects from each stratum

Snowball Sampling

Subjects are identified by asking existing subjects to identify the names of other potential participants. Often used when characteristic of interest is rare or hard to find subjects with characteristic otherwise

- statistical analysis
- descriptive statistics & inferential statistics

- Step 4: You will need to determine the appropriate __________________________ and conduct that analysis on your data to determine if you can reject or fail to reject the null hypothesis.

- What are the two different categories of statistics?

- likelihood
- alpha level; p-value
- hypotheses

Step 5 – Compare the P-Value to the Alpha Level:

- The _________________ that any one event will occur, given all possible outcomes.

- In stats we compare the probability of obtaining our results due to chance (__________________) to the probability of obtaining test results at least as extreme as the result actually observed (_______________)

- You’ll use these probabilities to evaluate your ______________________ and make a decision

- impossible
- unlikely
- equally likely as unlikely
- likely
- certain

Probability Ratios:

- 0 = ____________________

- 0.25 = _________________

- 0.50 = ____________________

- 0.75 = _________________

- 1 = _________________

1. Reject the null hypothesis -> this is GOOD (usually)!
2. Do not reject the null hypothesis

Testing will always result in one of two decisions ...

- SIGNIFICANT EFFECT
- SIGNIFICANT DIFFERENCE
- chance alone

- To say you reject the null hypothesis infers that a _______________________ exists
- Aka there is a statistically _______________________________- Aka the difference observed is not likely due to _____________________

- A priori set alpha level
- 0.05
- 95%
- 0.01

- __________________________ Level of significance OR probability of obtaining results due to chance
- Alpha typically ____________ (there is less than a 5% chance that the stat obtained or more extreme occurred under the null)
- ___________ confident in decision
- Medical field often uses alpha = ____________ (Less than 1% chance our result occurred under null)

- reject
- Type I error
- true
- false
- Type II error

- The alpha level indicates that we will ______________ H0 with an outcome that falls in the top 5% of the distribution
- The alpha level is ____________________. It is the probability of rejecting H0 when H0 is ___________
- The HA distribution represents the distribution if H0 is ________________
- ______________________: The area that represents failing to reject the H0 when H0 is false

- power
- size
- alpha level
- N

- __________________ = The probability of correctly rejecting H0

Ways to increase power:

- Increase the ___________ of the effect (administer a larger dose of medication)

- Increase your ____________________ (with caution)

- Increase your _________ (decreases the standard deviation of the sampling distribution)

- alpha value
- P value
- You reject the null hypothesis

- _______________________ = Probability of making a Type I error

We pick alpha

- ____________________ = Probability of significance

You calculate this, all statistical tests have P-value

- If the calculated P-value is less than the alpha = _____________________________

- Type I error
- correcting data

- ________________________ = Concludes that a significant difference exists when it really doesn't!
- Fixed by "________________________" = This means you keep lowering the P-value threshold based on how many tests you run

- difference
- REALLY EXISTED
- beta
- 0.2; 0.8
- Alpha affects power, which is 1 - beta

- Power of a statistical test = Ability of a test to find a _________________ when it exists- Opposite of Type I error = Failure to find significance when it _________________________
- _____________ = Probability of making a Type 2 error
- Typically, __________ for beta and __________ for power (Beta + Power = 1)
- Why is Type 2 error margin so much bigger than type 1?

68%, 95%, 99.7%

1 SD from mean, 2 SD from mean, 3 SD from mean

Sensitivity

- Test's ability to identify an individual with a disease as positive

- Example: If 100 patients known to have a disease were tested, and 43 test positive, then the test has 43% sensitivity.

Specificity

- Test's ability to identify an individual without a disease as negative

- Example: If 100 with no disease are tested and 96 return a completely negative result, then the test has 96% specificity.

Negative Predictive Value

The ability of a diagnostic test to correctly determine the proportion of patients without the disease from all the patients with negative test results.

Positive Predictive Value

The ability of a diagnostic test to correctly determine the proportion of patients with the disease from all the patients with positive test results.

In a test with high specificity, POS test rules IN a diagnosis

What does SpPin mean?

In a test with high sensitivity, NEG test rules OUT a diagnosis

What does SnNout mean?

- No!
- Sensitivity tells us the frequency in which a test is positive for patients who we know have the disease- Specificity tells us the frequency in which a test is negative for patients who we know DON'T have the disease

Are Specificity and Sensitivity the Best Measure of Test Accuracy?

Likelihood ratio

- Tests the utility of a diagnostic test
- Assesses how likely it is that a patient has a condition
- Probability that a test result is correct to the probability that a test result is incorrect

Sensitivity and Specificity to calculate

What does the likelihood ratio require?

= Sensitivity/(1 -Specificity)

What is the equation for a positive likelihood ratio?

= (1 - Sensitivity)/Specificity

What is the equation for a negative likelihood ratio?

- They are not impacted by the prevalence of the disease -> Specificity and sensitivity are impacted by disease prevalence
- They are intuitive
- They assess how good a diagnostic test is, and aid in selecting tests or sequences of test
- Can incorporate multiple test results
- Can use to calculate post-test odds and probability

Why use likelihood ratios?

"Probability of an individual with the condition having a positive test" (True positive) /"probability of an individual without the condition having a positive test” (False positive)

What does positive likelihood ratio in reality mean?

"Probability of an individual with the condition having a negative test" (False negative) /"probability of an individual without the condition having a negative” (True negative)

What does negative likelihood ratio in reality mean?