1/108
Looks like no tags are added yet.
Point Estimate
The estimate of units sold is a one-number "best-guess" units sold.
What is the goal with statistical analysis?
To reduce it where you can quantify it always.
Sampling Distribution
-Allows us to quantify uncertainty using just one sample.
-It is a theoretical distribution that represents the possible values of a statistic calculated from multiple samples.
Sampling Distribution of the Sample Mean
-the distribution of all possible sample means of size n from the sample population.
-is a theoretical probability distribution.
Central Limit Theorem (CLT)
For any population, the sampling distribution of the sample mean is approximately normal if the sample size n>30.
Based on CLT, the sampling distribution of the same mean is -?
-Normally distributed
- With mean of μ -Standard deviation of ⁄√𝜎 𝑛
What does ignoring dependencies in the data lead you to?
Underestimate uncertainty.
The null hypothesis (H0)
The belief that has been around for some time (the status quo) will be called the null.
H0: μ = 100
H0: β = 0
H0: μ1 - μ2 = 0
The alternative hypothesis (Ha)
What we want to prove.
HA: μ ≠ 100
HA: μ < 100
HA: μ > 100
Let's say a researcher has reason to believe the population mean is different from what has been expected....
-They set a hypothesis.
-They collect data
-They make use of a test statistic and p-value as evidence to reject H0.
-They make a decision between H0 and Ha and reject or fail to reject the H0.
Test Statistic
Measures the distance (the difference measured in units of errors) between the null hypothesis and your parameter estimates.
The correlation of data with a lagged version of itself is called
Autocorrelation
Suppose you have a time series model and the lag1 coefficient is B=1.2. What type of series do you have?
Diverging
Which of the following is the most useful type of series for modeling?
Mean-Reverting
Use logistic regression when
The response is binary.
What is a sampling distribution?
The distribution of a statistic.
If your sample size is n=250, what is the approximate shape of the distribution for the sample mean?
Normal
Dependencies in the data often lead to an
Underestimate of uncertainty.
How do you estimate the standard error of the sampling distribution using a bootstrap?
calculate the standard deviation of the bootstrap estimates
Which of the following is a pre-specified cut-off to determine statistical significance for a hypothesis test?
significance level
In hypothesis testing, a low p-value means
a rare result if the null were true
Increasing the sample size will
increase the chance of finding statistical significance
If a researcher collects more data or tries different analyses just to achieve statistical significance, this is known as
p-hacking
Suppose the p-value reported for a beta coefficient in summary.glm is
the predictor's coefficient is statistically different from 0
What are the 2 types of frequentist standard errors we learned?
Theoretical & bootstrap
The bootstrap is used to
estimate sampling distributions
If you set a seed when running a bootstrap, you will get
the exact same results if you run it several times
What is the Achilles' heel of the bootstrap?
when you have dependent observations
What is a benefit of increasing B, the number of bootstrap samples, in a bootstrap?
it will decrease Monte Carlo error
Which of the following is a way to allow subjective beliefs to be incorporated mathematically into a probability calculation?
Bayesian statistics
Using a very large set of predictors in a regression model will likely lead to
poor performance of the model on new data
If you have taken the log of the response variable in regression and use the regression to make a prediction, it's important to
use the exp() function on the predicted value from the predict() function
Which of the following is the correct interpretation for PPV (positive predictive value) of a medical test for COVID?
the chance you have COVID if you test positive
What method uses penalized deviance minimization?
regularization
What does the "free" argument accomplish?
it allows certain coefficients to be unpenalized
The vertical line in the plot indicates
the model chosen using the AICc (the corrected AIC), not the BIC
frequentist uncertainty
if we were to repeatedly draw samples of the same size from the same population, we might obtain slightly different sample means each time.
A trade group states that back-to-school spending averages $606.40 per family. If this is wrong, a different economic model is needed. Is there evidence the average spending is higher this year? State the hypotheses.
•H0: μ = $606.40
•HA: μ > $606.40
"If the P is low......"
the null must go
In hypothesis testing, a common way to use p-values is to compare them to a
significance threshold α
-commonly α = 0.05 or α = 0.01
The symbols like *** or * that you see next to the p-value in the R output indicates whether
each coefficient is significant at different thresholds.
p-hacking
where people look at many tests and report only those that are significant which have a p-value less than some cut-off, usually 0.05.
What does p-hacking lead to?
a high proportion of false discoveries in the published literature
How close is a sample statistic to a population parameter?
The statistic from our sample is probably off by some levels of error.
Point Estimate +/- Margin of Error
Confidence level
the probability that the true population parameter falls within the interval.
•Ex: 95 CI: 95% confidence level, means we want to be right 95% of the time
•α= 1- 95%
Margin of Error
critical value*se
$fit
1
33326
Point estimate
$se.fit
[1] 1330.901
Standard Error
> certpredfit+c(−2,2)∗certpredse.fit
[1] 30664.2 35987.8
What does this tell us?
We are 95% confident that the average sales price of certified used cars is between $30,664.20 and $35,987.80.
Bootstrap
•uses resampling with replacement from your current sample to mimic the sampling distribution.
•It allows you to create a version of the sampling distribution on your computer, so that you can draw from it as needed to understand the properties of your analysis.
Using set.seed will allow the user to
get the exact same sequence of random numbers generated afterwards each time. necessary if, for example, you need to replicate the exact sample to test the impact of code changes.
Monte Carlo variation
The variability in the bootstrap estimates
When choosing this sample size, you are trading between
computation time and precision.
-Larger B decrease Monte Carlo variation and increases computation time
nonparametric bootstrap
does not rely on any assumptions about the underlying distribution of the population.
Bootstrap provides..
robust standard errors. it estimates the sampling distribution for your estimator of interest.
getBeta
produces your statistic of interest.
statistical bias
the difference between the expectation of your estimator and the expectation of the target that you are trying to estimate.
Bootstrapping Regression Model
Advantage: The bootstrap doesn't make any parametric modeling assumptions and thus provides standard errors that are robust to many sources of mis-specification such as nonconstant error variance.
Disadvantage: The bootstrap takes more computation time.
Achilles' heel
the bootstrap makes no parametric modeling assumptions, but it has an Achilles' heel: it relies on an assumption of independence between observations.
How to fix Achilles' heel
instead of resampling individual observations, you resample blocks of observations that are potentially correlated with each other.
percentile bootstrap confidence interval
to calculate the bounds directly on your sample of bootstrap estimates.
Unbiased estimator
the expected value of the estimator is equal to the true value of the parameter being estimated.
Bias-adjusted
difference between the original price estimate and the estimated bias
Bayesian uncertainty
characterizes probabilities over models and parameters by appealing to the idea of subjective beliefs rather than repeated trials.
How to solve a linear regression problem?
Model (construct a linear equation model), Estimate the unknown coefficients, Predict (making an educated guess using our formula)
An underfit in the model is said to have
high bias
An overfit model is said to have
high variance
Out-of-sample data (test data)
is new data that your model hasn't seen before and needs to make predictions on.
We can use the gamlr library to run our lasso regression to predict..
fail from the inputs.
Sparse model matrix
Does not store any 0's to save on memory, since in many applications, your input matrix will contain mostly 0's.
lambda
the sequence of fitted λt penalties
alpha
the sequence of fitted intercept terms
beta
the sequence of fitted regression coefficients
In gamlr, the "free" argument lets you
exclude certain columns from being penalized in the regression fit by specifying their column indices or names
AIC
is an attempt to approximate the average OOS deviance on new data.
The AIC is the most commonly applied IC, but....
you should not use the AIC.
The BIC can be useful for..
model selection in small sample settings.
-it tends to underfit and choose overly simple models
drop(yhat)
removes sparse formatting
exp(drop(yhat))
exponentiate to get predicted prices.
When is a log-log model appropriate?
When the response and explanatory variable move multiplicatively together.
When should you use logistic regression?
Binary response (yes/no, 1/0, true/false)
Why do you need to exp() to predictions that are on the log scale?
-Because e^log(y) = y
-Puts prediction back on scale the data were measured in
What's the difference between frequentist statistics and Bayesian statistics?
-Frequentist relies on thought experiment of repeatedly sampling from a DGP.
-Bayesian incorporates prior (subjective) beliefs to update probability.
Se.fit=T
gets standard errors
Regularization
-Uses penalized deviance minimization
-Penalizes for complexity (number of predictors)
-Provides paths - sets of candidate models
Overfit vs underfit
-Overfit is high variance
- Underfit is high bias
How do you compute a bootstrap confidence interval?
use percentiles.
What B is and what it means to have a smaller versus larger B?
-B = number of bootstrap samples
-Bigger B lowers Monte Carlo variation
Why not always have a really large B?
Computational time
What is Set.seed used for?
to ensures someone can replicate the results of a procedure that uses random sampling
monte Carlo method
Uses repeated sampling to estimate sampling distribution
"95% confidence" versus "95% probability"
-Technically, once you compute the CI, it no longer has a 95% chance of containing the parameter (it either happened or it didn't)
-However, the procedure is such that if you compute many confidence intervals 95% of them will contain the parameter
What does the sample size do to the p-value?
Easier to achieve statistical significance (get a low p-value)
practical versus statistical significance
Statistical significance is low p-value
Significance level
A pre-specified cut-off for how low the p-value needs to be to say you think the alternative is true
Low vs High P-value
Low p-value means you observed a rare result if the null were true
-High p-value is a common result if the null is true
What does dependence among observations do to SE?
Makes SE smaller than it should be
What is a confidence interval estimating?
The true population parameter
Prediction intervals are wider than confidence intervals because:
They include variability of individual observations
What does set.seed() do?
Makes results reproducible