CRAM TIME BABBYYY (statistics)

What are the three Kolmogorov Axioms of probability?

1) $P(A) \geq 0$

2) $P(\Omega) = 1$

3) For disjoint events

$P(\cup Ai) = \sum P(Ai).

Define Conditional Probability $P(A|B)$

$P(A|B) = \frac{P(A \cap B)}{P(B)}$

State the Law of Total Probability

$P(A) = \sum{i} P(A|Bi)P(B_i)$

What is the formula and meaning of Bayes' Theorem?

$P(Bj|A) = \frac{P(A|Bj)P(Bj)}{\sum P(A|Bi)P(B_i)}$.

When are two events Independent?

When $P(A \cap B) = P(A)P(B)$

Difference between Mutually Exclusive and Independent events

Exclusive means they cannot happen together ($P(A \cap B) = 0$). Independent means one doesn't affect the other.

Define the Cumulative Distribution Function (CDF) $F_X(x)$

$F_X(x) = P(X \leq x)$. It is non-decreasing and ranges from 0 to 1.

What is the Probability Density Function (PDF) $f_X(x)$?

For continuous variables

Define the Expected Value (Mean) $E[X]$

$\int x f_X(x) dx$ (continuous) or $\sum x P(X=x)$ (discrete).

Define Variance $Var(X)$ and its shortcut formula

$Var(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2.

What is Standard Deviation $\sigma$

$\sigma = \sqrt{Var(X)}.

What is the Coefficient of Variation ($CV$)

$CV = \frac{\sigma}{|E[X]|}.

Bernoulli Distribution $Ber(p)$

Single trial: Success (1) with prob $p$

Binomial Distribution $Bin(n

p)$

Poisson Distribution $Poi(\lambda)$

Number of events in an interval with rate $\lambda$. $E[X] = Var(X) = \lambda$.

Geometric Distribution $Geo(p)$

Number of trials until the first success.

Uniform Distribution $U(a

b)$

Normal (Gaussian) Distribution $N(\mu

\sigma^2)$

ShutterstockStandard Normal Distribution $Z$

$Z \sim N(0

Exponential Distribution $Exp(\lambda)$

Time between events in a Poisson process. $E[X] = 1/\lambda$.

What is the Memoryless Property?

$P(X > s+t | X > s) = P(X > t)$. It means the remaining time until an event does not depend on how much time has passed.

What is a Marginal Distribution?

The distribution of one variable found by integrating/summing out others from the joint PDF/PMF.

Define Covariance $Cov(X

Y)$

Define Correlation $\rho_{XY}$

$\rho_{XY} = \frac{Cov(X

Independence of two variables $X

Y$

What is $E[aX + bY]$?

$aE[X] + bE[Y]$. (Always true).

What is $Var(aX + bY)$ for Independent variables?

$a^2Var(X) + b^2Var(Y).

State the Central Limit Theorem (CLT)

The sum of $n$ i.i.d. variables tends to a Normal distribution $N(n\mu

What is the Standard Error of the mean?

$SE = \frac{\sigma}{\sqrt{n}}.

What is a Point Estimator $\hat{\theta}$

A statistic used to estimate an unknown population parameter $\theta$.

Define Unbiasedness (Estimador não enviesado)

$E[\hat{\theta}] = \theta$.

Define Bias (Viés)

$E[\hat{\theta}] - \theta$.

Define Mean Squared Error (MSE)

$MSE(\hat{\theta}) = Var(\hat{\theta}) + [Bias(\hat{\theta})]^2.

What is the Likelihood Function $L(\theta)$?

The joint probability of the data viewed as a function of the parameter $\theta$.

What is the Maximum Likelihood Estimator (MLE)?

The value of $\theta$ that maximizes $L(\theta)$.

What is a Confidence Interval (CI)?

A range of values that has a probability $1-\alpha$ of containing the true population parameter.

What is a Pivotal Quantity?

A random variable that depends on the data and the parameter

When do we use the Student's t-distribution?

When testing the mean of a Normal population but the variance $\sigma^2$ is unknown and must be estimated from the sample.

What is the effect of Sample Size ($n$) on CI width?

Larger $n$ results in a narrower (more precise) interval

What is the CI for a Proportion $p$

$\hat{p} \pm z_{1-\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}.

What is the Null Hypothesis ($H_0$)

The status quo or the assumption of "no effect." We only reject it if there is strong evidence.

Define Type I Error ($\alpha$)

Rejecting $H_0$ when it is actually true (False Positive).

Define Type II Error ($\beta$)

Failing to reject $H_0$ when it is actually false (False Negative).

What is the Significance Level ($\alpha$)

The maximum probability of committing a Type I Error that we are willing to accept.

What is the p-value?

The probability of seeing data as extreme as ours

What is the Power of a test?

$1 - \beta$. The probability of correctly rejecting a false $H_0$.

What is a Rejection Region ($W$)

The set of values for the test statistic that lead to rejecting $H_0$.

What is Linear Regression?

A model that describes the relationship between a dependent variable ($Y$) and independent variables ($x$) using a straight line.

What are Residuals?

The difference between observed values and the values predicted by the model: $ei = yi - \hat{y}_i$.

Define the Least Squares Method

Picking estimates that minimize the sum of squared residuals: $\sum (yi - \hat{y}i)^2$.

What does the Coefficient of Determination ($R^2$) represent?

The proportion of the variance in $Y$ that is predictable from $x$.

What does a Slope ($\beta_1$) of zero mean?

It means there is no linear relationship between $x$ and $Y$

$x$ does not help predict $Y$.

What is Homoscedasticity?

The assumption that the variance of the residuals is constant across all values of $x$.

What is the distribution of the Sample Mean $\bar{X}$?

If the population is $N(\mu

State the Law of Iterated Expectations

$E[X] = E[E[X|Y]].

What is the distribution of the Sample Variance $S^2$?

For a Normal population

State Chebyshev's Inequality

$P(|X - \mu| \geq k\sigma) \leq 1/k^2$.

What is a Quantile of order $p$

The value $xp$ such that $P(X \leq xp) = p$.

Explain the Correction for Continuity

Adjusting boundaries by $\pm 0.5$ when using a continuous distribution (Normal) to approximate a discrete one (Binomial/Poisson).

What is Fisher Information?

A measure of how much information an observable random variable carries about an unknown parameter.

What are Degrees of Freedom ($df$)

The number of independent pieces of information in a statistic.

Significance vs. Power

Significance ($\alpha$) limits false alarms (Type I)

Power ($1-\beta$) is the ability to find a real effect.

Independence vs. Uncorrelated

Independence implies uncorrelated ($\rho=0$). Uncorrelated does not necessarily imply independence (could be non-linear).

What is the Standard Normal Density formula?

$\phi(z) = \frac{1}{\sqrt{2\pi}} e^{-z^2/2}.

Expected Value of a function $E[g(X)]$

$\int g(x) f_X(x) dx$ or $\sum g(x) P(X=x).

Chi-square Goodness-of-Fit Test

A test to see if observed sample frequencies follow a specific theoretical distribution.

Total Variance Decomposition in Regression

$SST = SSR + SSE$ (Total variation = Explained variation + Error variation).

What is Adjusted $R^2$?

A modification of $R^2$ that accounts for the number of predictors