3 Statistics probability and distances

What is the sample mean?

x_bar = sum(x_i) / n.

What is the sample median?

The middle ordered value, or the average of the two middle values for an even sample size.

When is the median more robust than the mean?

When extreme observations or strong skewness would disproportionately affect the mean.

What is the sample variance?

s² = sum((x_i - x_bar)²) / (n - 1), for n > 1.

What is the sample standard deviation?

The square root of sample variance; it measures spread in the original measurement units.

What is the interquartile range?

IQR = Q3 - Q1, the spread of the middle 50% of ordered values.

What do covariance and Pearson correlation describe?

Linear co-variation; correlation standardizes covariance by the two standard deviations.

What is Pearson correlation's range when defined?

From -1 to 1.

Does zero Pearson correlation establish independence?

No. Variables can have a nonlinear dependence with zero linear correlation.

Does correlation establish causation?

No.

Which plot is appropriate for the distribution of one numerical variable?

A histogram or density plot; a boxplot gives a compact summary.

Which plot helps inspect the relationship between two numerical variables?

A scatter plot.

Why should numerical summaries sometimes be computed within subgroups?

Pooling distinct groups can hide or create apparent patterns relative to within-group behavior.

What does multidimensional scaling aim to preserve?

Pairwise dissimilarities in a lower-dimensional representation as closely as its criterion permits.

Why can a two-dimensional MDS plot misrepresent cluster separation?

Projection distorts some distances and can hide structure present in higher dimensions.

What is Euclidean distance?

d(x,y) = sqrt(sum_j (x_j - y_j)²).

What is Manhattan distance?

d(x,y) = sum_j abs(x_j - y_j).

How does Minkowski distance generalize Euclidean and Manhattan distance?

d_q(x,y) = (sum_j abs(x_j-y_j)^q)^(1/q); q=1 gives Manhattan and q=2 gives Euclidean.

Why can measurement units dominate Euclidean distance?

Features with larger numerical scales contribute larger squared differences unless suitably scaled.

How is Jaccard similarity defined for binary attributes?

Shared presences divided by positions where at least one object is present; shared absences are excluded.

When is Jaccard preferable to simple matching?

When shared absence is uninformative, as with sparse purchase indicators.

What is simple matching similarity for binary vectors?

The proportion of positions with matching values, counting both shared presences and shared absences.

What is cosine similarity?

cos(x,y) = (x dot y) / (norm(x) norm(y)), for nonzero vectors.

What does cosine similarity emphasize?

Vector direction rather than magnitude.

Why should categorical codes not automatically be treated with Euclidean distance?

Arbitrary numerical codes impose artificial distances between categories.

[Supplement] How is conditional probability defined?

P(A given B) = P(A and B) / P(B), for P(B) > 0.

[Supplement] What is the addition rule for two events?

P(A or B) = P(A) + P(B) - P(A and B).

[Supplement] What characterizes independence of two events?

P(A and B) = P(A)P(B).

[Supplement] Are mutually exclusive events with positive probabilities independent?

No. Observing one makes the other impossible.

[Supplement] What does a Bernoulli variable represent?

One binary trial with success probability p.

[Supplement] What assumptions define a binomial count?

A fixed number n of independent binary trials, each with the same success probability p.

[Supplement] What are the mean and variance of Binomial(n,p)?

Mean np; variance np(1-p).

[Supplement] What is the binomial probability mass function?

P(X=x) = choose(n,x) p^x (1-p)^(n-x), for x=0,…,n.

[Supplement] What is a standard normal variable?

A normally distributed variable with mean 0 and variance 1.

[Supplement] Is a continuous density value the probability of observing that exact value?

No. Probabilities are areas over intervals; a single point has probability zero under a continuous distribution.

[Supplement] How is a chi-square variable constructed?

As the sum of squares of independent standard normal variables; their number gives its degrees of freedom.

[Supplement] How is a t variable constructed?

T = Z / sqrt(U/nu), where Z is standard normal and U is an independent chi-square variable with nu degrees of freedom.

[Supplement] How is an F variable constructed?

F = (U1/nu1)/(U2/nu2), for independent chi-square variables U1 and U2.

[Supplement] How do standard deviation and standard error differ?

Standard deviation describes variation among observations; standard error describes sampling variation of an estimator.

[Supplement] What is the estimated standard error of an independent sample mean?

SE(x_bar) = s / sqrt(n).

[Supplement] Does the central limit theorem say the original data become normal?

No. Under its conditions it concerns the sampling distribution of suitably standardized averages.