Probability and Statistics wk 1

A bernoulli random variable

Any random variable whose only possible values are 0 and 1

Random Variable

any rule that associates a number with a value in S. In mathematical language, a random variable is a function whose domain is the sample space and whose range is the set of real numbers

Discrete random variable

a random variable whose possible values either constitute a finite set or else can be listed in an infinite sequence in which there is a first element, a second element, so on (“countably” infinite)

Continuous random value

A random value is continuous if both of the following apply:

1. Its set of possible values consists of either all numbers on the number line (possibly infinite in extent e.g. -infinity to +infinity) or all numbers in a disjoint union of such intervals e.g ([0,10] U [20,30])

2. No possible value of the variable has a positive probability. That is P(x = c) = 0 for any possible value of c

Probability Distribution/Probability mass function (PMF)

Defined for every number $x$by $p(x)$= $P(X = x)$= $P(allw\in S:X(w)=x)$.

a family of probability distributions

The collection of all probability distributions for different values of the parameter is called a family of probability distributions

Parameter of a distribution

Suppose $p(x)$ depends on a quantity that can be assigned any one of a number of possible values, with each different value determining a different probability distribution.

cumulative distribution function (cdf), $F(x)$

$$F(x) = P(X \le x) = \sum_{y: y \le x} p(y)$$

• Key Idea: It accumulates (adds up) all individual probabilities $p(y)$for values up to and including $x$.

• Intuition: While $p(x)$ gives the probability of hitting exactly $x$, $F(x)$gives the probability of getting at most $x$.

Expected Value (Mean)

• Notation: $E(X)$, $\mu_X$, or $\mu$

• Definition: The long-run average outcome of a discrete random variable if the experiment is repeated many times.

• Formula:

  $$E(X) = \mu_X = \sum_{x \in D} x \cdot p(x)$$

• Formula Breakdown:

  • $E(X)$ / $\mu_X$: The expected value (mean) of the random variable $X$.

  • $\sum_{x \in D}$: Sum over every possible value $x$ in the domain set $D$.

  • $x$: A specific value that $X$ can take.

  • $\cdot$: Multiplication operator.

  • $p(x)$: The probability of $X$ taking the value $x$ (the PMF).

Expected Value of a Function of $X$

• Notation: $E[h(X)]$or $\mu_{h(X)}$

• Definition: The long-run average value of a transformation/function $h(X)$applied to a random variable.

• Formula:

  $$E[h(X)] = \sum_{D} h(x) \cdot p(x)$$

• Formula Breakdown:

  • $E[h(X)]$: The expected value of the transformed function $h(X)$.

  • $\sum_{D}$: Sum across all values in the set of possible outcomes $D$.

  • $h(x)$: The function applied to the specific outcome value $x$(e.g., $x^2$, $3x + 1$).

  • $p(x)$: The original probability of obtaining that specific outcome $x$.

Variance and Standard Deviation of $X$

• Definition:

  • Variance: The average squared distance of each outcome from the mean $\mu$.

  • Standard Deviation: The square root of variance, measuring spread in the original units of measurement.

• Formulas:

  $$\text{Variance: } V(X) = \sum_{D} (x - \mu)^2 \cdot p(x) = E[(X - \mu)^2]$$

  $$\text{Standard Deviation: } \sigma_X = \sqrt{\sigma_X^2}$$

• Formula Breakdown:

  • $V(X)$ / $\sigma_X^2$: Variance of $X$.

  • $\sum_{D}$: Sum over all possible outcomes in domain $D$.

  • $(x - \mu)$: The deviation of an individual value $x$from the mean $\mu$.

  • $(x - \mu)^2$: The squared deviation (ensures negative distances become positive).

  • $p(x)$: The probability weight of that specific outcome $x$.

  • $E[(X - \mu)^2]$: Equivalent expected value expression (the average of squared differences).

  • $\sigma_X$: Standard deviation of $X$.

  • $\sqrt{\phantom{x}}$: Square root function.

Binomial Experiment

A probability experiment composed of repeated trials that meet four specific conditions.

• Key Conditions Breakdown:

  • Fixed number ($n$): The number of trials is decided beforehand.

  • Dichotomous: Each trial has exactly two possible outcomes (Success or Failure).

  • Independent: The outcome of one trial has no effect on another.

  • Homogeneous: The probability of success ($p$) remains constant across all trials.

Expected Value of a Linear Function

• Notation: $E(aX + b)$ or $\mu_{aX + b}$

• Definition: The expected value scales linearly—multiplying the variable by a constant $a$multiplies the mean by $a$, and adding a constant shift $b$ shifts the mean by $b$.

• Formula:

  $$E(aX + b) = a \cdot E(X) + b \quad \text{or} \quad \mu_{aX+b} = a \cdot \mu_X + b$$

• Formula Breakdown:

  • $E(aX+b)/\mu_{aX+b}$ : The expected value of the transformed variable $(aX + b)$.

  • $a$: A constant multiplier (scaling factor).

  • $X$: The random variable.

  • E(X)$ / $\mu_X : The original expected value (mean) of $X$.

  • $b$: A constant added to every outcome (offset/shift).

Computational / Shortcut Formula for Variance

• Notation: $V(X)$ or $\sigma^2$

• Definition: An easier way to compute variance by subtracting the square of the mean from the expected value of $X^2$, avoiding individual distance subtractions.

• Formula:

  $$V(X) = \sigma^2 = \left[ \sum_{D} x^2 \cdot p(x) \right] - \mu^2 = E(X^2) - [E(X)]^2$$

• Formula Breakdown:

  • $V(X)$/ $\sigma^2$: Variance of X.

  • $\sum_{D} x^2 \cdot p(x)$: Sum of each squared value $x^2$multiplied by its probability $p(x)$.

  • $E(X^2)$: The expected value of $X^2$("the mean of the squares").

  • $\mu^2$ / $[E(X)]^2$: The square of the original mean ("the square of the mean").

Variance and Standard Deviation of a Transformed Variable $(aX + b)$

• Notation: Variance: $$v(ax + b) or $$\sigma_{ax+b}^2|

• Standard Deviation: $\sigma_{aX+b}$

• Definition:

  • Multiplying X by $a$increases variance by $a^2$and standard deviation by $\vert{}a\vert{}$.

  • Adding a constant b does not change variance or standard deviation because shifting all numbers leaves their spread unchanged.

• Formulas:

  $$V(aX + b) = \sigma_{aX+b}^2 = a^2 \cdot \sigma_X^2$$

  $$\sigma_{aX+b} = \vert{}a\vert{} \cdot \sigma_X$$

  $$\text{Special cases: } \sigma_{aX} = \vert{}a\vert{} \cdot \sigma_X \quad \text{and} \quad \sigma_{X+b} = \sigma_X$$

• Formula Breakdown:

  • $V(aX + b)$: Variance of the transformed variable.

  • $a^2$: The scaling factor squared (squaring removes any negative signs).

  • $\sigma_X^2$: The original variance of X.

  • $\sigma_{aX+b}$: Standard deviation of the transformed variable.

  • $\vert{}a\vert{}$: The absolute value of a (ensures standard deviation is always positive).

  • $\sigma_X$: The original standard deviation of $X$.

  • $b$: Disappears in the calculation because adding a constant shift does not affect spread.

A distribution

a distribution is a model of how an uncertain numerical quantity behaves.

What does random mean in this instance

Individual outcomes are uncertain. The pattern of outcomes can still be modelled.

What does PMF ask you

At one value: P(X = x)

What does CDF tell you

Up to one value: P(X ≤ x)

What does the Complement tell you

Above one value: P(X > x) = 1 − F(x)

What does higher variance mean

Higher variance means greater spread around the expected value.

Discrete

A countable set of numerical values. Example: number of alerts in one hour.

Continuous

Measurements on a scale. Example: latency, sensor error, response time.

Trap 1

Confusing one observed value x with the random variable X.

Trap 2

Reading a PMF as if it were a cumulative probability

Trap 3

Interpreting expected value as the next guaranteed outcome.

Calculation checklist before every equation

1. What is the random experiment?

2. What is the sample space

3. What is the random variable X

4. Which event is being talked about

5. Do I need the PMF, CDF, Expectation, or variance

Difference between PMF and CDF

PMF gives probability at a value, CDF gives probabilities up to values