Key Concepts in Probability and Statistics

Introduction to Probability

• Probability is the branch of mathematics concerning events and numerical descriptions of how likely they are to occur.

Important Properties of Probability

• Non-negativity: Probability cannot be negative.

• Range: The possible range of the probability of an event is [0, 1].

• Total Probability: The total probability of all possible outcomes is 1.

• Discrete vs Continuous:

  • If an event has a finite number of possible values, it has discrete probabilities.

  • If an event has an infinite number of possible values, it has a continuous probability distribution.

Classical Definition of Probability

• Terms:

  • Random Experiment: An experiment whose outcome is uncertain.

  • Outcome: The result of a random experiment.

  • Event: A collection of outcomes.

• Probability of Event A:

  • 
    $P(A) = \frac{n_A}{n}$

  • Where:

  • $n$ = Total number of possible outcomes

  • $n_A$ = Number of outcomes belonging to event A.

Sample Space and Events

• A sample space, S, contains all possible outcomes of a random experiment.

• Each possible outcome is a sample point, $s_k$, and an event corresponds to one or more sample points.

Axiomatic Definition of Probability

• A probability space consists of the following:

  1. Sample space S of elementary events.

  2. A set of events that are subsets of S.

  3. A probability measure $P[A]$ to each event A, defined by:

  • $P[S] = 1$

  • $0 \leq P[A] \leq 1$

  • For mutually exclusive events, $P(A \cup B) = P(A) + P(B)$.

Bernoulli Trials

• A Bernoulli trial (or binomial trial) has two outcomes, A or B. The probabilities remain consistent across trials:

  • Example: Flipping a coin.

  • Probabilities: $P(H) = \frac{1}{2}$, $P(T) = \frac{1}{2}$

• Formula for k successes out of n trials: $P(k) = C(n, k) p^k q^{(n-k)}$

  • Where: $q = 1 - p$

• Total probability of events equals 1.

Uniform Distribution

• In discrete uniform distribution, if K possible values exist:
  $P(X = x) = \frac{1}{K} ext{ for each value of } X$

Continuous Probability Density Function (PDF)

• Continuous random values are studied using PDFs, $f(x)$:

  • The area under $f(x)$ equals 1, and for a range [a, b]:
    P[a < x \leq b] = \int_{a}^{b} f(x) \, dx

Continuous Uniform Distribution

• For $X$ in the range [a, b]:
  f(x) = \begin{cases}
  \frac{1}{b-a} & \text{for } x \in [a, b] \
  0 & \text{otherwise}
  \end{cases}

Random Number Generator (RNG) in MATLAB

• MATLAB's rand() function generates uniformly distributed random values in [0, 1].

  • To scale to range [a, b]:
    $Y = (b - a) X + a$

• To reset RNG:

  • Use rng('default')

Histogram

• Histograms group data into bins to analyze distribution.

• To plot in MATLAB:

  • histogram(x, nbins) for a specified number of bins.

Mean and Variance

• Mean ($\mux$):
  $\mu = \frac{\sum</em>{i=1}^{n} x_i}{n}$

• Variance ($\sigma^2$) measures data spread:
  $\sigma^2 = \frac{1}{N} \sum<em>{k=1}^{N} (x</em>k - \mu_x)^2$

Cumulative Distribution Function (CDF)

• The CDF describes the probability distribution completely:

• For discrete random variables:

  • $F<em>X(x) = Pr(X \leq x) = \sum</em>{x<em>i \leq x} P(X = x</em>i)$

Properties of Normal Distribution

• The normal distribution is given by:
  $f(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} e^{ -\frac{(x - \mu)^2}{2 \sigma^2}}$

• Standard Normal Distribution: when $\mu = 0$ and $\sigma = 1$.

• 68.3%, 95.4%, and 99.7% of values lie within 1, 2, and 3 standard deviations from the mean, respectively.

• To calculate the CDF in MATLAB for normal distributions:

  • Use normcdf(x, mu, sigma)