ISYE 3770 Midterm 1

population

complete set of objects relevent to study

sample

subset of population

-use to make infrences about the whole

Methods to collect data

• retrospective

• observational

• designed experiments

Retrospective study

• historical data

avantages:

• existing data

• low cost

• doesnt interupt production

drawbacks:

• limited key factor varation

• missing data

• possible recording errors

• unmeasured variables

• hard to isolates effects/variables

observational study

• monitoring process/pop. with minimal interference, recording varibales of interest over period of time

advantages:

• minimal process distrubance

• potential to measure additonal variables

• improved data accuracy and reliability

drawbacks:

• limited to ability to manipulate factors

• difficulty isolating individual effects

designed experiments

• changing controllable variables, observing results, and using statistical principles to identify which factors cause observed effects

advantages:

• cause-effect relationships

• enables process improvement and validation of engineering theroies

drawbacks:

• time consuming

Models

describe how variables in a process are related

two types:

• mechanistic

• empirical

Mechanistic models

• based on known physical laws or mechanisms describing a process

  • ohm’s law (voltage=current x resistance)

• well understood underlying mechanism

limitation: real data often deviate due to uncontrolled factors

adding variability:

• real processes rarely behave as theory predicts

  • error term ( ε ) : accounts for environmental fluctuations, operator differences, and measurement noise

  • include error term to account for variability

empirical models

• developed from observed data when no simple theory exists (data relationships)

• mechanisms are unclear

  • Product Quality = β₀ + β₁(Conveyor Speed) + β₂(Operator Interventions) + β₃(Machine Temp) + ε

  • Coefficients (β’s) are estimated using regression analysis

• include error term to account for variability

• help understand and improve real processes

  • production line

probability model

help quantify risk and uncertainty in decisions

examples:

• sample size selection

• process monitoring

• quality control

discrete model - countable events (defects)

continuous model - measurable data (concentrations)

Why probability models are needed?

• sampling always involves chance → results may not represent reality

• measure uncertainty and control the risk of wrong decisions

examples:

• detecting a defect

• acceptable risk

probability

• study of randomness and uncertainty

• methods to quantify likelihood

random experiment

any activity whose outcome is subject to uncertainty

sample space (S)

set of possible outcomes from the experiment

• represented as tree diagram

event (E)

• any subset of sample space (S)

• simple - one outcome of interest

  • E_1={(0,0)}, … E_n = {(6,6)}

• compound - mult outcomes of interest

  • A={(0,0), (0,1)}

complement - A’

outcomes in sample space S that are not in A

Let 𝑨 = 𝟎, 𝟏, 𝟐, 𝟑, 𝟒 , 𝑩 = 𝟑, 𝟒, 𝟓, 𝟔 , 𝑪 = 𝟏, 𝟑, 𝟓

𝑨′ = 𝟓, 𝟔

Union - AUB

outcomes in A or B or both

Let 𝑨 = 𝟎, 𝟏, 𝟐, 𝟑, 𝟒 , 𝑩 = 𝟑, 𝟒, 𝟓, 𝟔 , 𝑪 = 𝟏, 𝟑, 𝟓

𝑨 ∪ 𝑩 = 𝟎, 𝟏, 𝟐, 𝟑, 𝟒, 𝟓, 𝟔 = 𝑺

intersection - A∩B

outcomes in both A and B

Let 𝑨 = 𝟎, 𝟏, 𝟐, 𝟑, 𝟒 , 𝑩 = 𝟑, 𝟒, 𝟓, 𝟔 , 𝑪 = 𝟏, 𝟑, 𝟓

𝑨 ∩ 𝑩 = 𝟑, 𝟒

P(A)

prob of A in S

must follow axioms of probabiliy

• For any event 𝑨, 𝑷 𝐀 + 𝑷 𝐀′ = 𝟏, from which 𝑷 𝐀 = 𝟏 − 𝑷 𝐀′

• For any event 𝑨, 𝑷 𝑨 ≤ 𝟏

  • When events 𝑨 and 𝑩 are not mutually exclusive, simply adding 𝑷(𝑨) and 𝑷(𝑩) double-counts their intersection

• (addition rule): For any two events 𝑨 and 𝑩 not mutually exclusive, 𝑷 𝑨 ∪ 𝑩 = 𝑷 𝑨 + 𝑷 𝑩 − 𝑷(𝑨 ∩ 𝑩)

• For any three events 𝑨, 𝑩, and 𝑪, 𝑷 𝑨 ∪ 𝑩 ∪ 𝑪 = 𝑷 𝑨 + 𝑷 𝑩 + 𝑷 𝑪 − 𝑷 𝑨 ∩ 𝑩 − 𝑷 𝑨 ∩ 𝑪 − 𝑷 𝑩 ∩ 𝑪 + 𝑷(𝑨 ∩ 𝑩 ∩ 𝑪)

Axioms of prob

axiom 1

P(A) >= 0

Axiom 2

P(S)=1

Axiom 3

If 𝑨𝟏, 𝑨𝟐, … is an infinite collection of disjoint events, then

𝑷 (𝑨𝟏 ∪ 𝑨𝟐 ∪ ⋯ )= Sum from i=1 to infinity of P(A_i)

relative frequencies

• probability

• as experiemnt is repeated, rel frequency of an event stablizes

• long run rel frequency defines the P(A) probability

counting techniques

find number of outcomes in sample space or events

permutation

• ordered arrangement of all elements in set

• The number of permutations of 𝒏 different elements is 𝒏! where: 𝒏! = 𝒏 × 𝒏 − 𝟏 × 𝒏 − 𝟐 × ⋯ × 𝟐 × 𝟏

• The number of permutations of subsets of 𝒓 elements selected from a

set of 𝒏 different elements is 𝑷𝒏,r = 𝒏 × (𝒏 − 𝟏) × (𝒏 − 𝟐) × ⋯ × 𝒏 − 𝒓 + 𝟏 =

𝒏!/(𝒏 − 𝒓 )!

permuation of similar objects (not all different)

• The # of permutations of 𝒏 = 𝒏𝟏 + 𝒏𝟐 + ⋯ + 𝒏𝒓 objects of which 𝒏𝟏 are

of one type up to 𝒏𝒓 of a 𝒓𝒕𝒉 type is 𝒏! /(𝒏𝟏! 𝒏𝟐! … 𝒏𝒓!)

combination

• selection of r elements when order does not matter

• Cn,r = (n r) = n!/ r!(n-r)!

• if random experiemnt where may or may not be replaced → combine combination with multication rule

mult rule (counting techniques)

• squence of k steps, number of ways to complete step 1 are n1, nuber of ways to complete step 2 are n2

• total number of ways to complete operation = n1xn2x..nk

conditional prob

• prob depends on what is known

• new info may change prob

P(A|B)

prob of A given B

• B is conditioning event

𝑷 (𝑨 )= 𝑵 (A) / N

For any two events 𝑨 and 𝑩 with 𝑷(𝑩) > 𝟎

P(A|B) = 𝑷(𝑨∩𝑩) / 𝑷(𝑩)

• P(A|B) is proportional to 𝑷(𝑨 ∩ 𝑩)

multiplication rule for 𝑷(𝑨 ∩ 𝑩) : 𝑷 (𝑨 ∩ 𝑩 )= P(A|B)× 𝑷(𝑩)

Bayes theorem - law of total prob

• with enough conditional prob, overall event prob can be determined

• For any event 𝑩, we can write 𝑩 as the union of the part of 𝑩 in 𝑨 and the part of 𝑩 in 𝑨′. That is: 𝑩 = (𝑩 ∩ 𝑨) ∪ (𝑩 ∩ 𝑨′)

• Because 𝑨 and 𝑨′ are mutually exclusive events, so are 𝑩 ∩ 𝑨 and 𝑩 ∩ 𝑨′, leading to the total probability rule for two events: P(B) = P(𝑩 ∩ 𝑨) + P(𝑩 ∩ 𝑨’) = P(B|A)P(A) + P(B|A’)P(A’)

• total prb rule for mult events: P(B) = P(𝑩 ∩ E1) + … + P(𝑩 ∩ Ek) = P(A|E1)P(E1) +…+P(A|Ek)P(Ek)

• 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴 | 𝐵)𝑃(𝐵) = 𝑃(𝐵 ∩ 𝐴) = 𝑃(𝐵 | 𝐴)𝑃(𝐴)

• 𝑃(𝐴 | 𝐵) = 𝑃(𝐵|𝐴)𝑃(𝐴) / 𝑃(𝐵)

• 𝑃(E1|B)= 𝑷 (𝑩 |𝑬𝟏 )𝑷(𝑬𝟏) / 𝑷 (𝑩| 𝑬𝟏) 𝑷 (𝑬𝟏) + ⋯ + 𝑷 (𝑩| 𝑬𝒌) 𝑷(𝑬𝒌) for P(B)>0

independence

• Two events A and B are independent when learning that one event occurred does not change the probability of the other

• Two events 𝑨 and 𝑩 are independent if 𝑷( 𝑨| 𝑩) = 𝑷(𝑨) and are dependent otherwise

• 𝑨 and 𝑩 are independent if 𝑷(𝑨 ∩ 𝑩) = 𝑷(𝑨) × 𝑷(𝑩)

• 
  

• Two events A and B are mutually exclusive if they

  cannot occur at the same time, i.e. 𝑷 𝑨 ∩ 𝑩 = 𝟎

random variable

for given subspace (S), RV is any rule that associates a number with each outcome in S

• RV is function whose domain is the sample space and range is the set of real numbers

• denoted with upercase letters (X)

• lowercase letter (x) id given outcome of X

2 outcomes → S={S,F}

RV in this sample space → X(S)=1, X(F)=0

bernoulli RV - RV whose only possible outcomes are 0 and 1

discrete - possible values form finite set or countable infinite list

continous - possible values lie in one interval or union of intervals

• no possible vlaue of te varibale has positive prob → P(X=c)=0

prob distr for discrete RV

• how total prob of mass 1 is distr among possible values of X

  • f(x)

• prob mass function (PMF) of discrete rv: 𝒇 (𝒙 )= 𝑷 (𝑿 = 𝒙)

  • f(x)>=0, and sum of all x f(x) = 1

PMF - prob mass function

• distr total prob mass (=1)

• prob histogram shows how prob mass allocated across values

  • all rectangles share same base width

  • rectangle above each value of x with f(x)>0

  • height is proportional to f(x)

parameter of prob distr

• fixed numerical value that determines how prob distr assigns prob to outcomes

• NOT random

• changing parameter changes shape of distr

• parameter (p)

  • if p inc → success is more likely

• 𝑷( 𝑿 = 𝟏) = 𝒑

• 𝑷 (𝑿 = 𝟎) = 𝟏 − 𝒑

• f(x) = (1-p)^x-1 p for x=1,2…

cumulative dist function CDF

• how prob accumulate up to x

• The CDF 𝑭(𝒙) of a discrete random variable 𝑿 with PMF 𝒇(𝒙) is defined for every number 𝒙 by: 𝑭 𝒙 = 𝑷 (𝑿 ≤ 𝒙) = SUM_y:y<=x for f(y)

• For any 𝒙, 𝑭(𝒙) is the probability that the observed value of 𝑿 will be at most 𝒙

CDF

• non-decreasing

• right cont : each horz step included its left endpoint and extend t the right

• limits: lim_x→-inf F(x)=0

  • lim_x→+inf F(x)=1

• For any numbers 𝒂 ≤ 𝒃: 𝑷 (𝒂 ≤ 𝑿 ≤ 𝒃) = 𝑭 (𝒃 )− 𝑭(𝒂−)

  • a- is largest possible value of X less than a

• If 𝑿 can only take integer values, 𝒂 and 𝒃 are integers; then: 𝑷( 𝒂 ≤ 𝑿 ≤ 𝒃) = 𝑭 (𝒃) − 𝑭 (𝒂 − 𝟏)

• Special case: 𝒂 = 𝒃 (probability at a single value) :

  • 𝑷( 𝑿 = 𝒂) = 𝑭( 𝒂 )− 𝑭 (𝒂 − 𝟏)

mean

center or middle of prob distr

-Two different distributions can have the same mean and variance

μ= E(X)= SUM_x for 𝑥𝑓(𝑥)

• E(X) is weighted average

• the point at which a bar balances

variance

measure of dispersion in distr

-Two different distributions can have the same mean and variance

σ² = V(X) = E(X-μ )² = SUM_x for (X-μ )² f(x) = SUM_x for x²f(x) - μ²

standard deviation

σ = square root of σ²

• prefered over variance

• σ is in same units as the data (variance is in unites²)

• X units away from the mean

• easier to compare across data sets

binomial distr

• series of repeated random independent trials

• rv X i= number of sucesses in n trials

• parameters: n trials, p=prob of success

• symmetric shape skewked pepending on n and p

• counts number of sicesses in a fixed number of tri

• each trial is success or failure (2 outcomes)

  • success meets criterion

  • prob of success is constant

  • number of trials n is fixed

• only 2 possible outcomes → bernouilli trial

general prob, 0<p<1

P(X=x) = (n x)(p^x)(1-p)^n-x

• no closed form for CDF

• CDF: F(X<=x)= SUM_i=0 to x for (n i)(p^i)(1-p)^n-i

• mean: E(X)=np

• variance: V(X): np(1-p)

general prob formula

(n x) = n! / x!(n-x)!

general prob

P(X=x) = (n x)(p^x)(1-p)^n-x

geometric distr

• series of repeated random independent trials

• counts number of trials until 1st sucess

• rv Y=trial number of 1st success

• parameter: p = success prob

• shape: always skewed right

• in bernoulli trials → f(x) = (1-p)^x-1 (p) for x=1,2,…

• CDF: F(x)= 𝑷 (𝑿 ≤ 𝒙) = 𝟏 − (𝟏 − 𝒑)^x

• mean: E(X)= 1/p

• variance: V(X)= (1-p)/p²

lack of memory

• success can be started at any trial w/o changing prob dist of rv

  • system does not wear out and lacks memory

  • prob of finding defect in next n chips is always the same

  • distr forgets the past: 100 defect free chips doesnt chage prob of next 100 chips beign dfect free

  • each inspection is ind with constant defect prob

  • system doesnt improve or degreafe during operation

negative binomial distr

• X= number of trials required to obtain r successes

• Total # of trials to obtain 𝒓 successes is 𝑿𝟏 + 𝑿𝟐 + ⋯ + 𝑿𝒓

• lack of memory property, each rv follows a geometric distribution with same value of 𝒑

• shape: right skewed, more skew as p dec

• bernoulli trials → f(x) = (x-1 r-1) (1-p)^x-r (p^r) , x=r,r+1,….

• CDF: P(X<=x)= SUM, k=r to x for (k-1 r-1) (1-p)^k-r (p^r)

• mean: E(X): r/p

• variance: V(X): r(1-p)/p²

Poission distr

• number of events occuring in fixed interval (time, space, volume)

• independent trials

• at known average rate

• randomly through interval

  • defect per batch

• shape: right skewed or symmetric, as λ inc → gets more symmetic

• for λ>0 → f(x)= (λ^e e^-λ) / x! for x=0,1…

• CDF: P(X<=x)= SUM, k=r to x for (λ^e e^-λ) / k!

• mean: E(X): λ

• variance: V(X): λ

  • mean and variance are the same

when to use: need all 4

• count discrete events

• random in time or space

• events are rare relative to opportunity

• events dont influence each other

• per unit quantities

• rare event in cont domain

• rate based process

prob density function

• f(x)>=)

• ∫−∞ to +∞ 𝒇 (𝒙 )𝒅𝒙 = 𝟏

• 𝑷 (𝒂 ≤ 𝑿 ≤ 𝒃 )= ∫ 𝒂 to 𝒃 𝒇( 𝒙) 𝒅𝒙 = area under 𝒇(𝒙) from 𝒂 to 𝒃 for any 𝒂 and 𝒃

histograms - approx of pdf

• area under bar = rel frequency = est of the prob that measurement falls in the interval

• area under f(x) over interval = true prob meas falls in interval

• density est at xf(x) = 0

  • assigns mass only over interval → prob at point =0

• 𝑷 (𝒙𝟏 ≤ 𝑿 ≤ 𝒙𝟐) = 𝑷 (𝒙𝟏 < 𝑿 ≤ 𝒙𝟐) = 𝑷 (𝒙𝟏 ≤ 𝑿 < 𝒙𝟐) = 𝑷(𝒙𝟏 < 𝑿 < 𝒙𝟐)

cumulative distr function

• F(x)= ∫−∞ to x f(u)du for −∞ <x<+∞

• density: d/dx (F(x)= ∫−∞ to x f(u)du) = f(x)

  • as long as derv exists

mean of cont rv

E(X)=∫−∞ to +∞ xf(x)dx

variance of cont rv

V(X)=∫−∞ to +∞ (X-μ )² f(x)dx = ∫−∞ to +∞ x²f(x)dx - μ²

standard deviation does not change for cont rv

normal (gaussian) distr

• model for cont measurements

• quality control, measurement sys, central limit theorem (many small ind effects add up to normal distr)

• parameters: − ∞ < μ < ∞ , and 𝝈>0

density: 𝒇( 𝒙) = 𝟏/ ( 𝝈 square root (𝟐𝝅)) • 𝒆^(−(𝒙−𝝁)² / 𝟐𝝈² ). for − ∞ < 𝒙 < ∞

mean: E(X): μ

variance: V(X): 𝝈²

notation: N(μ,𝝈²)

• from symmetry: 𝑷 (𝑿 < 𝝁) = 𝑷( 𝑿 > 𝝁 )

• shape: symmestric about 𝝁 and bell shaped

• center of bell is mean and mediam of distr

• 𝝁 is location parameter (changing 𝝁 shifts side to side)

• 𝝈 is scale parameter (changing 𝝈 strenches or compreses curve w/o chanigng shape)

  • lower 𝝈 → taller bell

standard normal distr

• Computing 𝑷(𝒂 ≤ 𝑿 ≤ 𝒃) requires integrating the pdf which cannot be done by standard integration techniques

• The probability has been calculated for 𝝁 = 𝟎 and 𝝈 = 𝟏 and tabulated for certain values of 𝒂 and 𝒃 - standard normal rv

• use table for other mu and sigma probs

• 
  

• CDF (Z): 𝚽( 𝒛) = 𝑷(𝒁 ≤ 𝒛)

• If 𝑿 is a Normal rv with 𝑬 𝑿 = 𝝁 and 𝑽 𝑿 = 𝝈², the rv: 𝒁 = (𝑿 − 𝝁)/𝝈

Normal approx of binomial distr

• if X is binimal distr with parameters n and p

  • Z=(X-np)/ (square root(np(1-p)))

    • np=E(X)

    • NP(1-p) = V(X)

      • good approx of E(X) and V(X) >5

• continuity correction:

  • P(X<=x) = 𝑷(𝑿 ≤ 𝒙 + 𝟎. 𝟓) = P(Z<= (x + 0.5 -np)/ (square root(np(1-p)))

  • P(X>=x) = 𝑷(𝑿 >= 𝒙 - 𝟎. 𝟓) =P(Z<= (x - 0.5 -np)/ (square root(np(1-p)))

normal approx to poisson distr

• If 𝑿 is a Poisson rv with 𝑬 𝑿 = 𝝀 and 𝑽 𝑿 = 𝝀 → Z= (X-𝝀)/ (sqaure root(𝝀))

• same continutiy correct ad binomial

  • approc is good if 𝝀>5

uniform distr

notation: U(A,b)

application: reandom number generation in simulations, and arrival time withon known interval

density: f(x) = 1/(b-a). for a<=x<=b

mean: E(X): (a+b)/2

variance: V(X)= (a+b)² / 12

exponential distr

• The rv 𝑿 that equals the distance between successive events from a Poisson process with mean number of events 𝝀 > 𝟎 per unit interval is anexponential rv with parameter 𝝀 and pdf

density: f(x)= 𝝀𝒆^(−𝝀𝒙), for 𝟎 ≤ 𝒙 < inf

Mean: E(X): 1/𝝀

variance: V(X): 1/𝝀²

• lacks memory

• For an Exponential rv 𝑿: 𝑷(𝑿 < 𝒕𝟏 + 𝒕𝟐 | 𝑿 > 𝒕𝟏) = 𝑷(𝑿 < 𝒕𝟐)

weibull distr

• model lifetimes and failure behavoirs (durability and preformance)

• shape parameter allows to represent inc, dec, constant failure rates

• parameters:

  • scale: 𝜹 > 𝟎

  • shape: 𝜷 > 𝟎

density: f(x)= (𝜷 / 𝜹) (𝒙 / 𝜹)^(𝜷−𝟏) e𝒙𝒑 [− (𝒙 / 𝜹)^𝜷 ]. for 𝒙 > 𝟎

CDF: 𝑭 (𝒙) = 𝟏 − 𝒆^[-(x / 𝜹)^𝜷 ]

mean: E(X): 𝜹𝚪( 𝟏 + (1/𝜷))

variance: V(X)= 𝜹²𝚪 (𝟏 + (2/𝜷))- 𝜹² [ 𝚪 (𝟏 + (1/𝜷))]²

bathtub curve (failure rates)

• early life failure (decreasing failure rate, 𝜷 < 𝟏)

  • Cause: Manufacturing defects, poor quality components, assembly errors

• Useful Life (constant failure rate, 𝜷 = 𝟏)

  • Cause: Random events, external shocks, and human error

  • failure is not related to the age of the component

• Wear-Out Failures (increasing failure rate, 𝜷 > 𝟏)

  • Cause: Material fatigue, corrosion, and natural aging

not included: lognormal

lognormal