Poisson Processes
Poisson Processes
Poisson processes are used to model natural phenomena and provide analytical results. They are mathematically simple and offer a relevant fundamental understanding.
The Poisson Distribution
The Poisson distribution with parameter \mu > 0 is given by:
, for
The mean and variance are:
Sum of Poisson Variables
Theorem 4.1.1: If X and Y are independent random variables with Poisson distributions with parameters and respectively, then X + Y has a Poisson distribution with parameter .
Proof:
Since X and Y are independent:
, for
Composition of Poisson with Binomial
Theorem 4.1.2: Let N be a Poisson random variable with parameter , and conditional on N, let M have a binomial distribution with parameters N and p. Then the unconditional distribution of M is Poisson with parameter .
The Poisson Process
A Poisson process describes how the count X of rare events changes over time t.
Definition 1 (Poisson process): A Poisson process of intensity or rate \eta > 0 is an integer-valued stochastic process for which:
For any sequence of instants t0 = 0 < t1 < t2 < … < tn, the process increments are independent and stationary random variables.
For and t > 0, the random variables have a Poisson distribution with rate :
, for.
If X(t) is a Poisson process of rate \eta > 0, then its mean and variance are given by:
Exercise 4.2.1
Defects occur along an undersea cable according to a Poisson process of rate per mile.
What is the probability that no defects appear in the first two miles of the cable?
X(2) has a Poisson distribution with parameter , and so:
Given that there are no defects in the first two miles of cable, what is the conditional probability of no defects between mile two and three?
Here we use the independence of and . So it follows that the conditional probability is the same as the unconditional probability:
Exercise 4.2.2
Customers arrive in a certain store according to a Poisson process of rate per hour.
Given that the store opens at 9.00 AM, what is the probability that exactly one customer has arrived by 9.30 and a total of five have arrived by 11.30 AM?
We are asked to determine . Using the independence of and , we reformulate the request as:
Non-homogeneous processes
A generalization of the Poisson process definition is to relax the stationarity hypothesis, by letting the rate be a function of time: . The probability of a single event occurring in an infinitesimal interval h of time is proportional to :
The probability of k events happening in a time interval would then be given by:
The Law of Rare Events
The Poisson distribution is the “discrete analog” of the normal distribution. The Law of Rare Events states that the total number of these rare events that do happen follows (approximately) a Poisson distribution.
Law of Rare Events, fixed probability case
Consider an experiment with a low and fixed probability p << 1 of success, which is repeated a high number N of times. In this case, the total number of successes after N trials follows a binomial distribution: , for In the limit of rare events and infinite trials , with a fixed success rate Np = \mu > 0, will follow the Poisson distribution: , for
Law of Rare Events: general case
Theorem 4.3.1: Let be independent Bernoulli random variables, where:
P[\nui = 1] = pi
P[\nui = 0] = 1 - pi
and let Sn = \nu1 + ··· + \nun. The distribution of is given by:
which differs from a Poisson distribution with rate by at most:
In particular, if all and is kept fixed, in the limit , , and so does the RHS of 4.4.
An analog of the Law of Rare Events holds for stochastic processes, stating that the total counts of events generated by many independent processes can be approximately described by a single Poisson process.
Properties of Poisson Processes
Sum of Poisson processes
Theorem 4.4.1: Let and be two independent Poisson Processes with rates . Then, the variable that counts both of them is a Poisson process itself with rate .
Proof:
To prove that X(t) is a Poisson process, we verify the requirements in definition 1:
At the starting time t = 0, the number of events counted for both processes will be zero (), and so their sum:
Since and have stationary and independent increments separately, so does X that is their sum.
Given the fact that the random variables and are Poisson distributed with parameter and and are independent of each other, their sum is therefore a Poisson distribution with parameter as a consequence of theorem 4.1.1.
Filtering a Poisson process
Theorem 4.4.2: Let X(t) be a Poisson Process with rate and let each event be independently marked as either type 1 with probability p, or type 2 with probability . Then, the events of the type 1 and 2 follow two independent Poisson Processes with rates and .
Proof:
As before, we start by verifying the 3 requirements of definition 1:
When we start counting since there are no events at all, so
Since X has stationary and independent increments and the marking of events occurs independently, then and inherit from X the stationarity and independence of their respective increments.
The joint distribution of the number of arrivals in the two sub-processes before a fixed time t is:
From the last point we know that and are independent random variables when they are evaluated at the same instant t.
Other distributions
Depending on which aspect of the process we focus on, different distributions emerge.
Inter-arrival times
Let’s consider the arrival times (or waiting times) at which a new event occurs, and the count X goes up by 1. We define the inter-arrival times (or sojourn times) as the difference between two consecutive arrival times:
Clearly:
For the inter-arrival times, we already know the answer:
Theorem 4.5.1: Inter-arrival times are i.i.d. exponential random variables with rate .
Waiting times
Theorem 4.5.2: The waiting time , i.e. the time needed for the n-th event to occur, has the gamma distribution whose probability density function is:
If we fix the number n of events occurring in the time interval (0,t), then the joint probability of the arrival times is that of an ordered sequence of uniformly chosen points.
Uniform distribution
Suppose we choose independently n points uniformly in the interval (0,t).
The joint distribution of is given by the product of n uniform pdfs.
Let’s call the sequence of ordered , with 0 < W1 < W2 < ··· < W_n < t. Note that the Wi are not independent of each other, since they must satisfy the ordering.
Let us consider the probability of and respectively being inside two small intervals and .
f{W1,W2} (w1, w2) \Omega w1 \Omega w2 = P[w1 < W1 < w1 + \Omega w1, w2 < W2 < w2 + \Omega w_2]
= P[w1 < U1 < w1 + \Omega w1, w2 < U2 < w2 + \Omega w2] + P[w1 < U2 < w1 + \Omega w1, w2 < U1 < w2 + \Omega w2]
P[w1 < U1 < w1 + \Omega w1, w2 < U2 < w2 + \Omega w2] = \frac{\Omega w1}{t} \frac{\Omega w2}{t}
Generalizing this argument up to n elements, the number of permutations of n elements is n!, whereas the joint pdf will be proportional to . So, the joint probability density function for is given by:
, for 0 < w1 < w2 < … < wn < t
Theorem 4.5.3: Let be the ordered occurrence times in a Poisson process of rate \eta > 0. Let us condition on , that is the fact that in interval (0,t) we observe exactly n events. Given their number, the arrival times of n events have the joint probability density function:
, for 0 < w_1 < ··· < wn < t
Proof:
Let’s first assume that all ’s are distinct.
P[wi < Wi < wi + \Omega wi, i = 1, …, n|X(t) = n] = f{W1,…,Wn|X(t)=n}(w1, … , wn)\Omega w1 ··· \Omega wn + o(\Omega w1, … , \Omega wn)
P[wi < Wi < wi + \Omega wi, i = 1, …, n and zero arrivals everywhere else in [0,t] | X(t) = n]
Suppose now that n events have happened in (0,t). Then, the probability of k < n events happening in (0, u), with u < t, is given by a Binomial distribution:
Theorem 4.5.4: Let X(t) be a Poisson process with rate . Given the fact we know that in the interval (0,t) we have n arrivals, that is X(t) = n, we want to find the probability that the number of arrivals in a subset 0 <u<t is 0 < k < n. Then, in formulas:
Theorem 4.5.5: Let be two concurrent independent Poisson processes with rates . Given the total number of arrivals in interval (0,t) i.e. , the probability of having k arrivals in the first process is:
This probability is given by a binomial distribution. In fact, the probability p1 of a generic event belonging to 1 is the ratio between the rate of 1, and the total rate of both processes.
Theorem 4.5.6: Let and be two independent Poisson processes with rates in the interval (0,t). Let s be a subset of t s.t. 0 0}. Let be the total number of particles created up to time t, and let count the number of alpha particles existing at time t.
We want now to find the number of particles present at time t: so we want to compute given that at the beginning the timer was zero, i.e. . We moreover condition on the number n of particles emitted up to time t, that is , where W1, …, Wn < t are the times when particles were created.
Then, for each particle emitted, we have that the particle k still exists if and only if Wk + Yk > t: the sum of its arrival and service times must be greater that the actual time t. Let us introduce the indicator function such that indicates whether a particle still exists at time t:
\mathbb{1}{Wk + Yk > t} = \begin{cases} 1 & \text{if } Wk + Yk > t \ 0 & \text{if } Wk + Y_k < t \end{cases}
Summing on all indicator functions corresponding to all particles, we then obtain the probability that the number of existing particles is equal to m, conditioned on the total number of particles created up to time t that is n.
P(M(t) = m | X(t) = n) = P(\sum{k=1}^{n} \mathbb{1}{Wk + Yk > t} = m | X(t) = n)
Note that is the probability of accepting exactly n events from n + m trials, where the success probability of each trial is p, which is given by a Binomial distribution. is the same statistics we would have by dealing with ordered version of i.i.d. random variables in (0,t).
{Wk + Yk > t} does not depend on the order of and we have the condition , the theorem (4.11) allows us to replace the Wi’s with the same number of i.i.d. uniform random variables ’s in the interval (0,t], not facing any issue.
P(\sum{k=1}^{n} \mathbb{1}{Wk + Yk > t} = m | X(t) = n) = P(\sum{k=1}^{n} \mathbb{1}{Uk + Yk > t} = m)
\sum{k=1}^{n} \mathbb{1}{Uk + Yk > t} = m becomes now independent of the total number of arrivals , since we are already considering it by taking the sum. Moreover both and are i.i.d., so each of indicator function is a binary random variable independent of all others. The sum of these n indicator function is thus binomial with parameters n and p that is computed as:
p = P(Uk + Yk > t) = \frac{1}{t} \int{0}^{t} P(Yk > t - u)du = \frac{1}{t} \int{0}^{t} (1 - G(t - u))du = \frac{1}{t} \int{0}^{t} (1 - G(z))dz
Where in the last step we just introduced a new variable .
Now that we have obtained the probability of the binomial distribution we can rewrite relation above as:
In order to remove the condition we marginalize over the distribution of X that we know is Poisson. Given that we have a binomial distribution of parameters and n is Poisson distributed itself, if we want to find the unconditional distribution of we obtain a new Poisson, where the new is then scaled according to p of the binomial. Mathematically:
, for
As becomes the expected service time, since the integrand is , that is the tail of the distribution .
For a generic t, the value of the integral depends on the details of the specific distribution , whereas in the long run it depends only on the mean .
This implies that in the long run two distributions will converge, despite they are different, if both have the same mean.
Let us re-write the equation above using inference terminology:
Shot Noise process
A Shot Noise process models electrical effects that are produced by the random arrival of electrons to an anode.
Let assume electrons arrive to an anode according to a Poisson process {X(t);t > 0} of constant rate
An arriving produces a current whose intensity per unit of time after arrival is given by the impulse response function .
The intensity of the current I(t) will be the superposition of the impulse response functions, that are generated by electrons arrived up to time t:
P(I(t) < x) = P(\sum{k=1}^{X(t)} h(t - Wk) < x) = \sum{n=0}^{\infty} P(\sum{k=1}^{X(t)} h(t - W_k) < x | X(t) = n) P(X(t) = n)
Given their number n:
\sum{n=0}^{\infty} P(\sum{k=1}^{n} h(t - Uk) < x) P(X(t) = n) = \sum{n=0}^{\infty} P(\sum{k=1}^{n} h(Uk) < x) P(X(t) = n) = P(\sum{k=1}^{\infty} h(Uk) < x)
The expected value of a random sum is the product of the expected value of the number of terms (Poisson distributed), times the common expected value for each term (uniformly distributed).
Binomial theorem
Theorem 4.6.1: Let [X(t)] be a Poisson process of rate \eta > 0. Then for 0 <u<t and 0 < k < n,
Proof:
Dual version
P(X(s) = k | X(t) = n) \text{ 0 < n < k, 0<t<s)
= P(X(s-t) = k-n) = \frac{ e^{-\eta (s-t)} {\eta (s-t)}^{k-n} }{ (k-n)! }\sigma = |\sigma i|\sigma kPkm = \lim_{n\rightarrow} P(Xn = k , Xn+1 = m)\lim_{n\rightarrow} P(Xn+1 = j|X0 = i) = \sigma jn \rightarrow \infty\lim{n\rightarrow} P(Xn = k , Xn+1 = j|X0 = i) = \lim{n\rightarrow} P(Xn-1 = k , Xn = j|X0 = i) = \sigma kP_{kj}P(X1(2) = 1|X1(3) = 2)P(X1(3) = 2|X1(2) = 1)P(X1(1) = 1|X1(2) + X2(2) = 3)P(X1(2) + X2(2) = 3|X1(1) = 1)$$
Note that in both cases the chain is regular and it does not depend on the initial state.
P.A.S.T.A. property
How can we be sure that times sampled according to some specific rules, no matter they were upon arrivals or departures, are representative of the long run behaviour of our process?
Let us define the two probabilities pn(t) and an(t):
pn(t) = P{N(t) = n}
an(t) = P{N(t) = n | an arrival occurred just after time t}
pn(t) - seen by an external observer
an(t) - seen by a given user
P.A.S.T.A. stands for Poisson Arrivals See Time Averages. Time averages are the statis-tics seen by an external observer, and in the long run we know that Poisson arrivals will see the same statistics, thus concluding there is no bias