3 Outlier detection

[Anomaly fundamentals] What is an outlier?

An object that deviates substantially from normal objects, possibly because it was generated by a different mechanism.

[Anomaly fundamentals] How does noise differ from an interesting outlier?

Noise is random measurement error or variation; an outlier may reveal a meaningful unusual event or mechanism.

[Anomaly fundamentals] Why should obvious data-quality problems be examined before anomaly detection?

Type errors, duplicate records, impossible ranges, or inconsistent categories can create artificial anomalies.

[Anomaly fundamentals] What simple checks for numerical features are suggested in the slides?

Check data types and numerical ranges for implausible or inconsistent values.

[Anomaly fundamentals] What simple checks for categorical features are suggested in the slides?

Inspect unique entries for misspellings, inconsistent encodings, or unexpected categories.

[Anomaly fundamentals] Why can duplicated rows or columns cause problems in anomaly detection?

They can distort frequency, density, or distance calculations and create misleading evidence of normality or abnormality.

[Anomaly fundamentals] Name applications of outlier detection from the slides.

Data cleaning, intrusion/fault detection, fraud detection, disaster monitoring, astronomy, and genetics.

[Anomaly fundamentals] Are all outliers mistakes that should be deleted?

No. Some are valuable rare events or discoveries; their treatment depends on the task.

[Anomaly fundamentals] What is the practical idea behind novelty detection?

Learn expected behavior from reference data and identify previously unseen patterns; a meaningful new pattern may later be incorporated into the normal model.

[Anomaly fundamentals] What three anomaly types are distinguished in the slides?

Point/global anomalies, contextual anomalies, and collective anomalies.

[Anomaly fundamentals] What is a point or global anomaly?

An individual object that differs significantly from the data set as a whole.

[Anomaly fundamentals] What is a contextual anomaly?

An object that is unusual within a specified context, even if its value is not globally unusual.

[Anomaly fundamentals] What are contextual attributes?

Attributes defining the comparison context, such as time, season, or location.

[Anomaly fundamentals] What are behavioral attributes?

Attributes whose values are evaluated for abnormality within the chosen context, such as temperature.

[Anomaly fundamentals] Why might the same temperature be normal in one record and anomalous in another?

Its expected range can depend on season or location, which changes the relevant context.

[Anomaly fundamentals] What is a local density outlier?

A point whose density is unusually low relative to its nearby comparison points, even if a global rule would not flag it.

[Anomaly fundamentals] How are contextual and local density outliers related?

Both use a restricted reference context, but contextual anomalies can use explicit time/location conditions rather than only geometric neighbors.

[Anomaly fundamentals] What is a collective anomaly?

A set of objects whose joint pattern is abnormal even when individual members are not abnormal by themselves.

[Anomaly fundamentals] Why can a sequence be collectively anomalous even if every individual value is plausible?

The unusual ordering, repetition, or coordination can violate the expected pattern.

[Anomaly fundamentals] What is an example of a collective anomaly from the slides?

An abnormal heartbeat subsequence or coordinated denial-of-service traffic among computers.

[Anomaly fundamentals] What is a central challenge in defining a global outlier?

Choosing a meaningful measure of deviation from normal data.

[Anomaly fundamentals] What is a central challenge in defining a contextual outlier?

Defining a useful context against which behavior should be judged.

[Statistical detection] What assumption underlies statistical or model-based outlier detection?

Normal objects follow a specified or estimated probability model; objects poorly explained by that model are potential outliers.

[Statistical detection] How can a Gaussian model identify possible anomalies?

Estimate its parameters from reference data, evaluate each object's density, and flag sufficiently low-density objects.

[Statistical detection] Does a low Gaussian density prove that an object is an anomaly?

No. It indicates poor fit to that model; the model itself may be inappropriate for normal data.

[Statistical detection] Why is the distributional assumption important in statistical anomaly detection?

A misspecified normal-data model can treat legitimate behavior as abnormal or fail to recognize real anomalies.

[Statistical detection] What is a parametric anomaly-detection model?

A model from a prespecified distribution family with a fixed set of parameters, such as a Gaussian with mean and variance.

[Statistical detection] What is a nonparametric density model?

A flexible estimate learned from the data without a fixed finite-dimensional distribution family, such as a histogram or kernel density estimate.

[Statistical detection] Does nonparametric mean that no settings or parameters are used?

No. Histograms have bin choices and KDE has settings such as bandwidth; nonparametric does not mean parameter-free.

[Statistical detection] What is univariate data?

Data with one measured attribute or variable per object.

[Statistical detection] What is the maximum-likelihood estimate of a univariate Gaussian mean?

mu_hat = (1/n) * sum_i x_i.

[Statistical detection] What is the maximum-likelihood estimate of a univariate Gaussian variance?

sigma_hat^2 = (1/n) * sum_i (x_i - mu_hat)^2.

[Statistical detection] Does the Gaussian maximum-likelihood variance use n or n-1 in the denominator?

It uses n. The usual unbiased sample-variance estimator instead uses n-1.

[Statistical detection] What is the Gaussian log-likelihood for n independent univariate observations?

log L(mu,sigma^2) = -(n/2)log(2*pi) - (n/2)log(sigma^2) - [sum_i (x_i-mu)^2]/(2*sigma^2).

[Statistical detection] What is a z-score?

z = (x - mu)/sigma. It expresses a value's signed deviation from the mean in standard-deviation units.

[Statistical detection] How is a two-sided three-sigma anomaly rule written?

Flag a value when |z| > 3, under an appropriate approximately Gaussian model.

[Statistical detection] Roughly what fraction of a Gaussian distribution lies within three standard deviations of its mean?

About 99.7%, so points beyond that range are rare under the Gaussian assumption.

[Statistical detection] What does a negative z-score mean?

The observation is below the estimated mean; the magnitude tells how many standard deviations below it lies.

[Statistical detection] In the temperature slide, what mean and standard deviation are quoted?

mu = 28.61 and sigma approximately 1.51. The slide uses these values to label 24 as beyond three standard deviations.

[Statistical detection] Using the temperature slide's quoted mu=28.61 and sigma=1.51, what is the z-score of 24?

(24-28.61)/1.51 is about -3.05. This reproduces the slide's intended below-minus-three interpretation, subject to its parameter inconsistency.

[Statistical detection] What happens if the Gaussian parameters are recomputed from the ten temperatures actually printed on the slide?

For {24.0,28.9,28.9,29.0,29.1,29.1,29.2,29.2,29.3,29.4}, mu=28.61, ML variance=2.3849, and sigma approximately 1.5443.

[Statistical detection] Does 24 exceed the strict three-sigma cutoff using the temperatures actually printed on the slide?

No. Its recomputed z-score is about -2.985. It remains the most unusual listed value, but the strict |z|>3 conclusion differs from the slide's quoted calculation.

[Statistical detection] Why is a density value not literally the probability of an exact continuous observation?

For a continuous variable, probabilities are integrals over ranges; a probability density is a local model-fit quantity, not a point probability.

[Multivariate clarification] What is multivariate outlier detection?

Detecting unusual objects using two or more attributes jointly, rather than evaluating only one variable at a time.

[Multivariate clarification] Why can separate univariate checks miss a multivariate outlier?

Each coordinate can look ordinary while their combination is unusual because of relationships among the attributes.

[Multivariate clarification] What is squared Mahalanobis distance from a mean vector?

D_M^2(x) = (x-mu)^T S^(-1)(x-mu), where S is the covariance matrix. The distance itself is sqrt(D_M^2).

[Multivariate clarification] What does covariance contribute to Mahalanobis distance?

It accounts for feature scales and correlations, measuring deviation relative to the distribution's spread in different directions.

[Multivariate clarification] What basic steps turn multivariate deviations into a one-dimensional score?

Estimate the mean vector and covariance, compute each object's squared Mahalanobis distance, and assess unusually large values using a suitable reference rule.

[Proximity methods] What is the main intuition of proximity-based outlier detection?

An outlier has an unusual relationship to its neighbors, such as being far from them or much less dense than they are.

[Proximity methods] What are the two proximity-based detection families in the slides?

Distance-based methods and density-based methods.

[Proximity methods] Why is the distance measure critical to proximity-based anomaly detection?

It defines who counts as a neighbor; an inappropriate measure produces misleading neighborhoods and anomaly scores.

[Proximity methods] Why can a tightly grouped set of anomalies fool a nearest-neighbor detector?

The anomalous objects are close to one another, so their nearest-neighbor distances may not be large.

[Proximity methods] What is the r-neighborhood rule for distance-based outlier detection?

Flag an object if too few other objects lie within radius r, or equivalently if a sufficiently large fraction lie outside it.

[Proximity methods] What do r and pi represent in the slides' distance-based definition?

r is the distance threshold; pi is the required fraction of objects that must lie farther than r for the point to be considered an outlier.

[Proximity methods] How can the kth-neighbor distance be used as an outlier score?

Compute d_k(p), the distance to p's kth nearest other object. A larger value suggests greater isolation.

[Proximity methods] What threshold rule can be applied to kth-neighbor distance?

Flag p if d_k(p) > r for a selected k and radius threshold r.

[Proximity methods] Why can a global distance cutoff miss a local anomaly?

The anomalous point may be close in absolute distance to a dense group, yet much less dense than its immediate neighbors.

[Proximity methods] Why can a normal point in a sparse cluster look suspicious to a global distance cutoff?

Its neighbors are far away in absolute distance even though that spacing is normal for the local cluster.

[Proximity methods] What relative comparison does local density-based detection make?

It compares a point's estimated density with the densities of its neighbors.

[Proximity methods] What is k-distance(p)?

The distance from p to its kth nearest other data object under the chosen distance measure.

[Proximity methods] What is the k-distance neighborhood N_k(p) used for LOF?

N_k(p) = {o in D, o != p : d(p,o) <= k-distance(p)}, including all ties at the kth-neighbor distance.

[Proximity methods] Can an LOF k-distance neighborhood contain more than k points?

Yes. Ties at the kth-neighbor distance can add extra neighbors.

[Proximity methods] Does the point itself count in a standard LOF neighbor set?

No. LOF neighborhoods use other objects, unlike the DBSCAN MinPts convention in these slides.

[Local Outlier Factor] What does LOF stand for?

Local Outlier Factor.

[Local Outlier Factor] What does LOF measure conceptually?

How sparse a point's local neighborhood is relative to the neighborhoods of its neighbors.

[Local Outlier Factor] What reachability-distance notation is used in these flashcards?

reachdist_k(p,o) = max(k-distance(o), d(p,o)), where p is the point being evaluated and o is one of its neighbors.

[Local Outlier Factor] Whose kth-neighbor distance appears in reachdist_k(p,o)?

The neighbor o's kth-neighbor distance, not p's. This avoids the inconsistent arrow notation in the worked slide.

[Local Outlier Factor] Why take a maximum in reachability distance?

It prevents extremely small pairwise distances from producing unstable local density estimates; the neighbor's k-distance supplies a smoothing floor.

[Local Outlier Factor] Is reachability distance generally symmetric?

No. The two points may have different k-distances, so reachdist_k(p,o) can differ from reachdist_k(o,p).

[Local Outlier Factor] What is local reachability density lrd_k(p)?

The inverse average reachability distance from p to its neighbors: lrd_k(p) = |N_k(p)| / sum_{o in N_k(p)} reachdist_k(p,o).

[Local Outlier Factor] How does local reachability density change as average reachability distance grows?

It decreases: density is the reciprocal of that average distance.

[Local Outlier Factor] What is the standard normalized LOF formula?

LOF_k(p) = (1/|N_k(p)|) * sum_{o in N_k(p)} [lrd_k(o)/lrd_k(p)].

[Local Outlier Factor] What is an equivalent ratio-of-densities expression for LOF?

LOF_k(p) = average_{o in N_k(p)} lrd_k(o) / lrd_k(p).

[Local Outlier Factor] What does LOF approximately equal to 1 mean?

The point has about the same local density as its neighbors.

[Local Outlier Factor] What does LOF substantially greater than 1 suggest?

The point is less dense than its neighbors and may be a local outlier.

[Local Outlier Factor] What does LOF below 1 suggest?

The point is locally denser than its neighbors.

[Local Outlier Factor] Is every normalized LOF value slightly above 1 automatically a meaningful anomaly?

No. The value indicates a relative density difference; a decision threshold or ranking policy must reflect the application.

[Local Outlier Factor] Why can LOF detect anomalies near clusters of different densities?

It uses a relative local-density comparison rather than one absolute distance cutoff for all points.

[Local Outlier Factor] What are the main steps for computing LOF?

Compute k-distances and tie-inclusive neighbor sets; compute reachability distances; compute local reachability densities; average neighbor-density/point-density ratios.

[Local Outlier Factor] What does k control in LOF?

The neighborhood scale used for density estimation and comparison; changing k can change which patterns look anomalous.

[Local Outlier Factor] Why is a larger standard LOF associated with a lower density at the evaluated point?

Its density appears in the denominator, while its neighbors' densities appear in the numerator.

[Local Outlier Factor] What normalization is omitted in the slides' product-of-sums LOF example?

The product [sum neighbor lrd] * [sum reachdist] must be divided by |N_k(p)|^2 to equal standard normalized LOF.

[Local Outlier Factor] Can |N_k(p)|^2 always be omitted without changing the ranking of different objects?

No. Ranking is preserved only when all compared objects have the same neighborhood size. Ties can make sizes differ.

[Local Outlier Factor] Why should the reference value LOF=1 not be applied directly to the slides' unnormalized example scores?

Those scores are scaled by |N_k(p)|^2 and are not on the standard LOF scale.

[LOF worked example] In the LOF example a=(0,0), b=(0,1), c=(1,1), d=(3,0), what distance and k are used?

Manhattan distance with k=2, excluding the point itself from its neighbor set.

[LOF worked example] What is Manhattan distance between two points?

d(x,y) = sum_j |x_j-y_j|; in 2D, |x_1-y_1| + |x_2-y_2|.

[LOF worked example] For a=(0,0), b=(0,1), c=(1,1), d=(3,0), what are the six Manhattan distances?

d(a,b)=1, d(a,c)=2, d(a,d)=3, d(b,c)=1, d(b,d)=4, and d(c,d)=3.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. For the four-point Manhattan example with k=2, what are the k-distances?

k-distance(a)=2, k-distance(b)=1, k-distance(c)=2, and k-distance(d)=3.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. For the four-point Manhattan example with k=2, what are the neighbor sets?

N_2(a)={b,c}; N_2(b)={a,c}; N_2(c)={a,b}; N_2(d)={a,c}.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. In the four-point example, why is reachdist_2(a,b)=1 but reachdist_2(b,a)=2?

reachdist_2(a,b)=max(k-distance(b),d(a,b))=max(1,1)=1; the reverse uses k-distance(a)=2.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. In the four-point example, what is lrd_2(a)?

2/[reachdist_2(a,b)+reachdist_2(a,c)] = 2/(1+2) = 2/3.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. In the four-point example, what is lrd_2(b)?

2/[reachdist_2(b,a)+reachdist_2(b,c)] = 2/(2+2) = 1/2.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. In the four-point example, what are lrd_2(c) and lrd_2(d)?

lrd_2(c)=2/(2+1)=2/3; lrd_2(d)=2/(3+3)=1/3.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. In the four-point example, what is standard LOF_2(a)?

[(1/2)/(2/3) + (2/3)/(2/3)]/2 = (3/4+1)/2 = 7/8 = 0.875.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. In the four-point example, what is standard LOF_2(b)?

[(2/3)/(1/2) + (2/3)/(1/2)]/2 = 4/3, about 1.333.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. In the four-point example, what is standard LOF_2(c)?

7/8 = 0.875, by the same density-ratio calculation as for a.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. In the four-point example, what is standard LOF_2(d)?

[(2/3)/(1/3) + (2/3)/(1/3)]/2 = 2.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. Which point is the top-ranked LOF outlier in the four-point example?

d, because it has the largest normalized LOF, equal to 2.

[LOF worked example] Data: a=(0,0), b=(0,1), c=(1,1), d=(3,0); k=2; Manhattan distance. How do the slide scores 3.5, about 5.3, 3.5, and 8 relate to normalized LOF?

All four neighbor sets have size 2, so divide the product-of-sums scores by 4 to get 0.875, about 1.333, 0.875, and 2. The ranking is unchanged.

[LOF worked example] What inconsistency appears in the reachability bullets of the worked LOF slide?

The bullets use a's k-distance where the neighbor's is needed, while the numerical lrd(a)=2/(1+2) follows the standard neighbor-based calculation.