EEE-6512_Exam3_Lecture20

What is a pattern according to Watanabe

The opposite of chaos and an entity vaguely defined that could be given a name

What is a pattern class

A collection of similar objects

What is intra class variability

Variation among patterns within the same class

What is inter class variability

Variation between patterns from different classes

What is a pattern class model

A mathematical description of a class such as a probability density function

What are the key objectives of pattern classification recognition

Hypothesize models that describe each pattern class and assign a novel pattern to the class associated with the best fitting model

What is the difference between classification and clustering

Classification uses known categories while clustering creates new categories

What is another name given for classification in the lecture

Recognition or supervised classification

What is another name given for clustering in the lecture

Unsupervised classification

What question motivates pattern recognition in the lecture

How do we assign a novel pattern to the most appropriate class

What are some example biometric patterns shown in the lecture

Fingerprint and iris and voice and face and hand and signature

What does the lecture say about examples of patterns outside images

Insurance or credit card applications and dating services and web documents can all be patterns

What is the main idea of template matching in pattern recognition

Match a pattern against a stored template while accounting for allowable pose and scale changes

What assumption does template matching make according to the lecture

Small intra class variability

Why is learning difficult for template matching

Difficult for deformable templates

What is the main idea of statistical pattern recognition

Represent patterns in feature space and model each class statistically

What does statistical pattern recognition focus on

The statistical properties of patterns such as probability densities

What is the main idea of syntactic pattern recognition

Represent complicated patterns using simple primitives and describe them with logical rules or grammars

What is a weakness of syntactic pattern recognition

Primitive extraction is sensitive to noise and describing patterns in terms of primitives is difficult

What is the basic idea of artificial neural networks in this lecture

They are inspired by biological neural networks and use dense interconnections of simple computational elements

Why does the lecture motivate artificial neural networks with biology

Humans solve complex recognition tasks quickly which suggests massive parallelism is important

What numbers are given for biological neural systems in the lecture

About 10^10 to 10^12 neurons and about 10^3 to 10^4 interconnections per neuron and about 10^14 total interconnections

What property of artificial neural nodes is emphasized in the lecture

They are nonlinear

What is a multilayer ANN according to the lecture

A feed forward network with one or more hidden layers between the input and output nodes

What can a three layer neural network generate according to the lecture

Arbitrarily complex decision regions

What training algorithm is mentioned for multilayer ANNs

Back propagation

What weakness of artificial neural networks is listed in the lecture

Parameter tuning and local minima in learning

What are the major pattern recognition approaches compared in the lecture

Template matching and statistical pattern recognition and syntactic pattern recognition and artificial neural networks

What is the first stage of the PR system shown in the diagram

Data acquisition and sensing

What is the second stage of the PR system

Pre processing

What is the third stage of the PR system

Feature extraction

What stage links features to categories during training

Model learning and estimation

What stage assigns a pattern to a category during recognition

Classification

What is the final stage after classification in the PR system

Post processing leading to a decision

What happens in data acquisition and sensing

Measurements of physical variables are collected

What issues are important in data acquisition and sensing

Bandwidth and resolution and sensitivity and distortion and SNR and latency

What happens in pre processing

Noise is removed and patterns of interest are isolated from the background

What happens in feature extraction

A new representation is found in terms of features

What happens in model learning and estimation

A mapping between features and pattern categories is learned

What happens in post processing

Confidence is evaluated and context can be exploited and experts can be combined

What example problem is used to illustrate the complexity of pattern recognition

Sorting fish on a conveyor belt

What two fish categories are used in the PR example

Sea bass and salmon

What preprocessing steps are listed in the fish classification example

Image enhancement and separating touching or occluding fish and finding the boundary of each fish

What feature is first suggested for distinguishing sea bass from salmon

Length

What prior knowledge motivates using length as a feature

A fisherman said sea bass is generally longer than salmon

Why is length alone not a perfect feature

There are many fish for which sea bass is not longer than salmon

What second feature is tried in the fish example

Average lightness of the fish scales

Why is average lightness a better feature than length in the example

It seems easier to choose a threshold even though it still does not give perfect classification

What two possible classification errors are described in the fish example

Calling a salmon a sea bass and calling a sea bass a salmon

Why does cost of error matter in classification

Different mistakes can have different practical consequences

How does the fish canning example illustrate unequal error costs

Customers buying salmon object strongly to sea bass in their cans while sea bass customers may tolerate occasional salmon

Why might we use more than one feature at a time

Single features may not give the best performance and feature combinations can improve recognition

What two features are combined in the fish decision boundary example

Lightness and width

What is a decision boundary

A boundary in feature space that partitions the space into class regions

What is the goal when choosing a decision boundary

Minimize classification error

Do correlated features always improve performance

No correlated features do not improve performance

What are other drawbacks of adding more features

Some may be difficult to extract and computationally expensive

What is the curse of dimensionality according to the lecture

Adding too many features can paradoxically worsen performance

Why does the curse of dimensionality arise when each feature is divided into M intervals

The total number of cells becomes M^d and grows exponentially with the number of features

Why does the amount of training data need to grow in high dimensions

Because each cell should contain at least one training point

What is model complexity in the lecture

The complexity of the classifier or decision model used to separate classes

Why can complex models be dangerous

They can fit the training data perfectly but fail to generalize to new data

What is overfitting

Good performance on training data but poor performance on novel data because the model is too complex

What is generalization

The ability of a classifier to produce correct results on novel patterns

How can generalization performance be improved according to the lecture

Use more training examples and prefer simpler models

Why do more training examples help generalization

They provide better model estimates

Why do simpler models often help generalization

They are less likely to overfit the training data

What prior knowledge enters the design cycle according to the lecture

Invariances can guide feature choice and model choice

What are the steps in the design cycle shown in the lecture

Collect data and choose features and choose model and train classifier and evaluate classifier

What challenge is listed first in the lecture's challenge slide

Noise and segmentation

How can noise affect pattern recognition

Noise reduces the reliability of measured feature values

How can knowledge of the noise process help

It can be used to improve performance

Why is segmentation a challenge in pattern recognition

Individual patterns must be segmented and the correct elements must be grouped together

What is the data collection challenge in PR

Knowing whether the training and testing examples are adequately large and representative

What questions are raised under feature extraction in the challenge section

Which features are most promising and how many should be used and whether features can be learned automatically

What kinds of features should be favored according to the challenge slide

Features robust to noise and features that lead to simpler decision regions

What should pattern representations satisfy

Patterns from the same class should look similar and patterns from different classes should look dissimilar

What transformations should pattern representations ideally be invariant to

Translations and rotations and size and reflections and non rigid deformations

Why are missing features a challenge

Certain features may be unavailable such as under occlusion and the classifier must still be trained and make decisions

What question is raised under model selection

How to know when to reject one class of models and try another

What two issues are emphasized in the overfitting challenge slide

Choosing model complexity appropriately and finding principled ways to choose that complexity

When should domain knowledge be incorporated

When there is not sufficient training data

What is analysis by synthesis in the domain knowledge slide

Modeling how each pattern is generated

What OCR example is used for incorporating domain knowledge

Assuming characters are sequences of strokes

What is classifier combination according to the lecture

Using a pool of classifiers to improve performance

Why is classification error treated as a risk

Because different misclassification errors can have different costs

What question is asked under classification error

How to incorporate knowledge about different risks and whether the lowest possible risk can be estimated

What factors are listed under computational complexity

Number of feature dimensions and number of patterns and number of categories

Why are brute force methods often impractical

They may give perfect classification results but usually require too much time and memory

What tradeoff must be considered in computational complexity

The tradeoff between performance and computational cost

Why are general purpose PR systems difficult to design

Different tasks may require different features and different solutions and different tradeoffs

What higher order difference should you know between classification and clustering

Classification assigns patterns to known categories while clustering discovers new groupings without predefined labels

What higher order difference should you know between training accuracy and generalization

Training accuracy measures performance on seen data while generalization measures performance on novel data

What higher order lesson does the fish example teach about feature selection

A feature that seems reasonable may still be weak and combining better chosen features can improve decisions

What higher order lesson does the lecture give about adding more features

More features are not always better because correlated or costly features and high dimensionality can hurt performance

What higher order lesson does the lecture give about model complexity

A more complex model can reduce training error but may increase error on unseen data through overfitting

What kind of professor style question is likely from this lecture about pattern recognition approaches

Questions comparing template matching and statistical and syntactic and neural network approaches with their assumptions and weaknesses

What kind of professor style question is likely from this lecture about the fish example

Questions about feature thresholds and unequal misclassification costs and why multiple features improve decisions

What kind of professor style question is likely from this lecture about generalization and dimensionality

Questions about why more features can worsen performance and how simpler models and more training data improve generalization