CP467 - CNN's

What is image classification?

The task of assigning an input image one label from a fixed set of categories

What are the steps for image classification?

1. Collect a dataset of images and labels
2. Use Machine Learning to train a classifier
3. Evaluate the classifier on new images

What is a Neural Network?

A computational model inspired by the way human brains operate. Consists of interconnected units or nodes called neurons

How do neural networks learn?

They learn from large amounts of data by adjusting the connections between neurons, in a style that the brain would learn

What are the three layers in a neural network?

1. Input layer
2. Hidden layers
3. The output layer

What is a precptron?

A single-layer neural network. Can be thought of as a single neuron

What is the activation function?

Used as a decision-making body at the output of a neuron

What are the six activation functions?

1. Sigmoid
2. tanh
3. ReLU
4. Leaky ReLU
5. Maxout
6. ELU

What is a convolutional neural network (CNN)?

Multi layer neural network. It often will include a pooling layer

What is the point of the pooling layer?

It aims to reduce the resolution of the input volume for the subsequent layers. Reduces computational work

What is a fully connected layer?

Have connections to all activations in the previous layer. These layers act as classifiers

What is ImageNet Large Scale Visual Recognition Challenge (ILSVRC)

Image classification challenge to created model that can correctly classify an input image into 1000 separate object categories

What is gradient descent?

Computes the gradient of the loss function with respect to all training examples in the dataset for each update

What is stochastic gradient descent (SGD)?

Computes the gradient of the loss function with respect to one randomly chosen training example in the dataset for each update

What is mini-batch gradient descent?

Gradients are computed on small batches of data instead of the entire dataset or a single sample

What is weight decay (L2 regularization)?

Technique to prevent overfitting. Penalizes large weights by adding a term properties to the square of the magnitude of the weights to the loss function

What is regularization?

A technique used to prevent overfitting in models

What are the 3 key points about AlexNets architecture?

1. Has 8 layers (not including pooling layers)
2. Uses ReLU for nonlinearity functions
3. Trained on two GTX 580 gpus for 5-6 days

What is the Modified National Institute of Standards and Technology (MNIST) Dataset?

A collection of 70,000 greyscale images of handwritten digits (0-9)

What is the Canadian Institute for Advanced Research (CIFAR) dataset?

1. CIFAR-10: 60,000 colour images in 10 classes with 6,000 images per class
2. CIFAR-100: 60,00 colour images in 100 classes with 600 images per class

What are the most popular frameworks to implement CNN's?

1. PyTorch
2. TensorFlow

What are the steps for image classification using a CNN?

1. Define the model
2. Define loss function and optimizer
3. Load and prepare dataset
4. Train the model
5. Test the model

What is transfer learning?

The process of fine-tuning a pre-trained model