CNN Notes

Convolutional Neural Networks (CNN / ConvNets)

• Goal: Learn a small, efficient model for image recognition.

• Question: Are all connections in a fully connected network necessary?

• Concept: Sharing parameters (weights) across connections.

Scaling Issues with Regular Neural Networks

• Issue: Regular Neural Nets don't scale well to full images due to the high number of parameters.

• Example: CIFAR-10 images (32x32x3) require 3072 weights for a single neuron in the first hidden layer.

  • $32 * 32 * 3 = 3072$

• Large Images: 200x200x3 images would require 120,000 weights per neuron, leading to rapid parameter increase and overfitting.

  • $200 * 200 * 3 = 120,000$

• Solution: CNNs arrange neurons in 3 dimensions (width, height, depth) to handle images more efficiently by exploiting spatial structure.

Learning Image Patterns

• Key Idea: Patterns (e.g.,

• Goal: Learn a small, efficient model for image recognition.

• Question: Are all connections in a fully connected network necessary?

• Concept: Sharing parameters (weights) across connections.

Scaling Issues with Regular Neural Networks

• Issue: Regular Neural Nets don't scale well to full images due to the high number of parameters.

• Example: CIFAR-10 images (32x32x3) require 3072 weights for a single neuron in the first hidden layer.

  • $32 * 32 * 3 = 3072$

• Large Images: 200x200x3 images would require 120,000 weights per neuron, leading to rapid parameter increase and overfitting.

  • $200 * 200 * 3 = 120,000$

• Solution: CNNs arrange neurons in 3 dimensions (width, height, depth) to handle images more efficiently by exploiting spatial structure.

Learning Image Patterns

• Key Idea: Patterns (e.g., textures, shapes) are extracted through convolutions in CNNs.