Introduction to Computer Vision: Fundamentals, History, and Image Theory

Module Overview and Academic Structure

SCC 322: Computer Vision is led by Dr Bryan M. Williams at Lancaster University. The module is positioned within a broader hierarchy of Artificial Intelligence, branching from Machine Learning and Deep Learning, specifically relating to architectures like CNNs, RNNs, and Transformers. It shares boundaries with Natural Language Processing and is supported by foundations in Mathematics, Image Processing, Signal Processing, Physics, and Pattern Recognition. Academic performance in this field is tracked through high-impact publications; notable venues include the IEEE/CVF Conference on Computer Vision and Pattern Recognition (h5-index: 450, h5-median: 702), Nature (h5-index: 490), and the International Conference on Learning Representations (h5-index: 362).

The teaching structure consists of two weekly lectures held on Mondays (15:00–15:50) and Thursdays (16:00–16:50) in Mngt LT18, supplemented by five lab groups (SCC.322/P01/01 through 05) held throughout the week in Infolab locations B81, C77, and C79. Students are expected to achieve specific learning outcomes, including the ability to illustrate current research issues, design and analyze algorithms, and construct contemporary computer vision components such as feature extraction, object detection, 3D imaging, and convolutional neural networks. Assessment is split between coursework (30%), which is issued in week 14 and submitted in week 20, and a final exam (70%) in Term 3. Students must adhere to strict academic integrity guidelines; plagiarism is defined as passing off others' work as original, including minor code modifications or sharing partial solutions in group chats. All materials are provided via Moodle, and the content is designed to meet UK accessibility requirements.

Definition and Scope of Computer Vision

Computer Vision is defined as an interdisciplinary field focused on enabling computers to gain high-level understanding from digital images and videos. While Image Processing focuses on low-level manipulation such as quality enhancement, deblurring, and feature extraction, Computer Vision seeks to answer higher-level questions: "What is in the image?", "Where is it located?", and "What is the context or action occurring?" The core goals of the field involve recognizing objects, patterns, and scenes; tracking motion and actions; reconstructing 3D structures from 2D images; and generating or editing visual content.

Computer Vision handles complex tasks such as object localization and recognition, character recognition, image enhancement (deblurring), segmentation, depth estimation, and 3D interpretation. It further extends to pose estimation and managing missing information. It is essentially an ill-posed task because understanding images requires interpreting 3D information from 2D projections, which involves inherent ambiguity. To overcome this, vision systems—both biological and artificial—rely on as many cues as possible to decide on the content of an image.

Human Vision Takeaways and Optical Illusions

Human perception provides critical insights into the limitations and requirements of computer vision systems. The Checker Shadow Illusion, created by Edward H. Adelson, demonstrates that the human vision system does not make decisions based on absolute intensity values. Instead, it relies heavily on context to interpret light and dark, proving that a computer vision system cannot simply rely on local object intensity. This is further illustrated by the Cornsweet Illusion, which influences how edges and gradients are perceived.

Other psychological phenomena include the Necker Cube, which shows that a single 2D drawing can be interpreted as having multiple orientations (e.g., different faces appearing as the "front"), and the Rubin Vase, which illustrates the ambiguity between foreground and background (two faces vs. a vase). These cases confirm that images are fundamentally ambiguous. Challenges for computer vision include variations in viewpoints, changes in illumination, occlusions (as seen in René Magritte's The Blank Signature), deformations, noise (illustrated by satellite image reconstruction or Gustav Klimt's Die Jungfrau), and intra-class variation (recognizing different types of chairs as the same category).

Historical Evolution of Computer Vision

The history of the field began with the first photograph, "View from the Window at Le Gras," taken by Joseph Niepce in 1826. This was achieved using a camera obscura with a polished pewter plate coated in bitumen of Judea, requiring an eight-hour exposure and a wash of lavender oil and white petroleum. The field progressed through several distinct eras:

1960s–1990s: Focused on Geometry and Early Algorithms, including edge detection, feature identification, stereo vision, and optic flow.
2000s: The Traditional Vision Pipeline era, characterized by SIFT, SURF, and HOG features, the Bag of Visual Words model, classical segmentation, and 3D reconstruction. Optimization often involved minimizing energy functions such as $\text{arg min} { \lambda_1 \int_{\Gamma} |u(x) - c_1|^2 dx + \lambda_2 \int_{\Omega\setminus\Gamma} |u(x) - c_2|^2 dx + \mu \int_{C} |\nabla u(x)| dx }$ .
2012–2020: The Deep Learning era, sparked by AlexNet winning the ImageNet competition, leading to the dominance of CNNs in classification, detection, and segmentation.
2020s–Present: The era of Transformers, Diffusion, and Multimodality, featuring Vision Transformers, Foundation Models, VLMs (Vision-Language Models), and LMMs (Large Multimodal Models).

Modern Applications of Computer Vision

Computer vision is integral to many modern infrastructures. In autonomous systems, it enables driving and robotics through lane detection, depth estimation, and SLAM (Simultaneous Localization and Mapping). In healthcare, vision systems assist in disease screening, surgical tracking, and patient monitoring, emphasizing accuracy and explainability. Security applications include face recognition and crowd counting, though these are subject to ethical constraints and privacy regulations, often requiring on-device processing and anonymization.

Forensics uses vision for identification, such as the H-Unique project which identifies suspects from hand images. Other industrial applications include quality control in manufacturing, footprint analysis in retail, crop health monitoring in agriculture, and motion capture in content creation. Multimodal systems now integrate vision with language for captioning and robotics.

Mathematical Representation of Images

Technically, an image is viewed as a matrix or an array of pixels. A black and white (B/W) image is a grid of intensity values. For computer vision theory, an image is defined as a function $u: \mathbb{Z} \times \mathbb{Z} \rightarrow \mathbb{R}$ where the domain $\Omega = \mathbb{Z} \times \mathbb{Z}$ represents rows and columns and the output is the intensity value. For videos, the function is $u: \mathbb{Z} \times \mathbb{Z} \times \mathbb{Z} \rightarrow \mathbb{R}$ , adding a time/frame dimension.

Images are stored in discrete space (grids), whereas theoretical work often uses continuous space (the real line). Common storage formats include:

8-bit format: $u \in {0, 1, \dots, 255}$ , represented as a matrix of size $m \times n \times p$ .
16-bit format: $u \in {0, 1, \dots, 65535}$ .
Double/Float format: $u \in [0, 1]$ , where single precision (32-bit) provides approximately 7 decimal places and double precision (64-bit) provides approximately 15.

Basic image processing includes scalar multiplication, represented as $\alpha u(x, y)$ . When $0 < \alpha < 1$ , the image darkens; when $\alpha > 1$ , the image brightens.

Colour Spaces and Foundations

Colour photography founded on Three Colour Photography, first demonstrated by Thomas Sutton in 1861 based on James Clerk Maxwell’s 1855 work and the Young-Helmholtz theory. This captured images using red, green, and blue filters.

RGB and CMYK

RGB (Red-Green-Blue) is an additive colour model used in digital displays, where colors are added to a black background. CMYK (Cyan, Magenta, Yellow, Black) is a subtractive model used in printing, starting with white and adding ink that absorbs light. The conversion from RGB to CMYK is:
$K = 1 - \max(R, G, B)$
$C = \frac{1 - R - K}{1 - K}$
$M = \frac{1 - G - K}{1 - K}$
$Y = \frac{1 - B - K}{1 - K}$
Note: Initial calculation for K uses normalized values $R, G, B \in [0, 1]$ .

HSV

HSV (Hue, Saturation, Value) is more intuitive and introduced in the 1970s. Hue represents perceived colour, Saturation represents colourfulness relative to brightness, and Value represents brightness itself. Given normalized $R', G', B'$ and $\Delta = \max(R', G', B') - \min(R', G', B')$ , the Value $V = \max(R', G', B')$ .
Saturation $S = \begin{cases} 0 & \text{if } V = 0 \ \frac{\Delta}{V} & \text{otherwise} \end{cases}$ .
Hue $H$ is calculated piecewise:
$H = \begin{cases} 0 & \text{if } \Delta = 0 \ \frac{1}{6} (\frac{G' - B'}{\Delta} \text{ mod } 6) & \text{if } R' = V \ \frac{1}{6} (\frac{B' - R'}{\Delta} + 2) & \text{if } G' = V \ \frac{1}{6} (\frac{R' - G'}{\Delta} + 4) & \text{if } B' = V \end{cases}$ .

CEILAB ( $L^<em>a^</em>b^*$ )

Defined by the CIE in 1976, this is a perceptually uniform space based on the opponent model of human vision. $L$ represents perceptual lightness (0 to 100), $a^<em>$ represents the Green-Red axis, and $b^</em>$ represents the Blue-Yellow axis. Conversion from RGB involves:

Normalizing: $R' = \frac{R}{255}$ .
Gamma Correction: $C_L = \begin{cases} (\frac{C' + 0.055}{1.055})^{2.4} & \text{if } C' > 0.04045 \ \frac{C'}{12.92} & \text{otherwise} \end{cases}$ .
Conversion to XYZ space via matrix: $\begin{pmatrix} X \ Y \ Z \end{pmatrix} = \begin{pmatrix} 0.4124 & 0.5745 & 0.1804 \ 0.2126 & 0.7151 & 0.0721 \ 0.0193 & 0.1191 & 0.9503 \end{pmatrix} \begin{pmatrix} R_L \ G_L \ B_L \end{pmatrix}$ .
Final conversion using reference points $X_n, Y_n, Z_n$ and function $f(t) = \begin{cases} t^{1/3} & \text{if } t > 0.008856 \ 7.787t + \frac{16}{116} & \text{otherwise} \end{cases}$ .
$L^* = 116f(\frac{Y}{Y_n}) - 16$
$a^* = 500(f(\frac{X}{X_n}) - f(\frac{Y}{Y_n}))$
$b^* = 200(f(\frac{Y}{Y_n}) - f(\frac{Z}{Z_n}))$ .