01 - Graphics Pipeline

Pixel

Short for "picture element". The smallest controllable dot on a display. An LCD is a rectangular array of these cells.

LCD (Liquid Crystal Display)

A rectangular array of cells containing liquid crystals, large molecules whose optical properties change when an electric field is applied. Small wires through the cells apply the field.

Resolution

The size of the display's pixel array, e.g. 1920 x 1080.

Native resolution

The physical pixel resolution an LCD is built with. Driving it at a different resolution forces rescaling, which makes the image look blurry.

Refresh rate

How many times per second the image is sent to the screen, at least 60 Hz. Very important for CRTs, much less for LCDs. The graphics card generates the signals.

RGB

How colour is represented in graphics: three components, red, green and blue (e.g. 0-255 each, or 0.0-1.0 in shaders). Each pixel stores one RGB triple.

Why RGB is used

It matches how screens produce colour: every pixel emits a mix of red, green and blue light.

Colour as a model of perception

Colour representation is a practical model of human perception, not a copy of reality: we have 3 types of cone cells, brightness perception isn't linear, and different spectra can look like the same colour.

Gamut

The range of colours a device can reproduce. A display's RGB gamut is a triangle inside the horseshoe-shaped region of all visible colours.

RGB gamut limitation

RGB can't reproduce every visible colour, because some colours would need negative amounts of a primary. Printers (CMYK) have a different, usually smaller, gamut.

Scanning pattern (scan-out)

Pixels are sent to the screen in scanline order: one row at a time, left to right, starting at the top-left and working down.

Blanking (retrace)

The period between scan lines (horizontal) and between frames (vertical) when no pixel data is sent. It comes from CRT electron beams, and modern displays keep the same scan-out pattern.

Raster

The 2D grid of pixels that makes up an image.

Frame buffer

Graphics memory that stores the colour of every pixel on the screen: the raster held in memory. Rendering writes into it and the display scans it out.

Double buffering

Drawing into a hidden back buffer while the front buffer is displayed, then swapping them (glfwSwapBuffers) when the frame is done. It prevents flicker and tearing.

Rendering

The process of converting the geometric model into the pixels in the frame buffer.

Model

The digital (usually mathematical) representation of an object's geometry that the computer stores and manipulates. In the pipeline it's made of triangles.

Real-time vs offline rendering

Real-time (games) must produce many frames per second and favours speed. Offline (film effects) favours image quality and doesn't care about speed.

Speed vs quality trade-off

It's very hard to get both in rendering. OpenGL favours speed at the expense of quality, while global illumination favours quality.

Local illumination

A point's colour depends only on that point, the light sources and the viewer. Fast and pipeline-friendly. Used by OpenGL (e.g. the Phong model).

Global illumination

Lighting that considers every object in the scene (shadows, reflections, light bouncing between surfaces). High quality but slow. Examples: ray tracing, radiosity.

Graphics pipeline

The sequence of stages that turns a 3D model into pixels. Each stage specializes in one kind of computation, which makes hardware (GPU) implementation practical.

Why models use triangles

They're simple, always flat, easy to transform, project and fill, and enough of them approximate curved surfaces. More triangles = smoother but more memory and work.

Vertex processing

The first, geometric part of the pipeline, done per vertex in the vertex shader: modeling transformations, viewing, projection, lighting/colour, and clipping.

Fragment processing

The second, per-pixel part of the pipeline, done in the fragment shader: find the pixels a triangle covers, compute their colour, interpolate depth, hidden surface removal.

Why vertex processing is per vertex

Transforming and projecting a vertex doesn't depend on any other vertex. Only clipping needs the whole triangle.

Why fragment processing parallelizes

Every pixel's computation is independent of the others, so the GPU can process many fragments at the same time.

Coordinate space chain

Local (object) -> World -> View (camera) -> Clip -> Screen. Vertex programs transform each vertex along this chain.

Projection

Mapping 3D triangles onto the 2D screen plane. The course uses perspective projection because it matches how eyes and cameras see.

Camera definition

Eye position, viewing direction, up direction, and near and far planes (plus the projection).

View frustum

The truncated pyramid of space a perspective camera can see, between the near and far planes. Only triangles inside it can appear on screen.

Clipping

Removing or trimming triangles relative to the view frustum: fully inside = keep, fully outside = discard, crossing = keep only the visible part. Unclipped triangles would land in the wrong places and confuse the hardware.

Filling (rasterization)

After projection, setting every pixel inside the triangle's 2D outline. Done with efficient algorithms in GPU hardware.

Fragment

A candidate pixel-sized piece produced by rasterization. In this course fragments = pixels, though they can differ in advanced setups (e.g. multisampling).

Hidden surface problem

Many triangles can project onto the same pixel. Only the surface closest to the viewer should colour it.

Depth (z) buffer

Extra memory, alongside the frame buffer, holding the current depth of each pixel. A new pixel is drawn only if it's closer than the stored depth, which then gets updated.

Depth of a vertex

A third coordinate produced by projection: the distance from the viewer. Interpolated across the triangle like colour and used by the z-buffer.

Lighting model + material

Triangles have no colour of their own, just geometry. Colour comes from lights (position/direction, colour) combined with material properties (how the surface reflects light).

Normal vector (role)

A vector perpendicular to the surface. It determines how light reflects off it, so it's essential for lighting.

Flat shading

One colour per triangle. Fast, but looks faceted.

Gouraud shading

Compute the colour at each vertex, then interpolate the colours across the triangle. The old OpenGL default. Can miss highlights.

Phong shading

Interpolate the normal vector across the triangle and compute the colour at every pixel. More realistic and what OpenGL does now, but more expensive.