Luminance & Contrast: Introduction, Texture and Gloss, Transparency, Contrast and Lightness

Luminance & Contrast: Introduction

• Learning Objectives:
  • Describe the relationship between illuminance, reflectance, & luminance and their variations in nature.
  • Describe how contrast is specified and what influences it.
  • Describe how the sensitivity of the eye to contrast varies with the coarseness or fineness (spatial frequency) of the viewed object: the contrast sensitivity function (CSF).
  • Explain visual acuity and describe the factors that influence it.
  • Explain lateral inhibition and its effect on vision.

Light and Objects

• Light: electromagnetic radiation visible to the eye.
• Need to consider how light interacts with objects to generate patterns of light that form images within our eye.
• How can this “pattern” be quantified?

Illuminance and Luminance

• Illuminance (E): incident light.
• Luminance (L): emitted or reflected light.
• For a Lambertian reflector, L = \frac{Er}{π}.
  • Reflectance (r).
  • Luminance of a surface increases if:
    • Illuminance increases.
    • Reflectance of surface increases.
  • Units:
    • Illuminance = lux.
    • Luminance = candelas/square metre (cd/m^2).

Typical Reflectances

• MgO, BaSO4: 0.98
• White paper: 0.80 to 0.95
• White paper with blank ink: 0.05
• Soot: 0.02
• Biological tissue is wavelength-dependent (e.g., retina is more reflective in the red due to blood).
• Range of reflectance in the natural world: ~0.05 to 0.95.
  • Would give a 20:1 range of luminances (1.3 log units), given a fixed level of illumination.
• Range of illuminances is huge:
  • Candle (at one metre) = 1 lux.
  • Sun on earth’s surface = 50,000 lux.

Logarithmic Units

• Values are commonly expressed as log units due to the vast range of values seen in vision science.
• Examples:
  • 0.001 in log_{10} units is -3.
  • 0.01 is -2.
  • 0.1 is -1.
  • 1 is 0.
  • 10 is 1.
  • 100 is 2.
  • 1,000 is 3.
• If something increases by 2 log units, it goes up 100x.
• If something decreases by 3 log units, it goes down 1,000x.

Luminance Levels

• Various luminance levels and their corresponding activities:
  • Retinal damage: 14 (log cd/m^2)
  • Tungsten filament: 10
  • White paper, full sunlight: 8
  • Fluorescent tube: 6
  • Candle flame: 4
  • Comfortable reading: 2
  • Read newsprint with difficulty: 0
  • White paper, moonlight: -2
  • White paper, moonless night sky: -4
  • Visual threshold: -6
• Photopic vision involves cones, scotopic vision involves rods, and mesopic vision involves both.
• Rods saturate at higher luminance levels.

Contrast

• Adaptation allows perceptions to be scaled to the overall prevailing luminance condition.
• Much of the time, we need to distinguish small differences in luminance on a largely uniform background.
• Perception is often critically dependent on the contrast of the object.
• Contrast = \frac{ΔL}{L}, where:
  • \Delta L = the difference in luminance.
  • L = the background luminance.
• In general, we detect an object if its contrast exceeds a certain threshold (commonly in the order of 0.5 – 2%).

Grating Contrast

• Grating contrast can be specified as: \frac{L{max} – L{min}}{L{max} + L{min}}.

Line Visibility

• A very thin white line on a black background might be visible, whereas a black line of identical width on a white background may not.

Optics of the Human Eye

• The optics of the human eye are not perfect; even a very small light spot will be imaged as a small blob on the retina.
• 1’ = 1 minute of arc = 1/60th of a degree.

Resolution

• Resolution is the ability to see critical detail in a scene, rather than simply detecting something is there (e.g., resolving two stars rather than one).
• The point spread function is a key factor in determining the eye’s resolving capacity for high spatial frequencies.

Visual Acuity

• 6/6 visual acuity = at a test distance of 6 meters (numerator), you could read a letter whose critical detail subtends 1 minute of arc when the letter is at 6 metres.
• 6/12 visual acuity = at a test distance of 6 meters (numerator), you could read a letter whose critical detail would subtend 1 minute of arc if the letter were placed 12 metres away (i.e., when viewing such a letter at 6 metres, its critical detail subtends twice this, or 2 minutes of arc).
• 6/12 = vision required for driving.

Contrast Sensitivity Function (CSF)

• The contrast sensitivity function describes how sensitivity varies with spatial frequency.
• High sensitivity: visible at low contrast.
• Low sensitivity: requires high contrast to be visible.

Spatial Frequency

• Images contain multiple spatial frequencies.
• Low spatial frequency content: coarse luminance variations (e.g., large objects, overall shape).
• High spatial frequency content: fine luminance variations (e.g., fine structure, details).

Lateral Inhibition

• The retina is tiled with multiple, overlapping light-sensitive areas called receptive fields.
• Many receptive fields show centre/surround opponency, formed by the wiring between various retinal cells.
  • Illuminate field uniformly: little change in neural signal from baseline firing rate, as centre (excitation) & surround (inhibition) influences cancel.
  • Shine light on centre = increase neural signal from receptive field.
  • Shine light on surround = decrease neural signal from receptive field.
• The idea that one area can influence the activity of a neighbouring area is called lateral inhibition.

Receptive Field Stimulation

• Receptive field maximally stimulated when viewing a grating of similar dimension to the field.
• As grating spatial frequency decreases, centre & surround more equally stimulated by grating bar:
  • Retinal response decreases.
  • Need higher contrast stimulus to reach perceptual threshold.
  • Poor sensitivity to large gratings in CSF due to lateral inhibition.

Hermann Grid Illusion

• Classical Explanation:
  • More lateral inhibition for fields at cross rather than bar.
  • Don’t see at fovea, as fields sizes are very small.
• But:
  • Illusion quickly collapses when intersections are slightly curved, although lateral inhibition should be similar.
  • Suggests alterative theory based on edge straightness in grid

Role of Lateral Inhibition

• Helps recode spatial information from the retina more efficiently by enhancing edges.
  • The shape of the edge defines the shape of the object.
• Don’t have vast numbers of neurons in the middle saying “This bit’s grey!” “Yes, this bit’s grey too!” “So’s this bit”.
  • Helps reduce redundancy in responses.
• Enhances those parts of the image that are of particular use & suppresses those that are less important.
• Lateral inhibition is employed widely throughout the sensory system to help detect change.

Line Drawings

• Lateral inhibition helps explain why a line drawing of an object is accepted by the visual system as a representation of the real thing.
• Real object (e.g., a face) = area of colour, light & shade.
• Line drawing = thin black line corresponding to transitions between areas of colour, light & shade.
• Line drawing instantly accepted, as retina is sending essentially the same sort of information as it would if confronted with the real image.

Summary

• Luminances of surfaces are determined by a combination of the amount of illumination, as well as the surface reflectances.
• A critical perceptual aspect of an image is its contrast.
• The human eye’s sensitivity to contrast varies with spatial frequency.
• Lateral inhibition is important in highlighting areas of change in our field of vision, such as at object edges.

Example MCQ

• Which of the following quantifies the distribution of light on the retina when looking at a very small light source?
  • Answer: a) Point spread function
• A receptive field with an excitatory centre and an inhibitory surround will respond best to a grating:
  • Answer: c) when the white bar of a grating falls on the centre, and the grating’s bar width approximately matches the centre width.

Luminance & Contrast: Texture and Gloss

• Learning Objectives:
  • Describe the utility of texture, and potential ways the visual system may both segregate and perceive texture.
  • How texture processing may relate to the phenomenon of crowding.
  • The nature of specular reflections, gloss and models for how gloss is perceived.
  • The use of specular highlights in art & illusion.

What is Texture?

• Surface structures typically have patterns of bumps & dips we can feel with our fingers:
  • Tactile texture (e.g., rock, bark, skin, wall paint).
• Surface variations also produce variations in light intensity reaching our eyes:
  • Visual texture.
• Visual texture can also come from variations that don’t give tactile texture:
  • Rock composition (quartz vs mica).
  • Water waves.
  • Patterns of surface colour (e.g., paint).

Texture vs Parts

• Q: Why are surface variations on a tree bark “texture”, but surface variations on a face (eyes, nose, mouth) “parts”?
• Swap face parts (e.g., eyes & mouth):
  • Turns into new object: dramatic change in appearance.
• Swap areas of bark (not identical, but appear to be the same “stuff”):
  • Minimal influence on perceived texture.
• Texture is:
  • Relatively homogenous, or at least slowly varying.
  • Stuff more compactly represented by its statistics (its aggregate properties) than by the configuration of its parts.

Utility of Texture

• Provides a cue for:
  • Shape & orientation of surfaces.
  • Identifying material object or surface is made from.
  • Perceiving coherent groups & regions in an image.
• Study:
  • To study, need to be able to generate textures with various statistical properties.
  • Challenging, prior to the advent of computing.
  • Julesz (1962, 1962): access to computers & algorithms to generate random textures.

Texture Segmentation

• Visual system “perceptually organises” images:
  • Transforms individual feature estimates into coherent regions, structures & object.
  • Texture similarity used as a cue for this (along with grouping by proximity, feature similarity, good continuation, etc.).
• Visual system can perform texture segmentation very quickly:
  • Perceive boundary between different textures in <0.2 s.
  • Boundary not literally in image: Gestalt phenomenon (whole is different from the sum of its parts).

Texture Segmentation: Pixel Statistics

• If two textures differ sufficiently in their average luminance, segmentation occurs:
  • Difference in 1st order luminance statistics.
• First-order statistics are based on the first order histogram:
  • Quantifies how commonly pixels of various grey levels (luminances) occur in image.
  • Doesn’t tell you where the pixels are in space.

Pixel Statistics and Texture Segmentation

• Differences in 1st order pixel statistics are not necessary for texture segmentation
  • e.g. differences in line orientation between two textures are as effective as luminance differences
• Textures can also differ in 2nd order statistics:
  • based on looking at pairs of pixels, giving orientation information
  • examines the frequency with which certain pairs of pixel intensities occur as a function of the distance & direction between the pair
• Julesz Conjecture:
  • “Whereas textures that differ in their first- and second-order statistics can be discriminated from each other, those that differ in [higher order] statistics usually cannot” (Julesz, 1975)

Texture Segmentation: Statistics of Textons

• Several counterexamples of Julesz Conjecture
  • difference in 2nd order pixel statistics neither necessary nor sufficient for texture segmentation, therefore
• “Texton” theory
  • rather than simply orientation, segmentation depends on other “textons” such as closure, curvature, line endpoints, junctions, etc.
  • intuitively appealing, but difficult to quantify: often based on verbal descriptions of image features rather than actual measurements [“word models”]

Texture Segmentation: Image processing-based models

• Based on simple image processing operations like those that occur in early vision
  • texture segmentation arises from this processing
• Models have similar structure (LNL):
  • Filtering of image [L: linear, e.g. response increases when luminance is above background, decreases when it is below]
  • Non-linear operator [N: non-linear, e.g. response increases as luminance either increases or decreases from background level]
  • Further filtering [L: linear] & Final decision stage [often acts as a coarse-scale edge detector]
• Make testable predictions about texture segregation
  • in contrast to word models
• Agree with certain behavioural data

Texture Segmentation as an Artifact

• Is, therefore, texture segmentation an artifact of early visual processing?
  • probably not ideal, as would like to identify boundaries in an intelligent — not capricious — way
• Possible that statistical & image processing methods both used, e.g.:
  • use statistics to first see if image characteristics are significantly different in two areas
  • if so, apply image processing-based segregation to determine border
• Some evidence this is the case
  • e.g. segmentation worse as texture variability increases

Analysis of Texture Perception

• One approach
  • have observers judge similarity of various textures
  • look to see if critical dimensions exist in judgement data
• Rao & Lohse (1996)
  • had participants rate textures based on 12 dimensions
  • on analysis, identified three most significant dimensions to describe natural textures

Analysis of Texture Perception

• Another Approach:
  • Analyse texture with preferred model
  • Use model’s representation to generate a new instance of the texture
  • Model classed as successful if the new texture appears to be “made of the same stuff” as the original
• Some models work well in certain circumstances:
  • e.g. Portilla & Simonceli (2000)
  • less successful for purely periodic textures (tiles), binary or pen-and-ink textures, & textures that are collections of small objects (e.g. pile of jellybeans)

Peripheral Vision: Crowding

• Crowding: an impairment in the ability to resolve a target when it is surrounded by irrelevant targets: e.g. reading a single letter, versus reading a letter in a string of letters
• Crowding most prominent in peripheral vision, where visual acuity is better for single letters than for strings of letters:
  • letters become lost in jumble
  • is as if letter’s features (e.g., bars, curves) become untethered & incorrectly bound to neighbouring letter’s features
  • crowded letter “only seems to have a ‘statistical’ existence”

Crowding and Texture Processing

• Proposed that crowding represents texture processing in the periphery, & so excessive feature integration
  • assumes that texture processing operates by default in the periphery
• Interestingly, humans make same sort of errors with peripheral vision as is predicted by some texture processing models

Gloss

• Surface lightness typically understood as the percept relating to the diffuse reflectance of a surface
• Perception of gloss typically understood as relating to the specular (mirror-like) component of reflection
  • diffuse e.g. painted wall - light ray reflected in all directions
  • specular e.g. mirror - light ray reflected in single direction

Reflection Size and Shape

• Size of reflection will depend upon size of light source, and shape of specularly reflecting surface:
  • Flat: reflected image same size as light source
  • Convex: reflected image smaller than light source
  • Concave: reflected image larger than light source
• Higher curvature = more magnification or minification

Models for Gloss Perception

• Argued that perceived gloss well predicted by strong positive skew in first-order image histogram
• Not quite so simple: specular highlights & reflections must appear in ‘right place’ on surfaces to elicit gloss
  • Physically, highlights cling to regions of high surface curvature

Models for Gloss Perception

• Perceived gloss also depends on complex interactions between light field & surface shape:
  • Recent modelling quite successful, however, at accounting for much of the variance in observers’ perception of gloss
• Specular highlights might also have role in maintaining stable colour percepts under changes in illumination (colour consistency)

Exam and MCQ Examples

• Textures are best represented by: b) their statistical properties, with the border between adjacent textures being distinguishable in under 0.2 seconds.
• Which words best complete the blanks in the following sentence? “A ____ reflecting surface that is ____ in shape will produce an elongated highlight that is a visual cue to gloss.” d) specularly, cylindrical

Luminance & Contrast: Transparency

• Learning Objectives
  • Describe the nature of physical transparency, and differentiate it from translucency
  • Explain the principal concepts surrounding the perception of transparency in 2D images, including double-belongingness and scission
  • Explain the conditions for perceptual transparency to arise

Physical Transparency

• Object that allows light to pass through without being scattered
  • may reduce light passing through, however
• Transmittance = amount of light out / amount of light in
  • may be wavelength dependent (= coloured filters)

Translucent Materials

• Contrast physical transparency with translucent materials, that also scatter light passing through

Perceiving Transparency in Images

• “the phenomenal possibility of seeing something through something else and shifting attention from what is in front to what is behind, along the same line of sight” (Gerbino, 2012)

Visual System Decomposition

• Visual world consists of objects & surfaces, distributed in a three-dimensional environment
• Retinal visual inputs = no such objects & surfaces, just two-dimensional arrays of light intensities
  • Intensities depend on many variables (e.g. surface reflectance, illumination intensity & position, object orientation, etc.)
• visual system must decompose pattern of intensities into the separate contributions responsible for the retinal image
  • enormously difficult, given the infinite potential combination of variables that could account for pattern

Scission and Double-belongingness

• Double-belongingness” causes a scission (n. a cutting):
  • where grey superposition region is separated into two components:
    • a transparent grey, belonging to the bar
    • an underlying white surface, belonging to the cross
  • separation occurs despite retina being illuminated (locally) with a single intensity
• Double-belongingness in (a) depends upon:
  • Locally: good continuation of contours at crossing points
  • Globally: improvement in form regularity

Physical & Phenomenal Transparency Relationship

• In (a), see two opaque grey squares on different lightness backgrounds
• In (b), move squares to touch at lightness border: see single, large transparent filter
  • physical transparency therefore not necessary for phenomenal transparency
• In (a), don’t see transparency, even though display could be produced by transparent filter on bicoloured background
  • physical transparency not sufficient for phenomenal transparency

Conditions for Perceiving Transparency

• Geometric Conditions
  • Topological & Figural conditions
• Photometric Conditions
  • concern the presumed reflectance or luminance of various objects

Geometric Conditions

• To support perceived transparency, p & q regions should group together to form a layer
  • should also group with adjacent regions a & b to form a background surface partly occluded by layer
• Double-belongingness of 2 of the 4 regions depends on geometrical constraints

Geometric Conditions: Topological

• For two subregions to appear as a compelling transparent layer, each subregion must contact:
  • the other (reciprocal contact constraint)
  • only one of the remaining regions

Geometric Conditions: Figural (local)

• X-junctions = strong cue for transparency
• T-junctions = typically indicates occlusion by opaque surface
  • Possible for an illusory contour to create an implicit X-junction: when L2 is between L1 & L3 [L = luminance of each area]

Geometric Conditions: Figural (local and global)

• Good continuation at X-junctions critical local factor for vivid transparency
• Computed measures of the relative complexity of mosaic vs transparency solutions for pattern segmentation correlate with subjective transparency judgements.
• Negative turning angles in layer boundary decrease perception of transparency, despite good local continuation being preserved

Photometric Conditions

• Consider the appearance of helicopter blades versus a fan.
• Metelli’s Episcotister Model for Transparency:
  • attempt to interpret transparent objects by relating them to the appearance of a rapidly rotating disc, with a gap

Metelli's Model Equations

• Assume regions p & q are mix of background reflectance & episcopotister reflectance in proportion of \alpha and 1-\alpha:
  p = \alpha a + (1-\alpha)t
  q = \alpha b + (1-\alpha)t
• Can then solve for \alpha and t:
  \alpha = \frac{p-q}{a-b}
  t = \frac{aq-bp}{a+q-b-p}

Metelli's Model Conditions

• Condition One (Polarity constraint):
  • p – q must have the same sign as a - b
  • i.e. if region a is darker than region b, then region p must be darker than region q
• Condition Two (Magnitude constraint);
  • | p – q | must not be greater than | a – b |
  • i.e. central region must have a smaller reflectance difference than surround

Metelli’s Model: Limitations

• Some perceptions not predicted by model:
  • e.g. black episcotister appears more transparent that white, despite identical physical transmittance a
• Evidence that transparency more related to image contrast in the transparent layer (Singh & Anderson, 2002)

Transparency in Outline Patterns

• Intertwined line patterns can give the appearance of double-belongingness within a region:
  • The nature of line pattern alteration can give the perception of other properties of the transparent layer.

Spatial Distortions & Transparency: water

• Appropriate dynamic (but not static) image distortions give a strong impression of transparent water
• Nature of deformation must match that from refraction (how light is bent) by water:
  • e.g. lower — rather than higher — spatial frequency deformations give a stronger better transparency impression

Summary of Transparency

• Transparency seen when an area of an image has “double-belongingness”
• Physical transparency is therefore not necessary for phenomenal transparency (useful, as can be exploited in painting!)
• Several geometric and photometric conditions exist that make seeing transparency likely
• Models (e.g. Metelli’s) give some insights into how to analyse transparency, although do not perfectly predict our perceptions
• Motion can potentially override cues that would — on their own —suggest transparency is not present!

Luminance & Contrast: Contrast and Lightness

• Learning Objectives
  • Understand how adaptation scales visual sensitivity over a large range of image luminances
  • Define and distinguish lightness & brightness
  • Define lightness constancy & identify examples of it
  • Understand mechanisms behind simultaneous contrast
  • simultaneous contrast illusion
  • Identify border contrast effects
  • Mach bands
  • Craik-O’Brien-Cornsweet illusion

Range of Luminances

• Range of reflectance in the natural world ~ 0.05 to 0.95, i.e. a 20:1 ratio
• Range of illuminances is huge:
  • candle (at one metre) = 1 lux
  • Sun on earth’s surface = 50,000 lux
• Range of luminances (proportional to reflectance times illuminance) is huge

Adaptation

• Illumination of objects may change dramatically
  • visual system more concerned with encoding reflectances of objects than their luminances, which change radically with prevailing illumination
• Key to minimizing the influences of fluctuations in illumination is the notion of adaptation

Local Adaptation

• Reflectance
• Brightly illuminated black (A) might have a luminance greater than dimly illuminated white (B): Still looks black, however!
  • suggests that adaptation can occur locally

Lightness vs Brightness – Key terms

• lightness: perceived reflectance
• brightness: perceived luminance
• perceived lightness is akin to perceived reflectance
• perceived brightness is akin to perceived luminance

Lightness Constancy

• Lightness of real objects illuminated in a natural environment is largely independent of the overall level of illumination
  • e.g. a book appears to have same lightnesses (print & page) under bright or dim light

Anchoring Theory

• In a variety of experimental contexts, the highest luminance is perceived as white
• Likely to be more complex, though: “anchoring of perceived luminance to white” may be a consequence of the particular experimental conditions explored

Simultaneous Contrast

• An object of moderate reflectance may appear lighter or darker according to whether it is surrounded by a region that is considerably darker or brighter than the object itself

Lateral Inhibition and Contrast

• Helmholtz (1867) - blackness results from the absence of light: “a spot in the visual field that sends no light to the eye is seen black”
  • did not realise that blackness required the simultaneous (or immediately successive) presence of objects of different luminance
• Chevreul (1839) - provided practical demonstrations of how simultaneous contrast may affect the sensations of lightness and colour of juxtaposed surfaces

ON-center & OFF-center Cells

• ON-centre & OFF-centre cells = two antagonistic mechanisms responsible for brightness & darkness sensations
• In addition to ON-centre cells seen previously, can also have the opposite: OFF-centre cells

Border Contrast Effects

• Mach bands
• Craik-O’Brien-Cornsweet illusion

Relevance to Art

• Rembrandt simulates the very large (>100 fold) range of scene luminances despite limited range (~20 fold) of pigment reflectances by using both gradual background changes & local abrupt changes
• Remember: local adaption = local image area scaled to have lightnesses comparable to typical ~20 fold range of object reflectances (and, therefore, ~20 fold range of luminances) when illuminated by a single source
• Gradual shifts in scene luminance between areas not well signalled by the eye and so don’t need to be accurately represented
• Altering the lightness of an object by manipulating the background in the opposite direction is known as countershading.

Summary of Contrast and Lightness

• Vast variation in luminance potentially exist across visual scenes
  • but, expect only ~20 fold variation in luminances in any small area with a single source of illumination
• Adaptation can shift comparatively small range of visual responses of visual system to match the luminances in any area of image
  • Light and dark represent the extremes of this scales range
  • Range commonly anchored to highest local luminance
• Lateral inhibition effects can alter perceived luminance of an area:
  • Exploited in illusions (Mach Bands / Craik-O’Brien-Cornsweet illusion)
  • Exploited to highlight luminance difference in visual art (e.g. countershading)

Example MCQ

• What process makes us relatively insensitive to gradual changes in luminance, and what visual illusion exploits this?
  • b) Lateral inhibition; Craik-O’Brien-Cornsweet illusion
• Which of the following is a process that allows the visual system to retain its sensitivity across a vast range of luminances?
  • d) Adaptation