PSY280 Lecture 8: Motion & Intro to Colour

What information does visual motion provide?

1. Helps recognize objects

2. Helps distinguish depth relations

3. Helps us interact with the environment

4. Helps direct our attention

3. Environmental Interaction- What is vection? What is optic flow?

• Vection: perception of self-motion due to optic flow

• Optic flow helps us know which way we are headed

  • Example pilot uses this to land a plane, using optic flow cues to judge where they are going to land

  • It's the expansion of the image on your retina as you move forward or the contraction of it as you move backward, but it changes in the expansion of the retinal image as you move through space

• Also underlies phenomenon called vection

  • If show motion may feel like moving but not actually

  • In movie theatre, sitting up close, IMAX movie, feel like flying or moving but are actually stationary.

  • In car/train and car next to you starts moving, that's next to you.

3. Environmental Interaction- What is Optic Flow & Tau?

• Need to know when something is going to collide with you (like a baseball or a boxing glove)

• Could calculate this based on how far something is & how fast its moving: Time to collision = distance/speed

  • Problem is the visual system is bad at estimating distance

• Useful for knowing when something going to reach/hit me.

• One possibility, slow, clunky inefficient way-to calculate it based on how far something is and how quickly it is moving.

  • We do not do this

  • Use high school physics formula time to collusion=distance/speed

  • Very slow & clunky way, not way brain does this

  • If use this garbage at figuring/estimating out distance

  • Use a lot of depth cues but not very precise

  • Instead if want to know when something will hit me a more efficient way is to do faster computer & use rate at which the image expands, use our optic flow information to figure out when something will hit me.

What is Tau?

• Optic flow gives us access to Tau (τ), which is related to time to collision but is easier for the visual system to calculate than physical distance

• Tau is inversely proportional to the rate that the retinal image expands

• Objects expanding quickly will reach you sooner than objects expanding more slowly

• Slow rate of expansion: it will take a long time for this baseball to reach me

• Fast rate of expansion: it will take a short time for this baseball to reach me

• Many organisms use Tau, including gannets, flies, and pigeons (who have specialized cells that respond at a fixed interval before contact)

3. Environmental Interaction

• Motion capture can influence reaching, pointing & throwing

• Need to factor in motion to figure out where I'm reaching

• Motion influences reaching, pointing and throwing

• Fo example in basketball if you have people who are adding optic flow by adding these arrows & moving them, there's motion back here that can throw people off

• Because you have this very strong optic view, the environments moving this way, maybe I need to counteract that.

• But motion is very important for impacting reaching, throwing & pointing motion

4. What is Guided attention?

• When have motion can’t help but pay attention to it. Motion captures attention.

• For example, look for the backwards ‘R’ among a bunch of ‘Rs’. This is very hard to do. However when added motion to the image, captured attention instantly.

Interim Summary

• Motion is critical to many aspects of vision

  • Object recognition (structure from motion)

  • Depth perception (parallax)

  • Navigation (optic flow)

  • Attentional Guidance (visual search)

How do we perceive motion? Is motion directly measured or inferred from position change?

•  One possibility is motion is directly measured by our visual system

  • Cells selective for motion & have distinct physiological mechanisms for sensing motion directly

  • Neurons in brain sensitive to movement

  • Something we can potentially directly tap into via the responses of our cells.

  • Have evidence that motion is something directly measured not inferred by our visual system.

  • We have some evidence for these low-level detectors. So the idea is that there is some evidence that we have neurons in our brain that are directly sensitive to movement & operate at very early stage of visual processing

 

• Another possibility is the slower & clunkier route, which is that we might have to infer motion from position change.

  • Need to do a comparison. This object was here at time 1 and this object was here at time 2. So here it's like an A-B comparison.

  • I've noticed that this thing changed position so I infer because it has changed position that there must be movement

  • Indirect, clunky, included inference

• Evidence for low level detectors

  • Motion aftereffect, we perceive motion due to adaptation

    • Looking at something for prolonged time, decrease in firing rate of neurons that respond to that stimulus

    • Can do same thing and apply it to movement

• Kinematograms, motion then shape

Direct measurement of motion- What is Motion After-Effect (MAE)?

• Star at blue dot in middle of screen, motion is going inward. Then shown photo of stationary Buddha but it looks like its moving outward to you. Looks like it is expanding

• Motion is in opposite direction of what you adapted to.

• Looking at something stationary but moving

• Motion directly measured & NOT inferred from position change.

• Adapting to motion & seeing something opposite of what you adapted to. Looking at something stationary yet perceiving movement. This suggests motion really cannot be inferred from position change because I have something that isn't moving at all, there is no physical position change happening with the Buddha & yet you are seeing movement

  • NOT inferred from position change but motion is directly measured

  • No physical movement but seeing movement.

What is Motion After Effect (MAE)?

•  Similar to tilt aftereffect

• Have different populations of neurons- some neurons prefer inward, some prefer outward.

Pre-test

• When looking at something not moving/static, they are both firing a little but but not a lot, firing at their spontaneous firing rate. They are in balance, perception stationary, nothing moving

Adaptation

• When looking at movement, physical stimulus moving inward, inward cells are happy and have a high firing rate for inward motion

• Outward motion cells at their spontaneous rate

• Now tipped in favour of inward preferring neurons, perception of inward movement

• While adapting seeing inward. Overtime, when adapt firing rates will decrease. Fatigue analogy.

Post-test

• After adapted for prolonged period, looking at static image, inward preferring cells have adapted out and stop responding.

• Outward preferring neurons spontaneous level of activity.

• Outward at spontaneous level, inward stop responding. Balance of system is tipped, in favour of outward motion

• Therefore perceive outward orientation

Explaining the Motion After Effect: What is the Medial Temporal Area? What is some fMRI evidence?

• Motion directly measured

• Motion processed at many different stages of visual processing, certain areas particularly sensitive

• MT (medial temporal area), V5, motion selective area

• Important for our conscious experience of motion

• Many cells in the brain respond to motion but all of those cells send cells to this area, area MT

• Compare two different conditions to see where in the brain the motion after effect is happening, what kind of responses do we see in a situation where there's no actual movement & yet we have this experience, our conscious experience is of movement

• One condition participants adapting to motion in one direction (inward movement) then look at stationary image of Buddha

•  In the second condition have random motion, neither inward or outward, flicker, random. Movement but not consistently in one condition

• Would expect motion after-effect in condition at the top and NO motion aftereffect in the one at the bottom, since its random and not moving consistently in one way.

• Compare how area MT responds to aftereffect for each condition & then look at statis Buddha image. 

Result from fMRI MAE

• Red line is the condition in which I expect to get a motion aftereffect, the dashed line represents where have shift from adapting to looking at the stationary image.

• Everything to left of dashed line is during adaptation. Right switched over to stationary Buddha image.

• Area MT responds very strongly when adapting, at dashed line have change over to static image.

• In one condition expect to get motion in one direction, and in blue don’t expect to get motion aftereffect. Red motion aftereffect perceived

• In area MT see response while looking at a stationary image but only in the condition where I perceive the motion aftereffect. Area MT is responding very strongly when I experience the aftereffect even though there's no movement.

• In the condition where there's no aftereffect activity drops pretty quickly, right away. The idea is that im seeing a stationary image, I haven't adapted to motion in one direction so there is no aftereffect.

Conclusion

• So area MT seems to be largely related to our experience of the motion aftereffect

Direct Measurement of Motion- Kinematogram

• Shape defined by motion

• Another piece of evidence that motion is something that we directly measure is something seen in kinematograms.

• See black and white TV static, now add movement to it, see a rectangle.

• When no motion there is no shape, when add motion add shape out of that motion.

• Motion and then shape

• In order to perceive shape, have to perceive motion/movement first.

• Motion directly measured, I cannot compare the position of the rectangle at one point in time to the other point in time unless I see the shape. Here this is just one point in time, nothing to infer/compare between A and B

• Shape as result of movement.

Stereograms vs. Kinematogram

• Kinematograms are similar to random dot stereograms, only with motion instead of disparity

• Random dot kinematogram are just like these stereograms except here with stereograms I had a left eye image & a right eye image.

• With kinematograms what I’m doing is just shifting them over time. Time 1, time 2, which is what lets me see the shape.

So is motion directly measured? Or inferred from position change of object?

• Saw motion where there was no change in position of object (motion aftereffect)

• Saw motion where there was no noticeable object (kinematogram)

• Therefore,  Motion directly measured by our visual system, not inferred

  • MAE

  • Kinematograms

Motion is Directly Measured

• Motion is directly measured, not inferred

• That means we have parts of the brain that are explicitly sensitive to motion (e.g, MT)

What are Receptive fields?

Receptive field: an area on the retina corresponding to an area in the visual field to which a cell is responsive

• An area out there in space corresponding to point in retina to which a neuron responds

Physiology of motion detection

 

• If I need to have a neuron that's directly sensing motion I have to have some way of coding or responding to the change in an objects position over time. That is essentially what movement is. I need some way of directly sensing that.

• Have V1 receptive field, set aside on & off regions. Looking at a picture, need some mechanism to tell me that I have one thing at one point in time & another thing at another point in time. Need some way of coding fact something is here on the receptive field on the left & then something is here on the receptive field on the right at different points in time.

• Build circuit for detecting motion

• Need to build circuit for motion detection.

• Hypothetical circuit for detecting motion-exists as mechanism for motion detection.

Physiology of motion detection- What is Reichardt Detector?

Reichardt Detector- Directionally Selective Unit

•  These motion detecting circuits are called Reichardt detectors

• They are simple neural circuits that our brains use for detecting movement

• One neuron not enough

• Have one neuron in peach, another in purple. This is my neuron & somewhere in the world is its receptive field

• Looking at bugs for example. Looking at bug in peach, if its in this receptive field, if it moves to purple receptive field, neuron 2 will respond

• Need some mechanism of combining those signals over time in order to sense motion.

• Let's say neuron 1 fires once the ladybug enters its receptive field & then it sends its signals off to another neuron. This neuron allows signal to be delayed a little but, now added a delay into my circuit & then that neuron passes the signals on to yet another neuron down here

• When lady bug enters this cells RF, neuron 2 responds & sends signals down to same neuron

• If these signals reach X signal at the same time, and these signals added together, that is what signals movement to our brain

• So this particular circuit will allow us to detect motion that is going rightward

Reichardt Detector Demo

 

• Light enters first cells RF, it fires, sends its signals to another neuron in which delays its responses for a little bit. Then the light passes through RF, then signals combined, when reach at the same time, this triggers an AP

• If do motion in opposite direction, that neuron will not respond. Depends on which side delay is on.

Reichardt Detector Demo

 

• Sensitive to rightward, will not respond to leftward motion

• If want to built another circuit this time sensitive to leftward motion, change which side delay on.

 

• The direction the circuit is sensitive to depends on which side has the delay.

Reichardt Detectors

• Mechanisms like the Reichardt detector can operate in the retina & the visual cortex (e.g, V1)

• Left to right

• Delay always on the side where the first input is received. So for leftward motion it would be on the left side.

• Exam Question: show diagram & ask which direction this circuit sensitive to, based on which side has the delay.

 

• Can operate in the retina & the visual cortex (V1)-complex & hypercomplex cells.

• Delay fixed

What if we presented two images successively in the RFs of a Reichardt detector at the cells preferred timing?

 

• Can trick this system into seeing motion where there is actually none.

• Have Reichart detector, can show something in RF of this cell and then this cell over here where there is no actual movement

• Have pic of dog in first RF in one point of time, I take it away and I put it over here at another point in time. Notice there is no actual movement but this will trigger my circuit to respond.

• Interpret as motion even though there is no motion.

• Trick the system.

• Work when have moving object but also if have two stationary flashes

What is apparent motion?

• Reichardt detector operates when a moving object is presented to two adjacent receptive fields sequentially

• But two stationary flashes of light should work just well

Examples of apparent motion

• Lights at entrance of theatres

• Vegas

• Airport Runway Lights

• Movies, TV, computers, cellphone displays, etc

  • When you’re watching movies or TV, all you’re seeing is a series of stills. Not actual movement, series of stills

  • Videos made up of set of still frames

• Christmas lights

• Animations just a series of frames

• Flip Book

 

• Impression of motion when don’t have physical movement

• Two different frames, frame 1 & 2. One in one point of time and another dot at another point of tie.

  • Called apparent motion.

  • Perceive movement when there isn't anything physically moving but triggering these detectors

Apparent motion using barrier grid animations (scaminations)

• Barrier grade animations

• Horse example, horse, as moved the grid looking like it was running

  • When slide over way running over way

  • When sliding it quicker, running quicker, when slow down the running slows down

 

• Cover & uncover parts of image to uncover different portions of the image

• Brain creating impression of motion from a series of still frames

Middle Temporal are (MT) or V5

• The middle temporal area (MT) or V5 (same area that liked MAE) also has neurons that are directionally sensitive & is important for our experience of motion

Motion Blindness- What is Akinetopsia?

• Damage to area MT (middle temporal lobe) results in Akinetopsia

• Lose ability to detect motion/movement

• Incredibly rare cases, has to happen on both sides

 

 

• Like living under a strobe light

• One person here at one point of time, another at another point of time

• Pouring coffee or tea, see motion when see level change, if don’t see motion, at one point in time its like oh my cup is empty, the next thing you know you spilled coffee or tea all over the table because you didn't detect that motion that led to that change.

• Have to rely on this comparison of one point in time to another point in time

Interim Summary

• Motion is critical to many aspects of vision

• Motion is directly measured, not inferred

  • MAE

  • Kinematograms

• Reichardt detectors

• Apparent motion

• Akinetopsia (Motion Blindess)

Low-Level, Mid-Level & High-Level Vision

The Importance of Perceiving Colour

• Foraging

  •  Camouflage

  • Helping us find foods to eat

• Finding mates

• Finding which food is most cooked

Colour is not just a physical property, but a psychophysical property

 

• How map on physical property to psychological state.

• Can only see small sliver of these wavelengths 400nm-700nm (visible light)

• Shorter on left, we perceive as blue

• Longer wavelengths, we perceive as red

• Photons themselves have no colour, it is how we perceive them.

• Nothing about photon itself that gives it its 'blueness or redness'

• It is psychophysical property: have some physical stimulus and mapping it to a psychological state.

  • So when we say colour is a psychophysical property it has to do with how we map on the physical property, that is the wavelength of those photons to our psychological states to us

  • Shorter wavelength impression of blue, nothing about photon itself, how the brain interprets it

Different Organisms are sensitive to different wavelengths

 

• Our peak sensitivity is in the visible spectrum but it also corresponds to the maximum output of the sun which is our light source

• Different organisms that can shift around a little bit

• Bees different range of wavelengths

• Bees shorter set of wavelengths, closer to 300nm-650nm

• We see wavelengths they cant, they can see wavelengths we can't

 

• Flowers look different in different wavelengths, allows bees to pick up on this

• Valuable to different organisms and different organism are sensitive to different wavelengths

Sunlight is composed of many different wavelengths

 

• Sunlight our main light source is composed of many different wavelengths

• Light from sun white light consists of whole range of different wavelengths 400-700nm

• Here we have white light going through glass prism (discovered by Newton), light refracted differently depending on the different wavelengths.

• Psychophysical property: he noticed it was 'in our heads'

• Colour not property of light itself but in our brain.

Objects Reflect Light at Different Wavelengths

 

• Why is tomato red? A blueberry blue?

• We have the sun which is our main light source, inputting a whole bunch of energy in the visible range but also outside of the visible range. Sun emitting lots of electromagnetic energy. All these photons bombarding from 700-400nm the tomato.

• Different objects reflect different kinds of wavelengths

• Tomato reflecting longer wavelengths which we interpret as red.

• Light that doesn't get reflected gets absorbed.

Reflectance Spectrum

Different Objects have Different Reflanctance Spectra

 

• What proportion of photons are reflected

• For the blueberry selectively reflect more short wavelength photons, not so many of long wavelength photons

How is colour coded?

• Photons themselves are NOT coloured. The light that hits your eye looks like something like this

How is colour coded? What are the 2 theories?

1. Trichromatic Theory

2. Opponent process theory

• Both are true, both describe how colour perception works but operate in different stages

• Colour processed first in trichromatic theory then opponent process stage/theory

What is Young- Helmholtz Trichromatic Theory?

• There are only 3 cones, each broadly tuned to wavelength

• Perceived colour depends on the relative strength of their activation

• Any colour can be formed by combining different amounts of 3 primary colours (corresponding to 3 cone types)

•  Observed that there are 3 kind of receptors & each is broadly tuned to one part of the spectrum or one range of wavelengths

  • Short, medium, long

  • Colour we perceive is determined based on response of each type of cone, short, medium and long wavelength cone & that determines the colour.

  • Have all this information, all these different wavelength photons are getting picked up by my cones. Idea is have some response from each type of cone, for my short, medium & long wavelength cone and the idea is that any colour I perceive can be combined by just combining different amounts of short, medium & long wavelengths. Or different amounts of red, green & blue.

Cone photoreceptors

 

• 3 cone types.

• Still pick up other colours not just blue, green or red. But what they maximally pick up on.

•  S, M & L better representative

  • Not actually coloured this way.

Cone photoreceptors

• Images of the 3 kinds of cones in the living human retina

• Colours used here are just the popular labels for each kind of cone

• Pattern of trichromatic cones differs greatly from person to person

• Our colour perception is largely similar, despite these differences. This is likely due to long-term auto calibration process.

• Distribution of cones in the retina, not actually coloured this way.

• In general people tend to have more long wavelength cones compared to medium wavelengths cones and more medium wavelength cones than short wavelength cones.

• Variation between person to person

• But people usually agree and colour perception tends to be similar broadly speaking across individuals

Young- Helmholtz Trichromatic Theory

• Perceived colour varies with ratio of responses of the three cones

• One response from short, another from medium, another to long. Blueberry respond most to short, least to long wavelength photons.

• Big triangle big response, little triangles little response.

Human Colour Vision has only 3 dimensions

• Perceived colour varies with ratio of responses of the three cones

•  Young-Helmholtz trichromatic theory proposed that the colour that we see just depends on how much activation do I have from each type of cone.

  • Ratio of response of 3 wavelengths gives us colour.

Human Colour Vision has only 3 dimensions

• Human vision measures only 3 values, one for each cone

• Your brain only knows what the cones tell it

• non-invertible: can’t recover the wavelength distribution

 

• Colour is noninvertible cannot work way backward

• Only information I have access to is how much of a response I have from each cone type, I can't solve backwards and figure out what kind of spectrum produced that pattern of responses

Colour is non-invertible

 

• Cannot recover the true reflectance spectrum out there in the world, only have 3 numbers to work with

• An infinite number of different spectra could produce the same response from the 3 cones, so the brain can’t interpret which spectra is correct

What are Metamers?

Metamers: different spectra, same colour appearance

•  We can think of metamers as being confusions, where we can mix up one colour for another because we don't have the receptors to distinguish them. We are reducing information down to 3 different numbers

• Have yellow light, by mixing green & red light

• Strong response from long wavelength cones & medium wavelength cones & not much of a response from my short wavelength cones

• Have exactly the same pattern of responses for two very different sources of light.

• One kind of a broad range and the other is fairly narrow

• Metamers as confusions, colours that we can mix up because they have different spectra but they look the same to us.

Metamers

• Brain only knows the ratio of cone responses (3 numbers) & can’t tell the difference here. These two physically different stimuli will look the same.

Metamers

• Lights that have different physical spectra, but the same perceived colour