W4L2 - Visual Imagery and Concepts

Kosslyn and Schwartz Theory

• A theory on the algorithmic level

• Describes how images are represented and processed

• Does not describe the brain

  • (i.e. what brain areas are doing the representation?)

The visual buffer

• A medium where mental images appear

  • Similar to the “visuo-spatial sketchpad” of working memory

• Like a TV screen – the visual buffer has a particular size and resolution

  • Higher resolution near the center

  • Also has a coordinate system

• Mental images appear in the visual buffer, but fade away if not refreshed

Representation - name the two types and explain

• Surface representation: the actual mental image that appears in your visual buffer

• Deep representation: information in long term memory about the visual properties of an object

What are the two types of Deep representation

• Propositional encodings: “A chair has a seat, legs, and a
  back”

• Literal encodings: “One time I saw a chair and it looked
  like this”

What are the four image processes and explain them

• Kosslyn and Schwartz imagine specific operations with names

• Image generation

  • Create a new surface representation (image) in the visual buffer, based on information from deep representations

• Image inspection

  • Study a surface representation to answer questions
    about it

  • E.g. “Does a rabbit have a tail?”

• Image transformation

  • Change a representation

  • E.g. rotate

Practice question: 

Bernadette is daydreaming and imagines a battleship. Because Bernadette is a military history nerd, she remembers that her favorite type of battleship has 3 masts. The knowledge of how many masts the battleship has is:
▪ A surface representation
▪ A propositional encoding
▪ A literal encoding
▪ An image transformation

NEED TO CONFIRM: A propositional encoding

Aphantasia

• Some people cannot use visual imagery

• People with less vivid imagery tend to have less vivid imagery across all senses, and also have less vivid episodic memories

• But they have no problem with visual working memory

What are concepts?

A concept is a category

• Objects (e.g. “dog”)

• Actions (e.g. “run”)

• Qualities (e.g. “purple”)

• Abstractions (e.g. “justice”)

Classic idea of concepts

• A list of defining features

  • “A dog has four legs, a tail, and barks”

• Membership is all or none

• But not all concepts have such a list...

Wittgenstein

Wittgenstein suggests “family resemblances.” A chair is more or less similar to
other chairs, but doesn’t necessarily share any particular feature in common.

EXAMPLES:

• A “game” does not have a list of defining features

  • “Fun to play” – but many other things are fun besides games

  • Games all have different objectives, rules, and participants

  • What do hockey, Pong, and Dungeons & Dragons actually have in common

• Concrete objects have this problem too

  • Chairs usually have four legs, but not always

  • Chairs usually have backs, but not always

  • What about a beanbag chair? A wheelchair?

  • “Used for sitting” – but so is a stool, a bench, a sofa…

What are three other mysteries of concepts?

• People define concepts partly by typicality

  • Quicker to define a robin as a bird than a penguin as a
    bird

• Some features are more central than others

  • A robin that can’t sing is still a robin

  • A robin that doesn’t have wings is a very strange robin...

• Some concepts are ad hoc

  • E.g. “things you would remove from a burning building”

  • You might not already have a list of these things in your mind

  • You might make one by imagining what you would do in that situation.
    

Perceptual simulation

EXAMPLE:

Wu and Barsalou (2009): asked subjects to list features of an object

• Features of a watermelon:

  • Green, oval

• Features of half a watermelon:

  • Red, full of seeds

THEREFORE: Maybe we are not storing a list of features

• Instead we are using sensory imagery

• We visualize a watermelon and then inspect the features of the image!

Statistical accounts

• Some concepts do not have sensory features that can be easily visualized

  • “Respect,” “justice”

• Lakoff & Kovecses: how do we represent “anger”?

  • Correlations with other words (“irritation,” “resentment,” “rage”)

• Experiences with being angry

• Perceptual simulations based on metaphor

  • “He blew his lid”

  • “She was letting off steam”

  • Anger as a heated fluid under pressur

Which theory is correct? Perceptual simulation or Statistical accounts

• It seems we use both perceptual simulations and statistical relationships

  • Both at once for the same concepts!

  • Abstract concepts may rely more on statistics

• Louwerse et al (2015)

  • When people are under time pressure, they use more statistics

  • When people are able to be slow and deliberate, they use more perceptual simulation

• The exact role and importance of both is still debated