Cog Sci - Module 5 - Representations and algorithms

Computational Theory of Mind

What the mind is is a computational system 

Functionalism

Mental states are identified by their causes and effects

Levels of analysis

• An approach to studying complex, computational systems 

  • Links between levels are often incomplete 

  • Not every cognitive scientist works on every level!

Difference between levels of analysis

Computational level - What information is computed and why?

Algorithmic level - How information is represented and computed.  

Implementation Level - The physical substrate that performs the computation. 

• where is the physical area the computation performed

Formal systems 

• A Formal system takes symbols, combines them into expressions, and manipulates them using processes 

Mental symbols have properties

 semantic properties - they are about or refer to things in the world 

Symbols in the mind are representations- they represent the world around us 

Representation

something that stands for something else 

Ex: the image/thought of a cat representing a real life cat 

Parts of a representation

1- Bearer

2- content 

3- grounding/intentionality 

4-interpretability 

What makes a mental representation a mental representation is that it has at least these 4 properties 

Bearer

• The bearer realizes the representation 

Content

• The representation has content - it stands in for something - semantic content 

Grounding/ intentionality 

• The mental content has a relationship to a real world referent 

• Real world referent the content is linked to 

interpretability 

The representation can be used in some computation

Referent

• The thing in the external world that the representation stands in for 
  

You can do all different things with this mental representation

• Decide you want to pet the cat 

• Memory recall 

• Describe the cat to your friends 

Concepts

• The building blocks of thought 
  

Symbolic - stands for an idea or object but does not have a genuine resemblance 

Ex: the word “CAT” doesn't have a physical resemblance to an actual cat 

• The word is a symbol that stands in the place of a cat, generally not any particular cat specifically 

Proposition

• Complex representations with sentence-like structure that can be true of false. 

• allow us to take the building blocks to reason about, talk and conclude things of them

Ex: “The cat is on the mat”

• You do not have to be thinking/talking in English in order to use a proposition 

Propositions can represent logical relationships

English sentence:     “Mary Loves John” 

First order logic:     LOVE (m,j) - real world 

Propositions can represent counterfactuals

• Counterfactual reasoning - propositions that are not necessarily true. - What could be true of the world 

Ex:

LOVE (m,j) - real world

LOVE (j,m) - possible worl

Mental map

• Representation of spatial layout that captures info like direction and distance 

• Capture something true/accurate to what is out in the world 

Ex: tolman and mental map, rats 

Types of mental representations

• concepts 

• proposition 

• mental map 

• mental image 

Mental image

isomorphism

 1 to 1 correspondence 

• Your mental representation preserves the structure of whatever it is you're representing 

• Your mental image of a cat will look like a cat

Aphantasia

• An inability to create voluntary visual mental image 
  

There are different levels of imagining/variations in the human brain

• Aphantasia is not a disability/disorder 

• Natural variability of how the mind/imagination works 

• Differences in representation mean different algorithms 

• What those differences are…

Visual memory experiment

the set up

• Aphantasics and controls draw images based on memory and direct perception 

The question 

• Are there differences in content and accuracy between groups? 

In the study people had to recreate/draw pictures of a room while looking at the reference, and then from memory. (ppl with and without aphantasia)

• Memory- drawing from memory 

• Perception - drawing from them looking at picture 

Draw a picture from memory

• Aphantasics: cannot get picture from memory, have to use different algorithms to access pic 

Visual memory experiment results

Aphantasics used

• Fewer objects 

• Less color 

• More verbal scaffolding (words,

  descriptions)

• Using support of linguistic

 description 

• Describe to yourself whats in

 that picture 

Aphantasics had 

• High spatial accuracy 

• Fewer false objects 

Mental rotation

• Manipulation over a mental image - rotate it in your mind 

Ways to solve problem/do mental rotation

Algorithm 1 

1. Cartesian coordinates 

2. Apply mathematical transformation

3. …

Algorithm 2 

1. 3D mental image 

2. Rotate image in real time until a visual match is made 

• People with aphantasia do not do this 

mental rotation aphantasia vs non aphantasia results

• People with aphantasia were slower to respond/process if the shapes were rotated 

Aphantasics were slower, but more accurate 

• Especially as the angle of rotation was bigger, making the task harder 

• The algorithm aphantasics are using instead of mental imaging is slower but more accurate

We don't always default to the easiest way something is represented to humans 

• We start off with the way they are presented 

• These algorithm and ways of solving are things that we can learn 

Mental Representations

Concepts - abstract symbols - building blocks of thought 

Propositions - complex expressions that are true or false 

Mental maps - representations of spatial layout 

Mental images - perceptual representations without sensory input 

Mental representations - clocks - analog

• Continuous spectrum 

Analog clock- hands always moving, there is never an exact time (you can extract an exact time but its always changing

Mental representations - clocks - digital

• Discrete categorical  

• Digital clock - numbers straightforward, no in between, either 12:06 or 12:07 

Mental representations - color - digital

- discrete categorical 

•  clear differences between the colors, categorical (type of distinction we can name

• Language - a digital format of representation, a digital format of categories, guide to categorize things 

Mental representations - color - analog

 - continuous spectrum 

• color gradient instead of discrete categories, no division between the color that we would label using language,

• you don’t know exactly when one color ends and one color stops 

• (Not associated with mental image, people with aphantasia do have analog representations)

approximate number system

• Detect differences between large sets without counting 

Ex: you don't have to count out to see whats more of what - purple or pink 

• Exact difference doesn't matter, Ratio matters  

• Weber’s Law: the discriminability of any two magnitudes is a function of their ratio 

Innate number sense

•  Humans and some animals are born with the innate ability to discriminate between sets - see how much they are without counting 

Subitizing system

• Discriminate very small sets without counting 

• Fast, automatic, and accurate discrimination of quantities up to 4 

• Discovered by William Stanley Jevons - threw beans in a box and quickly guessed how many were there 

• After 4 the guesses started to get inaccurate - as the amount got bigger, the amount he could guess got wider 

• Not about counting, about object discrimination 

Subitizing system - in infants

• Infant search patterns show that discrete object representation maxes out around three 

  • When you’re born the subitizing system maxes out at 3 

Subitizing system in infants experiment

 4 balls in a box, when taking out 4 or less, see if infants notice the difference 

• How many objects an infant can keep track of when its not directly visible to them?

Subitizing system in infants experiment - process

• Researchers took 2 or more/less balls out of the box without them seeing

• if the infants kept searching they knew there were more balls in box, if infant stopped, it means they didn't keep track of amount/know there were more in the box 

• seeing when their subitizing system stops working

Subitizing system in infants experiment - results

• For sets of 2 or 3 balls in total: infants searched more when 1 ball remained

• For sets of 4 balls in total: Infants stopped searching when 2 balls remained!
  

Subitizing system in infants experiment - conclusion

Infants can keep track of what we label as 2 or 3, but afterwards they cannot keep track anymore 

• They don't have the language to describe/name more than 3 

• after 4 it shifts to the idea of “some”

Once we cross the threshold of 3 or 4, we

don't use the subitizing system, and cannot keep count unless we start naming the numbers

• Shift from exact and discrete to proximate and fuzzy 

• For adults: 4, for infants: 3

Number selective neurons

• 1 neurons - fires when there is 1 thing noticed or seen 

• 2 neurons - fires when there is 2 things 

• When there is 5 things - 4 or 5 neurons may fire 

Two distinct signatures for number- selective neurons: 

• Equally precise for 4 and below 

• Progressively less precise for 5 and up  

Two distinct neural signatures for number- selective neurons: 

• Equally precise for 4 and below 

• Progressively less precise for 5 and up

You can match objects to quantities if you track them using something else

When having a number list

The amount you can count is matched up to the list you have

The power of symbols 

• Verbal count lists let us represent discrete quantities of any magnitude 

• Step 1 -memorizing list (numbers 1 to 10)  
  

• Step 2- learn what list corresponds to (1 means 1 item , 2 means 2 items)

Successor function

 for every one place in the list, it corresponds with every 1 thing in the world 

• Once you figure it out you can use the list to count real things

List

• Is given to you by your culture, language 

• Some languages have different count lists 

• One general list- if they count something by a particular set, they might use second count list 

• Also diff based on what they are counting- people, food 

For any way of doing things, there is a tradeoff between

• what it brings to the front vs information it doesn’t shed light on. Important in the processes of symbolization and representing things 

You were born with 2 ways to count

• Subitizing system and approximate number system 

• Maxes out at 3 

• List is learned - not innate 

What are the ingredients of a mental representation? 

• Representation - something that stands for something else

1. Bearer

   1. The bearer realizes the representation

2. Content

   1. The representation has content - it stands in for something - semantic content

3. Grounding/Intentionality 

   1. The mental content has a relationship to a real-world referent

   2. Real-world referent the content is linked

4. Interpretability 

   1. The representation can be used in some computation

Define and explain the differences between concepts, propositions, mental maps, and mental imagery.

• Concepts are the building blocks of thought

• Propositions are complex representations with sentence-like structures that can be true or false

  • Take building blocks to reason about, talk, and conclude things of them 

• Mental maps are representations of spatial layouts that captures information like direction and distance

  • They capture something true to what is out in the world

• Mental imagery 

  • Isomorphism - 1 to 1 correspondence

  • Your mental representation preserves sturcture of whatever it is youre representing - humans have but dont really need

What is aphantasia, and what does it tell us about mental representations and algorithms?

• Aphantasia - inability to create voluntary visual mental images 

There is a variety of different mental representations/ a spectrum and not everyone is the same

• People with aphantasia use different alogrithms to solve problems related to mental imaging

  • ex: drawing experiment, mental rotation experiment

  • there are tradeoffs to each algorithm choice

Give an example of representational/algorithmic trade-off.

• Choices come with tradeoff

• There is always a choice in what algorithms reuse to solve a particular problem 

• Any particular representation makes certain info explicit at the expense of info that is pushed into the background and might be hard to recover

  • Important in processes of symbolization and representing things

• John Von Neuman solved algorithm in harder way, we dont always default to easiest way something is represented to in humans

• E.g. 

  • Using spatial images -simple and flexible, but slower processing time which increases w angle of rotation

  • Use abstract code - Faster comparison, but requires rigid and complex representations

Explain what analog and digital representations are using examples.

• Analog representations - continuous spectrum

  • Analog clock - hands always moving, never exact time

  • Color - continuous  spectrum

    • Dont know where one ends and one stops

• Digital representations - discrete categorical

  • Digital clock - exact time, no in between

  • Color - discrete categorical

    • Clear differences between colors

    • Language - a digital format of representation, labels to categorize things

Describe the two innate number senses found in humans and other species. How do they compare?

• Approximate Number System 

  • Detect differences between large sets without counting

    • Innate Number Sense - humans and some animals are born w innate ability to discriminate between sets - see how much they are w/o counting

• Subitizing system - discriminate between very small sets w/o counting