cognitive science - information theory

who revolutionized the way we think about and quantify information?

Claude Shannon

what is information theory?

the field of study of the way we think about and quantify information

what term did Claude Shannon coin?

bit

what is a bit?

a way of quantifying information regardless of its form

what does Shannon say information is?

the resolution of uncertainty

do surprising or ordinary events contain more info?

surprising events

how much info do we receive when we are told that the outcome of a fair coin flip (P(heads) = 0.5) is heads?

1 bit

how much info do we receive when we are told that the outcome of a unfair coin (P(heads) = 1) is heads?

0 bits

how much info do we receive when we are told the outcome of an unfair coin flip (P(heads) = 0.9) is heads?

more than 0 but less than 1 bit

what is a binary memoryless source?

two outcomes represented by an alphabet of 0 and 1 and symbol probabilities of 0.5 each

what is Shannon’s equation for calculating the amount of self-information contained in a symbol?

I(ax) = -log2(px)

what is I(0) when px = 0.5?

-log2(0.5) or 1 bit

what is I(0) when px = 1?

-log2(1) or 0 bits

what is I(0) when px = 0.9?

-log2(0.9) or about 0.15 bitswha

what is information entropy?

information contained in a source, which is a weighted average of the self-information of each of the possible symbols

what is the equation for information entropy?

H(X) = -sigmapxlog2(px)

what is channel capacity?

theoretical maximum rate at which a message can be reliably transmitted in bits per second over a communication channel

do languages have the same amount of information per syllable?

no, because they differ in the number of possible syllables allowed

do languages with many possible syllables, like English, or languages with few possible syllables, like Japanese, contain more information in one syllable of speech?

languages with many possible syllables, like English

is the effective information rate different across languages?

no, it is relatively similar because of natural variation in speaking rates

what natural variations in speaking rates allow for effective information rate to be similar across languages?

languages conveying more info per syllable are spoken more slowly and languages conveying less information per syllable are spoken faster

what happens when you speak too quickly?

info processing overload

what happens when you speak too slowly?

inefficient use of channel

what is the uniform information density hypothesis?

language users prefer to distribute info uniformly over a message

according to the uniform information density hypothesis, what do speakers do when they are about to produce something very surprising?

they slow down, allowing themselves or the listener more time to process the information

what is Hick’s law?

the more possible responses there are, the longer it will take to choose the correct response

what does Hick’s law predict?

reaction time as a function of the number of choices

what is the equation for Hick’s law?

RT = a + b(log2(n))

what does a refer to in Hick’s law?

the constant time it takes to perceive stimulus and execute response, the intercept

what does b refer to in Hick’s law?

the rate of gain of information, the slope

what kind of increase is seen when there is an increase in choice RT with the log of the number of stimulus-response alternatives?

a linear increase

according to the Hick-Hyman law, when designing an interface where the user may need to make rapid decisions, do you want more or fewer interface elements?

fewer

what does Fitts’ law tell you?

how long it should take to execute a movement to a target

what is the equation for Fitts’ law?

MT = a + b(log2(2D/W))

what does MT refer to in Fitts’ law?

movement time

what do a and b refer to in Fitts’ law?

the intercept and slope, which are empirically determined

what does D refer to in Fitts’ law?

distance

what does W refer to in Fitts’ law?

width of the target

what does a smaller W mean for how long it should take to execute a movement to a target under Fitts’ law?

should mean more precision required to make movement

what does a longer D mean for how long it should take to execute a movement to a target under Fitts’ law?

makes precise movements harder

what does movement depend on?

relative precision, the ratio of D/W

under Fitts’ law, for which target would you expect a slower movement time, once closer but less wide or one further but wider?

movement time is the same because the ratio of D/W is the same in both cases

according to the Zheng & Meister reading, what is the information throughput of human behavior?

about 10 bits per second

according to the Zheng & Meister reading, what is the ratio of rate of sensory information input to rate of information throughput of human behavior?

about 100,000,000

according to Zheng & Meister, how many bits per second can a single neuron transmit?

about 10

what does Zheng & Meister say is the reason for the difference in peripheral and central information rates?

the peripheral system largely uses parallel processing while the central system seems to be strictly serial