Lecture 22: the antecedents of transformer models

what are the main aspects of a retrieval machine? (2)

• index of information that needs to be retrieved

• how to generate queries

why do certain argue that LLMs have the abilities to humans have (ex: language production)

because the same process would happen for humans and LLMs

what was the inspiration for transformer networks?

LSA

what is the complexity added to transformer models that distinguishes it from BEAGLE?

• BEAGLE learns about context based on words order

• transformer network are more complex, they can do parsing (determine the grammatical word of a sentence)

would you prefer instance models or transformer networks?

transformer networks: nothing work similarly but transformer models are simpler computationally and during training

true or false: transformer can learn control processes of cognition (executive functioning)

false: it seems to be a reflection of the training material