CGSC 3601 Midterm

What are the Sub-Disciplines of cognitive science?

1. Philosophy

2. Linguistics

3. Anthropology

4. Neuroscience

5. Computing

6. Psychology

When was the Cognitive Revolution?

The 1950s

What is a Symbol?

A language-like pattern that provides “distal access” (stands for something else in the mind, brain, or the world)

What was Newell’s definition of a Symbol?

1. Behave as an (almost) arbitrary function of the environment (universality)

2. Operate in real time

3. Exhibit rational, i.e., effective adaptive behaviour

4. Use vast amounts of knowledge about the environment

5. Behave robustly in the face of error, the unexpected, and the unknown

6. Use symbols (and abstractions)

7. Use (natural) language

8. Exhibit self-awareness and a sense of self

9. Learn from its environment

10. Acquire its capabilities through development

11. Arise through evolution

12. Be realizable within the brain as a physical system

13. Be realizable as a physical system

What was Newell’s definition of General Intelligence Action?

1. Has changing goals

2. Searches over possible ways to achieve the goals

3. Can abstract from the specifics of a situation for reasoning (i.e. must use symbols)

What is a Physical Symbol System?

A system that takes symbols, combines them into expressions and manipulates them (using processes) to produce new expressions.

What is the Physical Symbol System Hypothesis?

A physical symbol system has the necessary and sufficient means for general intelligent action

What are AI & Cognitive Models?

1. AI: Algorithms that solve practical problems accurately and efficiently

2. Cognitive Models: Simulate the mind/brain to produce human-like behaviour

How are AI & Cognitive Models evaluated?

1. AI: Based on accuracy, efficiency, and computing cost

2. Cognitive Models: How human-like it is

What is a Cognitive Model?

A theory of the human mind/brain

How does a MINERVA Model work?

1. Each memory is a list of features (vector)

2. Each new memory is put in a new row of the memory matrix

3. When cued, an echo is produced (a sum of all memories weighted by the cubed similarity to the cue)

What is Narrow AI?

An algorithm that solves a particular kind of problem

What is AGI?

An algorithm that is generally intelligent

What is Supervised Machine Learning?

When a computer learns from labeled examples with known correct answers

What is Unsupervised Machine Learning?

Giving a computer unlabeled data and letting it discover patterns or groups by itself

What is Generalization (in classification models)?

Performing well on new, unseen examples, not just memorizing the training data (can result in overfitting)

What is Overfitting?

A model that tries to maximize correctness on the training data may start memorizing small details and noise that do not matter in general

What are the Key Steps of supervised machine learning?

1. Model Training: The input data is labeled and the program has access to the answer

2. Model Testing: The model makes its prediction without seeing the actual answer BUT the human evaluating the model has access to the answer

3. Model Application: The model is used on data where the answer is not known

What should input Look Like?

A common form is features + outcome

How does the Holdout Method work (for training vs. testing)?

Randomly split data into a training and a testing set (e.g. 70/30, 80/20, etc)

How does the Bagging Method work (for training vs. testing)?

1. Training examples are chosen by random sampling with replacement(i.e. the same example can be chosen over and over)

2. Keep sampling until you have as many examples in the training set as the total data set (e.g. 100 examples, 80 examples, etc)

3. All examples NOT in the training setare in the test set

4. About 1/3 of the data is in the test set and 2/3’s is in the training set

How does k-fold cross validation work (for training vs. testing)?

1. The data is divided into K equal-sized parts (folds)

2. One part, K₁, is left out

3. The model is trained using the other parts (K₂, K₃, and K₄)

4. The model is tested against the left out part, K₁

5. This process is repeated with each different part, (K₂, K₃, and K₄) individually used as the testing data

6. The results of all tests are combined at the end

Do large data sets need more or less folds?

Less folds are needed for large data sets

What are Features in supervised machine learning?

Properties of the data

What is Feature Engineering?

Identification of features that “best” represent the target

What is Feature Selection?

Keeping the “best” subset of features

What is Feature Reduction?

Creating a subset of new features that summarize the important aspects of a full set of features

What are Limitations of the overall accuracy of a model (correct/total)?

It cannot tell you what kinds of mistakes the model is making

What is another term for TPR?

Sensitivity

What is another term for TNR?

Specificity

How do you calculate the FPR?

1 – specificity (TNR)

How do you calculate the FNR?

1 - sensitivity (TPR)

(*special case*) What are you calculating if you have the same amount of Actual Cases across two different cases?

The overall accuracy

How does a Baseline Model work?

Predicts the most commonly occurring class every time

What are the options for Finding the Best Model?

1. Choose the one with the highest accuracy

2. Choose the one with the highest accuracy when compared to the baseline model

How can you determine if one model is significantly better than another?

You can use null hypothesis testing (p value)

What is the Bayesian Approach to Reasoning?

Both humans and artificial agents act and/or make decisions based on beliefs about states in the world

What are the different types of Probability?

1. Prior Probability P(H): what you believe the probability is before seeing new evidence

2. Posterior Probability P(H|e): the updated probability after seeing new evidence

What does “P(H)” (prior probability) stand for?

Probability of the hypothesis before seeing the evidence

What does “P(e)” (marginal) stand for?

Probability of the evidence under all hypotheses?

What does “P(H|e)” (posterior probability) stand for?

Probability of the hypothesis given the evidence

What does “P(e|H)” (likelihood) stand for?

Probability of the evidence given that the hypothesis is true

What does “P(e |¬H)” stand for?

Probability of evidence being true given that the hypothesis is false

What is Fan Effect?

Participants are slower to judge pairs that contain items that occur in more pairs in the study set (i.e., have a higher fan)

What is Procedural Memory?

A production system/expert system that uses rules

What is Declarative Memory?

Based on Bayesian Probability (start with an initial belief and update it when you get new evidence)

What is a Naïve Bayes Classifier?

A Bayesian Network that finds the probability of a hypothesis H given evidence X for each relevant variable related to X (called attributes)

What is the goal of any Classifier?

To predict the outcome class based on a set of features (also called attributes)

When is a Naïve Bayes Classifier useful?

When you cannot separate one group from another using a straight line

What does it mean to Clamp a baysian network Node?

The value of the node is held constant instead of being allowed to change during processing

What does ACT-R do?

It is a platform for designing computational cognitive models of human behaviour

What components make up ACT-R?

1. Declarative memory (all facts)

2. Procedural memory (all rules)

3. Working memory (currently available facts)

What are the two different types of knowledge (in ACT-R?)

1. Facts

2. Rules (if all of the preconditions are met, the rule fires)

Where is knowledge (facts & rules) stored in ACT-R models?

The knowledge base

What are other terms for ACT-R systems?

Production systems or expert systems

How are preconditions met in ACT-R systems?

The preconditions must correspond to facts already in working memory

What are the steps for making an ACT-R Model?

1. Setup

2. Define an environment

3. Define an agent

4. Run the model

What are the steps for Defining an Agent in an ACT-R model?

1. Name the class

   • class Agent(ACT-R):

2. Create a working memory buffer

   • working_memory = Buffer ()

3. Initialize the working memory

   • working_memory.set(“slot1:value1”)

4. Define production rules

   • def rule1(working_memory=”slot1:value 1”)

     • print(“blah blah blah”)

     • 
       

How does ACT-R represent knowledge?

slot:value pairs (e.g. colour:red, size:large, etc.)

What does ! mean in def rule2(working_memory=”slot1:!value1”)

This rule will fire as long as slot1 doesn’t contain value1

What does ? mean in def rule2(working_memory=”slot1:?”)

This rule will fire regardless of what is in slot1

What is the difference between working_memory.modify vs. working_memory.set

1. Modify changes some of the slots (adds/changes)

2. Set changes all of the slots (replaces everything)

What does ?x mean in def rule2(working_memory=”slot1:?x”)

It matches to any value in slot1 and defines x to represent that value

How do you end an ACT-R simulation?

self.stop()

How do you create a Long-Term memory in ACT-R?

1. Create a buffer for holding retrievals from memory

   • recall = Buffer ()

2. Create a declarative memory using that buffer

   • memory=Memory(recall)

3. Define at least one production rule that stores memories

   • memory.add(“slot1:a”)

4. Define at least one production rule that requests memories

   • memory.request(“slot1:a”)

5. Define at least one production rule that uses the retrieval buffer

   • def memory_recall(recall=”slot1:a”)

What is Reinforcement Learning?

Learning what action to take in order to maximize a numerical reward signal

What are the requirements of Reinforcement Learning?

1. Sensation

2. Action

3. Goals/reward function

What do Agents need to make decisions?

Preferences for the various outcomes (also called states)

What are two things an Agent must consider when acting under uncertainty?

1. Utility

2. Proability

What is Decision Theory?

Probability theory + utility theory

What is Probability Theory?

The math used to describe and calculate uncertainty

What is Utility Theory?

A way of measuring how desirable outcomes are so that decisions can be compared

What does the Utility Function U do?

It assigns a value to each outcome based on its desirability to the agent

What does EU in EU(d) stand for?

Expected Utility

What does d in EU(d) stand for?

Decision

What does P(s) stand for in (SUM OF) P(s)U(s)

Probability of outcome s

What does U(s) stand for in (SUM OF) P(s)U(s)

Utility/value of outcome s

How does a Rational Agent choose an action?

It picks the one with the highest expected utility

What is Preference Elicitation?

Learning someone’s preferences so those preferences can be used in decision-making.

What is Policy (in reinforcement learning)?

Defines the agent’s way of behaving at a given time

What is Reward (in reinforcement learning)?

At each step, the agent immediately receives a positive or negative number from the environment

What is Value Function (in reinforcement learning)?

The total amount of reward an agent can expect to accumulate over the future

What is Q Learning?

A reinforcement learning method where the goal is to learn a policy that tells the agent what action to take in each state

What is a strategy used by Q-Learning?

1. Start by making random moves

2. After many moves, the agent will learn how to optimally beat the game (least number of moves)

3. Achieves this via a Q Table

What do the cells in Q Tables stand for?

The Q value in each cell of the table represents the estimated reward for taking the action A at the state S

What should Q-Table values be Initialized to?

0 for regular values or -1 for impossible values

What are the two options when choosing an action using a Q Table?

1. Select action with highest expected reward (exploitation)

2. Select action randomly (exploration)

What is the Standard Approach to Q Table navigation?

Use a predefined threshold called an epsilon, which controls how often an agent explores instead of using what it already thinks is best

What does the s in Q (s, a) stand for?

State

What does the a in Q (s, a) stand for?

Action

What is Q (s, a)?

Current reward + expected future reward

What does Y stand for in Q-Learning?

Discount rate

What is the range for Discount Rate (Y)

1. Y = 0: Only care about immediate reward

2. Y = 1: Care equally about immediate and far future rewards

Why introduce a Discount Rate?

Animal/human behaviour shows preferences for immediate rewards

What is credit assignment?

Deciding which earlier actions were responsible for a later reward or penalty

What does R(s) stand for?

Reward for a given state