Reinforcement Learning, Rewards, and Human Behavior

Reward

Feedback that tells an RL agent how good an outcome/action was.

Dense rewards

Frequent feedback given throughout the task.

Sparse rewards

Rare feedback, such as only win/loss at the end of a game.

Intermediate rewards

Rewards given at steps before the final outcome to guide learning.

Continuous rewards

Rewards available over a continuous range or ongoing process.

Model-based reinforcement learning

Uses or learns a model of the environment to plan actions.

Model-free reinforcement learning

Learns from experience without explicitly modeling the environment.

Q-learning

Learns action values based on expected future rewards.

Policy search

Learns a policy/rule for choosing actions.

Active reinforcement learning

Learns by exploring actions and updating values/policies.

Passive reinforcement learning

Learns values while following a fixed policy.

Inverse reinforcement learning

Infers a human’s goals/preferences/reward function by observing choices.

Taxi route prediction from GPS

Example of inferring destination/route preferences from observed human behavior.

Inverted pendulum

Control/RL benchmark problem, not the IRL human-behavior example here.

Chess neural network

Game-playing ML example, not the taxi-driver IRL example.

Condition-action rule for braking

Rule learned from sensor patterns and human braking behavior near stop signs.

Self-driving car agent

Autonomous agent that uses sensors, models/rules, and decisions in a safety-critical environment.