Read w3

6.1 Units of a Neural Network

Unit Overview

6.1 Units of a Neural Network

Why Non-Linear Activation Functions are Necessary

6.1 Units of a Neural Network

Common Activation Functions

6.3 Feedforward Neural Networks

Architecture & Terminology

6.3 Feedforward Neural Networks

Vectorized Layer Computation

6.3 Feedforward Neural Networks

Summary of Dimensions

Lexical Semantics

Goal

Lexical Semantics

Lemma, Word Sense, Synonymy, Antonymy, Hypo/Hypernymy

Lexical Semantics

Similarity vs Relatedness, Affective Meaning

Lexical Semantics

Vector Semantics: Intuition

Lexical Semantics

Count-based embeddings

Lexical Semantics

Dot Product & Limitation

Lexical Semantics

Cosine Similarity Metric

Lexical Semantics

Cosine Similarity vs Euclidean Distance

Lexical Semantics

Word2Vec Overview

Lexical Semantics

Word2Vec: SGNS

Lexical Semantics

Word2Vec: Embeddings Learned

Lexical Semantics

Word2Vec: Learning via Cross-Entropy Loss

Lexical Semantics

Semantic Properties of Embeddings

Lexical Semantics

Semantic Properties of Embeddings

Lexical Semantics

Semantic Properties of Embeddings

FO vs SO co-occurrence, Limitations

Lexical Semantics

Bias and Embeddings

Lexical Semantics

Debiasing Embeddings

Lexical Semantics

Evaluating Vector Models

Lexical Semantics

Summary

Findings of Paper

Decisive Superiority of Context-Predicting Models

Findings of Paper

Failure to Dominate Only on Selectional Preferences

Findings of Paper

Parameter Robustness vs. Brittleness

Findings of Paper

Optimal Architectural and Hyperparameter Insights

Findings of Paper

Not All Neural Models Are Created Equal (The CW Baseline)