3/12/2025 neural networks

neural vs biological neurons

• not inspired by brain

artificial neuron

• a lot of inputs with weights put into a formula to make different outputs

perceptron the architecture introduced by McCullough, Pitts

• a linear classifier with threshold output and a peculiar learning algorithm

• extensive hype

perceptron the book

• emphasizes

neural with hidden layer

• popular 1986

• train by backpropagating errors

• have a non-linearity between layers (allow arbitrary functions)

• new hype cycle

• disputes in ai and psy

misunderstood theorem discourages research into more layers

convolutional neural networks

max pooling

long short-term memory (no practical application)

deep learning explosion

2012 alexnet

2014 seq2seq with LSTM

2016 AlphaGo uses CNN and deep RL

2017 intro of transformers

2018 BERT

2022 GPT 3.5 first llm broadly available to the general public

2024 deep seek

Artificial Neural Networks (ANN)

• versatile, powerful, scalable models for classification and regression

• inspired by the networks of biological neurons but different

• many application

  • classifying images

  • speech recognition

  • video recommend defeating go champions