Major Milestones

300 BC; Aristotle

Introduced Associationism, started the history of human’s attempt to understand brain

1873; Alexander Bain

Introduced Neural Groupings as the earliest models of neural network, inspired Hebbian Learning Rule

1943; McCulloch and Pitts

Introduced MCP Model, which is considered as the ancestor of Artificial Neural Model

1949; Donald Hebb

Considered as the father of neural networks, introduced Hebbian Learning Rule which lays the foundation of modern neural network

1958; Frank Rosenblatt

Introduced the first perceptr1on which highly resembles modern perceptron

1974; Paul Werbos

Introduced Backpropagation

1980; Teuvo Kobonen

Introduced Self Orga19nizing Map

1980; Kunihiko Fukushima

Introduced Neocogitron which inspired Convolutional Neural Network

1982; John Hopfield

Introduced Hopfield Network

1985; Hilton and Sejnowski

Introduced Botzmann Machine

1986; Paul Smolensky

Introduced Harmonium which is later known as Restricted Boltzmann Machine

1986; Michael I. Jordan

Defined and intrduced Recurrent Neural Network

1990; Yann LeCun

Introduced LeNet, showed the possibility of deep neural network in practice

1997; Schuster and Paliwal

Introduced Bidirectioal Recurrent Neural Network

1997; Hochreiter and SChmidhuber

Introduced LSTM, solved the problem of vanishing gradient in recurrent neural networks

2006; Geoffrey Hinton

Introduced Deep Belief Networks, also introduced layer-wise pretraining technique, opened current deep learning era

2009; Salakhutdinov and Hinton

Introduced Deep Boltzmann Machines

2012; Geoffrey Hinton

Introduced Dropout, an efficient way of training neural networks