Lecture 19 - Machine Learning II

Artificial Neural Network

A computer algorithm inspired by how neurons in our brains work where the system is made of a number of simple, highly interconnected processing elements which process information by their dynamic state to respond to external input

• So it goes through a parallel processing of inputs rather than sequential

There are many different ANNs, they’re characterised by their:

• Type (feedforwards, etc)

• Structure (backpropagation, etc)

• Other learning algorithms it may use

Can be simple to extremely complicated (and powerful) such as convolution (where nearby “neurons” can affect one another) and deep learning algorithms (multiple neural hidden layers)

Applications of ANNs

• Pattern association & classification, regularity detection

• Image processing, speech analysis

• Quality assurance

• Robot steering

• And many more

• They can used to approximate any continuous and discontinuous function

Elements of ANNs

• Neuron/Node H (in hidden layers)

• Inputs x

• Outputs y

• Weight w (will tell % to take from inputs)

• Axons

At each node, transfer functions use all the inputs, x_i with varying weights w_i and translate them into an output y

• Can be any function (linear, sigmoid, hyperbolic, gaussian, etc)

• We can choose the functions and test different ones

Neural Architecture

The number of neurons per layer (which is hidden), the number of layers, and the specified connections for each layer

• The more hidden layers, the more complex behaviour it can emulate

Training ANNs

The dataset will be subdivided into three sections rather than the normal two (training & testing). There needs to be an addition validation dataset, which is required to stop the training process

1. A training instance is selected from the training dataset and inputted into the ANN

2. Inputs are propagated through the network to get an output

3. Error of output is computed (desired output - network output)

4. Error is backpropagated through the network to adjust weights

5. Back to step 1, another training instance is processed, until all are processed

6. In the end, the resulting trained ANN is checked with the validation dataset. If the final ANN is previous than the best, it is saved

Afterwhich, it’ll be tested on the unseen testing dataset to assess performance

Pros & Cons of ANNs

Pros:

• Can handle a partial lack of system understanding

• Can create “adaptive” models (that can “learn”)

• Very powerful

Cons:

• Results are as good as the training dataset (true for any algorithm honestly)

• Complicated, results in a black box and doesn’t help us understand

• Can be slow, energetically intensive, and expensive

• Struggle with extremes