Recurrent Neural Networks

What is Temporal Data?

past data influences current data

need previous data for correct predictions

What are the Approaches to Overcome Vanishing Gradient Problem?

• long short-term memory

• gated recurrent unit

What is Long Short-Term Memory (LSTM)?

Add gates to control information flow

• Forget Gates - what to erase from memory

• Input Gates - what to add

• Output Gates - what to output

What is Gated Recurrent Unit (GRU)?

• Combined forget and input gate into update gate

• Achieves fewer parameters and similar performance

What are Recurrent Neural Networks?

at each time step, the network calculates a hidden state (memory) and an output

What do the Time Steps look like in RNNs?

Each block passes its hidden state onto the next
Consider each layer to be a time step

What are some Variants of RNNs?

• many to many

• one to many

• many to one

What are some Examples of Many to Many RNNs?

• Diagnosis

• Chatbot

What are some Examples of One to Many RNNs?

• Image captioning

• Music generation

• Storytelling

What are some Examples of One to Many RNNs?

• Sentiment analysis

• Waveform