📌 What is an RNN?

A Recurrent Neural Network (RNN) is a type of neural network designed to handle sequential data by maintaining a memory of previous inputs.
Unlike ANNs/CNNs (which treat inputs independently), RNNs have connections that loop back, enabling them to remember context over time.

📌 Why RNNs?

• Regular ANNs can’t capture dependencies in sequences (e.g., predicting the next word in a sentence).
• RNNs solve this by keeping a hidden state that evolves with the sequence.

Example:

• Input: “I am going to the **”
• RNN can predict → “market” instead of random word because it remembers context.

📌 RNN Architecture

1. Input Layer
   • Sequential data (words, time series, audio frames).
2. Hidden State (hₜ)
   • The “memory” of the network that carries information from one time step to the next.
3. Recurrent Connections
   • Loops that pass information forward in time.
4. Output Layer
   • Produces predictions (next word, next value, classification).

📊 Formula

At time step t: ht=f(Wx⋅xt+Wh⋅ht−1+b)h_t = f(W_x \cdot x_t + W_h \cdot h_{t-1} + b)ht​=f(Wx​⋅xt​+Wh​⋅ht−1​+b) yt=Wy⋅ht+cy_t = W_y \cdot h_t + cyt​=Wy​⋅ht​+c

Where:

• xtx_txt​ = input at time t
• ht−1h_{t-1}ht−1​ = hidden state from previous step
• Wx,Wh,WyW_x, W_h, W_yWx​,Wh​,Wy​ = weight matrices
• fff = activation function (usually tanh or ReLU)

📌 Types of RNNs

1. Vanilla RNN
   • Basic form with short memory.
2. LSTM (Long Short-Term Memory) 🧠
   • Designed to handle long-term dependencies with gates (input, forget, output).
3. GRU (Gated Recurrent Unit)
   • A simpler alternative to LSTM, faster but effective.
4. Bidirectional RNN
   • Reads input both forward and backward for better context.

📌 Applications of RNN

• 📝 Natural Language Processing (NLP) → Text generation, translation, sentiment analysis
• 🎤 Speech Recognition → Convert speech to text
• 📈 Time Series Forecasting → Stock prediction, weather forecasting
• 🎶 Music Generation → Compose melodies based on previous notes
• 👁️ Video Analysis → Activity recognition in videos

📌 Advantages of RNN

✅ Captures sequential dependencies
✅ Works well for variable-length input/output sequences
✅ Suitable for speech, text, and time series

📌 Challenges of RNN

❌ Vanishing & Exploding Gradient → Hard to train on long sequences
❌ Training is slow (sequential by nature)
❌ Not great for very long-term memory (solved by LSTM/GRU)

📖 Analogy

Imagine reading a storybook 📖:

• You don’t just read one word → you remember past words to understand the current one.
• That’s how an RNN processes sequences — with memory of the past.