[Verse 1] When data flows in temporal streams Sequential patterns chase your dreams Vanilla RNNs start the fight But gradients vanish out of sight Each timestamp feeds the hidden state But long-term memory meets its fate [Chorus] Recurrent networks learn through time LSTM gates keep memories in line GRU simplifies the gated flow Attention shows us what to know Sequential data finds its way Through architectures built to stay [Verse 2] LSTM brings three gates to play Forget gate throws old thoughts away Input gate decides what's new Output gate controls the view Cell state carries information far Long dependencies are now the star [Chorus] Recurrent networks learn through time LSTM gates keep memories in line GRU simplifies the gated flow Attention shows us what to know Sequential data finds its way Through architectures built to stay [Bridge] Sequence to sequence translation Encoder builds representation Decoder generates each token Attention mechanism keeps us woken Bahdanau looks at every step Luong keeps alignments prepped [Verse 3] GRU combines the forget and input Reset and update gates throughout State-space models like Mamba rise Structured states before your eyes Choose RNNs for sequential flow Transformers when parallel you go [Chorus] Recurrent networks learn through time LSTM gates keep memories in line GRU simplifies the gated flow Attention shows us what to know Sequential data finds its way Through architectures built to stay [Outro] Time series forecast in your lab Financial data you can grab Sensor readings through the day LSTM models show the way Memory gates and attention bright Sequence models done just right
← Unit 3.2 — Convolutional Neural Networks (CNNs) | Unit 3.4 — Generative Models →