[Verse 1] In twenty-sixteen, Lillicrap found the key When gradient descent seemed impossible to be The backward pass through layers, weight transpose required But nature doesn't calculate what textbooks have desired They swapped out W transpose with matrix B so random Fixed and wild, no training, like a neural phantom Yet learning still emerged from this approximation A breakthrough in the quest for brain computation [Chorus] Random feedback, fixed matrix B Still teaches networks effectively Forward weights W adapt and align Making random signals work just fine Not exact but close enough to learn Biological gradients start to turn Random feedback, nature's clever way Approximate but guides us every day [Verse 2] The forward weights begin their dance of adaptation Slowly aligning with B through correlation W starts matching patterns in that random matrix Creating useful signals from chaotic practice The gradient becomes approximate, not precise But good enough to make the network think twice Error flows backward through these random channels Teaching without perfect mathematical handles [Chorus] Random feedback, fixed matrix B Still teaches networks effectively Forward weights W adapt and align Making random signals work just fine Not exact but close enough to learn Biological gradients start to turn Random feedback, nature's clever way Approximate but guides us every day [Bridge] Shallow networks thrive with this approach so bold But deeper architectures lose their golden hold The approximation starts to fade and drift When too many layers create a growing rift Sign-symmetric feedback brings us closer still Same signs as W transpose but different thrill Different magnitudes, same directional cue Closes gaps and makes the learning true [Verse 3] Not perfect equivalence, but nature doesn't need Mathematical precision to make neurons heed The cortical columns compute without exact math Using random feedback on their learning path Distributed units process information flows Without backpropagation that nobody knows Biology's solution, elegant and smart Approximate gradients, computational art [Chorus] Random feedback, fixed matrix B Still teaches networks effectively Forward weights W adapt and align Making random signals work just fine Not exact but close enough to learn Biological gradients start to turn Random feedback, nature's clever way Approximate but guides us every day [Outro] From Lillicrap's discovery we now understand Perfection isn't needed in this neural land Random matrices teach with approximate grace Showing us how brains learn in their natural space
← Matrices Can't Dance Backwards | Messy Wires Still Learn →