[Verse 1] In neural networks where energy flows free There's a theorem that sets the gradients free Scellier and Bengio found the way Equilibrium propagation saves the day When beta approaches zero in the night The gradient emerges crystal bright Two energy states we calculate The difference reveals our learning fate [Chorus] Equilibrium prop, the gradient's there In the limit of beta, beyond compare Local Hebbian updates at every synapse Exactly the gradient, no neural collapse From deterministic to Boltzmann's dance Energy-based learning gets its chance The mathematics whispers what we need Cortical columns plant the learning seed [Verse 2] Take the partial of loss with respect to theta That's our target, our learning meta But here's the magic in the formulaic spell Partial E over theta tells us well At x-star-beta minus x-star-zero Divided by beta, our learning hero As beta shrinks toward infinitesimal The gradient surfaces, unmistakable [Chorus] Equilibrium prop, the gradient's there In the limit of beta, beyond compare Local Hebbian updates at every synapse Exactly the gradient, no neural collapse From deterministic to Boltzmann's dance Energy-based learning gets its chance The mathematics whispers what we need Cortical columns plant the learning seed [Bridge] Stochastic or deterministic mode Both follow this elegant neural code Each synapse knows its weighted role In the grand computational goal Recurrent networks find their peace When energy gradients never cease The local and global harmonize Through equilibrium's compromise [Verse 3] No backpropagation through time required Just energy landscapes, beautifully wired The cortical columns compute with ease Distributed units that aim to please Hebbian learning at each connection Matches the gradient's true direction Biology and mathematics align In this theorem so divine [Verse 4] Temperature schedules guide the way As thermal noise begins to sway Free phase settles to its ground While clamped phase keeps target bound The energy function smooth and clean Differentiable in every scene From artificial to biological minds This principle unites and binds [Chorus] Equilibrium prop, the gradient's there In the limit of beta, beyond compare Local Hebbian updates at every synapse Exactly the gradient, no neural collapse From deterministic to Boltzmann's dance Energy-based learning gets its chance The mathematics whispers what we need Cortical columns plant the learning seed [Outro] When beta fades to nothing at all The gradient answers learning's call In cortical columns we trust and believe Equilibrium propagation helps us achieve
← Dancing Neurons in the Sacred Way | Energy Landscapes and Dancing Phases →