[Verse 1]
In neural networks where energy flows free
There's a theorem that sets the gradients free
Scellier and Bengio found the way
Equilibrium propagation saves the day
When beta approaches zero in the night
The gradient emerges crystal bright
Two energy states we calculate
The difference reveals our learning fate
[Chorus]
Equilibrium prop, the gradient's there
In the limit of beta, beyond compare
Local Hebbian updates at every synapse
Exactly the gradient, no neural collapse
From deterministic to Boltzmann's dance
Energy-based learning gets its chance
The mathematics whispers what we need
Cortical columns plant the learning seed
[Verse 2]
Take the partial of loss with respect to theta
That's our target, our learning meta
But here's the magic in the formulaic spell
Partial E over theta tells us well
At x-star-beta minus x-star-zero
Divided by beta, our learning hero
As beta shrinks toward infinitesimal
The gradient surfaces, unmistakable
[Chorus]
Equilibrium prop, the gradient's there
In the limit of beta, beyond compare
Local Hebbian updates at every synapse
Exactly the gradient, no neural collapse
From deterministic to Boltzmann's dance
Energy-based learning gets its chance
The mathematics whispers what we need
Cortical columns plant the learning seed
[Bridge]
Stochastic or deterministic mode
Both follow this elegant neural code
Each synapse knows its weighted role
In the grand computational goal
Recurrent networks find their peace
When energy gradients never cease
The local and global harmonize
Through equilibrium's compromise
[Verse 3]
No backpropagation through time required
Just energy landscapes, beautifully wired
The cortical columns compute with ease
Distributed units that aim to please
Hebbian learning at each connection
Matches the gradient's true direction
Biology and mathematics align
In this theorem so divine
[Verse 4]
Temperature schedules guide the way
As thermal noise begins to sway
Free phase settles to its ground
While clamped phase keeps target bound
The energy function smooth and clean
Differentiable in every scene
From artificial to biological minds
This principle unites and binds
[Chorus]
Equilibrium prop, the gradient's there
In the limit of beta, beyond compare
Local Hebbian updates at every synapse
Exactly the gradient, no neural collapse
From deterministic to Boltzmann's dance
Energy-based learning gets its chance
The mathematics whispers what we need
Cortical columns plant the learning seed
[Outro]
When beta fades to nothing at all
The gradient answers learning's call
In cortical columns we trust and believe
Equilibrium propagation helps us achieve