[Verse 1]
Start with energy function, two layers deep
Hidden units dancing where the gradients sleep
Free phase running wild, let the network breathe
Clamped phase holds the target, what we need to achieve
Derivatives flowing through the temporal gap
Beta times the difference, that's the learning map
First principles building what the cortex knows
Equilibrium Propagation, watch the magic unfold
[Chorus]
EP makes the neurons learn without the backward flow
Beta small and steady, let the gradients grow
Variance versus bias, find the sweet spot zone
Energy-based learning, cortical columns shown
Two phase dance forever, free then clamped again
EP makes the neurons learn like biological brain
[Verse 2]
MNIST digits calling, time to code the test
Import torch and numpy, put the theory to rest
Network architecture, weights initialized
Forward pass in free phase, neurons energized
Then we clamp the output, second phase begins
Measure state differences where the learning wins
Compare against backprop, convergence in the race
Who reaches the solution with the smoothest pace
[Chorus]
EP makes the neurons learn without the backward flow
Beta small and steady, let the gradients grow
Variance versus bias, find the sweet spot zone
Energy-based learning, cortical columns shown
Two phase dance forever, free then clamped again
EP makes the neurons learn like biological brain
[Bridge]
How small must beta shrink for estimates to shine
Point zero zero one or point zero zero nine
Plot the variance curves against the gradient truth
Empirical investigation, mathematical proof
Biological plausible, no backward traveling waves
Just local computations, the way the cortex behaves
[Verse 3]
Batch size experiments, learning rates to tune
Convergence trajectories beneath the research moon
Error bars expanding when beta grows too large
Bias creeping upward, variance takes charge
Sweet spot in the middle where the trade-offs meet
Accurate approximation makes the learning complete
Cortical columns singing in their distributed way
EP shows the future of how networks learn and play
[Verse 4]
Synaptic weights adjusting through the energy lens
No ghost of gradient past that backprop tends
Real-time computation, phase transitions clear
Mathematical beauty that the brain holds dear
[Chorus]
EP makes the neurons learn without the backward flow
Beta small and steady, let the gradients grow
Variance versus bias, find the sweet spot zone
Energy-based learning, cortical columns shown
Two phase dance forever, free then clamped again
EP makes the neurons learn like biological brain
[Outro]
From first principles rising to empirical ground
Equilibrium Propagation, the sweetest learning sound