[Verse 1] Start with energy function, two layers deep Hidden units dancing where the gradients sleep Free phase running wild, let the network breathe Clamped phase holds the target, what we need to achieve Derivatives flowing through the temporal gap Beta times the difference, that's the learning map First principles building what the cortex knows Equilibrium Propagation, watch the magic unfold [Chorus] EP makes the neurons learn without the backward flow Beta small and steady, let the gradients grow Variance versus bias, find the sweet spot zone Energy-based learning, cortical columns shown Two phase dance forever, free then clamped again EP makes the neurons learn like biological brain [Verse 2] MNIST digits calling, time to code the test Import torch and numpy, put the theory to rest Network architecture, weights initialized Forward pass in free phase, neurons energized Then we clamp the output, second phase begins Measure state differences where the learning wins Compare against backprop, convergence in the race Who reaches the solution with the smoothest pace [Chorus] EP makes the neurons learn without the backward flow Beta small and steady, let the gradients grow Variance versus bias, find the sweet spot zone Energy-based learning, cortical columns shown Two phase dance forever, free then clamped again EP makes the neurons learn like biological brain [Bridge] How small must beta shrink for estimates to shine Point zero zero one or point zero zero nine Plot the variance curves against the gradient truth Empirical investigation, mathematical proof Biological plausible, no backward traveling waves Just local computations, the way the cortex behaves [Verse 3] Batch size experiments, learning rates to tune Convergence trajectories beneath the research moon Error bars expanding when beta grows too large Bias creeping upward, variance takes charge Sweet spot in the middle where the trade-offs meet Accurate approximation makes the learning complete Cortical columns singing in their distributed way EP shows the future of how networks learn and play [Verse 4] Synaptic weights adjusting through the energy lens No ghost of gradient past that backprop tends Real-time computation, phase transitions clear Mathematical beauty that the brain holds dear [Chorus] EP makes the neurons learn without the backward flow Beta small and steady, let the gradients grow Variance versus bias, find the sweet spot zone Energy-based learning, cortical columns shown Two phase dance forever, free then clamped again EP makes the neurons learn like biological brain [Outro] From first principles rising to empirical ground Equilibrium Propagation, the sweetest learning sound
← Energy Landscapes and Dancing Phases | Towers in the Theta Waves →