[Verse 1] In the cortex where neurons dance and play Two phases govern learning's sacred way Free phase first, no output getting clamped Let the system roam where signals are stamped Energy flows through pathways undefined While correlations form in neural mind No teacher's hand to guide the firing rate Just natural patterns that neurons create [Chorus] Free then clamped, that's the learning game Beta strength controls the nudging flame Contrastive Hebbian makes the change Difference in correlations, weights rearrange When beta shrinks to zero's domain Backprop gradients we can reclaim Free then clamped, two phases reign In cortical columns' computational brain [Verse 2] Now comes the weakly-clamped phase's turn Output nudged toward targets we discern Beta parameter sets the nudging force Not too strong to knock us off our course Energy function grows by beta C Where C measures cost of what we see System balances its natural state With gentle pressure toward learning's gate [Chorus] Free then clamped, that's the learning game Beta strength controls the nudging flame Contrastive Hebbian makes the change Difference in correlations, weights rearrange When beta shrinks to zero's domain Backprop gradients we can reclaim Free then clamped, two phases reign In cortical columns' computational brain [Bridge] Delta W proportional to one over beta Clamped correlations minus free's meta Xi times Xj in angular brackets show How neurons fire together, high and low Mathematical beauty in this neural art Where physics and learning both play their part [Verse 3] The magic lives in taking the difference Between these phases' correlation fingerprints What changed when we applied the gentle nudge That's the signal that helps neurons budge Hebbian plasticity with a twist of fate Contrastive learning updates connection weight From biological circuits comes this inspiration For artificial intelligence's next generation [Verse 4] Through time the network learns to minimize The energy that in the system lies Attractor states emerge from this process Local minima where patterns find their rest Hidden layers capture structure deep While visible units the outputs keep Boltzmann machines and their thermal dance Give learning algorithms their second chance [Chorus] Free then clamped, that's the learning game Beta strength controls the nudging flame Contrastive Hebbian makes the change Difference in correlations, weights rearrange When beta shrinks to zero's domain Backprop gradients we can reclaim Free then clamped, two phases reign In cortical columns' computational brain [Outro] Energy landscapes shape the neural flow Distributed units help the knowledge grow In mathematics of the cortical kind We find the algorithms of the mind
← Electric Dreams and Backward Streams | When Beta Meets Zero →