[Verse 1]
In the cortex where neurons dance and play
Two phases govern learning's sacred way
Free phase first, no output getting clamped
Let the system roam where signals are stamped
Energy flows through pathways undefined
While correlations form in neural mind
No teacher's hand to guide the firing rate
Just natural patterns that neurons create
[Chorus]
Free then clamped, that's the learning game
Beta strength controls the nudging flame
Contrastive Hebbian makes the change
Difference in correlations, weights rearrange
When beta shrinks to zero's domain
Backprop gradients we can reclaim
Free then clamped, two phases reign
In cortical columns' computational brain
[Verse 2]
Now comes the weakly-clamped phase's turn
Output nudged toward targets we discern
Beta parameter sets the nudging force
Not too strong to knock us off our course
Energy function grows by beta C
Where C measures cost of what we see
System balances its natural state
With gentle pressure toward learning's gate
[Chorus]
Free then clamped, that's the learning game
Beta strength controls the nudging flame
Contrastive Hebbian makes the change
Difference in correlations, weights rearrange
When beta shrinks to zero's domain
Backprop gradients we can reclaim
Free then clamped, two phases reign
In cortical columns' computational brain
[Bridge]
Delta W proportional to one over beta
Clamped correlations minus free's meta
Xi times Xj in angular brackets show
How neurons fire together, high and low
Mathematical beauty in this neural art
Where physics and learning both play their part
[Verse 3]
The magic lives in taking the difference
Between these phases' correlation fingerprints
What changed when we applied the gentle nudge
That's the signal that helps neurons budge
Hebbian plasticity with a twist of fate
Contrastive learning updates connection weight
From biological circuits comes this inspiration
For artificial intelligence's next generation
[Verse 4]
Through time the network learns to minimize
The energy that in the system lies
Attractor states emerge from this process
Local minima where patterns find their rest
Hidden layers capture structure deep
While visible units the outputs keep
Boltzmann machines and their thermal dance
Give learning algorithms their second chance
[Chorus]
Free then clamped, that's the learning game
Beta strength controls the nudging flame
Contrastive Hebbian makes the change
Difference in correlations, weights rearrange
When beta shrinks to zero's domain
Backprop gradients we can reclaim
Free then clamped, two phases reign
In cortical columns' computational brain
[Outro]
Energy landscapes shape the neural flow
Distributed units help the knowledge grow
In mathematics of the cortical kind
We find the algorithms of the mind