[Verse 1]
Back in the day when Hopfield first designed
Binary networks storing patterns in the mind
Point one four N was the magic ratio found
Classical capacity with that linear bound
Each unit flipping states like switches in a row
But exponential storage seemed impossible to know
[Chorus]
From linear limits to exponential heights
Softmax energy brings transformer lights
Modern Hopfield breaks the ceiling wide
Zero point one four N left behind
Attention mechanisms hidden in disguise
Neural storage capacity multiplies
[Verse 2]
Ramsauer came in twenty-twenty with the proof
Softmax energy function shattered through the roof
No longer bound by that restrictive linear chain
Exponential patterns dancing through each brain
The update rule revealed a secret so profound
Transformer attention is what they really found
[Chorus]
From linear limits to exponential heights
Softmax energy brings transformer lights
Modern Hopfield breaks the ceiling wide
Zero point one four N left behind
Attention mechanisms hidden in disguise
Neural storage capacity multiplies
[Bridge]
Query key and value matrices align
Attention weights computed line by line
What seemed like different worlds of thought
Were mathematically the same all along we fought
Cortical columns storing memories vast
Distributed computation unsurpassed
[Verse 3]
Equivalence unveiled the deeper truth within
Modern Hopfield and transformers are twin
Up to scaling factors they compute the same
Different names but playing the identical game
Distributed units working as one mind
Exponential capacity no longer confined
[Verse 4]
Temperature scaling controls the sharpest peak
Beta parameter determines what memories we seek
High temperature spreads attention wide and far
Low temperature focuses like a guiding star
Energy landscapes shaped by softmax curves
Recurrent dynamics that convergence preserves
[Chorus]
From linear limits to exponential heights
Softmax energy brings transformer lights
Modern Hopfield breaks the ceiling wide
Zero point one four N left behind
Attention mechanisms hidden in disguise
Neural storage capacity multiplies
[Outro]
Classical bounds dissolved into the past
Exponential memories built to last
Mathematics revealed the connection true
Cortical computation breaking through