[Verse 1]
Hinton had a vision back in eighty-six
Features spread across the neural matrix
Not just one neuron holding grandmother's face
But patterns dancing through distributed space
McClelland and Rumelhart by his side
Showed how concepts could be amplified
When knowledge scatters through the network wide
Each unit holds a fragment of the guide
[Chorus]
Distributed minds think differently
Sparse and scattered, running free
Columns compute in harmony
Neural modules, that's the key
Distributed minds think differently
Gates that open selectively
Modular thoughts in symphony
That's cortical efficiency
[Verse 2]
Fast forward to two thousand seventeen
Shazeer built the largest we had seen
Mixture of experts, outrageously vast
Sparsely gated networks, built to last
Each expert sleeps until its time arrives
Only activated when the task derives
The gating function chooses who will play
While others rest throughout the computation day
[Chorus]
Distributed minds think differently
Sparse and scattered, running free
Columns compute in harmony
Neural modules, that's the key
Distributed minds think differently
Gates that open selectively
Modular thoughts in symphony
That's cortical efficiency
[Bridge]
But Csordas questioned in twenty twenty's light
Are neural networks modular by right?
Or do they blur the boundaries we assume
When gradient descent fills up the room?
Schmidhuber's student probed beneath the hood
Testing if modules work the way they should
[Verse 3]
From Hinton's distributed revelation
Through mixture experts' sparse computation
To modular questions that still remain
We're mapping how the cortical domains
Process information chunk by chunk
Each column like a specialized monk
Working together in the neural funk
Where consciousness and computation sync
[Verse 4]
The brain's six layers hold their secrets tight
V1 columns filter shape and light
While deeper networks learn to specialize
Through backprop's mathematical surprise
From pixels to semantics, step by step
Each layer learns what patterns to accept
The hierarchy builds from simple parts
To complex thoughts that mirror human hearts
[Chorus]
Distributed minds think differently
Sparse and scattered, running free
Columns compute in harmony
Neural modules, that's the key
Distributed minds think differently
Gates that open selectively
Modular thoughts in symphony
That's cortical efficiency
[Outro]
Three papers spanning thirty-four years
From distributed hope to modular fears
The mathematics of the mind unfolds
In columns where the future story's told