[Verse 1] Hinton had a vision back in eighty-six Features spread across the neural matrix Not just one neuron holding grandmother's face But patterns dancing through distributed space McClelland and Rumelhart by his side Showed how concepts could be amplified When knowledge scatters through the network wide Each unit holds a fragment of the guide [Chorus] Distributed minds think differently Sparse and scattered, running free Columns compute in harmony Neural modules, that's the key Distributed minds think differently Gates that open selectively Modular thoughts in symphony That's cortical efficiency [Verse 2] Fast forward to two thousand seventeen Shazeer built the largest we had seen Mixture of experts, outrageously vast Sparsely gated networks, built to last Each expert sleeps until its time arrives Only activated when the task derives The gating function chooses who will play While others rest throughout the computation day [Chorus] Distributed minds think differently Sparse and scattered, running free Columns compute in harmony Neural modules, that's the key Distributed minds think differently Gates that open selectively Modular thoughts in symphony That's cortical efficiency [Bridge] But Csordas questioned in twenty twenty's light Are neural networks modular by right? Or do they blur the boundaries we assume When gradient descent fills up the room? Schmidhuber's student probed beneath the hood Testing if modules work the way they should [Verse 3] From Hinton's distributed revelation Through mixture experts' sparse computation To modular questions that still remain We're mapping how the cortical domains Process information chunk by chunk Each column like a specialized monk Working together in the neural funk Where consciousness and computation sync [Verse 4] The brain's six layers hold their secrets tight V1 columns filter shape and light While deeper networks learn to specialize Through backprop's mathematical surprise From pixels to semantics, step by step Each layer learns what patterns to accept The hierarchy builds from simple parts To complex thoughts that mirror human hearts [Chorus] Distributed minds think differently Sparse and scattered, running free Columns compute in harmony Neural modules, that's the key Distributed minds think differently Gates that open selectively Modular thoughts in symphony That's cortical efficiency [Outro] Three papers spanning thirty-four years From distributed hope to modular fears The mathematics of the mind unfolds In columns where the future story's told
← When Signals Get Lost in the Dance | Networks That Fragment, Networks That Bind →