Connection to neuroscience

tango reggae, bengali egyptian, accordion ambient techno, shoegaze afro-funk · 5:29

Listen on 93

Lyrics

[Verse 1]
In the brain's cathedral, neurons fire and learn
But backprop's got a secret that makes biologists concerned
It sends the signal backward through symmetric pathways clean
But cortex doesn't mirror what our algorithms have seen
The weight transport's too perfect, like a highway built both ways
When biology uses shortcuts through its tangled neural maze

[Chorus]
Gradients flow like rivers, but the question's how they move
Forward pass or backward pass, which direction do we choose
Same equation, different dance, the math will find its way
Biological plausibility starts with gradients today
Same destination, different roads, the cortex knows the score
There's always more than one way to unlock the gradient door

[Verse 2]
The separate backward phases need their own distinct command
But living tissue can't divide its processes so planned
No teacher signal traveling down the exact reverse of tracks
The brain computes its learning while the forward signal acts
Simultaneous processing in the cortical domain
Not waiting for the echo to come traveling back again

[Chorus]
Gradients flow like rivers, but the question's how they move
Forward pass or backward pass, which direction do we choose
Same equation, different dance, the math will find its way
Biological plausibility starts with gradients today
Same destination, different roads, the cortex knows the score
There's always more than one way to unlock the gradient door

[Bridge]
Equilibrium propagation finds the balance in the flow
Predictive coding whispers what the next layer needs to know
Contrastive learning teaches through the difference and compare
While feedback alignment shows the gradients were always there

[Verse 3]
The columns hold their secrets in distributed arrays
Each unit solving pieces of the optimization maze
No central backward sweeping through the network's rigid spine
Just local computation making global patterns shine
The mathematics promise what biology can achieve
When gradients find new pathways we're ready to believe

[Chorus]
Gradients flow like rivers, but the question's how they move
Forward pass or backward pass, which direction do we choose
Same equation, different dance, the math will find its way
Biological plausibility starts with gradients today
Same destination, different roads, the cortex knows the score
There's always more than one way to unlock the gradient door

[Outro]
The foundation's in the knowing that the gradient's just math
Not married to the method or the computational path

← Exercises | Core concepts →