[Verse 1]
MNIST digits on the screen, twenty-eight by twenty-eight
Forward weights matrix W learns to classify their fate
But backward signals use a random matrix B instead
Can these misaligned pathways train a neural network's head?
[Chorus]
Track the alignment, watch it grow
W and B start to show
Feedback signals find their way
Even when they're led astray
Cortical columns compute and learn
Even when the signals turn
Random pathways still can teach
Neural harmony within reach
[Verse 2]
Start with shallow, two or three, layers stacked up neat and clean
Feedback alignment works just fine, performance sharp and keen
But push it deeper, five or six, seven layers high
Watch the random feedback fail as accuracy waves goodbye
[Chorus]
Track the alignment, watch it grow
W and B start to show
Feedback signals find their way
Even when they're led astray
Cortical columns compute and learn
Even when the signals turn
Random pathways still can teach
Neural harmony within reach
[Bridge]
At what depth does backprop win the race?
Test it layer by layer, find that breaking place
Plot the curves and trace the lines
Where feedback alignment declines
[Verse 3]
Sign-symmetric holds the key, positive and negative dance
Forward weights go left, feedback mirrors that stance
Same direction, opposite sign, a structured random play
Closing gaps between the truth and feedback's roundabout way
[Verse 4]
Measure how the gradients flow through tangled pathways deep
Does the sign constraint rescue what random feedback couldn't keep?
Plot the learning curves and see how much the gap gets small
When structure meets the chaos in cortical protocol
[Verse 5]
Initialize the weights with care, small and centered tight
Watch the early epochs where alignments catch the light
Layer by layer, epoch by epoch, the dynamics unfold
Random matrices learning tricks that never were foretold
[Bridge 2]
Direct feedback, indirect paths
Calculate the neural math
Training time and convergence rate
Which algorithm seals the fate?
[Chorus]
Track the alignment, watch it grow
W and B start to show
Feedback signals find their way
Even when they're led astray
Cortical columns compute and learn
Even when the signals turn
Random pathways still can teach
Neural harmony within reach
[Outro]
From shallow nets to towers tall
Test the limits, test them all
Sign-symmetric bridges built
Between the chaos and the gilt