[Verse 1]
In cortical columns where neurons align
Almeida-Pineda draws the design
Two phases dancing in computational space
Forward dynamics set the pace
X equals f of x and theta with input flowing
To equilibrium we're going
Fixed point iteration finds the state
Where neural patterns calculate
[Chorus]
Forward to fixed points, backward to learn
Jacobian transpose makes the gradients turn
Y equals J-transpose Y plus error correction
Contraction ensures convergence direction
Two phases, one algorithm, neural computation
Almeida-Pineda's elegant solution
[Verse 2]
Forward pass settles to steady state
Neural activities collaborate
Function f maps current state to next
With parameters and input context
Iteration cycles until motion ceases
Equilibrium brings neural peace
This forward phase computes the answer
Like a perfectly choreographed dancer
[Chorus]
Forward to fixed points, backward to learn
Jacobian transpose makes the gradients turn
Y equals J-transpose Y plus error correction
Contraction ensures convergence direction
Two phases, one algorithm, neural computation
Almeida-Pineda's elegant solution
[Verse 3]
Backward dynamics tell a different tale
Error signals must not fail
Y vector carries gradient information
Through the network's transformation
J transpose multiplies the flowing error
Each iteration brings it nearer
To the gradient we need to find
For learning rules refined
[Bridge]
Contraction is the secret key
Without it chaos runs free
Eigenvalues less than one
Make convergence surely come
Both phases iterate to rest
Fixed points put learning to the test
[Verse 4]
From recurrent networks to the brain
This algorithm breaks the chain
Of biological implausibility
With mathematical beauty
No unfolding through time required
Just two phases, simply wired
Nature's way of gradient descent
Through equilibrium content
[Chorus]
Forward to fixed points, backward to learn
Jacobian transpose makes the gradients turn
Y equals J-transpose Y plus error correction
Contraction ensures convergence direction
Two phases, one algorithm, neural computation
Almeida-Pineda's elegant solution
[Outro]
In the cortex where thoughts arise
Mathematics underlies
Forward settling, backward flowing
Keeping neural networks growing
[Final Chorus]
Forward to fixed points, backward to learn
Jacobian transpose makes the gradients turn
Y equals J-transpose Y plus error correction
Contraction ensures convergence direction
Two phases, one algorithm, neural computation
Almeida-Pineda's elegant solution