[Verse 1] In cortical columns where neurons align Almeida-Pineda draws the design Two phases dancing in computational space Forward dynamics set the pace X equals f of x and theta with input flowing To equilibrium we're going Fixed point iteration finds the state Where neural patterns calculate [Chorus] Forward to fixed points, backward to learn Jacobian transpose makes the gradients turn Y equals J-transpose Y plus error correction Contraction ensures convergence direction Two phases, one algorithm, neural computation Almeida-Pineda's elegant solution [Verse 2] Forward pass settles to steady state Neural activities collaborate Function f maps current state to next With parameters and input context Iteration cycles until motion ceases Equilibrium brings neural peace This forward phase computes the answer Like a perfectly choreographed dancer [Chorus] Forward to fixed points, backward to learn Jacobian transpose makes the gradients turn Y equals J-transpose Y plus error correction Contraction ensures convergence direction Two phases, one algorithm, neural computation Almeida-Pineda's elegant solution [Verse 3] Backward dynamics tell a different tale Error signals must not fail Y vector carries gradient information Through the network's transformation J transpose multiplies the flowing error Each iteration brings it nearer To the gradient we need to find For learning rules refined [Bridge] Contraction is the secret key Without it chaos runs free Eigenvalues less than one Make convergence surely come Both phases iterate to rest Fixed points put learning to the test [Verse 4] From recurrent networks to the brain This algorithm breaks the chain Of biological implausibility With mathematical beauty No unfolding through time required Just two phases, simply wired Nature's way of gradient descent Through equilibrium content [Chorus] Forward to fixed points, backward to learn Jacobian transpose makes the gradients turn Y equals J-transpose Y plus error correction Contraction ensures convergence direction Two phases, one algorithm, neural computation Almeida-Pineda's elegant solution [Outro] In the cortex where thoughts arise Mathematics underlies Forward settling, backward flowing Keeping neural networks growing [Final Chorus] Forward to fixed points, backward to learn Jacobian transpose makes the gradients turn Y equals J-transpose Y plus error correction Contraction ensures convergence direction Two phases, one algorithm, neural computation Almeida-Pineda's elegant solution
← Symmetric Dreams and Random Feedback | Fixed Point Magic →