[Verse 1]
In the cortex where the columns stand like towers
Processing information through computational powers
Each unit holds a secret, a distributed design
Where mathematics reveals how neural networks align
[Chorus]
DeepSets and Janossy, embed then aggregate
Any function on a set, this pattern we create
Procrustes takes the stage, transformations align
Linear maps connect what seemed like separate design
Key results converging, geometry preserved
Different models speaking languages we never heard
[Verse 2]
Take your elements scattered, map them one by one
Into common vector space where the magic's done
Then aggregate the features, sum or max or mean
The most elegant solution that you've ever seen
Per-element embeddings hold the power tight
Janossy pooling orders what was scattered light
[Chorus]
DeepSets and Janossy, embed then aggregate
Any function on a set, this pattern we create
Procrustes takes the stage, transformations align
Linear maps connect what seemed like separate design
Key results converging, geometry preserved
Different models speaking languages we never heard
[Bridge]
Huh and colleagues showed us twenty twenty-four
Large models trained apart still share a common core
Representations different on the surface gleam
But linear transformation bridges every dream
The underlying structure stays mathematically sound
While different architectures dance to the same ground
[Verse 3]
Orthogonal rotations, scaling up and down
Procrustes analysis wears the matching crown
Two matrices of features from different neural nets
Linear transformation pays off all the debts
The geometry within remains fundamentally true
Though the coordinate systems paint a different view
[Verse 4]
Invariance and equivariance guide the way
Symmetric operations in the neural ballet
When order doesn't matter, DeepSets shine so bright
Permutation freedom gives the model insight
Universal approximation theorem holds the key
Set functions learned with perfect symmetry
[Chorus]
DeepSets and Janossy, embed then aggregate
Any function on a set, this pattern we create
Procrustes takes the stage, transformations align
Linear maps connect what seemed like separate design
Key results converging, geometry preserved
Different models speaking languages we never heard
[Outro]
From cortical columns to the math that makes them flow
Distributed computation puts on quite a show
Sets and transformations, alignments crystal clear
The beauty of convergence draws the future near