[Verse 1] In the cortex where the columns stand like towers Processing information through computational powers Each unit holds a secret, a distributed design Where mathematics reveals how neural networks align [Chorus] DeepSets and Janossy, embed then aggregate Any function on a set, this pattern we create Procrustes takes the stage, transformations align Linear maps connect what seemed like separate design Key results converging, geometry preserved Different models speaking languages we never heard [Verse 2] Take your elements scattered, map them one by one Into common vector space where the magic's done Then aggregate the features, sum or max or mean The most elegant solution that you've ever seen Per-element embeddings hold the power tight Janossy pooling orders what was scattered light [Chorus] DeepSets and Janossy, embed then aggregate Any function on a set, this pattern we create Procrustes takes the stage, transformations align Linear maps connect what seemed like separate design Key results converging, geometry preserved Different models speaking languages we never heard [Bridge] Huh and colleagues showed us twenty twenty-four Large models trained apart still share a common core Representations different on the surface gleam But linear transformation bridges every dream The underlying structure stays mathematically sound While different architectures dance to the same ground [Verse 3] Orthogonal rotations, scaling up and down Procrustes analysis wears the matching crown Two matrices of features from different neural nets Linear transformation pays off all the debts The geometry within remains fundamentally true Though the coordinate systems paint a different view [Verse 4] Invariance and equivariance guide the way Symmetric operations in the neural ballet When order doesn't matter, DeepSets shine so bright Permutation freedom gives the model insight Universal approximation theorem holds the key Set functions learned with perfect symmetry [Chorus] DeepSets and Janossy, embed then aggregate Any function on a set, this pattern we create Procrustes takes the stage, transformations align Linear maps connect what seemed like separate design Key results converging, geometry preserved Different models speaking languages we never heard [Outro] From cortical columns to the math that makes them flow Distributed computation puts on quite a show Sets and transformations, alignments crystal clear The beauty of convergence draws the future near
← Platonic Truth in Neural Harmony | Platonic Shapes in Hidden Storms →