[Verse 1] Deep in neural networks, patterns start to form Representations clustering through each hidden storm Huh and Cheung discovered something quite profound Platonic shapes emerging where convergence can be found Different architectures, trained on separate tasks Still develop similar maps when we remove their masks Like ancient geometric forms that Plato once described Universal structures in the code we've inscribed [Chorus] CKA measures how these networks align Centered Kernel Analysis draws the line Between what's truly shared and what's unique Platonic representations that we seek UMAP reveals the manifold's true face Geometric structures hiding in high-dimensional space Reading representations like a language we can speak [Verse 2] Kornblith showed us metrics that could bridge the gap Between different neural networks and their feature map Linear CKA strips away the noise and scale While RV coefficients tell a different tale Similarity matrices become our guide To understand what networks learn deep inside When training converges on the same solution We see the platonic representation evolution [Chorus] CKA measures how these networks align Centered Kernel Analysis draws the line Between what's truly shared and what's unique Platonic representations that we seek UMAP reveals the manifold's true face Geometric structures hiding in high-dimensional space Reading representations like a language we can speak [Bridge] McInnes gave us UMAP for dimension reduction Preserving local neighborhoods through careful construction Topological data analysis shows the way To visualize what high dimensions want to say Cross-dataset generalization starts to make sense When universal patterns break down the defense [Verse 3] Cortical columns process information in streams These computational units fulfill biological dreams Distributed processing mirrors what we've learned About representation spaces where knowledge is earned The mathematics tells us there's a deeper truth That brains and artificial networks share the proof [Verse 4] From vision models learning edge and texture maps To language models bridging semantic gaps The same abstractions rise from different seeds As if geometry dictates what learning needs Perhaps intelligence itself has certain laws That guide how meaning from raw data draws [Chorus] CKA measures how these networks align Centered Kernel Analysis draws the line Between what's truly shared and what's unique Platonic representations that we seek UMAP reveals the manifold's true face Geometric structures hiding in high-dimensional space Reading representations like a language we can speak
← Towers Speaking Secret Languages | Hidden Layers Tell Their Secrets →