[Verse 1]
When data flows through neural nets, we need to aggregate
Average pooling takes the mean, identity's the gate
Zero starts us off just right, addition builds the sum
Associative and commutative, that's how monoids run
Max pooling finds the largest peak, negative infinity waits
As our identity element, while maximum operates
[Chorus]
Monoids everywhere, structure crystal clear
Rho of sum phi x-i, DeepSets theorem here
Pooling operations, attention mechanisms too
Graph networks passing messages, monoids pull us through
Associative combining, with identity so true
Mathematical foundations in everything we do
[Verse 2]
Attention weights the values deep, weighted average flows
Addition forms our operation, zero's where it goes
Graph neural networks spread the word, messages cascade
Sum aggregation builds the monoid, patterns never fade
But permutation invariance needs a special form
Decompose your function right, weathering the storm
[Chorus]
Monoids everywhere, structure crystal clear
Rho of sum phi x-i, DeepSets theorem here
Pooling operations, attention mechanisms too
Graph networks passing messages, monoids pull us through
Associative combining, with identity so true
Mathematical foundations in everything we do
[Verse 3]
DeepSets reveals the secret that permutation-invariant maps
Can always be decomposed as rho applied to sums perhaps
First phi transforms each element, then we add them all
Finally rho processes the result, answering the call
This theorem guarantees the form, no matter what you choose
If order doesn't matter, this structure you can use
[Verse 4]
String concatenation builds our text with empty string as start
Logical operations join with truth and false apart
Product monoids multiply dimensions, tuples combine with ease
Each component runs its own monoid, harmony in the breeze
From simple sets to complex graphs, the pattern stays the same
Associativity and identity, playing nature's game
[Bridge]
But what if we need something more, associative but skewed
Matrix multiplication shows us how it's viewed
Left times right not right times left, order matters here
Transformations cascading through, making patterns clear
Neural pathways might require this asymmetric flow
When sequence matters more than sets, that's the way to go
[Chorus]
Monoids everywhere, structure crystal clear
Rho of sum phi x-i, DeepSets theorem here
Pooling operations, attention mechanisms too
Graph networks passing messages, monoids pull us through
Associative combining, with identity so true
Mathematical foundations in everything we do
[Outro]
Cortical columns computing distributed and wide
Monoid structures guide us with mathematical pride