[Verse 1]
Brokers scattered cross the cluster nodes
Topics split in partitions down the roads
Replicas mirror data, three by three
Leader writes while followers guarantee
ZooKeeper coordinates the cluster state
But KRaft consensus seals a modern fate
[Chorus]
Producers push, consumers pull the stream
Partitions parallel the data dream
Brokers, topics, replicas align
Kafka's architecture by design
Log compacted, segments retained
Distributed messaging unchained
[Verse 2]
Consumer groups balance partition load
Each member claims a different episode
Offsets mark where reading last occurred
Sequential processing, never blurred
Connect plugins bridge external stores
Streams transform what flows through data doors
[Chorus]
Producers push, consumers pull the stream
Partitions parallel the data dream
Brokers, topics, replicas align
Kafka's architecture by design
Log compacted, segments retained
distributed messaging unchained
[Bridge]
Schema Registry validates the shape
Serialized messages can't escape
Retention policies purge the old
While compaction keeps the keys we hold
Throughput climbs when durability bends
Latency waits where consistencyends
[Verse 3]
Segments roll when size limits break
Cleanup threads determine what to take
Acks configure how writes confirm
Zero, one, or all brokers affirm
KRaft eliminates the ZooKeeper layer
Metadata lives where partition players
[Chorus]
Producers push, consumers pull the stream
Partitions parallel the data dream
Brokers, topics, replicas align
Kafka's architecture by design
Log compacted, segments retained
Distributed messaging unchained
[Outro]
From producers to the final sink
Every message forms a data link
Kafka orchestrates the flowing tide
Where scalability and order collide