Loading audiobook...

KRaft (KIP-500) Curriculum

Subject: KRaft (KIP-500) Curriculum

26 chapters

1. 1 Why KRaft Exists

[Verse 1]
In the old days Kafka had a friend
ZooKeeper helped from start to end
Broker registration, metadata store
Topic partitions, ACLs and more

[Verse 2]
Controller election, who's in charge today
ZooKeeper held the truth, showed the way
But running two systems side by side
Brought complexity we couldn't hide

[Chorus]
Why does KRaft exist, why the change
Two systems felt so strange
Metadata split in different places
Scaling up showed all the spaces
KIP five hundred came to save
One system, that's all we crave

[Verse 3]
Operations teams would pull their hair
Managing both systems everywhere
When metadata didn't match up right
Debugging took us through the night

[Verse 4]
Two hundred thousand partitions max
Hit the ceiling, felt the cracks
ZooKeeper's limits held us back
Scalability's what we lacked

[Chorus]
Why does KRaft exist, why the change
Two systems felt so strange
Metadata split in different places
Scaling up showed all the spaces
KIP five hundred came to save
One system, that's all we crave

[Bridge]
Simplify operations, that's the goal
Remove dependencies, take control
Self-managing metadata, all in one
Unified system, scaling's begun

[Chorus]
Why does KRaft exist, why the change
Two systems felt so strange
Metadata split in different places
Scaling up showed all the spaces
KIP five hundred came to save
One system, that's all we crave

[Outro]
No more ZooKeeper in the way
KRaft's here to save the day

2. 2 What is KRaft?

[Verse 1]
ZooKeeper's reign is ending now
A quorum built from Kafka's own
Controllers cluster, take a bow
Event streams carved in metadata stone
No external keeper anymore
The protocol lives within the core

[Chorus]
KRaft means Kafka Raft inside
Eating dogfood with controller pride
Metadata flows like events should be
Consensus native, finally free
KRaft, KRaft, the future's here
ZooKeeper fades, the path is clear

[Verse 2]
Three controllers form the quorum base
Event logs hold the cluster state
Each decision leaves a metadata trace
No more external keeper's weight
The architecture speaks in streams
Events and consensus share the same dreams

[Chorus]
KRaft means Kafka Raft inside
Eating dogfood with controller pride
Metadata flows like events should be
Consensus native, finally free
KRaft, KRaft, the future's here
ZooKeeper fades, the path is clear

[Bridge]
From ensemble to controller nodes
The paradigm completely shifts
Event-based consensus explodes
Through streams where metadata drifts
KIP five hundred leads the charge
Making Kafka's footprint less large

[Verse 3]
Controllers vote on every change
Event sourcing tells the tale
No coordination that's strange
Built-in Raft will never fail
Simplified deployment waits
While event logs coordinate

[Chorus]
KRaft means Kafka Raft inside
Eating dogfood with controller pride
Metadata flows like events should be
Consensus native, finally free
KRaft, KRaft, the future's here
ZooKeeper fades, the path is clear

[Outro]
Event-driven all the way
Metadata streams and controllers play
KRaft consensus, here to stay

3. 3 The Raft Consensus Protocol Primer

[Verse 1]
In the land of distributed nodes, chaos reigns supreme
Until one brave server steps up to lead the team
Raft protocol brings order from the scattered mess
Leader election starts when heartbeats go silent, I confess

[Chorus]
Leader logs and followers sync
Append entries in the chain, don't break the link
Safety first with term numbers climbing high
Raft consensus keeps our data unified

[Verse 2]
Leader sends append entries down the wire
Followers acknowledge or the leader must inquire
Majority votes commit each entry to the log
No split-brain scenarios in this algorithmic fog

[Chorus]
Leader logs and followers sync
Append entries in the chain, don't break the link
Safety first with term numbers climbing high
Raft consensus keeps our data unified

[Verse 3]
Kafka takes the textbook and rewrites the rules
Pull-based replication, not push-based tools
Epochs fence the old leaders when they resurrect
Controller quorum manages what ZooKeeper once protected

[Bridge]
ZAB versus Raft, two cousins in the night
ZooKeeper's broadcast protocol kept Kafka's metadata tight
Now KRaft mode eliminates that external dependency
Three nodes in Docker, check your quorum's harmony

[Verse 4]
Kafka metadata shell reveals the cluster state
Controller logs replicated, no more external fate
Bootstrap servers point to controllers now
Kraft mode simplifies what ZooKeeper used to allow

[Chorus]
Leader logs and followers sync
Append entries in the chain, don't break the link
Safety first with term numbers climbing high
Raft consensus keeps our data unified

[Outro]
Lab time: Docker Compose with three nodes standing guard
Verify quorum health, consensus isn't hard
KRaft protocol evolved from Raft's foundation
Distributed consensus for the streaming generation

4. 1 Node Roles

[Verse 1]
In the cluster architecture, three paths diverge tonight
Controller nodes stand guard with metadata insight
Process roles controller, that's their sacred code
Managing the quorum where cluster secrets flow

[Verse 2]
Broker nodes hustle with a different calling card
Process roles broker, working twice as hard
Client produce consume, that's their daily grind
Serving all requests while controllers guide the mind

[Chorus]
Controller, broker, combined they stand
Three node roles to understand
Quorum keepers, traffic dealers
Mixed deployment when you need both hands
Controller, broker, pick your command
Size your topology, make your stand

[Verse 3]
Combined nodes juggle both roles in single space
Process roles broker comma controller face
One JVM handles double duty load
Perfect for development when resources explode

[Verse 4]
Small clusters love the combined approach so clean
Fewer moving parts in your deployment scene
But scaling up demands you separate the crew
Dedicated roles when traffic pushes through

[Chorus]
Controller, broker, combined they stand
Three node roles to understand
Quorum keepers, traffic dealers
Mixed deployment when you need both hands
Controller, broker, pick your command
Size your topology, make your stand

[Bridge]
Metadata quorum needs its dedicated space
Client traffic flows at its own urgent pace
Choose your battles, plan your nodes
KRaft deployment stories unfold

[Chorus]
Controller, broker, combined they stand
Three node roles to understand
Quorum keepers, traffic dealers
Mixed deployment when you need both hands
Controller, broker, pick your command
Size your topology, make your stand

[Outro]
Process roles define your cluster's beating heart
Choose wisely how you set each node apart

5. 2 The Metadata Log (`__cluster_metadata` topic)

[Verse 1]
In the heart of every cluster beats a single sacred log
Where metadata flows eternal, never lost within the fog
One partition holds the secrets, replicated far and wide
Every change that shapes the system gets recorded here inside

[Chorus]
Underscore underscore cluster metadata
Single thread of truth that orchestrates the data
Topics born and topics vanish, brokers join the dance
Partitions shift their loyalties, configs get their chance
One log rules them all, compacted clean and bright
Snapshots freeze the moments, keeping memory light

[Verse 2]
Brokers wake each morning, hungry for the latest news
They consume this sacred channel, downloading all the clues
Building caches in their memory from the metadata they receive
ACL permissions, feature flags, everything they need to believe

[Chorus]
Underscore underscore cluster metadata
Single thread of truth that orchestrates the data
Topics born and topics vanish, brokers join the dance
Partitions shift their loyalties, configs get their chance
One log rules them all, compacted clean and bright
Snapshots freeze the moments, keeping memory light

[Bridge]
When the log grows heavy, compaction sweeps the floor
Duplicate keys vanish, keeping just what matters more
Snapshots capture history, a checkpoint in the stream
So brokers boot up faster from this metadata machine

[Verse 3]
Registration brings you welcome, deletion says goodbye
Reassignment moves the pieces underneath the cluster sky
Every record tells a story of the changes flowing through
The metadata log remembers what the cluster needs to do

[Outro]
Single source of wisdom, replicated safe and sound
In underscore underscore cluster metadata, truth is found

6. 3 The Quorum Controller

[Verse 1]
In KRaft's kingdom three controllers wait
One will lead while two remain in standby state
Through Raft consensus they must all decide
Who holds the scepter, who steps aside
Terms increment when elections start anew
Votes cast in secret, majority rules true

[Chorus]
Quorum Controller, one leader chosen
Epochs keep order, prevents what's broken
Terms and votes and majorities reign
Fencing the past when new power's gained
Controller dot quorum dot voters define
Which nodes can rule this distributed shrine

[Verse 2]
Configuration lists the voting members
Only these nodes the cluster remembers
Active controller handles all the writes
Standby followers wait for leadership rights
When network splits or leaders disappear
Raft algorithm makes the choice crystal clear

[Chorus]
Quorum Controller, one leader chosen
Epochs keep order, prevents what's broken
Terms and votes and majorities reign
Fencing the past when new power's gained
Controller dot quorum dot voters define
Which nodes can rule this distributed shrine

[Bridge]
Epoch numbers climbing ever higher
Each new leader gets a fresh identifier
Stale commands from previous regimes
Rejected swiftly, protecting our schemes
No dual control, no conflicting commands
One active ruler, that's how KRaft stands

[Verse 3]
Heartbeats pulse between the chosen three
Leader announces "I'm authority"
Followers acknowledge, reset their clocks
Fencing ensures that old leadership stops
Metadata flows through this careful dance
No split-brain chaos, no risky chance

[Chorus]
Quorum Controller, one leader chosen
Epochs keep order, prevents what's broken
Terms and votes and majorities reign
Fencing the past when new power's gained
Controller dot quorum dot voters define
Which nodes can rule this distributed shrine

[Outro]
Three nodes voting, one takes the crown
Raft keeps us safe when systems break down
Epochs ascending, order maintained
KRaft's quorum wisdom, perfectly ordained

7. 4 Broker Lifecycle in KRaft

[Verse 1]
When a broker wakes up in the cluster's embrace
It sends registration to the controller's place
BrokerRegistrationRequest with details to share
Controller responds with an epoch to wear
This timestamp badge marks its lifecycle start
Every heartbeat proves it's playing its part

[Chorus]
Registered, epoch assigned, heartbeats in time
Fenced or unfenced, walking the line
Controller shutdown, graceful and clean
Or sudden failure breaks the machine
Broker lifecycle, four phases true
Registration, beating, shutdown's due

[Verse 2]
BrokerHeartbeatRequest pulses through the wire
Response comes back if the link's not on fire
Missing heartbeats trigger the fence
Controller marks it gone from hence
Unfenced means healthy, taking requests
Fenced means blocked till the system resets

[Chorus]
Registered, epoch assigned, heartbeats in time
Fenced or unfenced, walking the line
Controlled shutdown, graceful and clean
Or sudden failure breaks the machine
Broker lifecycle, four phases through
Registration, beating, shutdown's due

[Bridge]
Kill the active controller, watch the dance begin
Leader election starts, who's gonna win
Metadata log replays the state
JMX metrics show if brokers wait
Logs reveal the timing's truth
Failover testing, laboratory proof

[Verse 3]
Controlled shutdown sends a proper goodbye
Graceful exit, resources won't die
Uncontrolled failure leaves things hanging loose
Emergency cleanup, cut the broker free
Epoch numbers climb with every restart
Four phases cycle, end back to start

[Outro]
From registration through the beating heart
To shutdown's choice, controlled or sharp
Broker lifecycle in KRaft's new way
Four phases dancing every single day

8. 1 The KRaft Protocol (Raft Implementation)

[Verse 1]
In the distributed world of Kafka's reign
KRaft protocol breaks the old chain
No more ZooKeeper holding us down
Leader election's the new game in town
Vote RPC starts the conversation
BeginQuorumEpoch seals our foundation

[Chorus]
Pull don't push, that's the KRaft way
Fetch-based replication every day
High-watermark rising, committed and true
Observer nodes watching what leaders do
Vote, fetch, commit, repeat the flow
KRaft protocol is how we grow

[Verse 2]
Traditional Raft would push logs out
But KRaft says followers pull, no doubt
Fetch requests come from every node
Pulling entries down the replication road
Less network pressure, more control
Pull model plays a starring role

[Chorus]
Pull don't push, that's the KRaft way
Fetch-based replication every day
High-watermark rising, committed and true
Observer nodes watching what leaders do
Vote, fetch, commit, repeat the flow
KRaft protocol is how we grow

[Bridge]
High-watermark tells us what's safe to read
Committed offsets meet every need
When majority confirms what we wrote
That's when we advance and take our note
Observer nodes learn but never vote
Non-voter followers stay remote

[Verse 3]
BeginQuorumEpoch starts a new age
Leader and followers on the same page
Tracking committed offsets with care
Making sure data's consistent everywhere
Vote RPC chooses who will lead
Fetch RPC gets the logs we need

[Chorus]
Pull don't push, that's the KRaft way
Fetch-based replication every day
High-watermark rising, committed and true
Observer nodes watching what leaders do
Vote, fetch, commit, repeat the flow
KRaft protocol is how we grow

[Outro]
From Vote to Fetch to high-watermark
KRaft lights up Kafka with its spark
No ZooKeeper needed anymore
KRaft protocol opens up the door

9. 2 Metadata Snapshots

[Verse 1]
When metadata logs grow thick and dense
Compaction tries but hits a fence
Old entries pile, the disk space bleeds
A snapshot's what the cluster needs
Controllers drowning in the weight
Of every change since initial state
Bootstrap crawling, startup slow
There's got to be a better flow

[Chorus]
Snap it, pack it, store it clean
Metadata snapshots crystalline
Trigger points and loading fast
Making restarts break the past
Snap it, pack it, lifecycle spins
Where compaction ends, snapshot begins

[Verse 2]
Generation triggers when the log
Hits thresholds set by config cog
High watermark or segment count
Decides when snapshots must amount
Format binary, compressed tight
Storage local, out of sight
Quorum metadata frozen still
Ready when the brokers will

[Chorus]
Snap it, pack it, store it clean
Metadata snapshots crystalline
Trigger points and loading fast
Making restarts break the past
Snap it, pack it, lifecycle spins
Where compaction ends, snapshot begins

[Bridge]
SnapshotAlterRequest flies
Through the wire, no disguise
Loading phase on broker start
Reads the snapshot, plays its part
Controller restarts lean and mean
Skips the ancient, loads the scene

[Verse 3]
Lifecycle dancing, create and store
Validate, transfer, then deploy for more
Brokers hungry for the state they need
Snapshots answer, snapshots feed
No more crawling through the ancient past
Startup lightning, recovery fast

[Outro]
When logs grow long and hope grows thin
Let the snapshot dance begin
Pack the metadata, seal it tight
KRaft snapshots make it right

10. 3 Metadata Records & Versioning

[Verse 1]
In the cluster's beating heart, metadata flows
TopicRecord births the streams, PartitionRecord grows
When changes strike the system, PartitionChangeRecord knows
Every shift and transformation as the data river flows

[Chorus]
Seven records rule the realm, memorize their names
Topic, Partition, Change - the streaming games
Register, Registration - broker's claims
Access, Config, Feature - versioning's flames

[Verse 2]
RegisterBrokerRecord brings new nodes online
BrokerRegistrationChange keeps the cluster aligned
AccessControlEntryRecord guards the gates
While ConfigRecord holds the rules that regulate

[Chorus]
Seven records rule the realm, memorize their names
Topic, Partition, Change - the streaming games
Register, Registration - broker's claims
Access, Config, Feature - versioning's flames

[Bridge]
ApiVersion speaks the tongue of compatibility
Forward and backward, graceful ability
Rolling upgrades dance through the night
Old meets new without a fight

[Verse 3]
FeatureLevelRecord marks what systems understand
Version negotiation, protocol's helping hand
Metadata versioning makes the magic real
Seamless transitions, upgrade's spinning wheel

[Chorus]
Seven records rule the realm, memorize their names
Topic, Partition, Change - the streaming games
Register, Registration - broker's claims
Access, Config, Feature - versioning's flames

[Outro]
From record types to version control
KRaft metadata plays its role
Compatibility's golden thread
Keeps the cluster's future fed

11. 4 Client-Side Implications

[Verse 1]
When clients need the cluster map
They knock on broker doors, not controller caps
Metadata flows through familiar gates
While KRaft rebuilds what coordination creates
No direct line to the leader's throne
Brokers bridge the gap, they're not alone

[Chorus]
Metadata still routes through brokers
Four implications, learn the tokens
Faster propagation, less delay
Controller hidden, brokers relay
Cluster ID and Describe API
KRaft revolution, here's the why

[Verse 2]
ZooKeeper's sluggish whispers fade away
KRaft's metadata races, cuts the delay
Propagation speeds like lightning strikes
Controller to broker, then client likes
The latency drops, performance climbs
Milliseconds matter in modern times

[Chorus]
Metadata still routes through brokers
Four implications, learn the tokens
Faster propagation, less delay
Controller hidden, brokers relay
Cluster ID and Describe API
KRaft revolution, here's the why

[Bridge]
Cluster ID marks your domain
DescribeCluster breaks the chain
kafka-metadata shows the snapshot view
kafka-dump-log reveals what's new
Decode the records, trace creation
Topic birth through log foundation

[Verse 3]
Lab time calls with tools in hand
Snapshot inspection helps you understand
Dump the log and read the tale
Every record leaves a trail
From controller writes to broker reads
KRaft plants efficiency seeds

[Chorus]
Metadata still routes through brokers
Four implications, learn the tokens
Faster propagation, less delay
Controller hidden, brokers relay
Cluster ID and Describe API
KRaft revolution, here's the why

[Outro]
Four truths carved in KRaft stone
Brokers serve what controller owns
Speed increased, design refined
Architecture redefined

12. 1 Migration Strategy Overview

[Verse 1]
From ZooKeeper's reign to KRaft's domain
There's no sudden switch, no overnight chain
Three point three unlocked the early door
But three point six brought production shore

[Chorus]
Bridge mode flowing, ZK to KRaft
Migration pathway, carefully crafted
ZK mode to ZK plus KRaft bridge
Then KRaft-only, cross that ridge
No big-bang cutover, smooth transition
Bridge mode strategy, perfect mission

[Verse 2]
First phase running ZooKeeper alone
Second phase they coexist, dual zone
ZK plus KRaft working side by side
Until KRaft-only becomes your guide

[Chorus]
Bridge mode flowing, ZK to KRaft
Migration pathway, carefully crafted
ZK mode to ZK plus KRaft bridge
Then KRaft-only, cross that ridge
No big-bang cutover, smooth transition
Bridge mode strategy, perfect mission

[Bridge]
Version four point zero seals the fate
ZooKeeper vanished, KRaft's final state
Three phases dancing through the years
Bridge mode conquers all your fears

[Verse 3]
Early access brave souls took the leap
Production ready when GA runs deep
From metadata chaos to consensus clean
Smoothest migration you've ever seen

[Chorus]
Bridge mode flowing, ZK to KRaft
Migration pathway, carefully crafted
ZK mode to ZK plus KRaft bridge
Then KRaft-only, cross that ridge
No big-bang cutover, smooth transition
Bridge mode strategy, perfect mission

[Outro]
ZK to bridge to KRaft complete
Migration mastery, can't be beat

13. 2 Step-by-Step Migration Process

[Verse 1]
Time to migrate from the old ZooKeeper ways
First we upgrade to a KRaft-compatible phase
Check your version numbers, make sure they align
Before we start the journey, everything must shine

[Chorus]
Six steps forward, no looking back
Deploy, enable, activate the track
Rolling restart, then finalize clean
KRaft migration, smooth as a machine
Dual-writing metadata, controllers take the lead
From ZooKeeper bridges to the future we need

[Verse 2]
Deploy those controller nodes with their special role
Format their storage with kafka-storage control
Run the script with careful hands, prepare the ground
New leadership waiting to be found

[Chorus]
Six steps forward, no looking back
Deploy, enable, activate the track
Rolling restart, then finalize clean
KRaft migration, smooth as a machine
Dual-writing metadata, controllers take the lead
From ZooKeeper bridges to the future we need

[Verse 3]
Set the migration flag to true
Zookeeper metadata migration, pushing through
Enable equals true, the bridge comes alive
Controllers activate, dual-writing will thrive

[Bridge]
Rolling restart time, broker by broker
Process roles switching, no longer a joker
From ZK-based handling to broker-only clean
The most elegant transition you've ever seen

[Verse 4]
Finalize migration, disable the bridge
Decommission ZooKeeper from ridge to ridge
The ensemble retires, its duty complete
KRaft controllers make the circle complete

[Chorus]
Six steps forward, no looking back
Deploy, enable, activate the track
Rolling restart, then finalize clean
KRaft migration, smooth as a machine
Dual-writing metadata, controllers take the lead
From ZooKeeper bridges to the future we need

[Outro]
Upgrade, deploy, enable the way
Activate, restart, finalize today
KRaft migration, perfectly planned
The future of Kafka is now in your hands

14. 3 Rollback & Risk Management

[Verse 1]
Migration's humming, bridge mode running
ZooKeeper's still the puppet master
But if the wheels start coming loose
You've got an exit, deploy it faster
Check your metadata logs for errors
Before the point of no return
When KRaft controllers take the throne
There's no path back, the bridge will burn

[Chorus]
Roll it back, roll it back
Before the metadata shifts to KRaft
Watch the metrics, catch the cracks
Migration's got your cluster's back
But past the line, there's no rewind
Controllers sealed, the fate is signed
Roll it back, roll it back
Before you cross that final track

[Verse 2]
Common failures love to hide
In partition reassignments
Network splits can break the flow
Controller elections violent
Monitor your throughput curves
Latency spikes tell the story
If consumer groups start lagging hard
Time to abort, forget the glory

[Chorus]
Roll it back, roll it back
Before the metadata shifts to KRaft
Watch the metrics, catch the cracks
Migration's got your cluster's back
But past the line, there's no rewind
Controllers sealed, the fate is signed
Roll it back, roll it back
Before you cross that final track

[Bridge]
Lab four teaches both the paths
Happy migration, rollback math
Test cluster's where you learn to fail
Practice makes the perfect trail
JMX metrics paint the scene
Under-replicated partitions scream
Broker health checks, topic state
Don't migrate if you're running late

[Verse 3]
Disk space errors, memory leaks
Quorum failures, network freaks
These are signs to stop the dance
ZooKeeper's still your safe romance
But once controllers claim their crown
Metadata lives in KRaft town
No going back to ZK's embrace
That bridge is gone without a trace

[Chorus]
Roll it back, roll it back
Before the metadata shifts to KRaft
Watch the metrics, catch the cracks
Migration's got your cluster's back
But past the line, there's no rewind
Controllers sealed, the fate is signed
Roll it back, roll it back
Before you cross that final track

[Outro]
Bridge mode's the safety net you need
Until KRaft proves it can succeed
Know your limits, trust the signs
Rollback saves you every time

15. 1 Sizing & Topology Planning

[Verse 1]
Three controllers rule the standard realm
Odd numbers keep the quorum at the helm
Five when clusters stretch across the sky
But more than that will make performance cry

CPU cores spinning, memory vast
Disk IOPS flowing, built to last
Eight gigs minimum, sixteen preferred
Network bandwidth, every packet heard

[Chorus]
Three or five, keep quorum alive
Odd numbers help the cluster survive
Combined or split, make your choice fit
Cross-rack placement, never submit
To single points where failures hit
KRaft controllers, strong and split

[Verse 2]
Combined nodes share the workload space
Brokers and controllers, same embrace
Dedicated splits them clean apart
Hardware focused, specialized art

Resource contention weighs the scale
Mixed deployment might make you fail
Separation costs but isolation pays
Choose your architecture, count the ways

[Chorus]
Three or five, keep quorum alive
Odd numbers help the cluster survive
Combined or split, make your choice fit
Cross-rack placement, never submit
To single points where failures hit
KRaft controllers, strong and split

[Bridge]
Availability zones spread the risk
Network partitions strike so brisk
Majority voting needs its space
Quorum scattered, not one place

Rack awareness guides the plan
Geographic distance, span by span
Fault domains carved with careful thought
Redundancy that can't be bought

[Verse 3]
Hardware specs define the floor
Monitoring disk, CPU, and more
Latency matters, milliseconds count
Storage performance, paramount amount

Topology maps the cluster's spine
Each controller in its design
Network segments, power grids
Infrastructure where chaos hides

[Final Chorus]
Three or five, keep quorum alive
Odd numbers help the cluster survive
Combined or split, make your choice fit
Cross-rack placement, never submit
Plan your sizing, get it right
KRaft topology, burning bright

[Outro]
Size your cluster, plan with care
Fault tolerance everywhere
Controllers balanced, quorum strong
KRaft will keep your data flowing long

16. 2 Configuration Reference

[Verse 1]
Starting up your cluster, first define your role
Process dot roles tells the system what you control
Controller manages the metadata state machine
Broker handles traffic, keeps the pipeline clean
Combined mode runs both in one efficient node
Node dot ID marks your place along the road

[Chorus]
Seven sacred settings for your KRaft design
Controller quorum voters in a numbered line
Listener names and metadata paths align
Fast disk storage keeps your cluster running fine
Snapshots trigger at twenty meg each time
Idle intervals measured in millisecond rhyme

[Verse 2]
Quorum voters listed with their host and port
One at host-one-nine-oh-nine-three for support
Two and three follow in the voting chain
Controller listener names keep the protocol sane
CONTROLLER spelled out caps, no need to guess
Fast dedicated disk prevents the storage stress

[Chorus]
Seven sacred settings for your KRaft design
Controller quorum voters in a numbered line
Listener names and metadata paths align
Fast disk storage keeps your cluster running fine
Snapshots trigger at twenty meg each time
Idle intervals measured in millisecond rhyme

[Bridge]
Max record bytes between each snapshot frame
Default twenty megabytes, you can change the game
Idle interval waits five hundred milliseconds flat
Before it sends updates through the network chat
Metadata log directory needs speed not size
SSD performance helps your cluster rise

[Verse 3]
Every node needs unique integer ID
No duplicates allowed in your topology
Process roles choose your server's destiny
Single purpose or combined functionality
These parameters form your foundation stone
Build them right and watch your cluster zone

[Outro]
Seven settings memorized, your cluster's ready now
KRaft configuration mastered, take your bow

17. 3 Monitoring & Observability

[Verse 1]
ActiveControllerCount tells the tale
One means healthy, zero means you'll fail
In your cluster there should be just one
Check this metric till the day is done

[Chorus]
Watch the numbers, read the signs
JMX metrics draw the lines
Raft-metrics hold the key
To your KRaft cluster's destiny
Commit-latency, current-leader too
Log-end-offset guides you through

[Verse 2]
Current-leader shows who's in command
Log-end-offset helps you understand
How far your metadata has grown
MetadataLoader seeds you've sown

[Chorus]
Watch the numbers, read the signs
JMX metrics draw the lines
Raft-metrics hold the key
To your KRaft cluster's destiny
Commit-latency, current-leader too
Log-end-offset guides you through

[Bridge]
Last-applied-offset tracks the state
Snapshot-loading shows you wait
Fenced brokers in your logs appear
Leader elections crystal clear
Snapshot failures raise alarms
Dashboard charts protect from harms

[Verse 3]
Grafana panels paint the view
Controller health in red and blue
Log patterns whisper warnings soft
When brokers fence or leaders scoff

[Chorus]
Watch the numbers, read the signs
JMX metrics draw the lines
Raft-metrics hold the key
To your KRaft cluster's destiny
Commit-latency, current-leader too
Log-end-offset guides you through

[Outro]
Monitor close, observe with care
KRaft cluster secrets hanging there
Dashboard wisdom, metrics true
Observability sees you through

18. 4 Security in KRaft Mode

[Verse 1]
Controllers gossip through encrypted wire
TLS certificates they require
No more plaintext floating in the air
SASL mechanisms handle authentication with care
Bootstrap servers exchange their secret keys
Mutual validation puts the network at ease

[Chorus]
Security flows through every node
Metadata locked with cryptographic code
ACLs live in the log, no ZooKeeper store
Authentication guards each connection door
Audit trails capture every change
KRaft security rearranged

[Verse 2]
Brokers authenticate with fresh credentials
Inter-cluster handshakes, now essential
Principal mappings route identity
GSSAPI, PLAIN, or SCRAM authority
Protocol adapters validate each request
Digital signatures put trust to the test

[Chorus]
Security flows through every node
Metadata locked with cryptographic code
ACLs live in the log, no ZooKeeper store
Authentication guards each connection door
Audit trails capture every change
KRaft security rearranged

[Bridge]
Gone are the ZooKeeper ACL days
Metadata log holds permission arrays
Create, delete, alter operations tracked
Timestamps and principals, nothing redacted
Compliance officers celebrate
Immutable records seal our fate

[Verse 3]
Bootstrap protocol negotiates the trust
Certificate chains verified, password robust
Cross-cluster replication needs secure lanes
Encrypted metadata flows through fiber veins
Observer nodes mirror without compromise
Security policies behind administrative eyes

[Chorus]
Security flows through every node
Metadata locked with cryptographic code
ACLs live in the log, no ZooKeeper store
Authentication guards each connection door
Audit trails capture every change
KRaft security rearranged

[Outro]
Controllers united in cryptographic dance
No single point where hackers advance
Distributed trust across the cluster wide
KRaft security, our fortress and guide

19. 5 Disaster Recovery

[Verse 1]
Controllers vanish in the digital night
Quorum scattered, metadata takes flight
Three nodes needed but only one remains
Recovery protocol breaking these chains
Check the logs first, assess what's broken
Find surviving nodes with tokens unspoken

[Chorus]
Backup, restore, reformat the core
Kafka-storage tool opens every door
Quorum loss hits but we know the way
Bootstrap fresh clusters, save the day
Metadata logged, disaster planned
KRaft recovery in your command

[Verse 2]
Network partitions split the brain
Half your cluster drives you insane
Disk space vanishing, metadata choked
Emergency procedures get invoked
Stop all brokers, freeze the state
Careful planning, don't hesitate

[Chorus]
Backup, restore, reformat the core
Kafka-storage tool opens every door
Quorum loss hits but we know the way
Bootstrap fresh clusters, save the day
Metadata logged, disaster planned
KRaft recovery in your command

[Bridge]
Format command with cluster ID
Random-uuid sets you free
Copy snapshots to safety zones
Before the system hits unknown
Monitor dashboards flash alerts
Prevention saves what failure hurts

[Verse 3]
Simulate chaos in your lab tonight
Controller crashes, test your sight
Network splits and disks run dry
Practice makes your skills fly high
Production dashboard tells the tale
Recovery wins when systems fail

[Chorus]
Backup, restore, reformat the core
Kafka-storage tool opens every door
Quorum loss hits but we know the way
Bootstrap fresh clusters, save the day
Metadata logged, disaster planned
KRaft recovery in your command

[Outro]
Five disaster scenarios mastered clean
KRaft resilience keeps systems lean
From quorum death to bootstrap birth
Recovery knowledge proves its worth

20. 1 KRaft Performance Characteristics

[Verse 1]
When ZooKeeper's reign came to an end
Metadata crawled through tangled threads
Five hundred milliseconds just to spread the word
KRaft controller cuts it down to ten instead
Quorum consensus dancing fast and lean
Propagation racing through the cluster scene

[Chorus]
KRaft performance blazing through the wire
Millions of partitions never tire
Metadata operations ten thousand per second
Controller throughput leaves the old ways reckoned
Latency drops while scalability climbs
KRaft performance optimized for modern times

[Verse 2]
Partition limits shattered past the ceiling
Million topics flowing without feeling strain
Controller failover measured now in heartbeats
Where ZooKeeper stumbled KRaft breaks the chain
Horizontal scaling past one thousand brokers
Network topology no longer chokes us

[Chorus]
KRaft performance blazing through the wire
Millions of partitions never tire
Metadata operations ten thousand per second
Controller throughput leaves the old ways reckoned
Latency drops while scalability climbs
KRaft performance optimized for modern times

[Bridge]
Tuning parameters for massive enterprise scale
Batch size optimization tells the tale
Controller queue depth carefully configured
Memory allocation precisely figured
JVM heap sizing for the metadata load
Performance monitoring shows what KRaft bestowed

[Chorus]
KRaft performance blazing through the wire
Millions of partitions never tire
Metadata operations ten thousand per second
Controller throughput leaves the old ways reckoned
Latency drops while scalability climbs
KRaft performance optimized for modern times

[Outro]
From milliseconds down to microseconds small
KRaft performance conquers all

21. 2 KRaft and Kafka Streams / Connect

[Verse 1]
No more ZooKeeper in our way
KRaft mode changes how we play
State stores still persist the same
But coordination's changed the game
Kafka Streams keeps running smooth
While underneath we've changed the groove

[Chorus]
KRaft Connect and Streams unite
No ZooKeeper in sight
State stores stay, workers coordinate
Eight four eight will demonstrate
New consumer groups align
In the KRaft design

[Verse 2]
Connect workers need to sync
How they coordinate and link
Leader election's built right in
To the Kafka core within
No external dependency
For worker group consistency

[Chorus]
KRaft Connect and Streams unite
No ZooKeeper in sight
State stores stay, workers coordinate
Eight four eight will demonstrate
New consumer groups align
In the KRaft design

[Verse 3]
Consumer groups get redesigned
KIP eight four eight defined
Server-side assignment's here
Makes rebalancing more clear
Streams applications benefit
From this protocol that's fit

[Bridge]
State store management stays true
RocksDB still works for you
But the metadata's moved around
To where Kafka logs are found
Everything's more unified
With KRaft as our guide

[Chorus]
KRaft Connect and Streams unite
No ZooKeeper in sight
State stores stay, workers coordinate
Eight four eight will demonstrate
New consumer groups align
In the KRaft design

[Outro]
From Connect to Streams we see
KRaft brings simplicity
One less service to maintain
Everything in Kafka's domain

22. 3 Multi-Tenancy & Quotas

[Verse 1]
When clusters grow and tenants multiply
Each needs their slice but can't monopolize
Metadata quotas keep the balance tight
Controllers watching every byte in flight
Rate limits guard the gates from overflow
While partitions dance in separate rows

[Chorus]
Multi-tenant, quota-bent
Keep the boundaries permanent
Isolate, allocate, never saturate
Controller's keeping order straight
MQR - Metadata, Quota, Rate
Three pillars holding cluster fate

[Verse 2]
Topic creation hits the ceiling fast
When quotas strike, the limits hold steadfast
Produce requests get throttled at the door
Consumer groups can't flood the broker core
Each tenant lives within their walled domain
The controller holds the master chain

[Chorus]
Multi-tenant, quota-bent
Keep the boundaries permanent
Isolate, allocate, never saturate
Controller's keeping order straight
MQR - Metadata, Quota, Rate
Three pillars holding cluster fate

[Bridge]
Resource pools divided clean
CPU slices, memory lean
Namespace barriers lock and key
Tenant A can't see Tenant B
Controller's brain distributes load
Down the KRaft consensus road

[Verse 3]
Authentication seals each tenant tight
Authorization checks every right
Network policies route the traffic flow
Resource tags tell the system what to know
Metrics isolated, logs compartmentalized
Shared infrastructure, privatized

[Chorus]
Multi-tenant, quota-bent
Keep the boundaries permanent
Isolate, allocate, never saturate
Controller's keeping order straight
MQR - Metadata, Quota, Rate
Three pillars holding cluster fate

[Outro]
When thousands share one cluster space
KRaft controllers set the pace
Boundaries drawn, resources measured
Multi-tenancy properly treasured

23. 4 The Road Ahead

[Verse 1]
ZooKeeper's reign has reached its final chapter
KRaft controllers emerge from scattered doubt
Dynamic quorum membership comes after
Adding nodes without restart throughout
KIP-853 reshapes the constellation
Hot-swapping guardians while systems breathe

[Chorus]
The road ahead, five hundred thousand partitions
Metadata streams like digital rain
Controller failover, broker transitions
Measuring heartbeats through KRaft's domain
No more coordination tax to pay
The road ahead shows us the way

[Verse 2]
KIP-966 brings eligible leaders
Replicas compete for command position
Smart selection filters out the bleeders
Optimized placement through stark precision
Benchmark Lab Six puts theory to test
Scale reveals where bottlenecks nest

[Chorus]
The road ahead, five hundred thousand partitions
Metadata streams like digital rain
Controller failover, broker transitions
Measuring heartbeats through KRaft's domain
No more coordination tax to pay
The road ahead shows us the way

[Bridge]
Tiered storage waits beyond the curve
Geo-replicated metadata served
Kafka Four-point-oh rewrites the rules
Post-ZooKeeper paradigm tools
Convergence timing, propagation speed
Architecture built for modern need

[Verse 3]
Controllers dance in seamless rotation
No downtime when membership shifts
Quorum logic guides transformation
While partition load gracefully drifts
Benchmark numbers tell the tale
Of systems built beyond the pale

[Chorus]
The road ahead, five hundred thousand partitions
Metadata streams like digital rain
Controller failover, broker transitions
Measuring heartbeats through KRaft's domain
No more coordination tax to pay
The road ahead shows us the way

[Outro]
KRaft consensus, future's foundation
Scalable truth through innovation

24. Official Documentation

[Verse 1]
Documentation waits in three sacred scrolls
KIP-500 breaks the ZooKeeper hold
Metadata quorum stands on its own
Controllers elect their rightful throne

[Chorus]
Read the docs, read them well
KIP-500 breaks the spell
Quorum votes, controllers lead
Metadata flows, ZooKeeper freed
Read the docs, make it clear
KRaft architecture's finally here

[Verse 2]
Official Kafka docs reveal the way
KRaft mode configuration saves the day
Bootstrap servers point to controller nodes
Cluster metadata in new episode

[Chorus]
Read the docs, read them well
KIP-500 breaks the spell
Quorum votes, controllers lead
Metadata flows, ZooKeeper freed
Read the docs, make it clear
KRaft architecture's finally here

[Bridge]
KIP-631 adds the missing piece
Quorum-based control brings sweet release
Three documents guide your migration quest
Official sources always know what's best

[Verse 3]
Format storage with the proper tool
Server properties follow KRaft rules
Process roles define what each node does
Controller quorum handles all the buzz

[Chorus]
Read the docs, read them well
KIP-500 breaks the spell
Quorum votes, controllers lead
Metadata flows, ZooKeeper freed
Read the docs, make it clear
KRaft architecture's finally here

[Outro]
Documentation wisdom never lies
Three KIPs unlock the enterprise
Official guides will see you through
KRaft implementation starts with you

25. Talks & Articles

[Verse 1]
There once was a time when Kafka relied
On ZooKeeper standing by its side
But scaling up meant complexity grew
Metadata scattered, performance fell through
Colin McCabe saw a better way
KIP-500 would save the day

[Chorus]
KRaft mode, no more ZooKeeper pain
Self-managing cluster, breaking the chain
Raft protocol keeps consensus strong
Leader election where it belongs
KRaft mode, streamlined and clean
The simplest Kafka you've ever seen

[Verse 2]
Jason showed us how Raft works inside
One leader chosen, others subside
Heartbeats flowing to stay alive
If leader fails, new one will thrive
Log replication keeps data safe
Majority wins, no time to waste

[Chorus]
KRaft mode, no more ZooKeeper pain
Self-managing cluster, breaking the chain
Raft protocol keeps consensus strong
Leader election where it belongs
KRaft mode, streamlined and clean
The simplest Kafka you've ever seen

[Bridge]
Confluent made it simple to see
Fewer moving parts means stability
Bootstrapping easier than before
No external dependencies to store
Metadata controllers lead the way
Built into Kafka, here to stay

[Verse 3]
Three key players in this new design
Controllers, brokers, working in line
Event-driven architecture flows
Self-contained system that truly knows
How to manage its own state
No ZooKeeper dependency weight

[Chorus]
KRaft mode, no more ZooKeeper pain
Self-managing cluster, breaking the chain
Raft protocol keeps consensus strong
Leader election where it belongs
KRaft mode, streamlined and clean
The simplest Kafka you've ever seen

[Outro]
From KIP-500 to production use
KRaft has set your clusters loose
No more ZooKeeper in the way
Welcome to Kafka's brighter day

26. Related KIPs

[Verse 1]
When metadata needs a home to stay
Five-nine-five builds the quorum way
Raft consensus keeps the truth in line
Controllers vote and data's aligned

[Chorus]
KRaft evolution, step by step we grow
Six-three-oh snapshots, seven-six-two knows
Rack awareness guides the leader choice
Eight-three-three production, eight-five-three membership voice
Nine-six-six eligible, replicas in the flow

[Verse 2]
Snapshots capture state when logs get long
Six-three-oh compression keeps performance strong
No more endless replay from the start
Compact images play the smarter part

[Chorus]
KRaft evolution, step by step we grow
Six-three-oh snapshots, seven-six-two knows
Rack awareness guides the leader choice
Eight-three-three production, eight-five-three membership voice
Nine-six-six eligible, replicas in the flow

[Verse 3]
Seven-six-two spreads controllers wide
Rack topology becomes our guide
Network failures won't bring chaos down
Geographic wisdom wears the crown

[Bridge]
Eight-three-three declares we're ready now
Production workloads, take the vow
ZooKeeper's legacy fades away
Modern architecture rules the day

[Verse 4]
Eight-five-three reshapes the member list
Dynamic changes can't be missed
Controllers join and leave with grace
Smooth transitions keep the pace

[Chorus]
KRaft evolution, step by step we grow
Six-three-oh snapshots, seven-six-two knows
Rack awareness guides the leader choice
Eight-three-three production, eight-five-three membership voice
Nine-six-six eligible, replicas in the flow

[Outro]
Nine-six-six selects who leads the dance
Eligible replicas get their chance
From metadata quorum to production might
KRaft's progression burns forever bright

Back to Home

KRaft (KIP-500) Curriculum | 93