Loading audiobook...

Kafka on Kubernetes with Strimzi

29 chapters

1. 2 Dashboards with Grafana

[Verse 1]
When your Kafka cluster's running wild and free
You need some eyes to watch and see
Grafana dashboards light the way
Monitoring metrics night and day
Import the examples that Strimzi provides
Three main dashboards to be your guides

[Chorus]
Broker dashboard shows the flow
ZooKeeper keeps the state we know
Connect dashboard tracks the streams
Custom metrics for business dreams
Visual data, clear and bright
Grafana makes your Kafka right

[Verse 2]
Broker dashboard tells the tale
Of throughput, latency, success and fail
Request rates climbing up and down
Partition leaders wearing the crown
Disk usage, memory consumption too
Every metric that matters to you

[Chorus]
Broker dashboard shows the flow
ZooKeeper keeps the state we know
Connect dashboard tracks the streams
Custom metrics for business dreams
Visual data, clear and bright
Grafana makes your Kafka right

[Verse 3]
ZooKeeper dashboard watches close
The ensemble health from coast to coast
Leader elections, sync delays
Outstanding requests through the maze
Session counts and connection state
Keep your cluster running straight

[Bridge]
Connect dashboard for the pipelines
Source and sink connector guidelines
Task status, error rates displayed
Connector health will never fade
Transform your data, stream it through
Dashboards show what connectors do

[Verse 4]
Custom dashboards for your needs
Business metrics, special feeds
Service level indicators shine
SLIs that match your bottom line
Order processing, user events
Metrics that make business sense

[Chorus]
Broker dashboard shows the flow
ZooKeeper keeps the state we know
Connect dashboard tracks the streams
Custom metrics for business dreams
Visual data, clear and bright
Grafana makes your Kafka right

[Outro]
Import, customize, and create
Dashboard views that truly relate
To your business goals and technical might
Grafana keeps your Kafka in sight

2. 5 Alerting

[Verse 1]
When your Kafka cluster starts to fail
Prometheus rules will tell the tale
Configure alerts before the storm
Keep your streaming platform warm
Broker down means traffic stops
Monitor before your system drops

[Chorus]
Alert, detect, respond with speed
Broker down is what we need to see
Under-replicated partitions cry
Disk usage climbing way too high
Consumer lag will slow you down
Four alerts to keep you sound

[Verse 2]
Under-replicated partitions spread
Data copies falling behind instead
When replicas can't keep the pace
Your fault tolerance loses face
Set the threshold, watch it close
This alert matters the most

[Chorus]
Alert, detect, respond with speed
Broker down is what we need to see
Under-replicated partitions cry
Disk usage climbing way too high
Consumer lag will slow you down
Four alerts to keep you sound

[Verse 3]
Disk usage fills up fast and tight
Ninety percent means red alert light
Messages pile up, nowhere to go
Storage limits bring you low
Monitor space before it's gone
Keep your Kafka running strong

[Bridge]
PagerDuty wakes you from your sleep
Slack channels where the alerts you keep
OpsGenie routing to the team
Integration makes the perfect scheme
Route the alerts where they belong
Keep your monitoring game strong

[Verse 4]
Consumer lag means falling behind
Processing slow, alerts you'll find
When consumers can't keep up the rate
Your real-time data starts to wait
Measure lag in time and count
Every second will amount

[Final Chorus]
Alert, detect, respond with speed
Broker down is what we need to see
Under-replicated partitions cry
Disk usage climbing way too high
Consumer lag will slow you down
Four alerts to keep you sound

[Outro]
Prometheus rules will guide your way
Keep your Kafka strong today
Alert before the system breaks
Monitoring is what it takes

3. 1 Prerequisites

[Verse 1]
Before you dive into Strimzi's might
You need a cluster running just right
Minikube for local development play
Kind for testing in a lightweight way
EKS on Amazon's cloud so wide
AKS with Azure by your side
GKE with Google's power and scale
OpenShift when enterprise won't fail

[Chorus]
kubectl and helm, your tools of choice
Command line magic, give them your voice
CPU and memory, storage too
Prerequisites met, we're breaking through
kubectl and helm, deploy with ease
Strimzi's waiting, bring it to its knees

[Verse 2]
Four CPU cores minimum to start
Two gigs of memory, that's just the part
For basic operations, don't go lean
Eight gigs or more keeps performance clean
Storage persistent, volumes that last
SSD preferred when you want it fast
Network policies, security tight
RBAC permissions, get them right

[Chorus]
kubectl and helm, your tools of choice
Command line magic, give them your voice
CPU and memory, storage too
Prerequisites met, we're breaking through
kubectl and helm, deploy with ease
Strimzi's waiting, bring it to its knees

[Bridge]
Check your cluster version, make sure it's current
API compatibility, keep it concurrent
Ingress controllers for external access
Load balancer ready for traffic that's massive
Resource quotas set for namespaces clean
Monitoring tools to see what's unseen

[Verse 3]
Helm three or higher, package management
Repositories added, no more embarrassment
kubectl configured, context is set
API server reachable, you're all set
Node resources labeled, zones defined
Persistent volumes, storage aligned
Prerequisites checked from top to bottom
Now Strimzi deployment, you've finally got them

[Chorus]
kubectl and helm, your tools of choice
Command line magic, give them your voice
CPU and memory, storage too
Prerequisites met, we're breaking through
kubectl and helm, deploy with ease
Strimzi's waiting, bring it to its knees

[Outro]
Foundation solid, cluster prepared
Kafka on Kubernetes, nothing compared
Prerequisites conquered, you're ready to go
Strimzi's full power, now let it flow

4. 1 Rolling Updates & Upgrades

[Verse 1]
When your Kafka cluster needs to change its face
Strimzi rolls the pods one by one with grace
No downtime chaos, no breaking the flow
Rolling restarts keep your data streams to go
Check the ready state before the next in line
StatefulSet magic keeps your uptime fine

[Chorus]
Roll it, upgrade it, keep the stream alive
Strimzi's got the power to help your cluster thrive
Operator first, then Kafka follows through
Protocol versions, format changes too
Roll it, upgrade it, stage by stage we climb
Zero downtime updates every single time

[Verse 2]
First upgrade your operator to the latest release
Download the YAML files, let the magic increase
Apply the new Custom Resource Definitions
Watch the controller handle all transitions
Backward compatibility keeps your configs sound
New features unlock when the operator's found

[Chorus]
Roll it, upgrade it, keep the stream alive
Strimzi's got the power to help your cluster thrive
Operator first, then Kafka follows through
Protocol versions, format changes too
Roll it, upgrade it, stage by stage we climb
Zero downtime updates every single time

[Verse 3]
Kafka version upgrades need a careful dance
Inter-broker protocol, give it half a chance
Log message format stays on the older side
Until all brokers finish their upgrade ride
Two-phase process keeps your data safe
Compatibility matrix shows the proper pace

[Bridge]
Canary deployments for the cautious mind
Test one broker, leave the rest behind
If everything's working, continue the flow
Staged rollouts help your confidence grow
Monitor metrics, watch the logs unfold
Rolling back is easy when problems take hold

[Chorus]
Roll it, upgrade it, keep the stream alive
Strimzi's got the power to help your cluster thrive
Operator first, then Kafka follows through
Protocol versions, format changes too
Roll it, upgrade it, stage by stage we climb
Zero downtime updates every single time

[Outro]
From operator changes to Kafka's new face
Strimzi orchestrates with elegant grace
Rolling updates keep your streams in motion
Kubernetes native, like waves in the ocean

5. 2 Scaling

[Verse 1]
When your cluster needs more power, time to scale it right
Add more brokers to the party, keep the data flowing bright
Edit your Kafka resource, bump the replica count
Watch the operator working, new pods it will mount

[Chorus]
Scale up, scale down, brokers join the dance
Reassign partitions, give each topic a chance
Cruise Control balancing, KafkaRebalance way
Connect workers scaling too, it's your Strimzi day

[Verse 2]
But adding brokers isn't magic, partitions stay the same
They won't spread automatically, that's not how we play the game
Manual reassignment needed, or let automation flow
Cruise Control will optimize, the best path it will show

[Chorus]
Scale up, scale down, brokers join the dance
Reassign partitions, give each topic a chance
Cruise Control balancing, KafkaRebalance way
Connect workers scaling too, it's your Strimzi day

[Verse 3]
Create KafkaRebalance custom resource with your goals
Add brokers mode selected, optimization it controls
Pending state it starts with, then proposal phase begins
ProposalReady status shows the rebalancing wins

[Bridge]
Connect clusters scale the same way
Worker pods increase or decrease
Change the replica field today
Watch your throughput find its peace

[Verse 4]
Scaling down needs extra care, remove brokers last to first
Reassign their partitions out, prevent the data burst
Rolling updates keep it smooth, zero downtime is the key
Strimzi handles all the details, scaling made so free

[Chorus]
Scale up, scale down, brokers join the dance
Reassign partitions, give each topic a chance
Cruise Control balancing, KafkaRebalance way
Connect workers scaling too, it's your Strimzi day

[Outro]
From three brokers up to ten
Or scaling back again
Strimzi makes it possible
Kubernetes and Kafka blend

6. 5 Disaster Recovery

[Verse 1]
When controllers start to fail and quorum breaks apart
Three becomes two becomes one, then silence in the heart
Of your KRaft cluster running wild without a guiding hand
Time to learn recovery before you lose command

[Chorus]
Back it up, restore it right, metadata logs secure
Kafka storage script in hand, bootstrap to be sure
Controller quorum lost tonight, but we know what to do
Five disaster steps ahead, we'll pull the cluster through

[Verse 2]
First assess the damage done, how many nodes remain
Check the metadata directory, what's broken, what's retained
Network partitions split the brain, disk space running low
Monitor your dashboard now to see which way to go

[Chorus]
Back it up, restore it right, metadata logs secure
Kafka storage script in hand, bootstrap to be sure
Controller quorum lost tonight, but we know what to do
Five disaster steps ahead, we'll pull the cluster through

[Bridge]
Kafka storage dot S H, format and begin again
Copy snapshots carefully, restore from backup when
Quorum majority is gone, elect new leaders strong
Production grade monitoring shows us what went wrong

[Verse 3]
Simulate the failures first in lab environment safe
Controller loss and partition, full disk scenarios chafe
Practice makes recovery smooth when real disasters strike
Dashboard shows the cluster health, alerts flash red alike

[Chorus]
Back it up, restore it right, metadata logs secure
Kafka storage script in hand, bootstrap to be sure
Controller quorum lost tonight, but we know what to do
Five disaster steps ahead, we'll pull the cluster through

[Outro]
When the smoke clears and nodes align
KRaft cluster back online
Lessons learned from failure's call
Ready now to handle all

7. 4 Lab: Multi-Cluster Replication

[Verse 1]
Two clusters standing in their namespaces apart
One primary, one secondary, playing their part
Deploy them both with custom resource definitions
Multi-cluster replication, that's our mission

[Chorus]
Mirror Maker Two connects the flow
Active-passive, watch the data go
Source to target, topics replicate
Failover ready, don't hesitate
Mirror Maker Two, the bridge between
Keeping clusters synchronized and clean

[Verse 2]
Configure the connector with source and target defined
Bootstrap servers pointing to clusters aligned
Topic patterns matching what we want to sync
Consumer groups and offsets, all in the link

[Chorus]
Mirror Maker Two connects the flow
Active-passive, watch the data go
Source to target, topics replicate
Failover ready, don't hesitate
Mirror Maker Two, the bridge between
Keeping clusters synchronized and clean

[Bridge]
When disaster strikes the primary down
Secondary cluster takes the crown
Consumer offsets translated through
Thanks to our Mirror Maker Two
Heartbeats flowing, checkpoints saved
Your data stream will be preserved

[Verse 3]
Simulate the failure, watch the switch
From active cluster without a hitch
Validate consumers find their place
Offset synchronization shows no trace
Of interruption in the data stream
Multi-cluster replication supreme

[Chorus]
Mirror Maker Two connects the flow
Active-passive, watch the data go
Source to target, topics replicate
Failover ready, don't hesitate
Mirror Maker Two, the bridge between
Keeping clusters synchronized and clean

[Outro]
Two namespaces, one solution bright
Strimzi makes replication right
Lab complete, you've learned the way
Multi-cluster works today

8. 2 KRaft and Kafka Streams / Connect

[Verse 1]
No more ZooKeeper in our way
KRaft mode changes how we play
State stores still persist the same
But coordination's changed the game
Kafka Streams keeps running smooth
While underneath we've changed the groove

[Chorus]
KRaft Connect and Streams unite
No ZooKeeper in sight
State stores stay, workers coordinate
Eight four eight will demonstrate
New consumer groups align
In the KRaft design

[Verse 2]
Connect workers need to sync
How they coordinate and link
Leader election's built right in
To the Kafka core within
No external dependency
For worker group consistency

[Chorus]
KRaft Connect and Streams unite
No ZooKeeper in sight
State stores stay, workers coordinate
Eight four eight will demonstrate
New consumer groups align
In the KRaft design

[Verse 3]
Consumer groups get redesigned
KIP eight four eight defined
Server-side assignment's here
Makes rebalancing more clear
Streams applications benefit
From this protocol that's fit

[Bridge]
State store management stays true
RocksDB still works for you
But the metadata's moved around
To where Kafka logs are found
Everything's more unified
With KRaft as our guide

[Chorus]
KRaft Connect and Streams unite
No ZooKeeper in sight
State stores stay, workers coordinate
Eight four eight will demonstrate
New consumer groups align
In the KRaft design

[Outro]
From Connect to Streams we see
KRaft brings simplicity
One less service to maintain
Everything in Kafka's domain

9. 4 Backup and Recovery

[Verse 1]
When disaster strikes your cluster down
Your topics hold the precious data crown
Configuration files and settings stored
Back them up before it's all ignored
Export your topic configs to a safe place
YAML files will keep your data's grace
Parameters and partitions defined
A backup strategy peace of mind

[Chorus]
Save it, Store it, Mirror it, Snap it
Four pillars keep your Kafka magic
Topics backed and offsets tracked
MirrorMaker keeps you on the right track
PVC snapshots when you need them most
Disaster recovery is your safety host

[Verse 2]
Consumer offsets tell the reading tale
Where each group left off without fail
Export the offsets to external store
JSON format holds what came before
When recovery time comes around
Restore those positions safe and sound
Your consumers pick up where they left
No message loss, no data theft

[Chorus]
Save it, Store it, Mirror it, Snap it
Four pillars keep your Kafka magic
Topics backed and offsets tracked
MirrorMaker keeps you on the right track
PVC snapshots when you need them most
Disaster recovery is your safety host

[Verse 3]
MirrorMaker Two across the wire
Replicates your streams through storm and fire
Source to target clusters synchronized
Real-time mirroring optimized
Cross datacenter failover ready
Keep your message flow rock steady
Connect framework powers the way
Disaster strikes but data stays

[Bridge]
Volume snapshots capture state
Persistent storage sealed by fate
Point in time recovery blessed
When your cluster faces its test
Storage classes make it clean
Best backup you've ever seen

[Chorus]
Save it, Store it, Mirror it, Snap it
Four pillars keep your Kafka magic
Topics backed and offsets tracked
MirrorMaker keeps you on the right track
PVC snapshots when you need them most
Disaster recovery is your safety host

[Outro]
Four strategies working as one team
Protecting your streaming data dream
Backup and recovery done right
Keeps your Kafka running through the night

10. 3 Multi-Tenancy & Quotas

[Verse 1]
In the KRaft controller world we share
Multiple tenants living everywhere
Need to keep them separated clean
Isolation is the key I mean
Metadata quotas set the boundary line
Rate limiting keeps performance fine

[Chorus]
Multi-tenant magic, keep them apart
Quotas guard the resources, that's the art
Rate limits flowing, controllers stay strong
Isolation patterns, nothing goes wrong
KRaft keeps it balanced, sharing done right
Multi-tenant magic, day and night

[Verse 2]
Topic creation has a ceiling count
Partition numbers, every byte does mount
Configuration changes need a brake
Too many requests, the system might shake
Controller resources split with care
Memory and CPU for tenants to share

[Chorus]
Multi-tenant magic, keep them apart
Quotas guard the resources, that's the art
Rate limits flowing, controllers stay strong
Isolation patterns, nothing goes wrong
KRaft keeps it balanced, sharing done right
Multi-tenant magic, day and night

[Bridge]
Namespaces separate the logical space
Access control keeps each tenant in place
Monitoring dashboards show who's using what
Resource allocation, every thread and slot
Controller elections stay fair and clean
Best multi-tenant system you've ever seen

[Verse 3]
Network bandwidth gets its limits too
Request queues managed, never overflow through
Tenant priorities can be assigned
Keep the noisy neighbors well confined
Graceful degradation when resources are tight
Every tenant gets their share just right

[Chorus]
Multi-tenant magic, keep them apart
Quotas guard the resources, that's the art
Rate limits flowing, controllers stay strong
Isolation patterns, nothing goes wrong
KRaft keeps it balanced, sharing done right
Multi-tenant magic, day and night

[Outro]
When you're building clusters meant to share
Remember quotas, isolation, care
Multi-tenant KRaft will see you through
Resource management tried and true

11. 3 Distributed Tracing

[Verse 1]
In our Kafka streams where messages flow
We need to trace each step to really know
From producer through the broker's heart
To consumer where the journey ends its part
OpenTelemetry becomes our guide
Instrumenting every single ride

[Chorus]
Trace it through, trace it through
Producer broker consumer too
Spans and tags light up the way
Jaeger shows us night and day
Every hop and every call
Distributed tracing sees it all

[Verse 2]
Configure Strimzi with the tracing flag
Set the service name and add the tag
Jaeger agent running in your pod
Collects the spans like a tracing god
Each message carries context in its head
Following the correlation thread

[Chorus]
Trace it through, trace it through
Producer broker consumer too
Spans and tags light up the way
Jaeger shows us night and day
Every hop and every call
Distributed tracing sees it all

[Bridge]
From the moment that you call send
To the broker where the logs extend
Through partitions and replicas
To consumer groups across the class
Timeline shows the latency
Bottlenecks for all to see

[Verse 3]
Zipkin's another choice you have
Both will make your debugging fab
Sampling rates to control the load
Too many traces can explode
Set your headers, configure right
Turn your black box crystal bright

[Chorus]
Trace it through, trace it through
Producer broker consumer too
Spans and tags light up the way
Jaeger shows us night and day
Every hop and every call
Distributed tracing sees it all

[Outro]
When your Kafka starts to lag
Check your traces, every tag
OpenTelemetry's your friend
Visibility from start to end

12. 3 Capacity Planning

[Verse 1]
When planning Kafka clusters on Kubernetes ground
Three factors multiply to keep your data sound
Retention times throughput and replication too
Storage requirements calculated just for you
Days times gigabytes times copies that you make
Equals total storage space your cluster needs to take

[Chorus]
ReTention Throughput Replication multiply
Storage sizing made simple reach up to the sky
Two gigs CPU four gigs RAM per broker start
Network bandwidth doubles when replicas depart
Plan your partitions wisely scale your Kafka heart

[Verse 2]
Each broker needs resources to handle all the load
Two CPU cores minimum to walk the data road
Four gigabytes of memory baseline for the game
More producers more consumers fan the resource flame
Scale up with your workload don't leave your brokers dry
Monitor and adjust as traffic volumes fly

[Chorus]
ReTention Throughput Replication multiply
Storage sizing made simple reach up to the sky
Two gigs CPU four gigs RAM per broker start
Network bandwidth doubles when replicas depart
Plan your partitions wisely scale your Kafka heart

[Bridge]
Network flows between the brokers replication stream
Double your producer bandwidth living the dream
Partition count guidelines keep performance tight
Ten thousand max per cluster keeps everything right
Balance across your brokers spread the load around
Strimzi makes it easier Kafka safe and sound

[Verse 3]
Start with topic size and growth rate that you see
Multiply by replicas two or maybe three
Cross-cluster traffic patterns network load design
CPU spikes with compression memory aligns
Test your calculations in development first
Before production traffic puts you to the test

[Chorus]
ReTention Throughput Replication multiply
Storage sizing made simple reach up to the sky
Two gigs CPU four gigs RAM per broker start
Network bandwidth doubles when replicas depart
Plan your partitions wisely scale your Kafka heart

[Outro]
Capacity planning victory Strimzi shows the way
Kafka clusters running strong every single day

13. 3 Cruise Control for Rebalancing

[Verse 1]
When your Kafka cluster's feeling unbalanced
Partitions scattered, performance challenged
Cruise Control comes to save the day
Optimizing assignments in every way
It watches your brokers, analyzes the load
Finds the perfect path down rebalance road

[Chorus]
KafkaRebalance resource, approval workflow
Full mode, add-broker, remove-broker go
Monitor progress, revert if needed
Cruise Control keeps your cluster completed
Partition assignment optimization
That's the Cruise Control sensation

[Verse 2]
Create a KafkaRebalance with your spec defined
Choose your rebalance mode, keep goals in mind
Full rebalance moves partitions all around
Add-broker spreads the load when new nodes are found
Remove-broker safely shifts before you scale down
Three modes to keep your cluster sound

[Chorus]
KafkaRebalance resource, approval workflow
Full mode, add-broker, remove-broker go
Monitor progress, revert if needed
Cruise Control keeps your cluster completed
Partition assignment optimization
That's the Cruise Control sensation

[Bridge]
Check the status, watch it run
ProposalReady when planning's done
Set refresh annotation to approve
Watch those partitions start to move
If something's wrong, don't hesitate
Revert the changes, don't be late

[Verse 3]
Monitor the progress through each phase
Ready, Rebalancing shows the ways
Completed means your work is done
But if you need to, hit revert and run
The approval workflow keeps control
While Cruise Control achieves your goal

[Chorus]
KafkaRebalance resource, approval workflow
Full mode, add-broker, remove-broker go
Monitor progress, revert if needed
Cruise Control keeps your cluster completed
Partition assignment optimization
That's the Cruise Control sensation

[Outro]
Balanced brokers, optimized load
Cruise Control on Kubernetes road
KafkaRebalance shows the way
For a better cluster every day

14. 5 Multi-Tenancy

[Verse 1]
In the Kafka world we need to share
Multiple tenants everywhere
Namespace walls keep data clean
Isolation strategies unseen
Dev and prod must stay apart
Resource boundaries from the start

[Chorus]
Multi-tenant, keep it separate
Quotas guard what we don't waste
Name your topics, set your ACLs
Shared or dedicated, choose it well
Multi-tenant, scaling right
Kubernetes keeps it all in sight

[Verse 2]
Produce and consume byte rates controlled
Request quotas, stories told
Throttling kicks when limits breach
Performance lessons that we teach
User principals get their slice
Resource management, roll the dice

[Chorus]
Multi-tenant, keep it separate
Quotas guard what we don't waste
Name your topics, set your ACLs
Shared or dedicated, choose it well
Multi-tenant, scaling right
Kubernetes keeps it all in sight

[Verse 3]
Topic naming with a plan
Team prefix, understand the span
Environment tags make it clear
Which data belongs to whom here
ACL patterns lock it down
Permission models all around

[Bridge]
Shared clusters save your cost
But isolation might get lost
Dedicated gives you control
Higher price but cleaner role
Choose your path with workload size
Traffic patterns, your compromise

[Chorus]
Multi-tenant, keep it separate
Quotas guard what we don't waste
Name your topics, set your ACLs
Shared or dedicated, choose it well
Multi-tenant, scaling right
Kubernetes keeps it all in sight

[Outro]
Strimzi operators help you scale
Multi-tenancy without fail
From namespace walls to quota guards
Your Kafka cluster playing cards

15. 4 GitOps & Infrastructure as Code

[Verse 1]
ArgoCD pulls your Strimzi configs down
From Git repos where your truth is found
Declarative state, no manual touch
GitOps workflow keeps your clusters in sync so much
Flux can watch your branches too
Auto-deploy when changes push through

[Chorus]
GitOps flows, Infrastructure as Code
A-R-G-O, F-L-U-X on the road
Helm and Kustomize, environments align
CI-CD pipelines keep your Kafka in line
Version control, review and merge
Infrastructure automation on the verge

[Verse 2]
Helm charts package all your needs
Values files plant the config seeds
Override defaults, customize your way
Production settings different than your dev today
Template engine renders clean
Best practices in every YAML scene

[Chorus]
GitOps flows, Infrastructure as Code
A-R-G-O, F-L-U-X on the road
Helm and Kustomize, environments align
CI-CD pipelines keep your Kafka in line
Version control, review and merge
Infrastructure automation on the verge

[Bridge]
Kustomize overlays, base plus patches
Dev test staging prod, each environment matches
Different namespaces, different scales
Different secrets, different tales
Topic management through automation
Connector configs cross every nation

[Verse 3]
CI pipelines validate your code
Before it hits the cluster road
Schema registry updates flow
Topic creation, let the automation go
Connector configs versioned tight
Pull requests make your changes right

[Chorus]
GitOps flows, Infrastructure as Code
A-R-G-O, F-L-U-X on the road
Helm and Kustomize, environments align
CI-CD pipelines keep your Kafka in line
Version control, review and merge
Infrastructure automation on the verge

[Outro]
From Git to cluster, smooth and clean
Best GitOps practices you've ever seen
Strimzi managed, infrastructure as code
Welcome to the automated road

16. 1 Strimzi on OpenShift

[Verse 1]
On OpenShift platform we deploy with care
Strimzi operators floating in the air
Routes expose our clusters to the world outside
While Security Context Constraints keep us safe inside
Red Hat AMQ Streams is the enterprise way
Commercial distribution for production day

[Chorus]
Routes, SCCs, OLM we say
OpenShift features pave the way
AMQ Streams for enterprise grade
Monitoring tools help problems fade
Strimzi on Red Hat's container stage
OpenShift makes Kafka's golden age

[Verse 2]
Operator Lifecycle Manager handles all the gear
Installing operators year after year
Service monitors connect to Prometheus stack
Grafana dashboards keep us right on track
Container security contexts lock things down
No root access wearing the admin crown

[Chorus]
Routes, SCCs, OLM we say
OpenShift features pave the way
AMQ Streams for enterprise grade
Monitoring tools help problems fade
Strimzi on Red Hat's container stage
OpenShift makes Kafka's golden age

[Bridge]
OpenShift Routes replace Ingress calls
HTTP and HTTPS through the firewall
AMQ Streams brings support and patches too
Enterprise features built for me and you
Pod Security Standards keep containers tight
Monitoring integration shining bright

[Verse 3]
ClusterServiceVersions define what we need
OLM installs and makes our cluster feed
Anyuid SCC when containers must run free
Restricted mode for better security
Prometheus rules fire when things go wrong
OpenShift Console keeps our cluster strong

[Chorus]
Routes, SCCs, OLM we say
OpenShift features pave the way
AMQ Streams for enterprise grade
Monitoring tools help problems fade
Strimzi on Red Hat's container stage
OpenShift makes Kafka's golden age

[Outro]
From upstream Strimzi to AMQ brand
OpenShift features help us understand
Enterprise Kafka running cloud native way
Strimzi on OpenShift every day

17. 2 Kafka Bridge (HTTP API)

[Verse 1]
When legacy systems need to talk to streams
And REST APIs fulfill your dreams
Deploy the Bridge with a simple CRD
HTTP to Kafka, seamlessly

KafkaBridge custom resource you define
Replicas and bootstrap servers align
Port eight zero eight zero opens the door
RESTful messaging like never before

[Chorus]
Bridge the gap, REST to streams
HTTP calls fulfill your dreams
Post to topics, get from queues
Legacy systems, serverless too
Bridge the gap, make it flow
Kafka Bridge is the way to go

[Verse 2]
Producing messages through HTTP POST
Content type JSON is what you need most
Topics endpoint takes your payload in
Consumer groups let the reading begin

IoT devices send their sensor data
Serverless functions process metadata
No native client libraries required
Just simple REST calls as desired

[Chorus]
Bridge the gap, REST to streams
HTTP calls fulfill your dreams
Post to topics, get from queues
Legacy systems, serverless too
Bridge the gap, make it flow
Kafka Bridge is the way to go

[Bridge]
But remember the performance cost
Native clients won't be lost
HTTP overhead adds delay
For high throughput, consider the native way

Stateless bridge means no consumer state
Long polling helps but don't be late
Scale your bridge pods when traffic grows
Monitor metrics, watch how it flows

[Verse 3]
Use cases span from web to cloud
Integration patterns, flexible and proud
REST microservices join the stream
Kafka Bridge completes the team

Configure cors for browser calls
Enable metrics to monitor it all
Authentication and authorization too
Security features see you through

[Chorus]
Bridge the gap, REST to streams
HTTP calls fulfill your dreams
Post to topics, get from queues
Legacy systems, serverless too
Bridge the gap, make it flow
Kafka Bridge is the way to go

[Outro]
From CRD deploy to REST API
Kafka Bridge lifts your systems high
HTTP to streams, the bridge you need
Strimzi makes integration succeed

18. 3 Schema Registry Integration

[Verse 1]
In Kubernetes where Kafka streams
We need more than just the data flowing
Schema Registry joins the scene
To keep our message structure growing
Apicurio or Confluent's way
Both can serve your validation needs
Deploy them right beside Strimzi
And watch your data quality succeed

[Chorus]
Register, validate, evolve with care
Avro, Protobuf, JSON everywhere
Backward, forward, full compatibility
Schema Registry brings reliability
Version control for every message sent
Evolution without breaking what was meant

[Verse 2]
Avro brings us binary tight
Compact serialization strong
Schema first approach feels right
Code generation tags along
Protobuf from Google's land
Language neutral, fast and clean
While JSON Schema takes a stand
For readable validation schemes

[Chorus]
Register, validate, evolve with care
Avro, Protobuf, JSON everywhere
Backward, forward, full compatibility
Schema Registry brings reliability
Version control for every message sent
Evolution without breaking what was meant

[Bridge]
Backward compatibility reads the old
Forward compatibility writes the new
Full means both directions hold
None means breaking changes through
Transitive checks the chain complete
From version one to latest state
Your schemas never skip a beat
When evolution rules you create

[Verse 3]
Deploy Registry in the same namespace
Network policies keep it secure
Service discovery finds its place
Through Kubernetes DNS for sure
Schema subjects organize your types
Version numbers track each change
Consumer groups avoid the gripes
When schemas shift within their range

[Chorus]
Register, validate, evolve with care
Avro, Protobuf, JSON everywhere
Backward, forward, full compatibility
Schema Registry brings reliability
Version control for every message sent
Evolution without breaking what was meant

[Outro]
From producer to consumer flow
Registry stands between
Ensuring that the data show
Stays valid, typed, and clean

19. 4 Kafka Streams & ksqlDB on Kubernetes

[Verse 1]
Deploy your streams beside the Strimzi cluster running
State stores need persistence when the data keeps on coming
Mount those volumes carefully, let Kubernetes manage
Stateful sets for storage, keep your data from damage

[Chorus]
Stream, store, scale and query
Deploy, persist, interact, worry-free
Kafka Streams on K8s dancing
State management advancing
Stream, store, scale and query
That's the Strimzi mastery

[Verse 2]
Interactive queries let you peek inside the state
HTTP endpoints serving data that your streams create
Read-only views of local stores, no need to duplicate
Query your topology live while processing doesn't wait

[Chorus]
Stream, store, scale and query
Deploy, persist, interact, worry-free
Kafka Streams on K8s dancing
State management advancing
Stream, store, scale and query
That's the Strimzi mastery

[Bridge]
Scale out horizontally, partition key determines
Which instance holds your data as the workload churns
Standby replicas ready when your primary node fails
Kubernetes orchestration keeps your streaming on the rails

[Verse 3]
Application instances spread across the cluster wide
Each one owns specific partitions, state stores reside
When you need to find the data, route to proper host
Metadata tells you exactly which pod matters most

[Chorus]
Stream, store, scale and query
Deploy, persist, interact, worry-free
Kafka Streams on K8s dancing
State management advancing
Stream, store, scale and query
That's the Strimzi mastery

[Outro]
Streams and storage unified
Kubernetes by your side
Strimzi makes it simplified
Real-time processing pride

20. 5 Custom Operator Extensions

[Verse 1]
Start with custom reconcilers, extending what Strimzi can do
Write your own controllers that watch the cluster through and through
Override default behaviors, add your business logic in
Custom resource definitions help your journey to begin
Import the Java libraries, build upon the solid ground
Program cluster management where automation can be found

[Chorus]
Five extensions we can make, C-R-U-D operations take
Custom reconcilers wake, when resource changes we create
Java libs help automate, programmatic control state
Open source collaborate, to the project contribute

[Verse 2]
Inherit from abstract classes, override the reconcile method
Watch for status changes, let your custom logic spread
Use the Strimzi client libs, Kubernetes API calls
Fabric8 and operator SDK help when complexity crawls
Handle create update delete, manage your resource lifecycle
Custom annotations and labels make your extensions more precise

[Chorus]
Five extensions we can make, C-R-U-D operations take
Custom reconcilers wake, when resource changes we create
Java libs help automate, programmatic control state
Open source collaborate, to the project contribute

[Bridge]
Contributing back upstream, fork the repository clean
Submit your pull requests, follow guidelines pristine
Write your tests and documentation, community code review
Make the project better for everyone who's using it too

[Verse 3]
Config maps and secrets, manage them programmatically
Status conditions and events, update them systematically
Error handling gracefully, retry with backoff delay
Metrics and observability show your operator's working way
Package in container images, deploy with Helm or YAML files
Custom operators running smooth across your Kafka cluster miles

[Chorus]
Five extensions we can make, C-R-U-D operations take
Custom reconcilers wake, when resource changes we create
Java libs help automate, programmatic control state
Open source collaborate, to the project contribute

[Outro]
Extend Strimzi's power, make it fit your special needs
Custom operator mastery plants those automation seeds

21. Appendix A: Quick Reference — Strimzi CRD Cheat Sheet

[Verse 1]
Nine resources to orchestrate your Kafka world
Each CRD has a purpose when deployed and unfurled
Kafka brings the cluster with ZooKeeper by its side
Spec dot kafka, zookeeper, entityOperator guide

[Chorus]
K-A-F-K-A leads the way
Node pools, topics, users every day
Connect and Bridge will route your streams
Nine CRDs fulfill your Kafka dreams

[Verse 2]
KafkaNodePool manages roles in KRaft design
Spec dot roles and replicas keep your nodes in line
Storage configuration sets the persistence right
KafkaTopic handles partitions, replicas, config bright

[Chorus]
K-A-F-K-A leads the way
Node pools, topics, users every day
Connect and Bridge will route your streams
Nine CRDs fulfill your Kafka dreams

[Verse 3]
KafkaUser brings authentication to the scene
Authorization controls what users can see
KafkaConnect builds clusters with replicas and config
Individual connectors need their class and logic

[Bridge]
MirrorMaker2 mirrors clusters far and wide
Spec dot clusters, mirrors side by side
KafkaBridge opens HTTP doors
Replicas and http specs explore

[Verse 4]
KafkaRebalance calls on Cruise Control's might
Mode and goals make partition distribution right
From Kafka to Rebalance, nine resources strong
Each one serves a purpose in your streaming song

[Chorus]
K-A-F-K-A leads the way
Node pools, topics, users every day
Connect and Bridge will route your streams
Nine CRDs fulfill your Kafka dreams

[Outro]
Strimzi's got the resources you need
Custom definitions help you succeed
Kubernetes and Kafka working as one
Your streaming platform's never been done

22. Appendix B: Recommended Learning Resources

[Verse 1]
When you're starting your Kubernetes journey with Kafka streams
Strimzi dot I-O documentation's where you'll find your dreams
Official guides and tutorials, installation made clear
Reference docs and examples that will steer you from here

[Chorus]
Six resources, learn them well
Strimzi GitHub, docs that tell
Blog and Kafka's native guide
CNCF talks and Confluent's stride
Build your knowledge, piece by piece
Let these sources bring you peace

[Verse 2]
GitHub dot com slash Strimzi holds the source code treasure
Issues, pull requests, and samples you can use at leisure
Operators, bridges, and connectors all living in the repo
Helm charts and examples help your understanding grow

[Chorus]
Six resources, learn them well
Strimzi GitHub, docs that tell
Blog and Kafka's native guide
CNCF talks and Confluent's stride
Build your knowledge, piece by piece
Let these sources bring you peace

[Verse 3]
Strimzi dot I-O slash blog brings the latest news
Release notes and case studies, real-world tips you can use
Best practices and patterns from the community's voice
Making complex deployments feel like an easy choice

[Bridge]
Kafka dot Apache dot org documentation's foundation
Core concepts and protocols, streaming computation
While CNCF webinars showcase the cloud native way
And Confluent Developer teaches fundamentals every day

[Verse 4]
Search for Strimzi talks in CNCF's treasure trove
Conference presentations showing how the pros evolve
Developer dot Confluent dot I-O for Kafka's base
Schema registry, streams API, producer-consumer grace

[Chorus]
Six resources, learn them well
Strimzi GitHub, docs that tell
Blog and Kafka's native guide
CNCF talks and Confluent's stride
Build your knowledge, piece by piece
Let these sources bring you peace

[Outro]
From beginner to expert, these paths will light your way
Strimzi mastery awaits you, study every day

23. Appendix C: Lab Environment Setup

[Verse 1]
Setting up your lab today, three paths to choose your way
Kind is lightweight, runs in Docker, perfect for local play
Kubernetes in Docker wrapped, minimal resource trap
Development flows so smooth, when you're learning Kafka's groove

[Chorus]
Three nodes, four cores, sixteen gigs of RAM
Hundred gigabyte storage for your Kafka plan
Kind, Minikube, or the cloud above
Choose your path to Strimzi love

[Verse 2]
Minikube needs more power, eight gigs minimum hour
Virtual machine spinning up, Kubernetes buttercup
Memory allocation high, let your local cluster fly
Hypervisor underneath, makes your setup complete

[Chorus]
Three nodes, four cores, sixteen gigs of RAM
Hundred gigabyte storage for your Kafka plan
Kind, Minikube, or the cloud above
Choose your path to Strimzi love

[Verse 3]
Cloud managed is the third way, EKS, AKS, GKE
Small node pool to start you right, production-like insight
Amazon, Azure, Google's might, enterprise-grade flight
Scaling up when traffic grows, that's how the story goes

[Bridge]
Local development or cloud production
Choose your infrastructure reduction
SSD storage keeps it fast
Lab environment built to last

[Chorus]
Three nodes, four cores, sixteen gigs of RAM
Hundred gigabyte storage for your Kafka plan
Kind, Minikube, or the cloud above
Choose your path to Strimzi love

[Outro]
Lab environment set and ready
Kubernetes cluster running steady
Strimzi waiting for deploy
Time to learn and time to enjoy

24. 2 Installation de Strimzi

[Verse 1]
Trois chemins s'ouvrent devant nous pour déployer
Strimzi sur notre cluster Kubernetes
Helm avec ses charts tout configurés
Ou les manifestes YAML à télécharger

[Chorus]
Helm, YAML, OperatorHub
Trois options pour ton Kafka club
Cluster Operator va surveiller
Tous tes espaces à gérer
Installation, vérification
Strimzi en action

[Verse 2]
Option A, le chart Helm officiel
Repository ajouté, values personnalisés
Une commande et tout s'installe
Paramètres modifiables, déploiement facilité

[Chorus]
Helm, YAML, OperatorHub
Trois options pour ton Kafka club
Cluster Operator va surveiller
Tous tes espaces à gérer
Installation, vérification
Strimzi en action

[Verse 3]
Option B depuis GitHub releases
Télécharge les manifestes, applique kubectl
Fichiers YAML, contrôle précis
Chaque ressource définie, rien d'inutile

[Bridge]
OperatorHub sur OpenShift
Marketplace intégré, installation swift
OLM gère tout automatiquement
Opérateur déployé simplement

[Verse 4]
Cluster Operator maintenant actif
Vérifie les pods, status, logs descriptifs
Single namespace ou multi-espaces
Surveille un seul ou toute la place

[Chorus]
Helm, YAML, OperatorHub
Trois options pour ton Kafka club
Cluster Operator va surveiller
Tous tes espaces à gérer
Installation, vérification
Strimzi en action

[Outro]
Considère bien ton architecture
Un espace ou plusieurs, quelle structure
Strimzi maintenant opérationnel
Kafka sur Kubernetes fonctionnel

25. 3 Pourquoi Strimzi ?

[Verse 1]
Kubernetes gère les pods sans état si bien
Mais les données persistantes, c'est un autre chemin
Les volumes, les réseaux, la haute disponibilité
Apache Kafka a besoin de stabilité

Les charges avec état demandent plus d'attention
Ordre de démarrage, stockage, coordination
Sans opérateur, c'est la complication
Strimzi apporte la solution

[Chorus]
Pourquoi Strimzi, trois raisons à retenir
CNCF incubé, communauté à servir
Opérateur natif, cycle de vie maîtrisé
Kubernetes et Kafka réconciliés

[Verse 2]
Confluent Operator, solution commerciale
Serveurs physiques, méthode traditionnelle
Mais Strimzi brille par sa philosophie
Open source, portable, démocratisée

L'écosystème CNCF garantit la pérennité
Standards ouverts, interopérabilité
La communauté active assure l'évolution
Pas de vendor lock-in, pure innovation

[Chorus]
Pourquoi Strimzi, trois raisons à retenir
CNCF incubé, communauté à servir
Opérateur natif, cycle de vie maîtrisé
Kubernetes et Kafka réconciliés

[Bridge]
L'opérateur observe l'état désiré
Compare avec l'existant, fait converger
Custom Resources définissent la configuration
Reconciliation loop, automatisation

[Verse 3]
Déploiement déclaratif, infrastructure as code
Mise à jour progressive, rolling upgrade mode
Monitoring intégré, métriques exposées
Sécurité renforcée, secrets orchestrés

Du broker au topic, tout est contrôlé
Schema Registry, Connect déployé
Un seul YAML suffit à tout paramétrer
L'état converge vers ce qui est souhaité

[Chorus]
Pourquoi Strimzi, trois raisons à retenir
CNCF incubé, communauté à servir
Opérateur natif, cycle de vie maîtrisé
Kubernetes et Kafka réconciliés

[Outro]
État désiré, état observé
L'opérateur fait le travail automatisé
Strimzi, le choix de la modernité
Kafka cloud-native, réalité

26. 2 Configuration de l'accès externe

[Verse 1]
Dans Kubernetes on déploie notre Kafka
Les courtiers tournent mais comment les joindra
L'accès externe demande configuration
Écouteurs annoncés, première leçon

[Chorus]
Advertised listeners, c'est la clé
Pour que les clients puissent se connecter
TLS passthrough ou terminaison
Bootstrap partagé, bonne liaison
Configure bien tes noms d'hôte
Sinon ton cluster capote

[Verse 2]
Chaque courtier a son adresse unique
Les clients doivent la résolution magique
DNS configuré pour pointer correctement
Vers les pods Kafka qui bougent constamment

[Chorus]
Advertised listeners, c'est la clé
Pour que les clients puissent se connecter
TLS passthrough ou terminaison
Bootstrap partagé, bonne liaison
Configure bien tes noms d'hôte
Sinon ton cluster capote

[Verse 3]
TLS passthrough laisse passer le flux
Jusqu'au courtier qui déchiffre au plus juste
Terminaison TLS s'arrête au proxy
LoadBalancer gère la sécurité

[Bridge]
Service bootstrap pour commencer
Un seul point d'entrée à contacter
Puis chaque courtier son service aura
NodePort ou LoadBalancer au choix

[Verse 4]
Partagé ou dédié pour chaque nœud
Configuration selon tes vœux
External listeners bien déclarés
Dans ton Custom Resource déployé

[Chorus]
Advertised listeners, c'est la clé
Pour que les clients puissent se connecter
TLS passthrough ou terminaison
Bootstrap partagé, bonne liaison
Configure bien tes noms d'hôte
Sinon ton cluster capote

[Outro]
Strimzi facilite le paramétrage
L'accès externe n'est plus un mirage
Kafka sur Kubernetes maintenant
Accessible de l'extérieur simplement

27. 2 Authentification

[Verse 1]
Dans Strimzi trois méthodes pour s'authentifier
TLS mutuel avec certificats pour se connecter
SCRAM-SHA-512 avec nom d'utilisateur secret
OAuth 2.0 pour les services modernes parfaits

[Chorus]
Authentification, trois façons de protéger
TLS, SCRAM, OAuth pour sécuriser
KafkaUser CRD configure les identifiants
Par écouteur on définit les différents moyens

[Verse 2]
TLS mutuel c'est client et serveur
Échangent leurs certificats avec confiance et honneur
Le client prouve son identité par sa clé privée
Le serveur vérifie et l'accès peut commencer

[Chorus]
Authentification, trois façons de protéger
TLS, SCRAM, OAuth pour sécuriser
KafkaUser CRD configure les identifiants
Par écouteur on définit les différents moyens

[Verse 3]
SCRAM-SHA-512 utilise nom d'utilisateur
Avec mot de passe haché, sécurité supérieure
Salt et itérations renforcent le chiffrement
Kafka stocke les secrets de façon permanente

[Bridge]
OAuth 2.0 s'intègre avec les fournisseurs
Keycloak, Azure AD, Okta pour les utilisateurs
Token JWT validé par le serveur d'autorisation
Centralise la gestion de l'authentification

[Verse 4]
Chaque écouteur peut avoir sa méthode
Port neuf zéro neuf deux pour TLS qui protège
Externe avec OAuth pour les apps distantes
Interne avec SCRAM pour les connexions courantes

[Chorus]
Authentification, trois façons de protéger
TLS, SCRAM, OAuth pour sécuriser
KafkaUser CRD configure les identifiants
Par écouteur on définit les différents moyens

[Outro]
KafkaUser définit les droits et l'accès
Pour chaque utilisateur les règles sont fixées
Trois méthodes au choix selon le contexte
L'authentification Kafka devient plus complexe

28. 4 Stratégies de stockage

[Verse 1]
Dans Strimzi tu dois planifier
Comment tes données vont résider
JBOD c'est la configuration
Plusieurs disques en coalition
Chaque broker a son espace
Pour que Kafka trouve sa place

[Chorus]
Quatre stratégies à retenir
Pour ton stockage bien choisir
JBOD, classes, extension, test
Performances qui résistent
Gp trois, io deux, Premium SSD
Ton cluster va prospérer

[Verse 2]
AWS offre gp trois rapide
io deux quand tout va vite
Azure propose Premium SSD
Pour des débits garantis
Google Cloud a ses solutions
Selon tes résolutions

[Chorus]
Quatre stratégies à retenir
Pour ton stockage bien choisir
JBOD, classes, extension, test
Performances qui résistent
Gp trois, io deux, Premium SSD
Ton cluster va prospérer

[Verse 3]
Étendre les volumes c'est possible
Redimensionner sans risible
Kubernetes gère l'expansion
Avec tes spécifications
Surveille bien la capacité
Pour éviter la saturation

[Bridge]
Étalonne les performances
Mesure les latences
IOPS et débit ensemble
Pour que tout s'assemble
Benchmark ton système
Avant le problème

[Chorus]
Quatre stratégies à retenir
Pour ton stockage bien choisir
JBOD, classes, extension, test
Performances qui résistent
Gp trois, io deux, Premium SSD
Ton cluster va prospérer

[Outro]
Stockage bien configuré
Kafka peut s'épanouir
Dans ton cloud préféré
Strimzi va t'éblouir

29. Annexe D : Parcours de certification

[Verse 1]
Dans ton parcours Strimzi, trois certifs à viser
CCDAK pour Kafka, développeur confirmé
Producers et consumers, partitions à maîtriser
Schemas et sérialisation, tout doit fonctionner

[Chorus]
CCDAK, CKA, CKAD
Trois lettres d'or pour ton CV
Kafka plus Kubernetes
Strimzi expertise validée
CCDAK, CKA, CKAD
La route vers l'excellence tracée

[Verse 2]
CKA pour l'admin, clusters sous contrôle
Networking, storage, chaque nœud qui s'envole
Troubleshooting expert, RBAC configuré
La haute disponibilité, c'est ton métier

[Chorus]
CCDAK, CKA, CKAD
Trois lettres d'or pour ton CV
Kafka plus Kubernetes
Strimzi expertise validée
CCDAK, CKA, CKAD
La route vers l'excellence tracée

[Bridge]
CKAD développeur, applications déployées
Pods et services, tout est orchestré
Helm charts et operators, yaml maîtrisé
Dans l'écosystème cloud, tu peux naviguer

[Verse 3]
Version un point zéro, février vingt-six
Strimzi zéro trente-huit, les specs qui se fixent
Kafka trois point six, streaming moderne
Cette combinaison rend ton profil terne

[Chorus]
CCDAK, CKA, CKAD
Trois lettres d'or pour ton CV
Kafka plus Kubernetes
Strimzi expertise validée
CCDAK, CKA, CKAD
Validation solide, carrière assurée

[Outro]
Trois certifications, une expertise complète
Strimzi mastery, ta mission secrète

Back to Home

Kafka on Kubernetes with Strimzi | 93