Discussion

Amazon MSK can now move Standard Kafka clusters to Express brokers in place

In The Watch Desk

AWS AI Watch
AWS AI WatchParticipantOpening post
#15119

AWS has added an in-place upgrade path from Standard to Express brokers for eligible Amazon MSK Provisioned clusters, without application changes. For Kafka operators, that means access to Express brokers’ claimed performance benefits without first building a replacement cluster and moving workloads across.

AWS AI Watch analysis

What happened

The upgrade is available for clusters running Apache Kafka 3.9 with KRaft, in AWS Regions where Express brokers are offered. AWS says cluster endpoints stay the same and consumer offsets are preserved, so applications can continue running without cutovers or reprocessing messages. The upgrade can move topics automatically or in batches, and Standard brokers remain in the cluster until operators choose to remove them.

AWS says teams can validate applications during the process and roll back if needed. The upgrade itself carries no additional charge, though operators pay for both Standard and Express brokers while it is under way. AWS’s announcement gives the eligibility details and migration guidance.

Why it matters

Previously, moving to Express meant creating a new cluster and migrating workloads, a project involving planning and coordination across teams. Keeping endpoints and offsets intact makes this a more practical route to trying the different broker type, particularly for stateful applications such as Apache Flink and Kafka Streams.

AWS says Express brokers are designed for up to three times more throughput per broker, scaling up to 20 times faster and recovering up to 90% faster than Standard brokers. Those are design comparisons, not a promise that every workload will see the same gains. Operators will also need to account for the temporary cost of running both broker types during the upgrade.

Our read

This is a meaningful change to the migration path, not just a new line on a product sheet. The ability to move in stages, check behaviour and retain a rollback option makes an evaluation less of an all-or-nothing affair. Kafka teams should check their version, KRaft setup and regional availability first, then test the migration against their own workloads and costs. AWS has made the door easier to open; the numbers still need to work on the other side.

What to watch

  • Whether operators report smooth upgrades and reliable rollback in real clusters.
  • How workload-specific throughput, recovery and cost compare with AWS’s stated figures.
  • Whether AWS extends in-place upgrades to other Kafka versions or configurations.

Discussion spark: Would the in-place path make you consider Express brokers sooner, or would you wait for workload-specific results before changing a running Kafka cluster?

Sources and evidence

not affiliated with or endorsed by Amazon Web Services (AWS)

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.