Atlassian's Kafka switch: 145B events a day, no more Kinesis

Atlassian is moving off AWS Kinesis. The team is going to Kafka. The scale is big: 145 billion daily events. They say Kinesis couldn’t keep up. The story is from their engineering blog, “Scaling StreamHub: Transitioning from Kinesis to Kafka.”

Atlassian says the switch was not a small upgrade. It was a full migration. They had to move all the data. They had to make sure nothing was lost. They had to keep their tools running while the switch happened. That is a lot of moving parts.

Kafka is a message system built for scale. It handles a lot of traffic at once. Kinesis is AWS’s own system. It is fast, but Atlassian says it hit a wall at their size. 145 billion events a day is a lot of data. That is not a small side project. That is the main engine.

Atlassian does not say the old way was bad. They say it was good for a while, but not good enough for what they need now. They need speed. They need room to grow. They need to keep their platform stable. The blog post walks through how they planned the move, how they tested it, and what broke along the way.

For the comunidad, this is a reminder that even big companies hit limits. They have to change. They have to pick new tools. It is not always easy. It takes planning. It takes patience. It takes a team willing to break things to fix them.

We work with tools every day. Sometimes the tool we love stops working for us. That is normal. The fix is not to panic. It is to look at what we need and find a better fit. Atlassian did that. They looked at 145 billion events and said, “We need more.”

Why this matters for us: when big platforms shift, the tools we use every day change too, and we need to stay sharp so we are not left behind by the next upgrade.