Scaling session replay 200x

As the team lead for Session Replay, I owned the part of the platform that captured and replayed real user sessions. The mandate was simple to say and hard to do: take it from a system straining at 250k sessions a day to one that could absorb enterprise traffic, without losing a single event.

From on-prem to AWS

The platform was pinned to on-premises hardware we could not grow fast enough. I drove the migration onto AWS (after a short detour through Azure), rebuilding the collection and processing path on S3, Athena, and Kafka. Capacity went from 250k to over 50M sessions a day. Two hundred times the throughput, on a stack that got cheaper per session as it grew instead of more expensive.

I ran delivery and stayed in the Scala code the whole way. Owning the architecture and writing it are not separate jobs at that scale. The design decisions that matter most are the ones you only see from inside the code.

Real-time, with no dropped events

Session replay is unforgiving. Drop one event and the playback lies, and a customer watching a broken replay does not file a subtle bug, they churn. We designed the pipeline on Kafka and RabbitMQ so back-pressure degraded gracefully: when a downstream stage slowed, the system buffered and shed load predictably instead of dropping data on the floor. Ingest, processing, and storage scaled independently, so a spike in one did not topple the others.

What it set up

The 200x headroom is what made everything after it possible: the enterprise deals that needed the volume, and eventually the acquisition that folded this pipeline into a much larger platform. Scale you build early is leverage you spend for years.