Keeping data consistent across many services

A look at the event-driven approach we use to keep data consistent across services, and why idempotency and clear ownership matter more than any single tool.

Engineering

Engineering

The hard part isn't sending events

Publishing an event is easy. Making sure every service ends up consistent, even when messages arrive late, twice, or out of order, is the real work.

We lean on an event-driven design where each service owns its data and reacts to events from others, rather than reaching into shared tables. That keeps the services decoupled and lets each one scale on its own.

The details that make it reliable are unglamorous: idempotent consumers so a repeated event does no harm, clear ownership so there is one source of truth for each fact, and careful handling of ordering and failure.

The real challenge

Consistency is a promise, not a default

Sending an event is easy; agreeing on the truth is hard.

In a distributed system, the same message can arrive twice, out of order, or after a service has already moved on. The hard engineering isn't the messaging, it's designing so that none of those cases corrupts your data.

We lean on idempotency and clear ownership of each piece of state, so every service can process an event confidently, knowing a replay changes nothing it shouldn't.

How we keep the truth straight

Idempotent handlers

Processing the same event twice yields the same result.

Single ownership

One service owns each fact, so there's no ambiguity to reconcile.

Safe replays

Reprocessing a stream is a routine operation, not a risk.

How we keep it consistent

The principles that matter

  1. 1

    Clear ownership

    Each fact has exactly one service that owns it, so there is never a fight over the truth.

  2. 2

    Idempotent consumers

    Processing the same event twice produces the same result, so retries and duplicates are safe.

  3. 3

    Handle failure

    We design for the message that arrives late, fails, or never comes, not just the happy path.

  4. 4

    Observe everything

    When something drifts, we can see it quickly and trace it back to the cause.

Tools serve the principles, not the other way round

The specific queue or stream matters less than the discipline around it. Get ownership, idempotency and failure handling right, and most tools will do the job well.

Deep dive

The boring choices that make realtime reliable

Most of the hard work in realtime sync is not the socket layer, it is deciding what happens when two edits collide, a client replays after a week offline, or a consumer falls behind the event stream. We settle those rules first and write them down.

Events are versioned and idempotent, so a message delivered twice changes nothing the second time. That single property removes a whole class of 3 a.m. incidents, because retries become safe instead of scary.

We also cap fan-out deliberately: presence and typing indicators can be lossy, financial state cannot. Splitting the two onto different delivery guarantees keeps the fast path fast without gambling with the numbers.

Team for AppsProduct & Engineering

We build software for teams who want their tools to fit the way they actually work — web, mobile, AI and the systems that tie them together. We write here about what we learn shipping it.

Have a project in mind?

Tell us what you're building and we'll get back to you within one business day.