Design a chat system

Real-time chat surfaces every hard distributed-systems topic: stateful connections, presence, ordering, delivery guarantees, mobile push, fan-out. It's the perfect second system-design exercise after the URL shortener.

The big idea

You're solving four sub-problems at once:

High-level architecture

┌──────────┐     ┌─────────┐     ┌────────────┐     ┌──────────┐
│ Client   │ ⇄ WS │ Edge    │ ⇄  │ Chat svc   │ ⇄  │ Storage  │
│ apps     │     │ servers │ pub-sub  (stateless) │     │ (DB+blob)│
└──────────┘     └─────────┘     └────────────┘     └──────────┘
                       ▲
                       │ push notifications
                       ▼
                 ┌───────────────┐
                 │ APNs / FCM    │
                 └───────────────┘

Connections: WebSocket or SSE

Long-lived bidirectional streams are how messages reach clients in milliseconds.

Chat = WebSocket. Each connected device opens one. The connection is stateful — a server that holds the connection must know which channels the user is subscribed to.

Sharding the connection layer

Millions of concurrent WebSockets don't fit on one box. Shard the connection layer:

1. 1

   Route by user ID

   User 42's connection lives on shard hash(42) % N. The same user on two devices can be on the same shard (easy fan-out) or different (better load balance).

2. 2

   Track which shard hosts which user

   A "presence service" (Redis) stores user_id → shard_id.

3. 3

   Use a message bus between shards

   A message to a room with users on three shards is published once; each shard receives it from the bus and pushes to its local connections.

Message flow — one-to-one

The acknowledgement lets Alice's UI mark the message as sent. The server-assigned sequence number makes ordering and dedupe possible.

Group chats: the fan-out problem

A message to a 200-member channel must reach all 200. The trade-off:

Slack, Discord etc. use a hybrid: write fan-out into recipient timelines for small channels, read fan-out (with caching) for huge ones.

Storage

CREATE TABLE channels (
  id           BIGSERIAL PRIMARY KEY,
  kind         TEXT NOT NULL,                  -- 'dm' | 'group'
  created_at   TIMESTAMPTZ NOT NULL DEFAULT now()
);
 
CREATE TABLE channel_members (
  channel_id   BIGINT NOT NULL REFERENCES channels(id),
  user_id      BIGINT NOT NULL,
  joined_at    TIMESTAMPTZ NOT NULL DEFAULT now(),
  last_read_seq BIGINT NOT NULL DEFAULT 0,
  PRIMARY KEY (channel_id, user_id)
);
 
CREATE TABLE messages (
  channel_id   BIGINT NOT NULL,
  seq          BIGINT NOT NULL,                -- monotonic per channel
  sender_id    BIGINT NOT NULL,
  text         TEXT,
  attachments  JSONB,
  created_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
  PRIMARY KEY (channel_id, seq)
);

last_read_seq per member powers unread badges with one row read per channel.

Presence

"Is Bob online?" sounds simple, isn't.

Naive

Heartbeat every 10s to a Redis key with TTL=30s. Works for thousands of users.

At scale

Connection layer publishes "user X connected/disconnected" events; an aggregator de-dupes across devices. Final state goes in Redis or a presence-specific store.

Privacy

Many users want to hide presence. Make it a setting and respect it.

Delivery guarantees

Three signals you want to send the user:

• Sent — the server acked.
• Delivered — the recipient's client received it (it sends an ack back).
• Read — the recipient's UI displayed it (a separate event).

The mechanics are similar — both delivered and read send a small ack(seq) event back through the system, persisted alongside the message.

Offline + push

When a recipient is offline, the WebSocket can't deliver. Two paths:

1. 1

   Queue the message in their mailbox

   Wait for them to reconnect and replay.

2. 2

   Send a push notification

   Via APNs (iOS) or FCM (Android). The push wakes the app, which then connects and replays from last_read_seq + 1.

In practice

You'll never get this completely right on the first try. Build the simple version (single shard, one DB, in-process pub/sub) first; it'll handle thousands of concurrent users. The complexity above is for after you can prove you need it.