Catalog My learning Blog
Full-Stack Developer Course: React, .NET and SQL Server · Phase 10 · Lesson 93 · 10 min read

Design a Real-Time Chat and Notification System

On this page
  1. Learning Objective
  2. Concept Explanation
  3. Worked Example: Schema, an Idempotent Send with Atomic Sequencing, and Reconnect Gap Filling
  4. Real-World Examples
  5. Review Questions & Answers

Learning Objective

Design a chat system with conversations, presence, history, offline notifications and delivery guarantees, using SignalR with a managed backplane, per-conversation sequence numbers, idempotent sends, keyset-paged history and safe handling of untrusted message content and links.

Concept Explanation

Chat combines several hard problems. Messages must be delivered in order, exactly once from the user's perspective, in real time, to people whose connections drop and reconnect constantly, and stored durably so history is available on any device. Requirements: one-to-one and group conversations, presence, unread counts, history with infinite scroll, delivery to offline users through push notifications, and moderation. Non-functional: message latency under a second, no lost or duplicated messages, availability during deploys, and privacy, since these are private conversations. Assume 500,000 daily users, 100,000 concurrent connections at peak, and 50 messages per active user per day, giving 25 million messages a day, about 300 per second average and 3,000 at peak. That volume is modest for a database, so the difficulty lies in connection management, ordering, and reliability.

The real-time layer holds long-lived connections, and a single server can only hold so many. With several servers, a message sent by a user connected to server A must reach a recipient connected to server B, which is the backplane problem from Day 87. Azure SignalR Service solves it by holding the client connections itself, in units of about 1,000 concurrent connections, so your servers only handle the hub logic and can scale and deploy independently; 100,000 connections means roughly 100 units at the standard tier or the corresponding premium sizing. WhatsApp engineer Rick Reed described in 2012 running around two million concurrent connections on a single Erlang server, a reminder that connection handling can be extremely efficient in the right runtime, and that the numbers here are a design choice, not a limit.

Reliability comes from the message pipeline. The client generates a message identifier, sends it, and the server persists the message before broadcasting. The identifier makes the send idempotent: if the client retries after a dropped acknowledgment, the unique constraint on the conversation and identifier returns the original message instead of creating a duplicate. Ordering is achieved with a per-conversation sequence number assigned atomically in the database, using a single UPDATE that increments and returns the counter, so every message in a conversation has a strictly increasing number that clients use to order and to detect gaps. On reconnect, the client asks for everything after the last sequence number it has, so a lost real-time message is repaired from the database, and the real-time channel becomes an optimization on top of a reliable history API.

Presence and notifications are separate concerns. Presence, whether a user is online, is ephemeral and high-churn, so keep it in Redis with a short time to live refreshed by heartbeats, not in SQL. For users who are offline, the message pipeline publishes a notification event through the outbox to Service Bus, and a worker sends web push or mobile push through the platform services, batching and respecting user preferences. Storage in SQL Server is sufficient at this scale: a Messages table clustered on conversation and sequence gives efficient keyset pagination for history and can later be partitioned by conversation.

Security shapes chat more than most systems. Every hub method must verify the caller is a participant in the conversation, as in Day 59, including the history API, since guessing a conversation identifier must reveal nothing. Message content is untrusted input rendered to other users, so it must be escaped, as Day 58 covered, with any rich text sanitized. Link previews require the server to fetch a URL a user supplied, which is server-side request forgery territory and needs the safe fetcher of Day 50. Enforce message size limits and per-user rate limits to prevent flooding and spam, and consider abuse reporting and moderation tooling. End-to-end encryption, where only participants hold the keys, gives the strongest privacy but prevents server-side search, moderation, and multi-device history unless key management is added, so state the choice and its consequences explicitly. Define retention and deletion, including account deletion requests, from the start.

ConcernDesignReason
Connections at scaleAzure SignalR Service, hubs on stateless serversServers scale and deploy without dropping clients
OrderingPer-conversation sequence number, assigned atomicallyTotal order, gap detection
DuplicatesClient message id with a unique constraintIdempotent retries
ReliabilityPersist first, then broadcast; gap fill from history on reconnectReal time is an optimization on a durable store
PresenceRedis keys with TTL and heartbeatsEphemeral, high churn
Offline deliveryOutbox to Service Bus to push workersReliable, decoupled from the request
SecurityParticipant checks everywhere, escaped content, safe link previews, rate limitsPrivate data and untrusted content

Worked Example: Schema, an Idempotent Send with Atomic Sequencing, and Reconnect Gap Filling

The listings show the tables, the hub method, the history endpoint, and the React hook that repairs gaps.

SQL
CREATE TABLE dbo.Conversations (
    Id UNIQUEIDENTIFIER NOT NULL CONSTRAINT PK_Conv PRIMARY KEY,
    LastSeq BIGINT NOT NULL CONSTRAINT DF_Conv_Seq DEFAULT 0            -- incremented atomically per message
);
CREATE TABLE dbo.Participants (
    ConversationId UNIQUEIDENTIFIER NOT NULL, UserId UNIQUEIDENTIFIER NOT NULL, LastReadSeq BIGINT NOT NULL DEFAULT 0,
    CONSTRAINT PK_Participants PRIMARY KEY (ConversationId, UserId)
);
CREATE INDEX IX_Participants_User ON dbo.Participants (UserId) INCLUDE (ConversationId, LastReadSeq);

CREATE TABLE dbo.Messages (
    ConversationId UNIQUEIDENTIFIER NOT NULL,
    Seq            BIGINT           NOT NULL,
    ClientMessageId UNIQUEIDENTIFIER NOT NULL,
    SenderId       UNIQUEIDENTIFIER NOT NULL,
    Body           NVARCHAR(4000)   NOT NULL,
    SentAt         DATETIMEOFFSET   NOT NULL CONSTRAINT DF_Msg_Sent DEFAULT SYSUTCDATETIME(),
    CONSTRAINT PK_Messages PRIMARY KEY (ConversationId, Seq),                       -- clustered: history is a range scan
    CONSTRAINT UQ_Messages_Client UNIQUE (ConversationId, ClientMessageId)          -- idempotent sends
);
C#
[Authorize]
public sealed class ChatHub(AppDb db, ChatNotifier notifier, ILogger<ChatHub> log) : Hub
{
    public async Task<SendResult> Send(Guid conversationId, Guid clientMessageId, string body)
    {
        var me = Context.User!.GetUserId();
        body = body?.Trim() ?? "";
        if (body.Length is 0 or > 4000) throw new HubException("Invalid message.");         // size limit at the boundary

        // Authorization: participants only. Same error for missing and forbidden conversations.
        if (!await db.Participants.AnyAsync(p => p.ConversationId == conversationId && p.UserId == me))
            throw new HubException("Not found");

        // Idempotency: a retried send returns the original message instead of creating a duplicate.
        var existing = await db.Messages.AsNoTracking()
            .Where(m => m.ConversationId == conversationId && m.ClientMessageId == clientMessageId)
            .Select(m => (long?)m.Seq).FirstOrDefaultAsync();
        if (existing is { } seq) return new SendResult(seq, Duplicate: true);

        await using var tx = await db.Database.BeginTransactionAsync();
        // Atomic sequence assignment: one statement increments and returns the counter, so numbers never repeat or skip in a conversation.
        var next = (await db.Database.SqlQuery<long>(
            $"UPDATE dbo.Conversations SET LastSeq = LastSeq + 1 OUTPUT inserted.LastSeq AS [Value] WHERE Id = {conversationId}")
            .ToListAsync()).Single();                                   // ToListAsync: UPDATE ... OUTPUT cannot be wrapped in a subquery

        db.Messages.Add(new Message { ConversationId = conversationId, Seq = next, ClientMessageId = clientMessageId, SenderId = me, Body = body });
        db.Outbox.Add(OutboxMessage.For("chat.message.v1", new { conversationId, seq = next }));    // offline push happens via the outbox
        try { await db.SaveChangesAsync(); await tx.CommitAsync(); }
        catch (DbUpdateException) { return new SendResult(await LookupSeq(conversationId, clientMessageId), Duplicate: true); }   // lost a duplicate race

        // Persist first, then broadcast. Recipients receive the payload; clients still verify sequence continuity.
        await Clients.Group($"conv:{conversationId}").SendAsync("message", new MessageDto(conversationId, next, me, body, DateTimeOffset.UtcNow));
        return new SendResult(next, Duplicate: false);
    }

    public async Task Join(Guid conversationId)
    {
        var me = Context.User!.GetUserId();
        if (!await db.Participants.AnyAsync(p => p.ConversationId == conversationId && p.UserId == me)) throw new HubException("Not found");
        await Groups.AddToGroupAsync(Context.ConnectionId, $"conv:{conversationId}");
    }
}

// History and gap filling use the same authorized HTTP endpoint with keyset semantics on Seq.
app.MapGet("/api/conversations/{id:guid}/messages", async (Guid id, long? after, long? before, int? limit, AppDb db, ClaimsPrincipal user, CancellationToken ct) =>
{
    var me = user.GetUserId();
    if (!await db.Participants.AnyAsync(p => p.ConversationId == id && p.UserId == me, ct)) return Results.NotFound();
    var take = Math.Clamp(limit ?? 50, 1, 100);

    var q = db.Messages.AsNoTracking().Where(m => m.ConversationId == id);
    q = after is { } a ? q.Where(m => m.Seq > a).OrderBy(m => m.Seq)                       // gap fill: everything after what I have
                       : before is { } b ? q.Where(m => m.Seq < b).OrderByDescending(m => m.Seq)   // scroll back: older history
                       : q.OrderByDescending(m => m.Seq);
    return Results.Ok(await q.Take(take).Select(m => new MessageDto(m.ConversationId, m.Seq, m.SenderId, m.Body, m.SentAt)).ToListAsync(ct));
}).RequireAuthorization();
Shell
// React: connect, join, and repair gaps. Real-time is the fast path; the HTTP history API is the source of truth.
export function useConversation(conversationId: string) {
  const qc = useQueryClient();
  const lastSeq = useRef(0);

  useEffect(() => {
    const conn = new HubConnectionBuilder().withUrl('/hubs/chat').withAutomaticReconnect([0, 1000, 3000, 10000]).build();

    const merge = (msgs: Message[]) => qc.setQueryData<Message[]>(['messages', conversationId], (old = []) => {
      const bySeq = new Map(old.map((m) => [m.seq, m])); msgs.forEach((m) => bySeq.set(m.seq, m));   // dedupe by sequence
      const merged = [...bySeq.values()].sort((a, b) => a.seq - b.seq);
      lastSeq.current = merged.at(-1)?.seq ?? lastSeq.current; return merged;
    });

    const fillGap = async () => merge(await request(`/api/conversations/${conversationId}/messages?after=${lastSeq.current}&limit=100`, MessagesSchema));

    conn.on('message', (m: Message) => {
      if (m.seq > lastSeq.current + 1) void fillGap();                                  // a gap: we missed something, repair it
      else merge([m]);
    });
    conn.onreconnected(async () => { await conn.invoke('Join', conversationId); await fillGap(); });
    conn.start().then(() => conn.invoke('Join', conversationId)).then(fillGap);
    return () => void conn.stop();
  }, [conversationId, qc]);
}

// Send with a client-generated id so retries are safe, and render message text as text, never as HTML.
const send = (body: string) => conn.invoke('Send', conversationId, crypto.randomUUID(), body);

Link previews deserve one more note in the design: fetch them in a background worker with the validated, allowlisted fetcher, cap the response size and time, strip the result to a title and image URL, and store it, so the message send path never waits on or trusts an external site.

A send is authorized against conversation membership and deduplicated by a client message id. One transaction increments the conversation sequence number, stores the message and writes an outbox row, and only then is the message broadcast. A reconnecting client repairs gaps from the history API.
A send is authorized against conversation membership and deduplicated by a client message id. One transaction increments the conversation sequence number, stores the message and writes an outbox row, and only then is the message broadcast. A reconnecting client repairs gaps from the history API.

Real-World Examples

  • Rick Reed, an engineer at WhatsApp, described in a 2012 talk how a single Erlang server handled around two million concurrent TCP connections, using the language's lightweight processes and careful tuning of the operating system. It became a widely cited example of how much connection capacity is possible, and of why chat systems are often built around runtimes designed for concurrency.
  • Slack's engineering blog has described its real-time messaging architecture, in which clients keep WebSocket connections to edge servers and use a lightweight cache called Flannel to send only the data a client needs, replacing a design that sent the whole workspace state on connect. The redesign reduced load and startup time for large workspaces and illustrates that connection setup and state synchronization, not just message delivery, become dominant costs at scale.
Key TakeawaysTreat a chat app's real-time channel as a fast path over a durable store, so a dropped connection never loses a message. In practice, persist messages with atomic per-conversation sequence numbers and idempotent client identifiers, broadcast afterwards, and let clients fill gaps from a history API on reconnect. Use a managed connection service for scale, keep presence in Redis, deliver offline notifications through the outbox, and enforce participant checks, safe rendering, safe link previews, size limits, and rate limits on every path.

Review Questions & Answers

Timestamps from different servers and clients can be equal or out of order, so they do not give a reliable total order. An atomic increment in the database yields a strictly increasing number for every message in a conversation, which clients use to order messages and detect missing ones by looking for gaps.

The client generates a message identifier and a unique constraint on the conversation and identifier prevents duplicates. If the acknowledgment is lost and the client retries, the server finds the existing message and returns it instead of creating another, so the user sees exactly one message.

Presence changes constantly with connections and heartbeats, and it is meaningless after a short time, so it needs very fast writes and automatic expiry. Redis keys with a time to live provide both without the write load and cleanup cost that SQL would incur for such ephemeral data.

Generating a preview makes your server fetch a URL supplied by a user, which is server-side request forgery, so the fetch needs allowlisted schemes, validation of resolved addresses, no redirects, tight time and size limits, and preferably isolation in a background worker. The result should be reduced to safe fields and rendered as text and escaped attributes.