Catalog My learning Blog
System Design Roadmap: 100-Day Course for Engineers · Phase 9 · Lesson 90 · 5 min read

System Design: Secure, Observable SaaS Platform

On this page
  1. The problem: design a secure, observable SaaS platform
  2. Identity: OpenID Connect and federated sign-in
  3. Multi-tenant isolation models: pool, bridge and silo
  4. Authorization and tenant isolation
  5. API and network security
  6. Data protection and per-tenant keys
  7. Observability and noisy neighbors
  8. Deployment on Kubernetes
  9. What a strong SaaS platform design includes
  10. Check yourself

The problem: design a secure, observable SaaS platform

Design a B2B project management SaaS. Customers are companies (tenants) with 10 to 10,000 users each; larger customers want to sign in with their own identity provider. Data must be isolated between tenants, encrypted, and auditable. When something breaks, engineers must find out before customers do and diagnose it within minutes. Work through identity, authorization, API security, data protection, observability and deployment.

Identity: OpenID Connect and federated sign-in

  • Users sign in with OpenID Connect through a managed identity provider (Entra ID, Okta, Auth0). Enterprise tenants connect their own IdP via OIDC or SAML federation, so their employees use company single sign-on and are removed automatically when they leave.
  • The web app uses the authorization code flow with PKCE through a backend-for-frontend, which keeps tokens server-side and gives the browser an HTTP-only cookie.
  • Access tokens carry sub, tenant_id and scopes, live for minutes, and are validated by every service, including audience checks.

Multi-tenant isolation models: pool, bridge and silo

How much tenants share is the first decision, because it shapes everything after it:

ModelDataStrengthCost
PoolShared tables with a tenant_id columnCheapest, simplest to operate and upgradeIsolation depends on every query; noisy neighbors
BridgeA schema per tenant in a shared databaseClearer separation, per-tenant backup and restoreMigrations run per schema; connection pools multiply (Day 28)
SiloA database, or a whole deployment, per tenantStrongest isolation, per-tenant scaling and data locationMost expensive; operations multiply with tenant count

Most SaaS products pool small and medium tenants and offer silos to the largest customers, who often require it contractually. Design the code so the same service can run against either.

Authorization and tenant isolation

  • Roles per tenant (owner, admin, member, guest) plus project-level permissions.
  • Every query is scoped by tenant. Enforce it in one place: a data-access layer that always adds tenant_id, plus PostgreSQL row-level security as a second line of defense, so a missing WHERE clause can't leak another tenant's data.
  • Row-level security needs to know the current tenant, usually from a setting such as app.tenant_id. With connection pooling, set it with SET LOCAL inside each transaction, so it can't leak onto the next request that reuses the connection. A plain session-level SET on a pooled connection is how one tenant's setting ends up applied to another's queries.
  • Automated tests call every endpoint as tenant A with tenant B's ids and expect 404.

API and network security

  • The gateway handles TLS, token validation, per-tenant rate limits and request size limits.
  • Services validate input, bind to DTOs, and return only intended fields.
  • Zero trust inside: mTLS between services (via a mesh), explicit service-to-service policies, Kubernetes network policies with default deny, and per-service database credentials from a vault.

Data protection and per-tenant keys

  • TLS everywhere, including to the database and cache.
  • Managed database encryption at rest; per-tenant data keys in a KMS for attachments and sensitive fields. Deleting a departing tenant's keys renders their backups unreadable too.
  • Secrets in a vault, rotated; nothing sensitive in logs.
  • An audit log of security events (logins, permission changes, exports), append-only and retained per customer contracts.

Observability and noisy neighbors

  • OpenTelemetry in every service: traces with W3C context propagation, metrics, and structured logs carrying trace_id and tenant_id.
  • Metrics and alerts on the four golden signals per service, and SLOs per user journey (sign in, load a board, save a task) with burn-rate alerts (Day 78).
  • Per-tenant views: a noisy tenant or a tenant-specific failure must be visible, so tenant id is a log and trace attribute (not a metric label, to avoid cardinality explosion).
  • Tail sampling keeps every error and slow trace.

Noisy neighbors need their own controls: per-tenant rate limits and concurrency limits on expensive operations such as exports (Day 72), per-tenant dashboards of request volume and cost, and a path to move a tenant that outgrows the pool onto its own shard or silo.

Deployment on Kubernetes

Services run as containers on a managed Kubernetes cluster across three zones, with resource requests, readiness probes, autoscaling, PodDisruptionBudgets and zone spreading. Infrastructure and policies live in code and go through review.

A secure, observable SaaS platform: OIDC identity, gateway policies, tenant-scoped services with mTLS, encrypted data with per-tenant keys, and a shared observability pipeline.
A secure, observable SaaS platform: OIDC identity, gateway policies, tenant-scoped services with mTLS, encrypted data with per-tenant keys, and a shared observability pipeline.

What a strong SaaS platform design includes

  • Federated sign-in for enterprise tenants and short-lived tokens.
  • Tenant isolation enforced in more than one layer.
  • Zero trust between services, not only at the edge.
  • Key management, including per-tenant keys, and an audit log.
  • Observability designed in: traces, metrics, logs and SLO alerts that link to each other.
  • Reasons for each choice, tied to a threat or a requirement.
  • A stated isolation model, and a plan for the tenant who outgrows it.

Check yourself

Defense in depth. Application filters only work if every query remembers them, and one forgotten WHERE tenant_id = ... in a new report leaks another customer's data. Row-level security enforces the tenant filter in the database itself, so a missed filter returns nothing instead of someone else's rows.

With thousands of tenants, a tenant_id label would multiply every metric into thousands of time series and explode Prometheus's cardinality. Logs and traces store each event or request separately, so high-cardinality attributes like tenant id are cheap there and let you filter by tenant during investigations.

The setting stays on the pooled connection, so a request that skips or fails the SET runs with the previous tenant's value. Use SET LOCAL inside each transaction.

Their IdP disables the account; federated sign-in fails, refresh tokens are revoked or expire, and short-lived access tokens stop working within minutes.