Skip to content
Backend Infrastructure10 min

The tenant you set is not the tenant you get: context that leaks across the async boundary

Tenant identity held in a request-scoped variable — AsyncLocalStorage on the app side, SET LOCAL on the database side — quietly detaches from the request when work crosses an async boundary or a pooled connection.

Failure modeMulti-tenancy is enforced by an ambient, request-scoped variable rather than an explicit parameter. When execution crosses an async boundary, a shared event loop, or a pooled database connection, the variable can carry the wrong tenant — so one tenant's request reads or writes another tenant's data.

Commercial risk

Cross-tenant leakage in a SaaS is the failure that ends contracts. Not a slow page or a broken feature — one customer seeing, or being billed for, another customer's data. It is simultaneously a security incident, a regulatory breach, and a trust event you may not recover from.

What makes it lethal is that it passes every test. Single-tenant test suites never exercise the concurrency that triggers it; the leak needs two tenants' requests interleaving on the same process or the same connection, which is a property of production load, not of the unit test.

Ambient request context is one of the most convenient patterns in modern backends. Instead of threading a tenant ID through every function signature, you stash it once — in Node's `AsyncLocalStorage`, in a Postgres session variable via `SET LOCAL` — and every layer beneath reads it implicitly. It keeps signatures clean and the wiring invisible. That invisibility is also the failure mode: when the context detaches from the request, nothing in the code looks wrong, and the isolation you thought you had is gone.

There are two distinct places this detachment happens — one in the application runtime, one in the database connection layer — and both produce the same catastrophic outcome. Both are worth understanding precisely, because the fix is not to abandon ambient context but to bound it correctly.

1. AsyncLocalStorage across the observable boundary

`AsyncLocalStorage` propagates a value through the async call graph rooted at `run()`. It works because Node tracks async context across awaits and native promise chains. It stops working when execution hops to a context it can't track — certain RxJS scheduling, a callback fired from a shared timer, a continuation resumed by a different request's `Promise.all`. When that happens, `getStore()` inside your handler can return the tenant from whichever `run()` is currently on the stack, which under concurrency may not be yours.

The concrete shape: tenant A's request enters `tenantStorage.run(A, ...)` and, deep in the chain, publishes a message or issues a query that reads the ambient tenant. Meanwhile tenant B's request has its own `run(B, ...)` in flight on the same event loop. If B's continuation interleaves at the wrong instant, A's downstream work reads B — and emits an event, or a query filter, stamped with the wrong tenant. No error; just A's data now flowing under B's identity.

// The value is ambient — nothing downstream names the tenant it uses.
tenantStorage.run(tenantContext, () => {
  return next.handle().subscribe(/* ... */);
  // If the observable schedules a continuation outside this async context,
  // getStore() inside it may resolve to a different concurrent request's tenant.
});

// Somewhere far below, reading the ambient value:
const tenantId = tenantStorage.getStore()?.tenantId; // whose request is this, really?

2. SET LOCAL across a pooled connection

The database side has the same shape with a different mechanism. Row-level security keyed on `current_setting('app.current_tenant')` is a strong isolation primitive — but only if that setting is reliably attached to the tenant whose query is running. `SET LOCAL` scopes the value to the current transaction, which is correct, precisely as long as every query runs inside the transaction that set it.

Connection pooling is where this breaks. If a query slips outside the transaction — an autocommit path, a helper that grabs its own connection — `SET LOCAL` has no effect and `current_setting` reads whatever the pooled connection last held, or nothing. Worse, `current_setting('app.current_tenant')` returning NULL doesn't fail loudly: under RLS, `tenant_id = NULL` matches no rows, so instead of a leak you get silent emptiness that the app may misread as "this tenant has no data." And if a pooled connection is reused without resetting the setting, the next tenant inherits the previous tenant's context.

-- RLS is only as good as the setting it trusts.
CREATE POLICY tenant_isolation ON content_assets
  USING (tenant_id = current_setting('app.current_tenant')::uuid);

-- Correct ONLY if every query runs inside this transaction:
BEGIN;
SET LOCAL app.current_tenant = '...';   -- scoped to THIS transaction
SELECT * FROM content_assets;           -- filtered
COMMIT;

-- A query on a pooled connection outside the txn: SET LOCAL never applied,
-- current_setting is stale or empty, and RLS silently misbehaves.

3. Bind the context, verify it, and never trust it as input

The application fix is to keep the ambient context genuinely bounded to the request and to stop trusting anything that sets it from the outside. Establish `run()` at the true entry point and be deliberate about any library that schedules across async boundaries — where a boundary is unavoidable, capture the tenant into a local and pass it explicitly rather than re-reading the ambient store on the far side. And never derive the tenant from a client-supplied header or an unverified token: a tenant claim from a decoded-but-unsigned JWT is attacker-controlled input, not identity.

The database fix is to make the setting inseparable from the query. Acquire a connection, set the tenant, and run the work in one bounded scope, resetting on release so no pooled connection carries a tenant into the next checkout. Then add the assertion the ambient pattern lacks: a cheap guard that fails closed if `current_setting('app.current_tenant')` is missing, so a query that lost its context errors instead of silently returning the wrong rows — or no rows.

The deeper principle is that isolation you cannot see is isolation you cannot trust. Ambient context is fine for convenience, but the tenant boundary is too important to be implicit everywhere; at the points where data actually crosses it, make the tenant explicit and assert on it.

// Bind tenant + connection + work in one scope; reset on release.
async function withTenant<T>(tenantId: string, fn: (tx: Tx) => Promise<T>): Promise<T> {
  assertVerifiedTenant(tenantId);            // from a *verified* token, never a header
  const conn = await pool.acquire();
  try {
    await conn.query("BEGIN");
    await conn.query("SET LOCAL app.current_tenant = $1", [tenantId]);
    // Fail closed if the setting ever comes back empty inside the txn.
    const [{ current }] = await conn.query(
      "SELECT current_setting('app.current_tenant', true) AS current",
    );
    if (current !== tenantId) throw new TenantContextLost(tenantId);
    const out = await fn(conn);
    await conn.query("COMMIT");
    return out;
  } finally {
    await conn.query("RESET app.current_tenant"); // don't leak into the next checkout
    pool.release(conn);
  }
}

Ambient request context — `AsyncLocalStorage`, `SET LOCAL` — is clean, convenient, and exactly as trustworthy as the boundary it's confined to. When work crosses an async boundary the runtime can't track, or a query runs on a pooled connection outside the transaction that set the tenant, the context detaches and your isolation quietly inverts. Nothing errors; the wrong tenant is simply served.

Bind the context to the request and the connection deliberately, reset it on release, and assert on it at the points where data crosses the tenant boundary — failing closed when it's missing rather than serving the wrong rows. And never let the tenant be set by unverified input. The convenience of implicit context is real; just don't let the most important boundary in a SaaS be the one thing nobody can see.

Need this level of structural integrity engineered for your system?

We audit and re-architect systems where a single one of these failure modes costs real money. Fixed scope, fixed price, defined delivery date.

Request a Fixed-Scope Architecture Blueprint