Blueprints.
Forensic deep dives into the failure modes that cost businesses real money — the architectural blindspots a working demo hides until scale, concurrency, or a regulator finds them first.
12 blueprints
- Backend Infrastructure9 min
Charged twice, enrolled twice: the webhook nobody made idempotent
Payment providers retry webhooks until you return 2xx, and message queues deliver at least once. Without a durable dedup key, the same event runs twice — and “twice” on a payment is a second charge.
Failure mode: A webhook or queue consumer processes each delivery as if it were unique, using a check-then-insert that races and a fragile exception-string catch instead of a durable, unique idempotency key. Redeliveries and at-least-once queues cause the same event to be applied more than once.
Read blueprint - Backend Infrastructure8 min
Decoded is not verified: the JWT that trusts whatever the client wrote
Splitting a token on dots and JSON-parsing the middle segment reads the claims without checking the signature. At that point the token is just a client-supplied object — and “role: admin” is a field the client can set.
Failure mode: An identity or authorization decision is made from JWT claims that were decoded but never cryptographically verified, or from a mutable profile record the caller can influence — so a forged token or an altered role grants access the system believes it authenticated.
Read blueprint - Backend Infrastructure10 min
The tenant you set is not the tenant you get: context that leaks across the async boundary
Tenant identity held in a request-scoped variable — AsyncLocalStorage on the app side, SET LOCAL on the database side — quietly detaches from the request when work crosses an async boundary or a pooled connection.
Failure mode: Multi-tenancy is enforced by an ambient, request-scoped variable rather than an explicit parameter. When execution crosses an async boundary, a shared event loop, or a pooled database connection, the variable can carry the wrong tenant — so one tenant's request reads or writes another tenant's data.
Read blueprint - Mobile & High-Liability Systems9 min
The save that swallowed the day: read-modify-write races in local-first apps
A whole collection serialised to one key, loaded, mutated in memory, and written back. Two saves in flight and the second silently erases the first — no error, no sync log, no way to know what was lost.
Failure mode: Local persistence stores a collection as a single serialised blob and updates it by load-mutate-store. Concurrent or interrupted saves overwrite each other or tear mid-write, silently discarding data that the user was told had been saved.
Read blueprint - Backend Infrastructure8 min
Fast but wrong, forever: the NO_EXPIRY cache and the bug that never expires
A TTL makes a broken refresh slow but correct — entries expire and rebuild. NO_EXPIRY with manual invalidation makes it fast but wrong, indefinitely, with no metric moving to say so.
Failure mode: A cache is populated with no expiry and cleared only by explicit invalidation. A deploy that computes a value incorrectly writes the wrong answer into a permanent cache, where it is served as correct until something outside the system notices.
Read blueprint - Backend Infrastructure8 min
202 Accepted is not “saved”: the write your UI lied about
The endpoint queues the write and returns 202. The app shows the value the user typed. If the queue never drains, the user believes something is true that the database never recorded — and acts on it.
Failure mode: A mutation is accepted onto a queue and acknowledged with 202, while the client optimistically renders the submitted value instead of the stored one. A dropped or delayed job leaves the user's view and the database permanently disagreeing.
Read blueprint - Backend Infrastructure8 min
The webhook that fired the truth from ten minutes ago
A queued listener receives an event carrying fully-loaded Eloquent models. By the time the worker runs, the models are a snapshot of the past — and that snapshot is what your integration partners receive as current state.
Failure mode: Events dispatched to the queue carry hydrated Eloquent models instead of identifiers. The worker deserialises a stale snapshot and acts on it, so downstream systems receive data that was already superseded before the job ran.
Read blueprint - Mobile & High-Liability Systems9 min
Reading the whole file into memory — and billing the user for it
An upload that loads the entire file into RAM and pushes it over whatever network happens to be active. Two failure modes in one line of code: an out-of-memory crash and a reverse-billed data charge the user never agreed to.
Failure mode: A media upload reads the full file into memory and transmits it with no network-type check, no size guard, and no resumability — so large files trigger out-of-memory kills and metered-network uploads bleed the user's data bundle, silently and without consent.
Read blueprint - Mobile & High-Liability Systems10 min
The token you stored in plaintext is the breach you shipped
Access tokens written as plain columns in a local database, non-atomic keychain writes that tear on backgrounding, and refresh flows with no lock. Three ways mobile credential storage quietly hands over the account.
Failure mode: Authentication credentials are persisted without at-rest encryption, written non-atomically across separate keystore operations, and refreshed without concurrency control — so a lost device, a backgrounded app, or two parallel requests can expose or destroy the session.
Read blueprint - Backend Infrastructure7 min
When “=0” means “off”: config drift and the unenforced legal limit
A configurable rolling-period limit falls back to a permissive default when the value is zero or malformed. The control still appears to run — it just stops enforcing anything.
Failure mode: A compliance limit reads its window from configuration with a fallback that treats an invalid or zero value as “no filter.” A single stray environment variable disables a legally mandated control while the code path still executes and reports success.
Read blueprint - Backend Infrastructure8 min
Where the rand goes: float accumulation behind an ORM decimal cast
`'amount' => 'decimal:2'` looks like a money type. It is a display formatter. Every calculation between load and save still runs in binary floating point — and the error compounds.
Failure mode: An ORM `decimal:2` cast is applied only at serialisation. Internal arithmetic runs in IEEE-754 floats, so rounding error accumulates across many operations and the reported totals drift from the true sum.
Read blueprint - Backend Infrastructure9 min
The two transactions that lock each other: POS deadlocks and phantom inventory
A void and a sale, running at the same till, acquire the same two row locks in opposite orders. The database picks a victim, rolls it back, and your stock count silently drifts from the shelf.
Failure mode: Two concurrent transactions acquire the same row locks in opposite order. One is chosen as the deadlock victim and rolled back — but the rollback is invisible to the operator, so inventory quietly desynchronises from physical stock.
Read blueprint