Deploying daemonized queue workers safely

Prerequisites

• ▹A process supervisor (Horizon behind Supervisor, or systemd) that restarts workers automatically.
• ▹A queue backend (Redis) and a deploy pipeline you can add a restart hook to.

Procedure

1. 1

   Supervise the worker so it always comes back

   A worker that crashes must respawn without human intervention. Run it under a supervisor with autorestart, not in a bare `nohup`. The supervisor owns liveness; your job is to make the worker exit cleanly when it should.

   [program:horizon]
   process_name=%(program_name)s
   command=php /app/artisan horizon
   autostart=true
   autorestart=true
   stopwaitsecs=3600      ; let an in-flight job finish before SIGKILL
   stopsignal=TERM
   user=deploy

2. 2

   Bound each worker's lifetime to recycle before leaks bite

   PHP accumulates memory across jobs. Rather than chase every leak, recycle workers on a schedule: cap jobs, wall-time, and memory so each worker exits cleanly and the supervisor starts a fresh one. This turns slow leaks into a non-event.

   php artisan queue:work redis \
     --max-jobs=1000 \   # exit after N jobs; supervisor respawns fresh
     --max-time=3600 \    # ...or after an hour, whichever first
     --memory=256 \       # ...or if RSS crosses the limit
     --tries=3 --backoff=10

3. 3

   Restart gracefully on deploy — after new code is live

   The failure mode is a worker holding old code after release, or being killed mid-job. Signal a graceful restart only once the new code is in place: the worker finishes its current job, exits, and the supervisor respawns it on the new release. Never hard-kill a busy worker.

   # 1. release new code, run migrations
   # 2. THEN signal workers to finish current job and restart on new code
   php artisan horizon:terminate   # or: php artisan queue:restart
   # Supervisor respawns the worker, now running the new release.

4. 4

   Carry tenant context into every job

   A worker has no request, so `Auth::user()` and the resolved tenant are null. Jobs must carry the tenant explicitly and re-bind it before running — and take entity IDs, not hydrated models, so they act on current state, not a snapshot from dispatch time.

   class SendInvoice implements ShouldQueue, TenantAware
   {
       public function __construct(public int $tenantId, public int $invoiceId) {}
   
       public function handle(): void
       {
           // TenantAware middleware re-bound the tenant before we got here.
           $invoice = Invoice::findOrFail($this->invoiceId); // re-hydrated, current
           Mail::to($invoice->customer)->send(new InvoiceIssued($invoice));
       }
   }

5. 5

   Configure retries, backoff, and a dead-letter alarm

   Transient failures should retry with backoff, not immediately hammer a struggling dependency. Permanent failures must land in `failed_jobs` and page someone — a job that silently vanishes is worse than one that loudly fails.

   // app/Providers/AppServiceProvider.php
   Queue::failing(function (JobFailed $event) {
       Alert::page('job dead-lettered', [
           'job'        => $event->job->resolveName(),
           'connection' => $event->connectionName,
           'exception'  => $event->exception->getMessage(),
       ]);
   });