Why My Background Job Processor Deadlocked Under Real Load (And How CancellationToken Saved It)

A recurring inventory-sync job ran fine in testing but froze in production when multiple instances hit the same EF Core transaction. Tracing the deadlock revealed that two separate background services were updating overlapping rows without consistent lock ordering — classic deadlock territory. The fix wasn't fancy retry logic but passing a shared CancellationToken that let the second job abort cleanly instead of waiting on a lock, combined with idempotent upserts so aborted jobs could safely resume. The lesson: background jobs aren't fire-and-forget; they need cooperative cancellation, deterministic ordering, and idempotency to survive real concurrency.