Reliability

When You Can’t Even Raise the Ticket

On Wednesday 16 September, from around 07:50 UTC, Salesforce had an outage across multiple regions that lasted more than seven and a half hours. Sales, service and marketing teams around the world lost access to the platform that runs their day. Downstream products built on Salesforce were affected too.

One detail made this outage memorable. Customers couldn’t log a support case, because the support process itself ran on the affected platform.

The Circular Dependency

This is one of my favourite failure patterns, in the sense that a cardiologist might have a favourite heart condition. The system you’d use to report the problem, coordinate the response or tell customers what’s happening depends on the thing that’s broken. Some examples I’ve seen over the years:

  • Status pages hosted on the same infrastructure as the service. Google’s status page had trouble during its own June 2025 outage.
  • Incident chat channels in a SaaS tool that’s down.
  • Runbooks stored on the file share that’s encrypted by ransomware.
  • On-call paging that relies on the email system that’s failing.
  • Bridge-call dial-ins on the phone network that’s out. Optus and Verizon both apply here.

Graceful Degradation

The organisations that did best on the 16th were the ones whose integrations degraded gracefully. Their forms kept accepting input, queued it, and synced once the platform came back, rather than simply throwing errors at customers.

A few questions worth answering before your next SaaS outage:

  1. Where do we talk when our primary tools are down? Write it down, and make sure people can find it offline.
  2. Can our customer-facing systems queue work while a back-end SaaS platform is unavailable?
  3. What’s the vendor’s out-of-band support route, a phone number or a secondary portal, and do we have it to hand?
  4. Do we know our own recovery order for when the platform comes back and everyone hits it at once?

More Information