An incident runbook should help a team limit impact, communicate and restore a reliable state. Write it for an operator who did not build the affected component.
Start with the customer symptom
In a fictional scheduling tool, customers can read shifts but cannot save changes. State that scope instead of declaring everything unavailable. Check the start time, affected accounts and recent changes before several people apply competing fixes.
| Stage | Ownership and evidence |
|---|---|
| Assess | One person states confirmed impact |
| Stabilise | A reversible action is selected and recorded |
| Communicate | Updates separate facts from unknowns |
| Restore | The customer workflow is exercised |
| Close | Pending work and data are reconciled |
Prepare rollback limits
Identify the prior version and its dependencies. Data migrations may prevent a simple code rollback. Consequential irreversible actions still need the appropriate decision owner; urgency does not remove their effects. Keep a timeline so later investigation can distinguish cause from recovery attempts.
When is the incident over?
When service is restored and outstanding consequences are understood. A green availability chart does not prove queued operations have completed. Record remaining recovery work and review the conditions that allowed the incident. Prefer improvements to detection, isolation and recovery over a search for an individual to blame.
An incident spanning a shift change
Write a handover update containing confirmed impact, actions already attempted, current hypothesis and the next decision. Avoid passing only a chat history. The incoming responder should know which changes are active and which recovery steps must not be repeated.
From this guide to your project
Development ↗
A SaaS is a system: data, users, permissions, billing and operations. Its architecture should support the business model.
Tell us about your ideaExamples are educational and do not describe client results. Content prepared with AI assistance; see our editorial policy. Our method ↗