release 1.0.0: cost control, 41 tools, 29 skills, custom commands, auto-load
This commit is contained in:
1 parent
a22d8e13b1
commit
ffa9a02c26
97 files changed
+4791
-788
No files matched your search
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: incident
|
||||
description: Respond to a production incident. Use when something is down, degraded, or misbehaving in production and must be diagnosed and mitigated under time pressure.
|
||||
---
|
||||
|
||||
# Incident response
|
||||
|
||||
Mitigate first, diagnose second. Restore service, then find out why.
|
||||
|
||||
## Confirm and scope before touching anything
|
||||
|
||||
What is actually broken, for whom, since when? Check the signal, not the report: the dashboard,
|
||||
the error rate, the health endpoint. A wrong scope sends you chasing a symptom. State the impact
|
||||
plainly in one line before you start changing things.
|
||||
|
||||
## Recent change is the prime suspect
|
||||
|
||||
Most incidents follow a deploy, a config change, a flag flip, or a scaling event. What changed
|
||||
in the window before it broke? Check the deploy log and the diff. The fastest fix is usually to
|
||||
undo the last change, not to understand it.
|
||||
|
||||
## Mitigate, then understand
|
||||
|
||||
- Roll back the deploy, flip the flag off, fail over, scale up, restart the wedged process —
|
||||
whichever restores service fastest, even if you do not yet know the root cause.
|
||||
- A mitigation you can reverse beats a perfect diagnosis that takes an hour. Note what you did so
|
||||
it can be undone or made permanent later.
|
||||
- Do not deploy an unreviewed "fix" into the fire; it adds a second change to a system already
|
||||
misbehaving.
|
||||
|
||||
## Preserve evidence before it rotates away
|
||||
|
||||
Capture the logs, the error, the relevant metrics, a snapshot of the state — before a restart or
|
||||
a rollback destroys it. You will want it for the postmortem, and it may be the only copy.
|
||||
|
||||
## Communicate and follow up
|
||||
|
||||
Say what is broken, what you are doing, and when the next update is — to whoever is affected,
|
||||
in plain language, on a schedule. Afterwards: write the timeline, the root cause, and the
|
||||
follow-ups that stop it recurring. An incident with no follow-up is a loan against the next one.
|
||||
Reference in new issue
Block a user