The postmortem process: a practical guide for small teams

To run a postmortem on a small team, write one whenever an incident crosses a pre-set threshold, rebuild the timeline backwards from alerts, deploy logs, and Slack in UTC, find the root cause with 5 Whys, and ship corrective actions that each have one owner, a verb, and a deadline. Keep it blameless: every cause is a system failure.

The result is a written record of what happened, why it happened, and what prevents a repeat. This guide walks through trigger thresholds, timeline reconstruction, root cause analysis, and corrective actions that actually ship, all without an incident-management platform.

Key takeaways

• ▸Write a postmortem whenever an incident crosses a pre-set threshold, such as customer-visible downtime, a missed cron or heartbeat, or a repeat page with the same cause.
• ▸Reconstruct the incident timeline backwards from detection using alerts, deploy logs, and Slack, normalizing every timestamp to UTC.
• ▸Use 5 Whys plus a contributing-factors list instead of formal frameworks, and phrase every cause as a system failure, never a person failure.
• ▸Every corrective action needs one owner, a verb, and a deadline, and it is only done after the fix is verified against the failure mode.
• ▸Convert every monitoring gap a postmortem reveals into a heartbeat, threshold, or health check so the next incident is cheaper to diagnose.

What is a postmortem process?

A postmortem process is the routine you follow after an incident: reconstruct what happened, find the root cause, and ship corrective actions so the same failure cannot repeat. Unlike a medical autopsy, the patient is still alive, and the goal is prevention, not a verdict. Unlike a retro, it covers one incident with a fixed timeline, not a sprint's worth of feelings.

The Google SRE Book defines the standard: a blameless written document with an impact summary, a timeline, root causes, and action items, owned by someone who was close to the response (Postmortem Culture). For a small team, that means one doc, written within 48 hours, that names what alerted you, what you checked, and what you will change.

When should you write a postmortem? Trigger criteria for a small team

Not every page deserves a postmortem. Write one when the incident crossed a threshold you set in advance, so nobody debates it at 3am. If a page is a false positive, fix the alert and move on.

Trigger thresholds you can copy

In Faultline, most of these triggers are monitors you can set up once. A heartbeat check fails when a cron job or worker stops pinging, so the skipped nightly job opens an incident instead of waiting for a ticket. An HTTP monitor only opens a CRITICAL incident after 3 failed checks in a row, so the single noisy 5xx in the last row never pages anyone.

Why "the customer noticed" is the wrong bar

By the time a customer notices, you're already writing the postmortem under pressure. The better bar is "we learned about it from a human instead of a monitor." A late cron job with a healthy /healthz endpoint counts, even if nobody paged. Silent failures teach you the most, and they're the ones your uptime checks will never catch.

How do you run a blameless postmortem without it becoming theater?

Blameless means you analyze decisions with the information the responder had at the time, not what they know now. In practice: nobody is named as a cause, and every "why did X do that?" is rephrased as "what made that seem reasonable?" At a 5-person company this matters more, not less, because the person who caused the outage also fixes it.

The blameless framing rule

Write causes as system failures, not people failures. "Deploy went out without a health check" becomes "deploys have no gating check, so a bad image shipped to prod." If a sentence only makes sense with a name in it, rewrite it. The test: would this paragraph still be true if the person quit tomorrow?

Who runs it when there's no incident commander

The person who was on-call writes the draft, but someone else facilitates the meeting, ideally whoever has the least context on the incident. Rotate it. Cap the meeting at 45 minutes and end with owned, dated action items, because follow-through is where postmortems usually die. Google's SRE guidance on postmortem culture emphasizes the same point: a postmortem's value is its corrective actions, not the document.

How do you reconstruct the incident timeline when your monitoring missed it?

Sources to pull, in order

Work backwards from detection, not forwards from deploy. Pull in this order: the alert or heartbeat gap that started the incident, deploy logs (including cron definitions and ECS task definitions), Slack channels where on-call discussed it, and finally application logs for the window. Deploy logs come second because the most common question is "what changed?", and a diff between the last good and first bad task definition answers it faster than any log search.

If the alert came from Faultline, the first source is already written down. Every incident keeps its own timeline: when it was detected, acknowledged, and resolved, each status change, assignment, and escalation, notes from responders, and every runbook step with its result. You still pull deploy logs and Slack yourself, but you start from a record instead of memory.

Worked example: the late nightly customer-summary job

A page fires: the nightly customer-summary job is late. The uptime monitor shows a green /healthz and 14 overdue runs, which tells you the job has been failing since roughly midnight but nothing about why. You pull the cron history and see the last successful exit at 00:14, then the deploy log showing a queue consumer change shipped at 00:30. Slack confirms the first engineer noticed at 00:52 from a customer ticket, not the page. Your timeline now reads: 00:14 last good run, 00:30 deploy, 00:32 first missed heartbeat, 00:52 human detection, 01:05 page.

That 33-minute gap between first failure and first alert is the number your corrective actions should target, because no amount of log archaeology fixes a detection gap. A Faultline heartbeat on the job closes it: set the interval to the job's schedule and a grace period for slow runs, and the check fails as soon as both pass with no ping.

Timestamps, timezones, and the 14-overdue-runs trap

Normalize every timestamp to UTC before you sequence anything; an RFC 3339 timestamp with a Z suffix leaves no room for doubt. Cron logs, container logs, and Slack exports routinely disagree by an hour, and a timeline built on mixed timezones produces a root cause that survives one read-through and collapses on the second. The same discipline applies to overdue-run counts: "14 overdue runs" is a duration multiplied by an interval, so divide it back into elapsed time before you write it down. A timeline entry should state the event, the UTC timestamp, and the artifact it came from, in that order, every time.

Which root cause analysis technique fits a startup postmortem?

Skip fishbone diagrams. For a two-person team, 5 Whys plus a contributing-factors list covers 90% of incidents, and nothing formal covers the rest better than writing down what you actually know.

5 Whys done right

5 Whys works when each "why" points at a process, not a person. "The cron job failed" → "the worker OOMed" → "the queue consumer had no memory limit" → "no one reviews ECS task definitions" → "task definition changes ship without a second pair of eyes." That last answer is a fixable process gap. If your fifth why is "Dave made a typo," you stopped too early.

Contributing factors vs root cause

The single-cause trap: most outages need three things to go wrong at once. A silent failure that paged nobody for 40 minutes, plus alert fatigue that muted the channel, plus a deploy that landed Friday afternoon. Naive alerting that fires on every non-200 trains you to ignore the pager, so list every factor that had to be true, and attach a corrective action to each. One root cause, three contributing factors, four actions.

How do you write corrective actions that actually get done?

Most postmortems die in the doc. The fix: every action item gets an owner, a verb, and a deadline. "Improve monitoring" is a wish. "Add heartbeat checks to the three nightly cron jobs by Friday" is a task.

The action-item quality checklist

Before you close the meeting, run each item through four filters. It names one person, not a team. It starts with a verb: add, delete, automate, alert. It has a date attached, even if that date is "next sprint." And it is small enough to finish in under a day. If an item needs a quarter of platform work, split it or cut it.

Tracking to completion with what you already have (Slack, issues, cron)

You don't need Jira. Open one issue per action item in whatever tracker you already use, and paste the links into the postmortem doc. Then set a cron job: weekly, it posts open items to your incident channel with the owner's handle. A Slack reminder works too, but cron is harder to snooze. The point is that the list surfaces itself; nobody has to remember to look at the doc.

When to close the loop: verification, not completion

An item isn't done when the PR merges. It's done when you've verified the failure mode it addresses. If the action was "alert when the queue depth exceeds X," close it after the next incident (or a deliberate test) proves the alert fires and someone sees it. Mark items "shipped, unverified" in the meantime so the honest ones stand out.

Faultline applies the same rule to its own fixes. After a runbook runs, it checks the service again 2 minutes later and records the result on the incident timeline. If the service is still down, it pages on-call at CRITICAL instead of calling the fix done.

What should the postmortem document contain? A copy-paste template

The template

Keep it to one page with seven sections: title, severity, duration, impact (customers affected, money lost), timeline, root cause, and corrective actions with owners and due dates. The timeline should pull from real artifacts: alert timestamps, Slack threads, deploy logs, and heartbeat gaps. If your monitoring only caught the symptom late, say so. A late page on a nightly cron job with a healthy /healthz endpoint is a finding, not an embarrassment: write down what the monitor showed and when.

# <Title: what broke, in one line>

Severity:  SEV-<n>
Duration:  <first failure UTC> to <resolved UTC> (<minutes>)
Impact:    <customers affected>, <money lost>

## Timeline (UTC)
<HH:MM>  <event>  (<artifact: alert / deploy log / Slack link>)

## Root cause
<one system failure, no names>
Contributing factors:
- <factor>

## Corrective actions
- [ ] <verb> <what> (owner: @<one person>, due: <date>)

Letting Faultline write the first draft

When a Faultline incident resolves, it drafts a blameless post-mortem from the incident's own record: the timeline, the escalations, and every runbook run. The draft has Summary, Impact, Timeline, Root Cause Analysis, What Went Well, What Went Poorly, and Action Items sections. MTTA and MTTR are computed in code from the detected, acknowledged, and resolved times, so the numbers are exact, and anything the evidence does not support is marked “needs human input” instead of guessed. It never overwrites a post-mortem a person already wrote. Treat it as the draft the on-call person edits, then add the deploy logs and Slack context it cannot see.

What to leave out

Cut the action items with no owner, the "lessons learned" paragraph that restates the summary, and any blame language. Skip full log dumps; link to the Slack channel or ticket instead. If a section would not change what you do during the next page, delete it.

How do you make the next incident cheaper to diagnose?

Every postmortem finding should end as a monitoring change. If the root cause was a silent cron failure, add a heartbeat to that job with a deadline tied to its schedule. If the timeline had a 40-minute gap because nobody noticed, add a check at a 1-minute interval instead of 5. The corrective action isn't done until the check exists and pages the right Slack channel.

Turning findings into heartbeats and checks

Audit your last three postmortems and list every symptom monitoring missed. Convert each into a heartbeat, a queue-depth threshold, or a container health check. This is how MTTR drops: the next incident starts with an alert that names the failing component, not a customer ticket.

In Faultline, each of those is one monitor: a heartbeat for a cron job or queue worker, a Docker or ECS check for a container, or an HTTP, TCP, SSL, or DNS check for an endpoint. Every monitor feeds the same incident, escalation, and runbook pipeline. If a postmortem ends with a known fix, such as restarting an ECS service, save it as a runbook so the next incident offers it for one-click approval.

Where AI fits: querying incident state from Claude

During an incident, minutes go to context-gathering: which jobs are overdue, which queue is backing up, which container restarted. Exposing that state through an MCP server lets Claude answer "what changed in the last hour?" while you read the runbook. Our own tool, Faultline, does this through its MCP server: Claude can list open incidents, read a service's recent checks and timeline, see who is on call, and, only after you approve, run a runbook. Resolved incidents are remembered, so a diagnosis can point at the past incident that the same runbook fixed. The honest gap is that it won't replace your metrics store, so keep Datadog if you need deep history.

FAQ

Is a postmortem only for outages, or should near-misses get one too?

Near-misses deserve postmortems, especially silent failures. If a queue drained but messages dropped, or a cron job skipped with a healthy /healthz endpoint, that is a finding worth a document even though no customer paged. The trigger is a pre-set threshold, not customer impact, because silent failures teach the most and uptime checks will never catch them on their own.

How long after an incident should the postmortem happen?

Within 48 hours, while memory of what you checked is still accurate. Write the draft sooner if you can, because timeline details like Slack threads and deploy logs get harder to sequence as context fades. The meeting itself can follow the draft, but the document should exist before the week ends.

Do I need a tool like PagerDuty or incident.io to run a postmortem process?

No. One shared document, your existing issue tracker, and a cron job that posts open action items to your incident channel cover the whole process. The value sits in owned, dated corrective actions, not in platform features. Add tooling only when the manual routine breaks, not before.

Who owns the postmortem if there's no SRE team?

The person who was on-call writes the draft, and someone with the least context on the incident facilitates the meeting. Rotate that facilitator role. Cap the meeting at 45 minutes and leave with action items that each have one owner and a date, because follow-through is where small-team postmortems usually die.

How do you measure whether a postmortem process is working?

Track three things: the gap between first failure and first alert, how many action items ship by their deadline, and whether repeat incidents with the same cause recur. Then audit your last three postmortems and convert every symptom monitoring missed into a heartbeat or check. If detection gaps shrink, the process works.