TL;DR
Blameless Postmortems: Building a Learning Organization for Platform Teams without the fluff: focus on outcomes, measure them, and stop pretending slides are progress.
“If the lesson from an incident is ‘be more careful,’ you did not learn anything useful.”
Principles of Blamelessness
- Focus on systems, not individuals
- Seek context: how the system made the action seem reasonable
- Actionable outcomes: guardrails, automation, and design changes
Anatomy of a Useful Postmortem
- Summary, impact, timeline, detection, response
- Contributing factors (technical + organizational)
- What worked, what didn’t
- Action items with owners and due dates
Tie to SLOs and Error Budgets
- Link incidents to SLO breaches and budget burn
- Prioritize reliability work based on error budget policy
Platform Engineering Lens
- Capture gaps in golden paths and guardrails
- Add policies to prevent recurrence (admission, CI checks)
- Improve runbooks, dashboards, and on-call ergonomics
Process and Cadence
- Postmortems for Sev 1–2; light versions for Sev 3–4
- Weekly triage of action items; monthly leadership review
- Track closure rate, recurrence, and toil reduction
Anti‑Patterns
- Human error as a root cause; it’s a starting point, not the end
- Action items without owners or verification
- Treating postmortems as punishment—participation will vanish
Conclusion
Learning organizations turn incidents into durable improvements. Make postmortems safe, actionable, and tied to platform guardrails.
Long‑Form Addendum: Making Blameless Postmortems: Building a Learning Organization for Platform Teams Feel “Obvious” to Teams
“Good platforms remove decisions; they don’t add dashboards.”
1) Start With a Promise
Write a one-paragraph promise to engineers: what the platform guarantees (templates, security defaults, observability, rollbacks) and what it requires (standards, ownership, on-call discipline). Publish SLOs for the platform itself so teams can trust it.
2) The Golden Path Mechanics
- Scaffold: a template creates repo, pipeline, service skeleton, and dashboards.
- Guardrails: policies are enforced in CI/CD and at deploy time.
- Self-service: teams can create a service, namespace, secrets, and alerts without opening a ticket.
- Upgrades: templates carry upgrades forward; deprecations are announced with timelines.
3) What to Measure (Outcomes)
- Time to first deploy for a new service
- Lead time for change and change failure rate
- Platform ticket volume per team and time-to-close
- % of services on the default path and number of exceptions
- OKR progress tied to business outcomes (availability, onboarding speed)
4) Common Failure Modes
- Too many options: teams choose differently and you can’t support it. Fix with fewer, better defaults.
- Optional policies: “recommended” guardrails become ignored. Fix by enforcing with clear remediation.
- No ownership: nobody owns migrations or breaking changes. Fix with product ownership and a deprecation policy.
5) Checklist
- One default template exists per service type (API, worker, UI).
- Exceptions are time-bound and visible.
- Platform publishes SLOs and a changelog.
- Monthly review: adoption, tickets, and incident learnings feed back into templates.
Glossary (Tooltips)
- IDP: Where templates, guardrails, and paved roads live.
- DX: What improves when you remove toil and ambiguity.
- CI/CD: The path from commit to production.
- SLO: How you keep the platform honest.
- OKR: Helps align platform work to outcomes.
Appendix 1: Making the Standard Path the Fast Path
A Practical Product Loop
Platforms win by compounding: every week you make the default path easier and the exception path more expensive.
- Pick 1–2 templates as “blessed defaults.”
- Add guardrails in CI/CD and at deploy time, with actionable errors.
- Publish a changelog and deprecation policy so teams trust upgrades.
- Review adoption and exceptions monthly; prioritize work that reduces repeated tickets.
30/60/90 (Adoption and Reliability)
- 30 days: define platform SLOs; ship one golden path with docs + runbook; measure baseline tickets and lead time.
- 60 days: add self-service for the top ticket categories; introduce an exception registry (owner + expiry).
- 90 days: enforce defaults for new services; migrate the highest-risk legacy patterns; tie outcomes to OKRs.
Checklist
- One obvious default path exists (and it works reliably).
- Exceptions are time-bound and visible.
- Feedback loops exist: office hours, a roadmap, and real metrics.
Appendix 2: Making the Standard Path the Fast Path
A Practical Product Loop
Platforms win by compounding: every week you make the default path easier and the exception path more expensive.
- Pick 1–2 templates as “blessed defaults.”
- Add guardrails in CI/CD and at deploy time, with actionable errors.
- Publish a changelog and deprecation policy so teams trust upgrades.
- Review adoption and exceptions monthly; prioritize work that reduces repeated tickets.
30/60/90 (Adoption and Reliability)
- 30 days: define platform SLOs; ship one golden path with docs + runbook; measure baseline tickets and lead time.
- 60 days: add self-service for the top ticket categories; introduce an exception registry (owner + expiry).
- 90 days: enforce defaults for new services; migrate the highest-risk legacy patterns; tie outcomes to OKRs.
Checklist
- One obvious default path exists (and it works reliably).
- Exceptions are time-bound and visible.
- Feedback loops exist: office hours, a roadmap, and real metrics.
Appendix 3: Making the Standard Path the Fast Path
A Practical Product Loop
Platforms win by compounding: every week you make the default path easier and the exception path more expensive.
- Pick 1–2 templates as “blessed defaults.”
- Add guardrails in CI/CD and at deploy time, with actionable errors.
- Publish a changelog and deprecation policy so teams trust upgrades.
- Review adoption and exceptions monthly; prioritize work that reduces repeated tickets.
30/60/90 (Adoption and Reliability)
- 30 days: define platform SLOs; ship one golden path with docs + runbook; measure baseline tickets and lead time.
- 60 days: add self-service for the top ticket categories; introduce an exception registry (owner + expiry).
- 90 days: enforce defaults for new services; migrate the highest-risk legacy patterns; tie outcomes to OKRs.
Checklist
- One obvious default path exists (and it works reliably).
- Exceptions are time-bound and visible.
- Feedback loops exist: office hours, a roadmap, and real metrics.