Context#
Mid-2025: a 12-engineer B2B fintech serving Polish and DACH SMBs. Compliance-heavy product (banking license partner, dual-control deploys, pre-deploy QA gates). Lead Time P50 of 14 days. Founder-CTO felt the team had “lost its early-stage shipping speed” but couldn’t articulate why or where.
We were brought in for a 26-week DORA performance engagement.
Starting state#
Pre-engagement DORA baselines (90-day rolling):
| Metric | Starting value | SMB P50 (our benchmark) |
|---|---|---|
| Deployment Frequency | 1×/week | 3×/week |
| Lead Time for Changes (P50) | 14 days | 2.5 days |
| Change Failure Rate | 35% | 15% |
| MTTR | 18 hours | 3 hours |
Every metric was 2-5× worse than the SMB P50. The team’s engineering effort was consistent with a high-performance team — output was not.
Diagnosis (Phase 1, weeks 1-4)#
We ran the full DORA implementation playbook plus stakeholder interviews. Three structural drags emerged:
- PR review wait time was the dominant Lead Time component. Median wait from PR-open to first review: 22 hours. P85: 4 days. The team had no async review SLA.
- CI was 35 minutes wall-clock. Dominated by sequential test execution. CI was effectively the second-largest Lead Time component.
- CFR was driven by database migrations. 60% of failed changes traced back to migration issues. The team used single-step destructive migrations as default pattern.
Interventions (Phase 2, weeks 5-20)#
Three tactics, sequenced for compounding effect:
Tactic 1 — Async review SLA + #review-please channel (week 5)#
Rolled out an explicit SLA: PRs older than 4 working hours get pinged in #review-please. PRs older than 24 working hours escalate to the EM. Reviewer rotation moved from “whoever’s available” to a predictable weekly schedule with named primary and backup.
Effect at week 8: median PR review wait dropped from 22h to 6h. Lead Time P50 dropped from 14 days to 8 days as a knock-on effect.
Tactic 2 — CI parallelization (weeks 9-12)#
Refactored the test suite to support parallel execution. Migrated CI from sequential to 8-way parallel. Set up test sharding by historical execution time.
Effect at week 14: CI median dropped from 35 minutes to 7 minutes. Lead Time P50 dropped to 5 days.
Tactic 3 — Expand-contract migration pattern (weeks 13-20)#
Rewrote the team’s migration playbook around the expand-contract pattern. All new migrations followed: add new column → backfill → migrate reads → migrate writes → drop old column. Each step independently rollbackable.
This was the slowest intervention to ship — 7 weeks of paired work with the team’s senior engineers — but had the largest CFR effect.
Effect at week 22: CFR dropped from 35% to 14%. MTTR dropped from 18h to 3h (smaller blast radius per failed change).
Final outcomes (week 26)#
| Metric | Starting | Final | Change |
|---|---|---|---|
| Deployment Frequency | 1×/week | 3×/week | +200% |
| Lead Time for Changes (P50) | 14 days | 3 days | -78% |
| Change Failure Rate | 35% | 12% | -23pp |
| MTTR | 18 hours | 2 hours | -89% |
All four metrics within or better than the SMB P50 from our benchmarks dataset.
What didn’t work#
Two interventions we tried that produced no measurable improvement:
- A pre-deploy “release captain” role. Tried in week 8. Created decision-bottleneck on one engineer. Abandoned by week 11 — replaced with the async review SLA.
- Mandatory PR templates. Added in week 6, removed in week 14. Compliance was high but quality of PR descriptions did not improve. The team was already writing good PR descriptions; the template added overhead without adding signal.
We’re publishing both because case studies that only report wins lie about the work.
Costs#
| Cost item | Amount |
|---|---|
| Pixel of Software fee (26 weeks, time + materials with success criteria) | Mid-five-figure EUR (within typical range for our engagements) |
| Internal engineering time | ~22 engineer-weeks across the team |
| Tooling cost increase | €0 (no new tools) |
| Training spend | €0 |
The team’s CFO confirmed payback within 5 months on engineering output alone, before accounting for reduced incident-response load and faster customer issue resolution.
What the team says now#
(Quote from the founder-CTO, used with permission and reproduced verbatim from the consent letter.)
“The intervention I was most skeptical of — the migration pattern rewrite — turned out to be the highest-impact change. It also had nothing to do with what I thought the problem was. The diagnostic phase was worth the entire engagement on its own.”
Where they are now (12 months later)#
The team is currently 17 engineers. DORA metrics have remained stable since handover. The async review SLA has held; the migration pattern is now policy in their engineering handbook. They have not retained external DORA support since the engagement ended — by design.
Engagement type#
This was a paid engagement. The team’s pro bono criteria did not apply (the company is VC-funded and revenue-positive). For organizations qualifying under our pro bono program, the same playbook is available without cost: Pro Bono dla Polskiego Biznesu.
Related Reading#
- DORA Metrics for Startups service page — the productized version of this engagement.
- How to Reduce Change Failure Rate Without Slowing Delivery — the framework that drove Tactic 3.
- Live DORA dashboard — our own engineering team’s current DORA metrics, published as a credibility artifact.