← All guides
Building a Distributed Workflow Engine
Inspired by Conductor
Build a distributed workflow engine from scratch — a pure state machine, PostgreSQL as the source of truth, a transactional outbox, sharded Redis Streams, task leases and self-healing recovery.
11 parts · about 3.5 hours of reading · full source on GitHub
Start reading →What you'll build
- A workflow model with HTTP, code, switch, fork/join and wait nodes, and CEL templates
- A pure, unit-tested state machine that turns events into actions
- A coordinator that locks, decides, persists and commits one event per transaction
- A transactional outbox woken by LISTEN/NOTIFY, publishing to sharded Redis Streams
- Workers with database-backed leases, renewal and fencing by attempt number
- Recovery loops for dead workers, lost messages, timers and partition retention