Welcome to Beyond CRUDs
Most engineers know how distributed systems work. Few understand why they work that way — the trade-offs, the failure modes, the decisions behind the design.
This site is for engineers who want to go deep: distributed systems, databases, consensus, replication, and storage engines — not just as concepts, but as things you build and break yourself.
Guide
Building a Distributed Log in Go — a full guide to building a Kafka-style replicated log from scratch. 12 chapters, every layer from raw bytes to a running multi-node cluster.
- Byte-level record encoding, append-only segments, and a memory-mapped sparse offset index
- Binary wire protocol with length-prefixed frames and a typed codec
- TCP transport and RPC server with typed message routing
- Cluster membership and failure detection with HashiCorp Serf (gossip protocol)
- Distributed consensus and metadata with HashiCorp Raft — coordinator, FSM, and log-backed storage
- Pull-based replication, ISR tracking, and high-watermark advancement
- Topic management with leader assignment and replica coordination
- Producer and consumer client libraries with automatic leader discovery and ack modes
Every design decision is motivated by a real production concern. By the end you understand not just how to build it — but why Kafka is designed the way it is.
Blog
Deep dives on distributed systems internals — how production systems like Kafka actually work under the hood.