Blog
Deep dives on the concepts and internals behind distributed systems — Kafka, storage engines, consensus, and more.
Database Storage Engines — Physical Layout and Logical Design
How B+ Tree, LSM Tree, Columnar, Time-Series, and In-Memory storage engines organize data on disk — pages, buffer pools, MVCC, compaction, column encoding, timestamp/float compression, and write amplification on HDD vs SSD.
Read article →Database Indexes Deep Dive — From B-Trees to LSM Trees
A comprehensive guide to database indexing: hash indexes, B-Trees, clustered vs non-clustered, covering, composite, partial, full-text, spatial indexes in MySQL/PostgreSQL, and how LSM-based engines like Cassandra and CockroachDB index data.
Read article →Kafka Internals — Wire Protocol, Data Encoding, and the RecordBatch Format
How Kafka encodes data on the wire and on disk: the RecordBatch v2 format, varint encoding, message format evolution, compression framing, and schema management with Schema Registry.
Read article →Kafka Replication — ISR, Leader Election, and KRaft
How Kafka replicates data across brokers: the ISR protocol, leader election, follower fetch loop, unclean election trade-offs, and a deep dive into KRaft — Raft epochs, leader election, fetch-based log replication, the metadata log, and snapshots.
Read article →Kafka Consumers & Consumer Groups — Pull Model, Partition Assignment, and Rebalancing
How Kafka consumers work: the pull-based model, consumer groups, partition assignment strategies, the rebalancing protocol, offset management, and tuning parameters.
Read article →Kafka Broker Internals — Segments, Indexes, and Log Compaction
How the Kafka broker stores data on disk: log segments, sparse offset and time indexes, physical directory layout, retention, and log compaction.
Read article →How the Kafka Producer Works — And Why It's So Fast
A deep dive into the Kafka producer internals: batching, compression, filesystem optimisation, acknowledgement modes, and the tuning knobs that matter.
Read article →