Handmade PostgreSQL 5/5 — Distribution

Part five, the finale of the Handmade PostgreSQL campaign: one database becomes several. A second copy of your server follows the first and answers reads from it. A single table stops being a single heap and becomes a set of partitions the planner can skip.

This continues parts one through four. The server, the SQL engine, the storage and the indexes are the ones you already built — nothing here replaces them, everything here sits on top. The ladder opens by re-checking that the four parts below it still work.

The two commands

Unchanged, and still captured into session memory:

serve: <command>     started as:  <command> <port> <datadir>
sql:   <command>     started as:  <command> <port>

The reply format is the one frozen in part one — one acknowledgement line per statement, SELECT printing its rows |-joined first, failures printing ERROR: <text> without closing the session. It is not restated here; it has not moved.

What this part adds

Three additions, and they are the whole contract of part five.

A third argument makes a replica

<command> <port> <datadir> <primary_port>

Started with a fourth word, your server comes up as a replica of the primary listening on 127.0.0.1:<primary_port>. It binds <port> and keeps its own copy under its own <datadir> — the two servers never share a directory.

A replica answers SELECT like any server. A statement that would write — INSERT, UPDATE, DELETE, CREATE TABLE — replies with an ERROR: line and changes nothing. The replica is a copy, not a second opinion.

What it copies is everything: the rows already on the primary when it starts (the initial copy), the rows written while it is connected (the stream), and the rows written while it was down (the catch-up when it comes back on its own data directory). How you move them — a log you ship, pages you send, statements you replay — is yours to choose and yours to write down.

WAIT <n>;

WAIT 1;   ->   WAIT 1

Sent to the primary, WAIT <n>; blocks until at least <n> replicas have acknowledged everything committed so far, then replies WAIT <n> where <n> is how many acknowledged. This is the handle that makes replication testable without sleeping and hoping: a check writes, waits, and reads the replica.

WAIT never blocks forever. If the count it was asked for has not been reached after a few seconds it gives up and answers anyway — WAIT 0 with nobody following, or an ERROR: line. A WAIT that hangs is a failed WAIT.

Partitioned tables

CREATE TABLE <t> (<columns>) PARTITION BY RANGE (<col>) PARTITIONS <n>;   ->   CREATE TABLE

The key column is INT. The table's rows live in <n> partitions named <t>_p0<t>_p<n-1>, and each partition owns a contiguous, ascending slice of the key. The slices are fixed and deliberately boring, so that your engine and a check agree without inventing a bounds syntax: partition i owns the keys from i × 1000 up to but not including (i + 1) × 1000. Keys at or above <n> × 1000 land in the last partition; keys below zero land in the first.

  • INSERT routes each row to the partition its key falls in and replies as always, INSERT <n>.
  • SELECT * FROM <t>; reads every partition and returns every row in insertion order — a partitioned table still behaves like one table.
  • SELECT * FROM <t>_p3; reads that one partition, as a table in its own right.
  • EXPLAIN <select>; reports the plan and, for a partitioned table, prints one PARTITION <name> line for each partition the plan will scan, in partition order, before its EXPLAIN acknowledgement. A select filtered by the key with WHERE <col> = <value> prints exactly one — that is pruning. An unfiltered select prints all of them. Whatever else your EXPLAIN already prints from part four stays; only the PARTITION lines are graded.

Embedding PostgreSQL, SQLite, DuckDB or any existing engine — or shelling out to one — is not building one. The replication and the partitioning are yours, same as the socket, the parser and the pages before them.

The ladder

  • Set up: the commands, and parts one to four still standing (10)
  • A replica connects (20)
  • The replica starts with what was already there (40)
  • Writes stream to the replica (40)
  • The replica is read-only (20)
  • The replica catches up after being down (40)
  • WAIT acknowledges, and never hangs (40)
  • Rows route to partitions (40)
  • EXPLAIN prunes the partitions it does not need (60)
  • Review: how you built it (150, judged)
Sessions

0

Visibility

Public

Category

Reinvent the Wheel

Slug

handmade-postgresql-5-distribution

Duration

30 min

Judge reviews

~24 per session

Active session

No

Points

10–150

Tags
  • database
  • replication
  • partitioning
  • handmade-postgresql
  • campaign
  • 1

    Set up the project and re-declare the two commands

    10

    pts / check

    +10 pts per passing check · +10 for completing the task

    The finale of the campaign: replication and partitioning on top of the
    database you already have.

    serve: started as:
    sql: started as:

    Same two commands as every part before this one. AGENTS.md (or
    README.md) carries serve:, sql: and test:, and the platform
    captures all three into session memory. AGENTS.md wins when both files
    declare one.

    Part five adds a third argument to serve:<command> <port> <datadir> <primary_port> starts the server as a replica of the primary
    on <primary_port> — plus WAIT <n>; on the primary and
    PARTITION BY RANGE on CREATE TABLE. The project description has the
    full contract.

    This rung also re-checks parts one through four in one pass: the server
    binds and answers, the engine takes a table and rows, the storage
    survives a restart on the same data directory, and an index serves a
    lookup. Carry your repository forward and this rung pays on the first
    probe.

  • 2

    A replica connects

    20

    pts / check

    +20 pts per passing check · +10 for completing the task

    A fourth word on the command line is the whole switch:

    started that way, your server comes up as a replica of the primary on
    127.0.0.1:<primary_port>. It binds its own port, keeps its own data
    directory, and — this is what is graded here — accepts client
    connections and answers a read.

    The check starts a primary, creates a table on it, then starts a
    replica pointed at it and asks the replica for that table. The table
    came from the primary, so the replica has to have followed the primary
    far enough to know it exists; it is empty, so the answer is the empty
    result SELECT 0.

    A replica that never finishes connecting, or that refuses connections
    until some far-off moment, fails here. Come up, follow, serve.

  • 3

    The replica starts with what was already there

    40

    pts / check

    +40 pts per passing check · +10 for completing the task

    A replica almost never joins an empty primary. It joins one that has
    been taking writes for a while, and the first thing it has to do is get
    a copy of what is already committed — the base backup, the initial
    copy, whatever you want to call the moment before the stream starts.

    The check writes several rows to the primary, and only then starts the
    replica. Every one of those rows must be readable on the replica, in
    insertion order, with the same SELECT <n> count the primary would
    give.

    The check polls the replica for a few seconds rather than demanding the
    copy be instant, so a replica that accepts connections while it is
    still copying is fine — as long as it finishes.

  • 4

    Writes stream to the replica

    40

    pts / check

    +40 pts per passing check · +10 for completing the task

    The initial copy is the easy half. The stream is the other one: rows
    committed on the primary after the replica is already connected have
    to arrive on it, without anybody asking for a fresh copy.

    This is where WAIT earns its place in the contract:

    WAIT 1; -> WAIT 1

    Sent to the primary, it blocks until at least that many replicas have
    acknowledged everything committed so far, then answers with how many
    did. The check writes to the primary, sends WAIT 1;, and reads the
    replica immediately afterwards — no sleeping, no polling. If your
    WAIT returns before the rows are really on the replica, the read that
    follows will say so.

  • 5

    The replica is read-only

    20

    pts / check

    +20 pts per passing check · +10 for completing the task

    A copy that accepts writes is not a copy — it is a second database that
    looks like the first one until the moment it quietly stops being it.

    Every statement that would change data, sent to a replica, replies with
    an ERROR: line and changes nothing. As always in this campaign the
    session survives: three rejected writes in a row get three ERROR:
    lines, not a dropped connection.

    The check sends an INSERT, an UPDATE and a DELETE to the replica,
    then reads both servers. Both must still show exactly the rows the
    primary wrote — the replica because it refused, the primary because
    nothing reached it.

  • 6

    The replica catches up after being down

    40

    pts / check

    +40 pts per passing check · +10 for completing the task

    Replicas die. The machine reboots, the process is killed, the network
    stalls for a minute. What matters is what happens when it comes back:
    it should ask the primary for the part it missed and carry on — not
    throw away its data directory and copy the whole database again.

    The check writes a first batch, lets the replica take it, kills the
    replica, writes a second batch while nothing is following, and then
    restarts the replica on the same data directory. Both batches must
    be there afterwards, in order, and WAIT 1; on the primary must
    succeed again — the follower is back in the count.

    Only the outcome is graded, but the data directory is handed back to
    you on purpose. A replica that re-copies gigabytes because it missed
    three rows is a replica nobody will run.

  • 7

    `WAIT` acknowledges, and never hangs

    40

    pts / check

    +40 pts per passing check · +10 for completing the task

    WAIT <n>; is the only window a client has into replication:

    WAIT 1; -> WAIT 1

    Sent to the primary it blocks until at least <n> replicas have
    acknowledged everything committed so far, and answers with the number
    that did. With a replica connected and caught up, WAIT 1; answers
    WAIT 1.

    The second half is the part people forget. With no replica
    connected, WAIT 1; can never be satisfied — and a statement that
    waits forever takes the client with it. So it gives up: after a few
    seconds it answers with the count it actually got, WAIT 0, or with an
    ERROR: line explaining that nobody is following. Either is accepted;
    hanging is not.

    Pick your own timeout, keep it under ten seconds, and make it the same
    number in your code and in your documentation.

  • 8

    Rows route to partitions

    40

    pts / check

    +40 pts per passing check · +10 for completing the task

    A partitioned table is one table to whoever queries it and several
    tables to whoever stores it:

    CREATE TABLE () PARTITION BY RANGE () PARTITIONS ; -> CREATE TABLE

    The key column is INT. Partition i is named <t>_p<i> and owns the
    keys from i × 1000 up to but not including (i + 1) × 1000 — fixed
    slices, no bounds syntax to invent. Every INSERT goes to the
    partition its key falls in.

    Two things are graded. SELECT * FROM <t>; still returns every row in
    insertion order, exactly as an unpartitioned table would — partitioning
    is not allowed to be visible to a reader who did not ask. And each
    partition is readable on its own, SELECT * FROM <t>_p2;, holding
    precisely the rows whose key belongs to it and no others. Some
    partitions get several rows, some get one, some get none and answer
    SELECT 0.

  • 9

    `EXPLAIN` prunes the partitions it does not need

    60

    pts / check

    +60 pts per passing check · +10 for completing the task

    Routing rows into partitions buys nothing until the planner uses it. A
    query filtered on the partition key only has to look in one place, and
    it has to be able to say so.

    EXPLAIN SELECT * FROM WHERE v = 2317;

    For a partitioned table, EXPLAIN prints one PARTITION <name> line
    for every partition the plan will scan, in partition order, before its
    EXPLAIN acknowledgement. Filtered by the key, that is exactly one
    line, naming the partition that owns the value — <t>_p2 for a key of
    2317, since partition i owns i × 1000 up to (i + 1) × 1000.
    Unfiltered, it is all of them.

    Whatever else your EXPLAIN prints from part four — index choice, scan
    kind, costs — stays exactly as it was; only the PARTITION lines are
    read here. The plan has to be honest: a filtered EXPLAIN that still
    lists every partition, or names the wrong one, is a planner that has
    not learned anything from the layout underneath it.

  • 10

    Review: how you built the distribution

    150

    pt budget

    Open-ended — a panel of 5 judges splits a 150-pt budget

    The rungs above proved the copy follows and the partitions prune. This
    one asks how it is built, and a panel of judges reads the repository to
    answer: architecture, performance, code craft, tests, and the way the
    project is run.

    Nothing new to implement. This is the last rung of the campaign, so
    spend the time you have left where the panel looks:

    • Architecture. What is the replication protocol you invented?
      Write it down as a protocol: what a replica says when it connects,
      what the primary sends, what an acknowledgement is, how a replica
      tells the primary where it stopped. Is that protocol a layer of its
      own, or is it woven through the storage and the accept loop? And is
      partitioning a routing layer with a seam — a table that dispatches to
      child tables — or a special case bolted into INSERT and SELECT?
    • Performance. What does the stream cost the primary: a commit that
      blocks on a socket, a background sender, a queue that can grow
      without bound? What happens when the network stalls and a replica
      stops acknowledging — does the primary slow down, buffer forever, or
      drop the follower? What does WAIT do to a busy primary, and what
      does pruning actually save on a table with many partitions?
    • Code quality. Duplication, dead branches, functions that do four
      things. Primary and replica are one binary here; if that produced two
      parallel copies of the same logic, the judge will find both. The
      measurement probes report duplication; the judge reads the rest.
    • Tests. Do they start a primary and a replica and assert on
      convergence, or do they only test the pieces? Do they cover the
      failure modes — a replica that dies mid-stream, a WAIT with nobody
      listening, a write refused by a follower, a key on a partition
      boundary? A test suite that has never had two servers running has not
      tested this part.
    • Technical governance. A successor has to operate this. Write
      down how to promote a replica, what to do when one falls too far
      behind, what your WAIT timeout is and why, what the durability and
      staleness guarantees really are, and how to add or resize partitions.
      Declare your conventions in tooling — a formatter, a linter, the
      test: command that really runs.

    When you are done, write .ololo/distribution-done.md with a short
    description of what you built and the decisions you made (at least 10
    words).