# Building the manuscript and its evidence The manuscript has four distinct build layers. Keeping them separate prevents a render command from silently becoming a database experiment and makes every published figure traceable to its input. 1. **Capture evidence** — run literal `EXPLAIN` statements against PostgreSQL and update the tracked JSON audit manifests. 2. **Render assets** — turn tracked plan evidence and normalized teaching fixtures into SVG files under `assets/`. 3. **Compile Markdown** — concatenate the chapter sources into `ManuscriptCompiled/` for Obsidian. 4. **Export PDF** — ask Better Export PDF to render the compiled manuscript, then create and structurally verify the compressed editions. Run `make help` for the complete target list. ## Make one operation card at a time The common editorial loop should not require remembering which plan and trace scripts must run, nor should changing one card rewrite all 36 operation assets. Use the operation key printed by `make list-operations`: ```sh make list-operations make operation-assets OP=IndexOnlyScan ``` That one command rebuilds both target images for the card: - its graphical plan tree, from the literal plan already stored in `scratch/actual_operation_plans.json`; and - its plan performance signature, from the exact OTLP capture already attached in `artifacts/chapter4_runtime_evidence.json`. Non-wait execution time is labeled `CPU`; observed waits retain their exact `wait_event_type:wait_event` names. Runtime classes use distinct vector hatching as their primary encoding so the figures remain legible when printed in grayscale; color is supplementary. For narrower or broader iterations: ```sh make operation-plan OP=IndexOnlyScan make operation-trace OP=IndexOnlyScan make section-assets SECTION=Index make operation-assets-all ``` `SECTION` is the Operations folder name (`Index`, `ResultSet`, `Tuple`, and so on). Operation and section matching is case-insensitive. These targets render images only: they do not connect to PostgreSQL, recapture evidence, edit the Markdown cards, concatenate the manuscript, or invoke Obsidian. An explicitly requested card or section fails when a trace is still `capture-required`; the whole-ledger target reports known pending traces and renders everything that has evidence. This makes a missing Citus trace visible without blocking unrelated stock-PostgreSQL asset work. ## The ordinary Chapter 4 loop After editing Chapter 4 prose or a deterministic diagram fixture: ```sh make chapter-4 make review-plans ``` `make chapter-4` performs only reproducible local rendering, compilation, and validation. It does not start PostgreSQL, change planner settings, or refresh captured evidence. Open `scratch/preview_pgadmin_plans.html` when plan-tree appearance or audit consistency changed. Operation cards publish the graphical plan by default. Their literal `EXPLAIN` output remains beside the figure in a hidden `literal-explain-plan` comment so the evidence stays auditable without making the reader decode the same plan twice. A card may show both forms only when it contains an explicit comparison lesson and the marker `<!-- plan-presentation: text-visual-comparison -->`; Chapter 4 validation rejects an unmarked visible duplicate. When the compiled chapter looks correct, export separately: ```sh make pdf ``` Keeping PDF export separate avoids paying for Obsidian rendering while iterating on SVGs or prose. ## Rebuild the Chapter 2 MVCC storyboards The versioning diagrams are semantic storyboards, not hand-drawn exports. Their transaction IDs, tuple locations, version links, payload change, and observer outcomes live in: ```text artifacts/mvcc_versioning_storyboard.json ``` Render the two publication SVGs with: ```sh make mvcc-assets ``` The renderer validates the MVCC invariants before writing either asset. In particular, the updating transaction must become the old tuple's `xmax` and the new tuple's `xmin`; the old tuple's `t_ctid` must identify the successor; and a fresh successor must point to itself. The output contains SVG-native text and geometry only. It has no filters, embedded rasters, external fonts, timestamps, or nondeterministic identifiers, so rebuilding the same fixture produces the same bytes and keeps the PDF text selectable. For the complete Chapter 2 loop, use: ```sh make chapter-2 ``` The focused renderer tests parse both outputs as XML, assert their teaching labels and tuple invariants, and verify that the checked-in SVGs match the tracked fixture exactly. ## Rebuild the Chapter 5 durability diagrams The Chapter 5 figures use the same fixture-first contract as the MVCC storyboards. The `pg_control` cutaway keeps its file dimensions, selected identity/condition/continuation fields, CRC guard, and illustrative WAL coordinates in: ```text artifacts/crash_recovery_control_storyboard.json ``` The WAL durability-boundary figure keeps its two volatile products, durable destinations, and two ordering rules in: ```text artifacts/wal_durability_boundary.json ``` Render and validate it with: ```sh make chapter-5 ``` The control-file renderer rejects a REDO start that does not precede the checkpoint record, missing mnemonic field groups, or an invalid control-file size contract. The WAL-boundary renderer rejects the loss of either the WAL-before-page or WAL-before-acknowledgement rule. Both outputs are selective mental models rather than byte-for-byte layouts or startup procedures. Their native SVG output is deterministic, searchable, and free of external fonts, rasters, filters, timestamps, and random identifiers. The semantic contracts follow PostgreSQL 18's WAL and recovery rules. ## Rebuild the Chapter 6 buffer clock sweep The clock-sweep illustration keeps its eight example buffer descriptors, usage-count heat, independent pin and dirty properties, current hand position, and four selection decisions in: ```text artifacts/buffer_clock_sweep.json ``` Render, compile, and validate the chapter with: ```sh make chapter-6 ``` The renderer rejects usage counts outside PostgreSQL's zero-to-five range, a selected descriptor that is pinned or still hot, a missing decision, or a fixture that fails to expose the dirty-candidate write toll. The figure models the default replacement strategy rather than access-strategy rings. Its native SVG is deterministic, searchable, and free of external fonts, rasters, filters, timestamps, and random identifiers. The semantic contract follows PostgreSQL 18's `freelist.c`, `bufmgr.c`, and `buf_internals.h`. ## When literal plan evidence must change The files `scratch/actual_illustrated_plans.json` and `scratch/actual_operation_plans.json` are captured evidence. Do not edit their plan text to make a diagram agree with an expectation. For stock PostgreSQL plan cases, start the repository's normal populated audit database and pass its connection URI explicitly: ```sh make capture-plans AUDIT_DATABASE=postgresql://[email protected]/plan_audit ``` That target captures the literal plans first, refreshes the hidden provenance beside each operation card, then renders the plan assets and rebuilds the HTML review. It fails before capture when `AUDIT_DATABASE` is absent; there is no implicit production or developer-database default. The stock target merges only cases without extension requirements, so it preserves previously captured pgvector and Citus evidence. For pgvector and Citus cases, use the pinned disposable environments: ```sh make capture-extension-plans ``` The extension workflow records PostgreSQL and extension versions, fixture names, capture timestamps, and complete literal plan text. See `scripts/extension_audit/README.md` for ports and the `--keep-running` inspection option. ## What the wait-event timelines mean `make trace-assets` regenerates the teaching timeline SVGs from deterministic fixtures in: - `scripts/generate_plan_traces_svg.py` - `scripts/generate_pathological_traces.py` These fixtures normalize observed execution phases into legible teaching timelines. They demonstrate sequencing and attribution—such as a `Sort` moving from CPU to `BufFileWrite` and `BufFileRead`—but they are not raw exports from `pg_stat_activity` or `pg_wait_tracer`. They remain available for the dedicated wait-event teaching chapters, but they are not valid runtime evidence for a Chapter 4 operation card. Chapter 4 uses a strict capture contract: - literal `EXPLAIN (ANALYZE, BUFFERS)` excerpts for nodes, rows, buffers, spill methods, worker counts, and batch counts; and - an exact `pg_wait_tracer` capture exported through `pgwt-server execution_detail` and `tools/export_otel.py` before a wait-event diagram can be attached to an operation card. The manifest marks every card without that source as `capture-required`; it does not render a substitute diagram. After attaching a capture, rebuild only that graph with `make operation-trace OP=<key>`. The exact attachment procedure is in `scripts/wait_capture/README.md`. ## Target dependency map ```text live PostgreSQL ──capture-*──> tracked audit JSON ──plan-assets──> plan_tree_*.svg normalized teaching fixtures ──trace-assets──> trace_*.svg exact pg_wait_tracer OTLP ──trace-assets──> operation-card wait SVG chapter Markdown + assets ──manuscript──> ManuscriptCompiled/*.md ManuscriptCompiled/*.md ──pdf──> master PDF ──verify──> compressed PDFs ``` The validation target follows Chapter 4's actual operation-card transclusions, checks that those sources refer to real assets, parses every referenced SVG as XML, rejects the removed generic `Factors`, `Operates on`, `Workloads`, `Description`, and `Runtime signature` rows, and audits the literal-plan and exact-capture boundaries recorded in `artifacts/chapter4_runtime_evidence.json`. A wait SVG can appear on an operation card only when its raw capture is recorded in that manifest. It then runs the trace renderer unit tests. It deliberately does not sweep unrelated Operations library index pages: ```sh make check-assets ```