Technical documentation: de-identification engine and workflow score

These pages describe what the engine package and the reference runner do at this version: how data moves, what each detector does and how well it measures, which keys exist and what each one can open, what the report says and how it is judged, and how the workflow score is computed. They are written against the code, and everything a number depends on (thresholds, bands, the coverage matrix, benchmark results, model hashes, versions) is rendered from the files the code reads, so the docs can't drift from the engine without a test failing.

Docs version: 0.6.0

Pages

De-identification:

1. Architecture and flow: intake to delivery, every modality, the unpacking and normalization layer, engine and runner split, size limits.
2. Threat model: buyer, desk, runner host, supply chain, refresh differences, and the defense mapped to each.
3. Keys: per-product salt, deal data key, narrow lookup key, sealed identity map, crypto-shredding, the isolation contract and its tests.
4. Detectors: every rule tier, the ensemble and its merge logic, thresholds, the model manifest, benchmark results per entity and modality, the independent re-check.
5. Profiles and coverage: the five profiles, sensitive attributes, and the coverage matrix (HIPAA Safe Harbor's 18 identifiers, GDPR, PCI) mapped to rules and corpus cases.
6. HIPAA and related law: Safe Harbor, Expert Determination, 164.514(c) re-identification codes, BAA and Security Rule, breach handling, Washington MHMDA, California CMIA, 42 CFR Part 2, GDPR and PCI. What is in place and what isn't.
7. Measurement and the release gate: the miss bound, the release gate, canaries, red teams, the independent re-check, the motivated-intruder test, report bands.
8. Linkage: the identity graph, redaction-only-grows, the date offset, k-anonymity.
9. Operations: hold reason codes, failure modes, recall and deletion requests.
10. Report schema: the deid_report JSON schema, its versioning and changelog.

Workflow score:

11. Workflow score: the business-object graph, cases and traces, business-id canonicalization, every metric's formula, ranges and sample sizes, bands, linkage kept, the OCEL 2.0 export, the estimate model's status, privacy controls, known limitations.

Status words

These pages use three words for status, and mean them literally:

• Live: in the code at this version, exercised by tests, and on by default in the reference runner.
• Built, off: in the code and tested, but behind a switch a control plane hasn't turned on, or needing an input no control plane sends yet.
• Not built: described as a plan; no code does it.

Nothing here is a compliance claim. Where a page names a regulation, it says which mechanism maps to which requirement and whether that mechanism is live. Legal status (agreements signed, review by counsel, regulator filings) belongs to each product and is out of scope here, except where the page says what is missing.

Editions

• Public edition (these files). Models appear by alias and report name only ("detector B", "name model A", "speech model"), with license and the tree hash every report already carries. No model or vendor names, no download sources, no per-file weight hashes, no runtime package names, no credential-format rule ids that name a provider. tests/test_techdocs.py in the reference runner fails if any of them appears in these pages. The public edition also leaves out what would help someone attack a delivery rather than evaluate one: per-kind, per-modality and per-category measurements (it gives value-weighted totals), each coverage row's note on what it misses, the watermark method, and each defense's status at this commit.
• Auditor edition. The same pages plus the full model manifest from the lock files (names, versions, sources, per-file sha256, runtime packages and their hashes), every rule id, the package's allowed third-party imports, and everything the public edition leaves out above. It is shared under NDA only. It is never committed, never synced to another repository and never published: generate it locally.

python tools/build_docs.py --auditor                 # writes build/techdocs-auditor/ (git-ignored)
python tools/build_docs.py --auditor --out /secure/dir

The script refuses an output directory inside the repository unless git ignores it, and refuses anywhere inside the package (which every runner vendors). Each page it writes starts with an NDA banner. The public edition carries a sha256 of each lock file, so an auditor can check that the edition they were given matches the commit they were shown.

How the generated sections work

A generated section is everything between <!-- generated: NAME --> and <!-- /generated: NAME -->. Don't edit inside those markers; edit the source file and re-render. Everything outside the markers is prose.

python tools/build_docs.py          # re-render every generated section in place
python tools/build_docs.py --check  # exit 1 if any committed section differs from a fresh render

Sources: thresholds.json, profiles.json, coverage.json, sensitive.json, patterns.json (version and rule ids), bizids.json, report/bands.json, report/schema.json, report/CHANGELOG.md, workflow/bands.json, bench/baseline.json, the corpus files, the package's own constants (limits, caps, labels), and the reference runner's lock files (models.lock.json, models/av.lock.json, models/media.lock.json).

Checks, all in the reference runner's tests/test_techdocs.py, which CI's test job runs on every pull request:

• every committed page equals a fresh render (so a threshold, band, coverage row or model hash can't change without the docs changing in the same commit);
• the docs version equals the package's __version__ and scrub/pyproject.toml;
• every generated section name is known, closed, and used on some page;
• the public pages name no model, vendor, model source or runtime package (the deny list is built from the lock files at test time, plus a hashed list of extra names);
• every hold reason code the operations page lists exists in the code, and every literal hold reason in the scrub is on the page;
• the auditor edition carries every model name, and the script refuses an output path inside the repository that git doesn't ignore.

The package's own test (tests/test_techdocs.py inside the package, so every vendoring runner's CI runs it) checks that the docs version block matches the package it ships with and that every section is closed.

The pages live inside the package directory on purpose: the sync that vendors the package into other runners carries them with the code at the same commit, and the package's isolation test scans them like every other file, so they stay free of product names and infrastructure details.