Threat model
Five adversaries, what each can do, and the defense mapped to each. "Reference runner" means the reference implementation at this commit; control-plane defenses are named as such and documented by each product.
1. The buyer
Has: every delivered file, the signed report, their evaluation sample, as much time and outside data as they want (public records, data brokers, social media, other purchases). Wants: to re-identify a person, the seller's business, or the seller's customers; or to pass the data on.
| Attack | Defense |
|---|---|
| Read a direct identifier the scrub missed | Rules (detector A), ensemble (detectors B and C), name model A fallback, the decoded view, column rules, the inline residual gate, the independent re-check, and a person's review of a random output sample with a published miss bound |
| Join stand-ins across deals or products | Every stand-in, keyed hash and fingerprint is an HMAC under a per-deal salt; salts differ per product label; the isolation test checks that two keys give disjoint output |
| Recompute a stand-in from a guessed identifier | Needs the deal salt, which only the runner (for one job) and the control plane's narrow lookup key hold. Under random stand-ins there is no function to recompute: tokens are CSPRNG output stored only in the sealed map |
| Re-identify from quasi-identifiers (ZIP, age, dates, job title, city) | ZIP3 with restricted areas 000, age cap, keyed whole-week date offset or year only, k-anonymity measured per table and on the linked view and reported (kMin, shareUnderK); the person-linkage table ships only for people in a class of at least k |
| Re-identify the seller (owner identity) | The owner-identity pass over content, OCR text, paths and repository metadata, with every variant of the deal's terms; held media when the pass can't read it (blind_media); a red-team corpus that must be caught 100% |
| Identify a third-party company | third_party_companies: tokenize in strict and standard (b2b keeps them) |
| Read sensitive attributes (health, religion, minors…) | sensitive.py holds or flags per record and per file by profile; never-released classes (sexual content involving a minor, sexual imagery, escalations, voices) |
| Read metadata (EXIF, document authors, PDF info, git signatures, file times) | Stripped or rewritten by kind; file times follow the date rule |
| Match path names | Paths scrubbed like text; real paths only inside the buyer-encrypted manifest; reports and deliveries never carry the deal id |
| Pass the data on | A watermark per buyer org and version, and a Merkle root over every delivered file and record |
| Compare two versions (refresh differences) | See section 5 |
| Use the score or report to learn about individuals | Small cells: counts under 11 read <11; workflow numbers under 11 withheld; confidential deals get buckets only |
What each defense's status is at this commit (live, built behind a switch, or partial) is in the auditor edition.
2. The desk
Has: the control plane's console. Wants (or could be coerced) to: read raw values, copy them, or release a version that fails.
| Risk | Defense | Status |
|---|---|---|
| Reads raw values | The runner sends the desk only sealed review samples (scrub review items and output-sample windows), under the deal key; output windows are scrubbed text. Masking, time-limited reveal with a reason, and the access log are control-plane features | Runner side live; console controls per product |
| Releases a bad version | The release gate is computed from the counts the runner sent and the desk's recorded review; how misses, overrides and HIPAA decisions are recorded is the control plane's (see the product's runbook) | Control-plane feature, behind a switch |
| Looks up a person | Stand-in lookup uses the narrow lookup key, not the data key; it decrypts nothing and is logged | Control-plane feature |
3. The runner host
Has (if compromised): one machine, one job's claim, one deal's data for the job's lifetime.
| Risk | Defense | Status |
|---|---|---|
| Exfiltration over the network | Boot locks outbound traffic to the control plane, claims, re-locks to the hosts the job type needs (resolved once and pinned), drops DNS and IPv6, drops to an unprivileged user with no-new-privileges. The finish call reports the egress mode | Live in the reference runner; the control plane refuses release when egress was off in production (control-plane check, not verified here) |
| Hostile files exploiting a parser | Every untrusted-format parser runs sandboxed: no network namespace, read-only root, cleared environment (no job token), work directory masked except scratch, killed with the parent, per-tool CPU, address-space, file-size, output and wall limits. Healthcare formats parse in their own sandboxed child | Live; mode reported in the finish call |
| Malware or macros | Malware scan of every downloaded file with signatures baked into a weekly-rebuilt image; macro-enabled Office files held | Live |
| Data at rest on the host | Work directory on tmpfs (or an encrypted volume for large jobs that is destroyed with the job), swap turned off, wipe on exit; on plain disk, zero-overwrite before unlink (best effort) | Live; mode reported |
| Leaking a file name through a tool's error | Working copies sit under hashed names; archive members are renamed to hashes as soon as they're extracted | Live |
| Long-lived compromise | One job per machine; machine destroyed after | Live (deployment property of the reference runner) |
4. The supply chain
| Risk | Defense | Status |
|---|---|---|
| A tampered Python dependency | requirements.lock installed with --require-hashes in the image and CI |
Live |
| A swapped model | Every baked model pinned in the lock files by sha256 (wheels) and tree hash (installed files and weights); re-hashed before any data moves (jobs/models.py); a mismatch fails strict jobs and drops an ensemble member otherwise; reports carry the tree hash per alias and a modelsOk provenance flag |
Live |
| Weights fetched from a moved or poisoned source | Fetched at image build, every file checked against its pinned sha256; no fetch at runtime | Live |
| The engine itself reaching out | The isolation test fails the build on any network, process or host module, any environment read, and any open() outside a short allowlist (keys.md) |
Live |
| The image | Deployed by digest; reports record the image digest (imagePinned) |
Live |
| A vendored copy diverging | Vendoring runners pin the package by git tree hash; their CI fails when the vendored tree differs from the lock | Live |
5. Refresh differences
A buyer of two versions of the same deal (a refresh, or a patched version after a recall) can diff them. Without care, a value redacted in version 1 and missed in version 2 is exposed by the diff, and a value whose stand-in changes reveals that it is the same person.
Keyed stand-ins are a pure function of the deal salt and the value, and dates shift by one keyed offset per deal, the same every run. With the deal's identity map, values redacted in an earlier run are redacted again in later runs and person ids carry across runs. Each defense's status is in the auditor edition.
Out of scope
- The control plane's own security (authentication, key-encryption key handling, database, vault), each product's console, and the marketplace. Each product documents these.
- Legal re-identification risk certification. The engine measures and reports; Expert Determination is a human expert's judgment (compliance.md).
- Adversaries with the deal data key or the product's key-encryption key.