Skip to main content
Technical docs

Linkage

the identity graph, redaction-only-grows, the date offset, k-anonymity.

Linkage

Linkage is what lets a buyer follow one person, order or ticket across files, systems and refreshes without learning who or what it is. It is also the main re-identification risk once direct identifiers are gone, so each piece below has a limit.

  • Map format 1; modes keyed, random; identifier types paired: emails, phones, names, mail_senders, contributors, chat_users.
  • Values longer than 8 words aren't kept for re-redaction.
  • Quasi-identifiers tracked for k-anonymity (k = 5): zip, age, dob, gender, title, department, city, region, country, ethnicity, marital, language; at most 1,000,000 distinct tuples per table, 20 tables listed.
  • Date offset: 1 to 52 whole weeks either way, never zero.

Stable stand-ins

Within a deal, the same value always becomes the same stand-in: keyed stand-ins are an HMAC of the normalized value under the deal salt; random stand-ins are drawn once and read back from the map. Normalization makes formatting variants one value (digit kinds: NFKC digits only; compact kinds drop spaces, dots and dashes; the rest casefolded with whitespace collapsed). Across deals nothing is shared: each deal's salt is different, and every key and digest in the map is keyed by it.

Identity graph

identity.Graph runs when the claim carries the deal's identity map (the control plane's linkage switch), and is required under random stand-ins. Two passes:

  1. Pair. Identifiers seen together are filed as one person: a table row's email, phone and name columns, a mail header's display name and address, a chat profile, a git author's name and email. Strong identifiers (email in canonical form, phone in E.164, platform user id) cluster by union-find on keyed link keys.
  2. Scrub as usual. Every stand-in the scrub makes is filed under its identifier's link key.

Names are weak and link only when unambiguous: a full name one cluster carries; a first name alone when one known person has it; an initial plus last name with one match; a greeting in a mail thread when one participant has that first name (scoped to the thread). A second person with the same name withdraws the link.

Each cluster gets a person_<12 hex> id (CSPRNG). The person-linkage table (__linkage__/people.jsonl) maps it to the stand-ins its identifiers became; mentions.jsonl carries the thread-scoped greeting links. Under risk_checked the table ships only rows for people whose linked quasi-identifiers fall in a class of at least k; when more than half would be cut it is held whole. Under off (Safe Harbor) no table is made. The table goes through the residual gate, the owner-identity check and the re-check like any file.

The map only gains members: a merge keeps the older person id, so earlier deliveries keep their meaning. Two concurrent runs: the second to commit gets a conflict, reads the newer map, absorbs it (nothing dropped; for random stand-ins both tokens are kept as alternates and both delivered) and commits again.

Redaction only grows

Every value redacted in a deal is remembered as a keyed 64-bit digest (redacted), with digests of first words as anchors. On every later run, Graph.spans finds those values again in any text the scrub reads (digit runs of 7 or more by their digits; up to 8-word values by their anchors) and redacts them whatever the detectors find this time. A refresh can't un-redact what an earlier refresh caught, even after a detector or rule changes. Values longer than 8 words and digit runs under 7 aren't kept.

Without a map (linkage off for the deal), this doesn't run, and a refresh depends only on that run's detectors.

Date offset

Under deal_offset, every date the engine recognizes moves by the same whole number of weeks for the whole deal: HMAC-SHA256(salt, "scrub date offset v1") gives 1 to 52 weeks with the sign from one bit, never zero. Intervals, cross-person ordering and weekdays survive; the offset is never reported (one known pair would undo every shift). File times that ship follow the rule. Birth dates go to the year under every mode except keep.

The offset is a weak defense on its own: a buyer who knows one true date of one event in the deal recovers it. It hides calendar position from casual reading and from joins on exact dates with outside data; it doesn't withstand a buyer with one ground-truth event.

Business-object ids

With the deal's workflow switch on, order, invoice, ticket and other internal business ids become [LABEL_<10 hex>] over a canonical core (a short letter prefix, #, separators and leading zeros dropped), so INV-00042 in billing and invoice #42 in support join. Keyed under the deal salt, or from the map under random stand-ins. With the switch off, internal ids are left as they are. Public ids (parcel tracking numbers) take the ordinary stand-in on every scrub, so redaction-only-grows covers them; customer ids with a type prefix take the person-id stand-in while the switch is on; SKUs follow the profile's skus field. Classes: workflow-score.md.

k-anonymity

kanon.py groups each delivered table's rows by the quasi-identifiers that survive the scrub (ZIP3, birth year, age or 90+, gender, title, place and the rest listed above). The linked view does the same with one tuple per person across every row linked to them. The report gives the smallest class (kMin) and the share under k. This is a measurement, not a mitigation: beyond the profile's fixed rules (ZIP3, age cap, birth year), nothing is generalized or suppressed to reach k, except the linkage table's own risk check above. Coverage row gdpr.identity_factors is planned for that reason.

What linkage gives away

  • The linkage table tells a buyer which stand-ins are one person. That is its purpose and its risk; the k check bounds it on the quasi-identifiers the engine knows, not on behavior (the sequence of a person's actions).
  • Stable stand-ins across refreshes let a buyer accumulate a profile per stand-in over time.
  • Business-id joins expose the shape of the seller's processes (that's what the workflow score measures).

Engine docs at commit 1c5af8f39ebb. Plain-language versions: what happens to your data, the workflow score.