Redaction Policies as Code: Building and Sharing Policies with the Philterd Policy Library

A redaction policy decides which values leave your building and which do not. That is a consequential decision, and in most deployments it lives somewhere nobody can review: a form in an admin console, a config blob nobody has read since it was pasted in, an argument that happened in a meeting eighteen months ago and was never written down.

The Philterd policy library is a set of working policies you can use directly, but the more useful thing it demonstrates is the shape. A redaction policy is a file. Files can be diffed, reviewed, tested, versioned, and rolled back, and once yours is one, questions like “when did we stop redacting organization names, and who approved that” have answers.

What is in the library

Twenty-two policies across eight categories: healthcare, finance, legal, government, education, AI training, contact center, and general-purpose starting points. They cover the compliance frameworks people actually get asked about, including HIPAA Safe Harbor, GDPR, CCPA/CPRA, PCI DSS, GLBA, FERPA, and Bankruptcy Rule 9037. Everything is Apache 2.0, the same license as the rest of the toolkit.

The repository splits into philterd/ and community/. Policies under philterd/ are maintained by the core team and kept current with Philter releases. Policies under community/ are owned by whoever contributed them. Both are useful, and the split exists so you can tell at a glance which is which rather than having to guess at the provenance of something you are about to point at production data.

Each policy is validated against the redaction policy JSON schema that lives in the Phileas repository, so a policy that parses is a policy Philter and Phileas will both accept.

The two-file pattern

Every policy is a pair of files in the same directory. The .json is the policy Philter loads. The .md beside it is the part that makes the policy reviewable by a person.

The sidecar carries frontmatter describing what the policy is for, and then prose explaining the reasoning. Here is the head of the AI training policy:

---
title: "LLM Training Data Preparation"
category: "ai-training"
version: "1.0.0"
philterCompatibility: ">=3.0.0"
useCase: "Aggressive PII redaction for documents being fed into LLM training,
  fine-tuning, or RAG vector stores - preserves semantic structure with type tokens."
entities: ["PERSON", "PHONE", "EMAIL", "SSN", "CARD", "IP", "URL", "ORG", "LOCATION"]
exampleInput: "Patient John Smith was treated by Dr. Garcia at Mercy Hospital in Austin, TX."
exampleOutput: "Patient [PERSON] was treated by [PERSON] at [ORG] in [LOCATION], TX."
---

Two fields there do more work than they look like they do. The exampleInput and exampleOutput pair is a statement of intended behavior in a form a non-engineer can check: a compliance reviewer who will never read the JSON can read those two lines and tell you whether the policy does what they wanted.

The prose below the frontmatter is where the reasoning goes, and the AI training policy is a good example of why that matters. It biases toward over-redaction, with a looser name-confidence threshold than the general-purpose policy, because once a value is in model weights or a vector store it is effectively unrecoverable. An extra redacted token is cheap and a missed one is not. It also replaces values with semantic tokens like [PERSON] rather than asterisks, so the model still learns that a patient was treated by a physician at a facility instead of learning that asterisks appear at random. Neither decision is visible in the JSON. Both are the kind of thing you want written down before someone tunes the threshold back up.

The part that makes it code

Every policy in the library has to pass a golden-file test: a fixed input, a known expected output, checked on every change. This is the piece that separates a policy in version control from a policy that is genuinely treated as code.

Redaction policies fail quietly. Tighten a confidence threshold to cut false positives, and the thing you notice is that the noise went away. What you do not notice is the category of real detections that went with it, because nothing errors and nothing logs. A golden-file test turns that silent change into a failed build.

If you take one habit from the library into your own repository, take that one. A handful of representative documents with their expected output, checked in CI, catches the class of change that review alone does not.

Versioning and compatibility

Each policy carries its own semver in the sidecar frontmatter, and the repository is git-versioned, so any previous state of any policy is recoverable from history. Policies declare a philterCompatibility range, and the library targets Philter 3.0.0 and later unless a policy says otherwise.

Per-policy versioning matters more than it sounds. A repository-wide version tells you the library moved. A policy-level version, bumped on breaking changes, tells the person running the healthcare policy that the healthcare policy changed, which is the only question they were asking.

Using one

Download it, load it, use it by name.

curl -O https://raw.githubusercontent.com/philterd/pii-redaction-policies/main/policies/philterd/healthcare/hipaa-safe-harbor.json

curl -X POST http://localhost:8080/api/policies \
     -H "Content-Type: application/json" \
     --data @hipaa-safe-harbor.json

curl "http://localhost:8080/api/filter?p=hipaa-safe-harbor" \
     -H "Content-Type: text/plain" \
     --data "Patient John Smith was discharged on 2025-03-14."

For an embedded deployment, the same JSON loads into Phileas directly. If you would rather build a policy visually than by hand, the Redaction Policy Editor produces the same schema.

Contributing one

Pull requests are welcome, and the bar is the two-file pattern plus a passing golden-file test. If you have tuned a policy for an industry the library does not cover well, that policy is more useful in community/ than in your private repository, and it stays yours.

The honest part

These policies are starting points, not legal advice, and not a certification. A policy named for a compliance framework encodes an interpretation of that framework, which is not the same as compliance with it. Detection is probabilistic, so what a policy does on the library’s example input tells you very little about what it will do on your documents, which have their own formats, abbreviations, and edge cases.

Before you rely on one, measure it against your own data. Philter Scope scores a policy on precision and recall against a gold standard, and the answer it gives you is specific to your corpus rather than to someone else’s example sentence. Start from the library, then find out what you actually have.