Writing your first redaction policy

This guide takes you from an empty file to a working redaction policy in a handful of steps. By the end you will have detected an entity, applied the policy with Philter, changed how a value is redacted, added more identifiers, and silenced a false positive. For the field-by-field reference behind every choice here, keep the policy schema guide open alongside this one.

What you will build

A policy that finds email addresses, phone numbers, and Social Security numbers in text and redacts each one. We will start with a single identifier and grow it.

You will need a place to run the policy. The REST examples below assume Philter 4.0 on localhost:8080, with your API key in PHILTER_API_KEY. The quick start gets you both in a few minutes. If you are embedding Phileas as a library, the same JSON loads directly in your code.

Step 1: start with an empty policy

A policy is a JSON object. The only part you almost always need is identifiers, the map of things to detect. Start here:

{
  "identifiers": {}
}

This is valid, but it detects nothing. Let us give it something to find.

Step 2: detect your first entity

Add an emailAddress identifier with one filter strategy. The strategy says what to do when an email address is detected:

{
  "identifiers": {
    "emailAddress": {
      "emailAddressFilterStrategies": [
        {
          "strategy": "REDACT",
          "redactionFormat": "{{{REDACTED-%t}}}"
        }
      ]
    }
  }
}

The pattern is always the same: an entity key (emailAddress), and inside it a strategies array named after the entity (emailAddressFilterStrategies). The REDACT strategy replaces the value with redactionFormat, where %t expands to the filter type. An email address becomes {{{REDACTED-email-address}}}.

Step 3: apply the policy

Save the file as my-first-policy.json, then upload it to Philter and redact some text:

# 1. Save the policy to Philter as my-first-policy
curl -sk -X POST "https://localhost:8080/api/policies?name=my-first-policy" \
     -H "Authorization: Bearer $PHILTER_API_KEY" \
     -H "Content-Type: application/json" \
     --data @my-first-policy.json

# 2. Redact text using the policy, referenced by name
curl -sk "https://localhost:8080/api/filter?p=my-first-policy" \
     -H "Authorization: Bearer $PHILTER_API_KEY" \
     -H "Content-Type: text/plain" \
     --data-binary "Reach me at jane@example.com."

The response replaces the email address:

Reach me at {{{REDACTED-email-address}}}.

You now have a working redaction policy. Everything from here is refinement. After each change below, save the file and replace the policy in Philter, since creating it again under the same name is refused:

curl -sk -X PUT "https://localhost:8080/api/policies/my-first-policy" \
     -H "Authorization: Bearer $PHILTER_API_KEY" \
     -H "Content-Type: application/json" \
     --data @my-first-policy.json

Philter keeps each earlier version in the policy’s history.

Step 4: change how a value is redacted

REDACT is one of several strategies. Suppose you would rather hide the address without a label in its place. Switch the strategy to MASK, which replaces each character with the mask character:

"emailAddressFilterStrategies": [
  {
    "strategy": "MASK",
    "maskCharacter": "*"
  }
]

Reach me at jane@example.com. now comes back as Reach me at ****************.

Each strategy has its own supporting fields. The filter strategies section of the schema guide lists them all, including format-preserving encryption and hashing for when you need the output to stay usable downstream.

Step 5: add more identifiers

Detecting more entity types means adding more keys to identifiers. Here is the policy with phone numbers and SSNs, each with a strategy that suits it. We keep the last four digits of phone numbers and fully redact SSNs:

{
  "identifiers": {
    "emailAddress": {
      "emailAddressFilterStrategies": [
        { "strategy": "MASK", "maskCharacter": "*" }
      ]
    },
    "phoneNumber": {
      "phoneNumberFilterStrategies": [
        { "strategy": "LAST_4" }
      ]
    },
    "ssn": {
      "ssnFilterStrategies": [
        { "strategy": "REDACT", "redactionFormat": "[SSN REMOVED]" }
      ]
    }
  }
}

Different entities can use different strategies in the same policy. That is the whole point: you decide, per type, how aggressive the handling should be.

Step 6: silence a false positive

Detection is never perfect. Suppose your documents include template text with the placeholder SSN 123-45-6789, which is not anyone’s number. Tell the policy to ignore it. The ignored list applies across the whole policy:

{
  "ignored": [
    {
      "terms": ["123-45-6789"]
    }
  ],
  "identifiers": {
    "ssn": {
      "ssnFilterStrategies": [
        { "strategy": "REDACT", "redactionFormat": "[SSN REMOVED]" }
      ]
    }
  }
}

Now that term is never redacted, even though it matches the SSN detector. Template SSN 123-45-6789; patient SSN 078-05-1120. becomes Template SSN 123-45-6789; patient SSN [SSN REMOVED]. Use ignoredPatterns instead of ignored when you need to match a regex rather than a fixed list of terms.

Step 7: detect a custom identifier, and validate it

The built-in types cover common PII, but most organizations also have their own identifiers: an account number, a member ID, a national ID you need to redact. For those, identifiers also holds an identifiers array of your own regex-based identifiers. Each entry has a pattern, a classification label, and its own identifierFilterStrategies.

A regex matches a shape, though, not a real value. A pattern for a nine-digit national ID also matches any other nine-digit run. To keep only the genuine values, add an optional validator: a named, built-in checksum or structural check. The match is kept only if it passes. Here is a Canadian Social Insurance Number validated with the Luhn checksum:

{
  "identifiers": {
    "identifiers": [
      {
        "classification": "canada-sin",
        "pattern": "\\b\\d{3}[ -]?\\d{3}[ -]?\\d{3}\\b",
        "validator": "luhn",
        "identifierFilterStrategies": [
          { "strategy": "REDACT", "redactionFormat": "[REDACTED-CANADA-SIN]" }
        ]
      }
    ]
  }
}

A Luhn-valid number like 046 454 286 is redacted; a same-shaped 123 456 789 that fails the checksum is left in place. Built-in validators cover identifiers such as the Canadian SIN, Brazilian CPF and CNPJ, Spanish DNI, French NIR, IBAN, and SWIFT/BIC. For the full list and parameterized validators, see the custom identifiers section of the schema guide and Cutting false positives on national IDs.

Step 8: keep refining

A policy is something you tune over time as you see real text. Two habits help:

• Measure it. Philter Scope scores a policy on precision and recall against gold-standard data, so you can see exactly which detector needs work rather than eyeballing a few examples.
• Review it like code. A policy is a plain file. Keep it in version control, and review changes the way you review any other change.

Where to go next

• The policy schema guide is the full reference for every field and strategy.
• PhiSQL lets you write the same policies in a few readable lines that compile to this JSON.
• The policy library has production-ready HIPAA, PCI DSS, and other policies to adapt instead of starting from scratch.