Philterd

Contact Us

Tell us about your stack and the privacy problems you're trying to solve. We typically respond within one business day.

Prefer email? support@philterd.ai

Please do not enter PII or PHI in this form. If you need to share an example, use a sanitized one.

← All posts

Redact Both Sides of the LLM: Inputs and Outputs

By Philter · Redaction

Most teams that worry about PII and LLMs do one thing: they scrub the prompt before sending it to the model. That is necessary, and it is not enough. Prompt redaction and response redaction defend against different threats, and neither one covers for the other. If you only do one, you have a gap you probably cannot see.

This is the second post in a three-part series on PII redaction around LLMs. The first post made the case that models memorize training data and you cannot reliably un-train it. This one is about the runtime: once the model is deployed, where does redaction have to happen. The answer is both sides, and here is the way to think about it.

Inputs protect the provider boundary

When you redact the prompt, you are controlling what leaves your network. A clean prompt means no personal data lands in the provider’s logs, retention windows, training pipelines, or subprocessors, and it keeps the LLM call out of scope for a lot of your compliance obligations. This is the data-sovereignty case: sensitive data should not cross a boundary it does not need to cross.

That is the half most teams already understand, and we covered the mechanics of it in how to redact PII before sending prompts to an LLM . It is also where provider-side controls fall short, because a provider’s own guardrails run after your data has already arrived , which is the wrong side of the boundary to be useful for this.

But notice what input redaction does not address: it says nothing about what comes back.

Outputs protect everything downstream

A response can contain PII that the prompt never did. The model is not just echoing your input; it is generating, and it can generate personal data from several sources:

  • Memorized training data. Large models memorize and can emit verbatim sequences from their training set, including real names, emails, and phone numbers. (That is the whole subject of part one : it is demonstrated, and you cannot reliably un-train it.)
  • Retrieved context (RAG). If your application pulls documents into the prompt, those documents may carry PII straight into the answer, even when the user’s question was harmless.
  • Cross-context or cross-tenant leakage. State, caches, and shared context can surface one user’s data in another user’s session.

And the response does not just go to a person. It flows into logs, downstream tools, analytics, and other services, any of which may not be authorized to hold that data. Redacting the prompt does nothing for any of this. Only scanning the output does.

So the two sides are not redundant. Input redaction keeps your data in. Output scanning keeps the model’s data from getting out to the wrong place.

Both-sides airlock diagram. Inside your network, an application sends a prompt containing PII to the Philter AI Proxy, which redacts it to a pseudonymized prompt before it crosses the network boundary to the LLM provider. The provider's response returns through the proxy, which scans it, before reaching the application. The provider only ever sees pseudonyms.
Redact on the way in, scan on the way back. The provider only ever sees pseudonymized text.

The hard part: keeping the answer useful

The reason people stop at prompt redaction is that doing it well is awkward. If you replace every name with [REDACTED], the model loses the thread and the answer gets worse. Two techniques make both-sides redaction practical:

  • Consistent pseudonymization. Replace Jane Doe with a stable stand-in like [PERSON_1] everywhere she appears, so the model can still reason about who did what without ever seeing the real value. The relationships in the text survive (this is referential integrity); the identity does not.
  • Re-identification inside your boundary. For the authorized user, the real values can be restored on the response. Re-identification is a governed, audited operation your own application performs through Philter, with the keys and mapping held entirely inside your network, and it applies to the reversible encryption strategies. It is value-by-value and deliberate, not an automatic whole-document round-trip the provider could ever participate in. The provider only ever saw the pseudonyms.

That is the difference between a redaction toy and a redaction system: the conversation stays coherent for the model, the real answer is recoverable by the right person inside your perimeter, and the provider only ever sees tokens. You are responsible for validating that the detection holds on your own data, but the boundary property does not depend on the detector being perfect: a value that was pseudonymized before it left simply was not there to leak.

Defense in depth, honestly

Both-sides redaction is one layer of three, and each layer catches what the others miss:

  1. Redact the training data so memorized PII is never there to leak (part one).
  2. Redact the inputs so PII does not cross the provider boundary.
  3. Scan the outputs so PII the model introduces does not reach the wrong place.

And we should be honest about the limits: output scanning is a safety net, not a guarantee. No detector has perfect recall, which is exactly why you layer it rather than rely on any single control, and why you validate it against your own data rather than trusting a number from a vendor. Anyone selling you a one-step “make the LLM safe” button is overstating what is possible. The point of defense in depth is that the layers are independent, so a miss in one is not a miss in all.

How Philterd does it

This is the design of the Philter AI Proxy : a drop-in airlock that sits between your application and the LLM provider. Prompts are redacted on the way in, responses are scanned on the way back, and the provider only ever sees pseudonymized text. It speaks OpenAI, Anthropic Claude, Amazon Bedrock, Gemini, Ollama, and OpenAI-compatible wire protocols, so the both-sides design does not depend on which model you use, and pointing your client at it is a base-URL change rather than a rewrite.

For agent and developer workflows, Philter MCP brings the same redaction inside the tool loop. Tool-use and function-calling are their own PII surface on both sides: the arguments your model hands to a tool can carry personal data outward, and the results a tool returns can carry it back into the context. An MCP layer that redacts those calls treats the tool boundary the same way the proxy treats the provider boundary.

Underneath both, the redaction is Phileas and Philter : the same open source, self-hosted engine doing consistent pseudonymization and governed re-identification, running entirely in your environment. The detection models themselves were trained on synthetic and public data, so the privacy layer is not built by absorbing someone else’s records.

The takeaway

If you are putting an LLM in front of, or behind, anything sensitive, redact both sides. Input redaction sets your compliance boundary by keeping data off the provider. Output scanning protects your users, logs, and tools from PII the model generates on its own, from memorized data, from retrieved context, or from another tenant. The prompt is only half the problem.

The final post in this series puts the training-data half into practice: the clean-room process for turning raw, identifier-laden data into a training set you can measure before it leaves the room.

If you want the both-sides airlock in front of your own LLM traffic, start with the Philter AI Proxy , see the broader AI and LLM use case , or get in touch.