New: Philter One, a redaction box for your office, is coming soon. Learn more →

Contact Us

Tell us about your stack and the privacy problems you're trying to solve. We typically respond within one business day.

Prefer email? support@philterd.ai

Please do not enter PII or PHI in this form. If you need to share an example, use a sanitized one.

Redact PII before it reaches the LLM

Philter AI Proxy

Philter AI Proxy redacts the PII and PHI it detects in your large language model (LLM) prompts before they leave your network, and can scan responses on the way back. Point your SDK at it instead of the provider.

What the LLM provider actually sees

Prompts routinely carry names, emails, and account numbers. Philter AI Proxy redacts what its policy detects before the request leaves your network, so the provider receives the redacted text instead. Detection is probabilistic, so validate the results against your own prompts.

Your application sends

Summarize this ticket: customer Margaret CollinsPERSON (mcollins@example.comEMAIL, SSN 123-45-6789SSN) was charged twice.

The LLM provider receives

Summarize this ticket: customer {{{REDACTED-name}}} ({{{REDACTED-email-address}}}, SSN {{{REDACTED-ssn}}}) was charged twice.

Responses can be scanned on the way back, streamed responses included. Your code changes by one URL and one header.

Run it with Docker

The proxy ships as an image on Docker Hub at philterd/philter-ai-proxy, with the model it uses to find person names included. It needs no other service.

# 1. Generate the keys. Keep the encryption key: the database cannot be read without it.
export ENCRYPTION_KEY="$(openssl rand -hex 32)"
export BOOTSTRAP_API_KEY="$(openssl rand -hex 32)"

# 2. Run it, with a volume for its database
docker run -d --name philter-ai-proxy -p 8080:8080 \
  -v philter-ai-proxy-data:/var/lib/philter-ai-proxy \
  -e PHILTER_AI_PROXY_ENCRYPTION_KEY="$ENCRYPTION_KEY" \
  -e PHILTER_AI_PROXY_BOOTSTRAP_API_KEY="$BOOTSTRAP_API_KEY" \
  philterd/philter-ai-proxy:1.0.0

# 3. Create a key for your application
curl -k https://localhost:8080/admin/api/keys \
  -H "Authorization: Bearer $BOOTSTRAP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-app", "roles": ["proxy"]}'

The image serves HTTPS with a self-signed certificate until you configure your own. Every option is in the installation guide.

The LLM privacy layer your stack is missing

Drop-in for major providers

Speaks the OpenAI, Azure OpenAI, Anthropic, Gemini, Vertex AI, Ollama, and Amazon Bedrock APIs, plus OpenAI-compatible providers such as Mistral, vLLM, or LM Studio. Chat, Responses, embeddings, and token counting are redacted, and your SDKs keep working with a new base URL and one header.

Redaction inside the proxy

The proxy runs the Phileas redaction engine in-process, with a PhEye model for person names, so prompts are redacted without a call to another service. Names, SSNs, emails, account numbers: replaced according to your policy.

Responses scanned, streams included

Turn on response scanning per route to redact what the model sends back, block the response, or flag it. Streamed responses are scanned as they arrive, including tool-call arguments.

Authentication and access control

Every request needs an API key, and mutual TLS can be required too. Per-key scopes on providers, models, and paths, and per-key policy binding, give different clients different access without running separate proxies.

Philter-compatible policies

Policies are standard Phileas policies, managed through an admin API with the same paths as Philter's. Build them in the Policy Editor and use them in either.

Self-hosted

Runs inside your perimeter, as the last hop before LLM-bound traffic leaves your network and the first hop on the way back. Philterd never sees your prompts.

Observability & operations

The proxy exposes the signals your platform team needs to operate it.

Prometheus metrics

A /metrics endpoint exposes request counts, request and redaction latency, values found by type, and provider and redaction error counts, labeled by provider. A Grafana dashboard ships in the repository.

Structured audit log

Every request is recorded as a JSON entry: provider, model, policy, the API key's ID, how many values of each type were found in the request and response, status, latency, client IP, and trace ID. Never the text of a prompt or response.

Health endpoints

/livez and /readyz are liveness and readiness checks for your orchestrator or load balancer, with readiness waiting until the detection model is loaded. /health serves the standard Philterd health response for a load balancer or uptime monitor.

Frequently asked questions

If something here isn’t covered, get in touch and we’ll answer.

Which LLM providers does the proxy work with?
It speaks the OpenAI, Azure OpenAI, Anthropic Claude, Google Gemini, Vertex AI, Ollama, and Amazon Bedrock APIs, plus OpenAI-compatible providers such as Mistral, vLLM, or LM Studio. Switching providers is a configuration change, not a code change.
Do I need to run Philter alongside the proxy?
No. Since version 1.0 the proxy redacts in-process with Phileas, the library under Philter, and the Docker image includes the model it uses to find person names. Policies use the same format as Philter's, so a policy works in both, except that the proxy refuses a policy that names a PhEye service, generators, or files and environment variables the deployment has not allowed.
Do I have to change my application code?
Only its configuration. Point your existing SDK at the proxy's base URL and send your proxy key in the x-philter-proxy-key header. The same OpenAI or Anthropic client keeps working; the proxy redacts each prompt and forwards it to the provider.
Does the proxy redact model responses too, or only prompts?
Both, but not the same way. Every prompt is redacted on the way out, before it leaves your network. Scanning responses on the way back is turned on per route: the proxy can redact, block, or flag what it finds. Streamed responses are scanned as they arrive, holding back a little text so a value split across fragments is still scanned whole.
Does the proxy redact images, audio, or files?
No. The proxy redacts text. A request carrying images, audio, video, or files is refused by default, and a route can be set to forward such requests with the media unredacted. A request to an endpoint the proxy does not know how to redact is refused too, rather than forwarded unredacted.
What does the proxy use to detect and redact PII?
Phileas policies, run inside the proxy: patterns and validators for identifiers such as SSNs, card numbers, and email addresses, and a PhEye model for person names. Policies are managed through an admin API with the same paths as Philter's. Detection is probabilistic, so validate the results against your own data.
How does the proxy map to SOC 2 and HIPAA controls?
Its features line up with specific SOC 2 Trust Services Criteria and HIPAA Security Rule safeguards: API keys and mTLS for access control, the audit log for audit controls, TLS for transmission security, and redaction itself for confidentiality and minimum-necessary handling. The SOC 2 and HIPAA control mapping guide lays out the full mapping and the configuration steps to support your own attestations.
Does the proxy log prompt or response content?
No. The audit log records metadata only: provider, model, policy, the API key's ID, how many values of each type were found, status, latency, client IP, and a trace ID. The text of a prompt or response is never written to it. (Client IPs may be personal data under GDPR, so govern the log accordingly.) When you configure a context to keep replacements consistent across requests, the replacements are stored in the proxy's database, encrypted, and found by a keyed hash of each original value; the original values are not stored.
How does the proxy fit with an AI gateway like LiteLLM or Portkey?
Philter AI Proxy is a redaction layer, not an AI gateway. It does redaction and the audit trail that goes with it, and it deliberately leaves provider routing, failover, rate limiting, response caching, token quotas, and per-tenant billing to a gateway such as LiteLLM, Portkey, or Kong AI Gateway. The proxy is built to run alongside one, on the last hop before traffic leaves your network, so whatever the gateway routes has already been redacted. The Using with an AI Gateway guide covers the deployment pattern.
Is Philter AI Proxy open source?
Yes. Philter AI Proxy is open source under the Apache License, version 2, and the code is on GitHub.

Ready to use Philter AI Proxy?

Three ways to get going: deploy the open source yourself, spin it up from a cloud marketplace, or work with our team directly. Pick the path that fits.

View on GitHub