---
title: "Philter AI Proxy"
description: "Self-hosted proxy that redacts PII and PHI from prompts before they reach OpenAI, Anthropic, Gemini, Bedrock, Ollama, or OpenAI-compatible APIs, and scans responses on the way back."
url: "https://philterd.ai/philter-ai-proxy/"
---

Redact PII before it reaches the LLM

# Philter AI Proxy

Philter AI Proxy redacts the PII and PHI it detects in your large language model (LLM) prompts before they leave your network, and can scan responses on the way back. Point your SDK at it instead of the provider.

[Run it with Docker](https://philterd.ai/dev/start/philter-ai-proxy/)

[Star: 7](https://github.com/philterd/philter-ai-proxy)

Change one URL. Keep PII out of your LLM prompts.

```
# Before
client = OpenAI(api_key=KEY)

# After: point at the proxy and send a proxy key
client = OpenAI(
    api_key=KEY,
    base_url="https://philter-ai-proxy.internal:8080/v1",
    default_headers={"x-philter-proxy-key": PROXY_KEY},
)
# Same OpenAI client. Prompts redacted before they leave.
```

The proxy redacts each prompt, then forwards it with your provider credential.

[Documentation →](https://philterd.github.io/philter-ai-proxy/) [Release Notes →](https://github.com/philterd/philter-ai-proxy/releases) [GitHub →](https://github.com/philterd/philter-ai-proxy)

## What the LLM provider actually sees

Prompts routinely carry names, emails, and account numbers. Philter AI Proxy redacts what its policy detects before the request leaves your network, so the provider receives the redacted text instead. Detection is probabilistic, so validate the results against your own prompts.

Your application sends

Summarize this ticket: customer Margaret CollinsPERSON (mcollins@example.comEMAIL, SSN 123-45-6789SSN) was charged twice.

The LLM provider receives

Summarize this ticket: customer {{{REDACTED-name}}} ({{{REDACTED-email-address}}}, SSN {{{REDACTED-ssn}}}) was charged twice.

Responses can be scanned on the way back, streamed responses included. Your code changes by one URL and one header.

## Run it with Docker

The proxy ships as an image on Docker Hub at [philterd/philter-ai-proxy](https://hub.docker.com/r/philterd/philter-ai-proxy), with the model it uses to find person names included. It needs no other service.

```
# 1. Generate the keys. Keep the encryption key: the database cannot be read without it.
export ENCRYPTION_KEY="$(openssl rand -hex 32)"
export BOOTSTRAP_API_KEY="$(openssl rand -hex 32)"

# 2. Run it, with a volume for its database
docker run -d --name philter-ai-proxy -p 8080:8080 \
  -v philter-ai-proxy-data:/var/lib/philter-ai-proxy \
  -e PHILTER_AI_PROXY_ENCRYPTION_KEY="$ENCRYPTION_KEY" \
  -e PHILTER_AI_PROXY_BOOTSTRAP_API_KEY="$BOOTSTRAP_API_KEY" \
  philterd/philter-ai-proxy:1.0.0

# 3. Create a key for your application
curl -k https://localhost:8080/admin/api/keys \
  -H "Authorization: Bearer $BOOTSTRAP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-app", "roles": ["proxy"]}'
```

The image serves HTTPS with a self-signed certificate until you configure your own. Every option is in the [installation guide](https://philterd.github.io/philter-ai-proxy/installation/).

## The LLM privacy layer your stack is missing

### Drop-in for major providers

Speaks the OpenAI, Azure OpenAI, Anthropic, Gemini, Vertex AI, Ollama, and Amazon Bedrock APIs, plus OpenAI-compatible providers such as Mistral, vLLM, or LM Studio. Chat, Responses, embeddings, and token counting are redacted, and your SDKs keep working with a new base URL and one header.

### Redaction inside the proxy

The proxy runs the [Phileas](https://philterd.ai/phileas/) redaction engine in-process, with a [PhEye](https://philterd.ai/ph-eye/) model for person names, so prompts are redacted without a call to another service. Names, SSNs, emails, account numbers: replaced according to your policy.

### Responses scanned, streams included

Turn on response scanning per route to redact what the model sends back, block the response, or flag it. Streamed responses are scanned as they arrive, including tool-call arguments.

### Authentication and access control

Every request needs an API key, and mutual TLS can be required too. Per-key scopes on providers, models, and paths, and per-key policy binding, give different clients different access without running separate proxies.

### Philter-compatible policies

Policies are standard [Phileas](https://philterd.ai/phileas/) policies, managed through an admin API with the same paths as [Philter](https://philterd.ai/philter/)'s. Build them in the [Policy Editor](https://policies.philterd.ai) and use them in either.

### Self-hosted

Runs inside your perimeter, as the last hop before LLM-bound traffic leaves your network and the first hop on the way back. Philterd never sees your prompts.

## Observability & operations

The proxy exposes the signals your platform team needs to operate it.

### Prometheus metrics

A `/metrics` endpoint exposes request counts, request and redaction latency, values found by type, and provider and redaction error counts, labeled by provider. A Grafana dashboard ships in the repository.

### Structured audit log

Every request is recorded as a JSON entry: provider, model, policy, the API key's ID, how many values of each type were found in the request and response, status, latency, client IP, and trace ID. Never the text of a prompt or response.

### Health endpoints

`/livez` and `/readyz` are liveness and readiness checks for your orchestrator or load balancer, with readiness waiting until the detection model is loaded. `/health` serves the standard Philterd health response for a load balancer or uptime monitor.

## Frequently asked questions

If something here isn’t covered, [get in touch](https://philterd.ai/philter-ai-proxy/#) and we’ll answer.

Which LLM providers does the proxy work with?

It speaks the OpenAI, Azure OpenAI, Anthropic Claude, Google Gemini, Vertex AI, Ollama, and Amazon Bedrock APIs, plus OpenAI-compatible providers such as Mistral, vLLM, or LM Studio. Switching providers is a configuration change, not a code change.

Do I need to run Philter alongside the proxy?

No. Since version 1.0 the proxy redacts in-process with [Phileas](https://philterd.ai/phileas/), the library under [Philter](https://philterd.ai/philter/), and the Docker image includes the model it uses to find person names. Policies use the same format as Philter's, so a policy works in both, except that the proxy refuses a policy that names a PhEye service, generators, or files and environment variables the deployment has not allowed.

Do I have to change my application code?

Only its configuration. Point your existing SDK at the proxy's base URL and send your proxy key in the `x-philter-proxy-key` header. The same OpenAI or Anthropic client keeps working; the proxy redacts each prompt and forwards it to the provider.

Does the proxy redact model responses too, or only prompts?

Both, but not the same way. Every prompt is redacted on the way out, before it leaves your network. Scanning responses on the way back is turned on per route: the proxy can redact, block, or flag what it finds. Streamed responses are scanned as they arrive, holding back a little text so a value split across fragments is still scanned whole.

Does the proxy redact images, audio, or files?

No. The proxy redacts text. A request carrying images, audio, video, or files is refused by default, and a route can be set to forward such requests with the media unredacted. A request to an endpoint the proxy does not know how to redact is refused too, rather than forwarded unredacted.

What does the proxy use to detect and redact PII?

[Phileas](https://philterd.ai/phileas/) policies, run inside the proxy: patterns and validators for identifiers such as SSNs, card numbers, and email addresses, and a [PhEye](https://philterd.ai/ph-eye/) model for person names. Policies are managed through an admin API with the same paths as [Philter](https://philterd.ai/philter/)'s. Detection is probabilistic, so validate the results against your own data.

How does the proxy map to SOC 2 and HIPAA controls?

Its features line up with specific SOC 2 Trust Services Criteria and HIPAA Security Rule safeguards: API keys and mTLS for access control, the audit log for audit controls, TLS for transmission security, and redaction itself for confidentiality and minimum-necessary handling. The [SOC 2 and HIPAA control mapping guide](https://philterd.ai/guides/soc-2-and-hipaa-control-mapping-for-philter-ai-proxy/) lays out the full mapping and the configuration steps to support your own attestations.

Does the proxy log prompt or response content?

No. The audit log records metadata only: provider, model, policy, the API key's ID, how many values of each type were found, status, latency, client IP, and a trace ID. The text of a prompt or response is never written to it. (Client IPs may be personal data under GDPR, so govern the log accordingly.) When you configure a context to keep replacements consistent across requests, the replacements are stored in the proxy's database, encrypted, and found by a keyed hash of each original value; the original values are not stored.

How does the proxy fit with an AI gateway like LiteLLM or Portkey?

Philter AI Proxy is a redaction layer, not an AI gateway. It does redaction and the audit trail that goes with it, and it deliberately leaves provider routing, failover, rate limiting, response caching, token quotas, and per-tenant billing to a gateway such as LiteLLM, Portkey, or Kong AI Gateway. The proxy is built to run alongside one, on the last hop before traffic leaves your network, so whatever the gateway routes has already been redacted. The [Using with an AI Gateway](https://philterd.github.io/philter-ai-proxy/ai-gateway/) guide covers the deployment pattern.

Is Philter AI Proxy open source?

Yes. Philter AI Proxy is open source under the Apache License, version 2, and the code is on [GitHub](https://github.com/philterd/philter-ai-proxy).

## Related products

- [Phileas: The redaction library the proxy runs in-process. Its policies are the same ones the proxy uses.](https://philterd.ai/phileas/)
- [Philter: The redaction API for everything that is not LLM traffic. Policies move between Philter and the proxy unchanged.](https://philterd.ai/philter/)
- [Philter MCP: Bring redaction into agent and dev workflows while the proxy guards production.](https://philterd.ai/philter-mcp/)

## Ready to use Philter AI Proxy?

Three ways to get going: deploy the open source yourself, spin it up from a cloud marketplace, or work with our team directly. Pick the path that fits.

[View on GitHub](https://github.com/philterd/philter-ai-proxy)
