Drop-in for major providers
Speaks the OpenAI, Anthropic, and Amazon Bedrock wire protocols, plus any OpenAI-compatible provider such as Mistral, Cohere, vLLM, or LM Studio. Your existing SDKs don't need a single line changed; only the base URL.
Redact PII before it reaches the LLM
Philter AI Proxy: Redact PII Before It Reaches the LLM
Philter AI Proxy is an airlock for your large language model (LLM) traffic: prompts pass through to the model, but sensitive data never crosses your network boundary. It sits between your application and LLM providers for both generative AI and RAG, including OpenAI, Anthropic Claude, Amazon Bedrock, and OpenAI-compatible providers like Mistral, Cohere, and vLLM. Prompts get redacted before they leave your network; responses get scanned on the way back.
The LLM privacy layer your stack is missing
Inbound prompt redaction
Strips PII and PHI from prompts before they're forwarded to the model. Names, SSNs, MRNs, account numbers: all replaced according to your policy.
Authentication and access control
API key authentication and mutual TLS (mTLS) are both supported, independently or together. Per-key rate limiting and per-key policy overrides let you grant different access levels to different clients without running separate proxy instances.
Audit log for compliance
Every redaction is logged with timestamp, entity type, and direction (in/out). The exact paper trail HIPAA and GDPR auditors expect for AI workloads.
Policy-driven
Uses the same Phileas policies as the rest of the Philterd toolkit. Define once; apply across redaction, discovery, monitoring, and now AI traffic.
Self-hosted
Runs inside your perimeter. The proxy is the last hop before LLM-bound traffic leaves your network, and the first hop on the way back: the airlock that sensitive data cannot cross.
Observability & operations
Production-ready from day one. The proxy exposes the signals your platform team needs to operate it with confidence.
Prometheus metrics
A /metrics endpoint exposes request counts, redaction latency, token usage (prompt and completion), and error rates, all labeled by provider and model. Drop it into your existing Grafana stack without custom instrumentation.
Structured audit logs
Every request is written as a JSONL record: entity types redacted, direction (inbound or outbound), model, policy, document ID, latency, client IP, and HTTP status. The exact audit trail HIPAA and SOC 2 reviewers expect for AI workloads.
Health endpoint
/health checks Philter backend reachability and returns structured JSON. Wire it into your load balancer, Kubernetes liveness probe, or uptime monitor to catch backend connectivity issues before they affect clients.
Run it with Docker
The proxy ships as a multi-arch image on Docker Hub at philterd/philter-ai-proxy. Pull it, point a config file at your Philter instance and chosen provider, and run.
# 1. Pull the image from Docker Hub
docker pull philterd/philter-ai-proxy
# 2. Grab a starting config and edit it for your Philter and provider
curl -O https://raw.githubusercontent.com/philterd/philter-ai-proxy/main/config.example.yaml
mv config.example.yaml config.yaml
# 3. Run it: expose port 8080 and mount your config
docker run -p 8080:8080 \
-v "$(pwd)/config.yaml:/app/config.yaml:ro" \
-e PHILTER_PROXY_CONFIG=/app/config.yaml \
philterd/philter-ai-proxy
Prefer Compose? The repo ships a docker-compose.yaml: clone it and run docker compose up. Every configuration option is documented in the installation guide.
Frequently asked questions
If something here isn’t covered, get in touch and we’ll answer.
Which LLM providers does the proxy work with?
It speaks the OpenAI, Anthropic Claude, and Amazon Bedrock wire protocols, plus any OpenAI-compatible provider such as Mistral, Cohere, vLLM, or LM Studio. Switching providers is a configuration change, not a code change.
Do I have to change my application code?
No. Point your existing SDK at the proxy's base URL and everything else stays the same. The same OpenAI or Anthropic client keeps working; the proxy redacts each prompt and forwards it to your configured provider.
Does the proxy redact model responses too, or only prompts?
Both. Prompts are redacted on the way out, before they leave your network, and responses are scanned on the way back. The proxy is the last hop before LLM-bound traffic leaves your perimeter and the first hop on the way back.
What does the proxy use to detect and redact PII?
How does the proxy map to SOC 2 and HIPAA controls?
Its features line up with specific SOC 2 Trust Services Criteria and HIPAA Security Rule safeguards: authentication and mTLS for access control, the audit log for audit controls, TLS everywhere for transmission security, and redaction itself for confidentiality and minimum-necessary handling. The SOC 2 and HIPAA control mapping guide lays out the full mapping and the configuration steps to support your own attestations.
Does the proxy log prompt or response content?
No. The audit log records metadata only: provider, model, the entity types redacted, counts, direction, latency, client IP, document ID, HTTP status, and a trace ID. Message content and filtered text are never written to the log. (Client IPs may be personal data under GDPR, so govern the log accordingly.)
Do we need Philterd to sign a BAA to use the proxy?
No. The proxy is self-hosted and runs inside your perimeter, so Philterd never creates, receives, maintains, or transmits your PHI and is not a business associate. Redacting PHI before prompts reach the LLM provider can also keep that provider off your own BAA chain. See Why Philterd Doesn't Sign a BAA.
Ready to use Philter AI Proxy: Redact PII Before It Reaches the LLM?
Three ways to get going: deploy the open source yourself, spin it up from a cloud marketplace, or work with our team directly. Pick the path that fits.