Drop-in for major providers
Speaks the OpenAI, Anthropic, and Amazon Bedrock wire protocols, plus any OpenAI-compatible provider such as Mistral, Cohere, vLLM, or LM Studio. Your existing SDKs don't need a single line changed; only the base URL.
Redact PII before it reaches the LLM
Philter AI Proxy
Philter AI Proxy sits between your application and LLM providers including OpenAI, Anthropic Claude, Amazon Bedrock, and OpenAI-compatible providers like Mistral, Cohere, and vLLM. Prompts get redacted before they leave your network; responses get scanned on the way back.
The LLM privacy layer your stack is missing
Inbound prompt redaction
Strips PII and PHI from prompts before they're forwarded to the model. Names, SSNs, MRNs, account numbers: all replaced according to your policy.
Authentication and access control
API key authentication and mutual TLS (mTLS) are both supported, independently or together. Per-key rate limiting and per-key policy overrides let you grant different access levels to different clients without running separate proxy instances.
Audit log for compliance
Every redaction is logged with timestamp, entity type, and direction (in/out). The exact paper trail HIPAA and GDPR auditors expect for AI workloads.
Policy-driven
Uses the same Phileas policies as the rest of the Philterd toolkit. Define once; apply across redaction, discovery, monitoring, and now AI traffic.
Self-hosted
Runs inside your perimeter. The proxy is the last hop before LLM-bound traffic leaves your network, and the first hop on the way back.
Observability & operations
Production-ready from day one. The proxy exposes the signals your platform team needs to operate it with confidence.
Prometheus metrics
A /metrics endpoint exposes request counts, redaction latency, token usage (prompt and completion), and error rates, all labeled by provider and model. Drop it into your existing Grafana stack without custom instrumentation.
Structured audit logs
Every request is written as a JSONL record: entity types redacted, direction (inbound or outbound), model, policy, document ID, latency, client IP, and HTTP status. The exact audit trail HIPAA and SOC 2 reviewers expect for AI workloads.
Health endpoint
/health checks Philter backend reachability and returns structured JSON. Wire it into your load balancer, Kubernetes liveness probe, or uptime monitor to catch backend connectivity issues before they affect clients.
Ready to use Philter AI Proxy?
Three ways to get going: deploy the open source yourself, spin it up from a cloud marketplace, or work with our team directly. Pick the path that fits.