Designing a HIPAA Architecture on AWS
Building a HIPAA workload on AWS feels like a solved problem until your first audit. The cloud is HIPAA-eligible, you signed the AWS Business Associate Addendum, you turned on encryption, and the architecture diagram looks clean. Then someone asks where the patient text goes after it leaves the database, and the clean diagram develops a long tail: an analytics warehouse, a few vendor SaaS tools, a staging environment, and an LLM call to a model AWS never agreed to cover.
AWS publishes a careful, accurate page on what it offers here . It is worth reading in full, because the most important thing on it is what it does not promise. The AWS BAA covers the infrastructure surface. It does not de-identify your data, it does not extend to vendors outside AWS, and it does not put third-party model providers under your compliance umbrella. This post walks the architecture that fills those gaps, with de-identification as the load-bearing idea: the cleanest way to make a workload HIPAA-compliant is to make sure it no longer contains PHI.
What the AWS BAA actually covers
When you accept the AWS Business Associate Addendum (in AWS Artifact), you bring AWS into a business-associate relationship for the subset of services AWS designates as HIPAA-eligible. AWS maintains the HIPAA-eligible services list , and the practical rule is simple: PHI may only be processed, stored, or transmitted by services on that list, in the configurations AWS specifies.
Two clarifications people miss:
- “HIPAA-eligible” is not “HIPAA-compliant.” Eligibility means a service can be used in a compliant architecture if you configure it correctly. Compliance is your responsibility. The shared-responsibility model does not bend for healthcare.
- The BAA covers AWS, and only AWS. It says nothing about the third parties your application talks to. Every other vendor in the data path needs its own BAA, or the data reaching it must no longer be PHI.
The BAA is the floor, not the architecture. The architecture is what you build on top of it.
The boring foundation
Before anything clever, the unglamorous controls have to be in place, because these are the ones an auditor checks first:
- Network isolation. PHI-handling workloads sit in a VPC with private subnets, no public ingress to data stores, and VPC endpoints so traffic to eligible services stays off the public internet.
- Encryption at rest and in transit. KMS-managed keys (customer-managed where you need control over rotation and access policy) for every store that holds PHI, and TLS everywhere in transit. This is a hard requirement, not a recommendation.
- Audit logging. CloudTrail for control-plane activity, access logs on data stores, and a retention policy that survives an OCR investigation. You want to be able to answer “who touched this record, and when” without reconstructing it from memory.
- No PHI in logs or metrics. This is the control teams forget. An application log line, a CloudWatch metric dimension, an error message with a patient name in it: each is a copy of PHI in a place that was never scoped for it. Treat log and metric pipelines as data stores that must never receive PHI.
None of this is novel. It is the table stakes that everything below assumes is already true.
Where the eligible-services list quietly hurts you
The eligible-services list looks like a green light, but the fine print is where architectures go wrong:
- Region matters. Eligibility is frequently scoped to specific AWS Regions. A service that is eligible in one Region may not be in another. Always verify the Region you actually deploy in.
- Feature flags can take a service out of scope. A service can be HIPAA-eligible while a specific feature, integration, or model within it is not. Bedrock is the sharpest example, and it gets its own section below.
- The list changes. Services are added over time. The version you validated against at design time is not necessarily the version that governs you at audit time. Re-verify on a cadence.
The practical habit: for every service in your PHI path, confirm against the current AWS documentation that the service, the Region, and the specific configuration you use are all eligible. Write that verification down. It is the artifact you will want when someone asks.
De-identification as scope reduction
Here is the lever that the AWS page cannot pull for you. Every control above is about handling PHI carefully. De-identification is about not having PHI at all.
HIPAA Safe Harbor (45 CFR 164.514(b)) defines a de-identification standard: remove eighteen categories of identifiers, and the resulting data is no longer considered PHI. Data that is not PHI is not in HIPAA scope. That is the whole game. An analytics warehouse full of de-identified records does not need a BAA, does not need the same access controls, and does not expand your audit surface, because there is no PHI in it to protect.
The operational pattern is to run de-identification as a stage in the pipeline, before data lands anywhere outside the PHI perimeter. Philter and Phileas exist to make that stage real and automated rather than a script an analyst runs by hand. You apply a Safe Harbor policy as data flows from the raw, PHI-bearing zone into the de-identified zone, and downstream consumers only ever read the de-identified side.
We wrote a full piece on mapping each of the eighteen identifiers to specific capabilities and wiring them into healthcare pipelines: Automating HIPAA Safe Harbor: A Blueprint for Healthcare Data Pipelines . That post is the detailed companion to this one. Read it for the identifier-by-identifier mapping; read this one for where it fits in an AWS architecture.
The LLM blind spot
This is the new HIPAA problem, and it is the one most teams have not accounted for yet.
Hosted LLM providers are not on the AWS BAA. OpenAI’s and Anthropic’s direct APIs are not covered by your AWS BAA regardless of where your application runs. Calling them from an EC2 instance, a Lambda function, or an ECS task inside your HIPAA-scoped VPC does not extend any AWS coverage to them. The AWS BAA governs AWS services. A request that leaves AWS to reach a third-party model API is outside it entirely.
Amazon Bedrock is the partial answer AWS offers, and the nuance matters: Bedrock is HIPAA-eligible, but eligibility applies to specific models in specific Regions. Hosting a model through Bedrock is not the same as the model provider’s direct API being covered, and not every model available through Bedrock carries the same status. Verify the exact model and Region against the AWS documentation before you assume coverage.
So you have two clean options for using an LLM on PHI-adjacent workflows: use a Bedrock-eligible model in an eligible Region under your AWS BAA, or send the model only de-identified text so the question of coverage no longer applies. The second option is more flexible, because it lets you use whichever model is best for the task without making each provider a business associate.
That is exactly what the Philter AI Proxy does. It sits inline between your application and the model provider as an OpenAI-compatible endpoint. You point your existing client at the proxy URL instead of at the provider, and it redacts PHI out of the prompt before the request leaves your perimeter. The application code does not change. The model never receives PHI, so whether that model is on a BAA stops being the question that blocks the project. This is the difference between “we cannot use that model because of HIPAA” and “we can use any model, because no PHI reaches it.”
Reference architecture
Putting it together, here is the shape of a PHI pipeline on AWS with the de-identification boundary drawn explicitly. Everything above the boundary handles PHI and needs the full set of controls and BAAs. Everything below it handles de-identified data and is out of HIPAA scope.
Ingest (EHR / HL7 / FHIR feeds, scanned docs, intake forms)
│
▼
┌───────────────────────────────────────────────────────────────┐
│ HIPAA SCOPE (AWS BAA in place, eligible services only) │
│ │
│ Raw zone (S3, KMS-encrypted, locked to compliance) │
│ │ │
│ ▼ │
│ ┌─────────────────────────────┐ │
│ │ Philter (Safe Harbor │ │
│ │ policy, 45 CFR 164.514(b)) │ │
│ └──────────────┬──────────────┘ │
│ │ │
└───────────────────────────┼─────────────────────────────────────┘
de-identification boundary (data below is not PHI)
┌───────────────────────────┼─────────────────────────────────────┐
│ OUT OF HIPAA SCOPE (de-identified data only) │
│ │ │
│ ▼ │
│ De-identified zone (S3 / warehouse) │
│ │ │ │
│ ▼ ▼ │
│ Analytics warehouse Downstream consumers │
│ (Redshift, BI, ML) (vendor SaaS, dev/test) │
└─────────────────────────────────────────────────────────────────┘
Live LLM requests (still touching PHI-adjacent text):
Application ──▶ Philter AI Proxy ──▶ LLM provider
(redacts PHI (Bedrock model,
inline, in-perimeter) OpenAI, Anthropic)
Two things to read off the diagram. First, Philter is the only consumer of the raw zone that writes to the de-identified zone; ACLs prevent any other read path, so de-identified and PHI stores stay cleanly separated. Second, the LLM path gets its own inline redaction stage through the proxy, because real-time prompts are a separate flow from the batch pipeline and need their own boundary.
What to ask your auditor
A short checklist to pressure-test the architecture before someone else does:
- Is the AWS BAA accepted in AWS Artifact, and does PHI flow only through HIPAA-eligible services in eligible Regions?
- Is there a BAA in place with every other vendor in the PHI path, or is the data de-identified before it reaches them?
- Are de-identified data stores clearly separated from PHI stores, with access controls that enforce the boundary rather than relying on convention?
- Is PHI absent from logs and metrics, including application logs, error messages, and metric dimensions?
- For every LLM call, is the model on a BAA (a Bedrock-eligible model in an eligible Region), or does only de-identified text reach the model?
If the answer to any of these is “we think so,” that is the place to start. Our broader take on building defensibly is on the compliance page .
Where this fits
The AWS BAA gets you a compliant place to run. It does not get you out of HIPAA scope, and it does not follow your data to the warehouse, the vendor, or the model. De-identification is what does both. Wire Safe Harbor de-identification into the pipeline as an automated stage, draw the boundary explicitly, and keep PHI out of every place that does not strictly need it.
Healthcare and life-sciences teams are our largest consulting practice . We have done these deployments across health systems, clinical research organizations, and pharma, and the AWS architecture review is one of the most common ways teams start.