HIPAA-Compliant LLM Options Compared (2026): Azure OpenAI, Google, Private Deployment
The realistic HIPAA-compliant ways to run a large language model on protected health information are a BAA-covered managed service like Azure OpenAI or Google Vertex AI, and a private deployment inside your own infrastructure. The deciding question is whether PHI ever leaves your perimeter.
If you need to run a large language model on protected health information, the realistic HIPAA-compliant options as of June 2026 come down to two families: a managed cloud service that will sign a Business Associate Agreement, such as Azure OpenAI or Google Cloud Vertex AI, and a private deployment of an open or commercial model inside infrastructure you control. The deciding question between them is a single one: does PHI ever leave your perimeter? If it must not, a private deployment is the cleanest answer, because data that never travels cannot be exposed by someone else’s logging, retention, or training.
No model is compliant by itself
HIPAA compliance is a property of the deployment, not the model, even though the marketing rarely says so. There is no such thing as a HIPAA-compliant model in the abstract. The exact same model can be compliant inside a properly configured, BAA-covered environment and flatly non-compliant in a consumer app. So the right question is never “which model is HIPAA compliant,” it is “which deployment of which model, under what agreement and controls, is compliant for my workflow.”
The options, compared
Here are the realistic paths for running an LLM on PHI, side by side.
| Option | BAA available? | Where PHI lives | Training on your data? | Customization | Best for |
|---|---|---|---|---|---|
| Consumer ChatGPT | No | Vendor systems, retained | Possible by default | Minimal | Nothing involving PHI |
| Azure OpenAI (with BAA) | Yes | Your Azure tenant region | No, under enterprise terms | Moderate | Microsoft-cloud organizations |
| Google Cloud Vertex AI (with BAA) | Yes | Your Google Cloud project | No, under enterprise terms | Moderate | Google-cloud organizations |
| OpenAI API / Enterprise (with BAA) | Yes | OpenAI environment under contract | No, under enterprise terms | Moderate | Teams wanting managed frontier models |
| Private deployment (open or commercial model) | Not needed for an external processor | Inside infrastructure you control | No; nothing leaves | High; fine-tune on your own data | PHI-heavy workflows needing maximum control |
The pattern in that table is the point. As you move down, your protected data depends less on a third party’s promises and more on controls you own. With a private deployment, the BAA question for the model layer can fall away entirely, because there is no external processor of the PHI to sign one.
The compliance checklist any option must pass
Whichever path you pick, the deployment has to clear the same bar before it touches PHI:
- A signed BAA with every party that processes PHI, unless the data never leaves your control.
- Access controls that scope each user to the minimum-necessary records.
- A durable, queryable audit log of who accessed what and when.
- A written guarantee that your data will not train shared models.
- Encryption in transit and at rest.
- Data scoping so the system only ever sees the PHI a task actually requires.
A managed service can satisfy all six with the right configuration and contract. A private deployment satisfies several of them by construction, because the data simply does not leave.
About open-source and fine-tuning
Two questions come up constantly. First, open-source models like Llama are perfectly usable on PHI when deployed inside your own environment; being open source neither helps nor hurts compliance, which is determined by the deployment around it. Second, fine-tuning on PHI is safe when the training and the resulting model both stay inside infrastructure you control. The risk to avoid is shipping PHI to a third party to fine-tune a hosted or shared model, where the data leaves your perimeter and may persist in ways you cannot audit.
What the downside is worth
The reason to be deliberate about this is concrete. Healthcare is the most expensive sector in the world to suffer a data breach: IBM’s Cost of a Data Breach 2024 report put the average healthcare breach at 9.77 million dollars, the highest of any industry. HHS guidance on the underlying obligations is published by the Department of Health and Human Services. Against numbers like those, the operational cost of keeping PHI inside your perimeter looks small.
How to choose
If your workflow is PHI-heavy and the data should never leave your walls, choose a private deployment. If the workflow is sensitive but a contracted managed service is acceptable, a BAA-covered option like Azure OpenAI carries less operational burden. Both are legitimate, and the choice follows the sensitivity of the data and how much control you need. Picking the right option is only half the job; the other half is building a custom AI workflow on top of it that fits how your clinical and administrative teams actually work. That is the reasoning we walk through with every healthcare client, and it sits alongside our companion guide on whether ChatGPT itself is HIPAA compliant.
To map a specific clinical or administrative workflow to the right option, book a demo.
Frequently asked questions
- What LLMs are HIPAA compliant?
- No model is HIPAA compliant on its own, because compliance is a property of the deployment rather than the model. The compliant paths are managed services that will sign a Business Associate Agreement, such as Azure OpenAI and Google Cloud Vertex AI, and private deployments of open or commercial models inside infrastructure you control. The same model can be compliant in one setup and non-compliant in another, which is why the deployment is what matters.
- Is Llama or open-source HIPAA compliant?
- Open-source models like Llama can be used in a HIPAA-compliant way when deployed inside infrastructure you control, because the protected health information stays within your perimeter and no third party processes it. The model being open source is neither here nor there for compliance; what makes the deployment compliant is the access controls, audit logging, encryption, and the fact that PHI never leaves your environment.
- Can I fine-tune a model on PHI safely?
- Yes, if the fine-tuning happens inside infrastructure you control and the resulting model stays there. The risk is sending PHI to a third party to fine-tune a shared or externally hosted model, where the data leaves your perimeter and may persist in ways you cannot audit. Fine-tuning within a private deployment keeps both the training data and the resulting model under your control.
- Which is the most compliant LLM option for a hospital?
- For PHI-heavy workflows, a private deployment is the cleanest option because protected health information never leaves the hospital's perimeter, which removes an entire category of third-party risk. A BAA-covered managed service like Azure OpenAI is a legitimate option with less operational burden, and the right choice depends on how sensitive the workflow is and how much control the organization needs.
Working in a regulated environment? Let’s talk.
Book a demo