AI in Healthcare: BAA Compliance Before the OCR Guidance Drops

Introduction

Healthcare organizations are deploying AI faster than compliance frameworks can adapt. Clinical decision support, patient communication bots, automated coding, and document summarization—all processing data that may contain Protected Health Information (PHI).

HHS has signaled that forthcoming OCR guidance will address AI-specific HIPAA obligations. But guidance takes 12–18 months to publish after signaling. Enforcement does not wait.

If your AI vendor does not have a Business Associate Agreement (BAA) and your system sends PHI to their models, you have a compliance gap today—not when the guidance drops.

Section 1: When AI Processing Creates a BAA Obligation

A BAA is required when a vendor creates, receives, maintains, or transmits PHI on your behalf. For AI systems:

Scenario	BAA required?	Why
Sending patient notes to an LLM for summarization	Yes	PHI transmitted to vendor
Using an AI API with PHI in prompts	Yes	Vendor processes PHI
Storing embeddings of PHI-containing documents	Yes	Vendor maintains PHI-derived data
Using AI on de-identified data only	No	No PHI involved
Running open-source models on-premise	No*	No third-party BA (*you are the covered entity)
AI vendor accesses PHI for model training	Yes	PHI used beyond your authorized purpose

The most common gap: teams use general-purpose AI APIs (without BAAs) for tasks that inadvertently include PHI in prompts.

Section 2: The BAA Checklist for AI Vendors

Before sending PHI to any AI vendor, verify:

Signed BAA covering AI/ML services specifically (not just general cloud services),
Data processing terms: PHI used only for your authorized purpose, not model training,
Subprocessor disclosure: who else touches your PHI (cloud providers, model hosts),
Data retention limits: how long PHI is stored, deletion procedures,
Breach notification: timelines and procedures if PHI is compromised,
Audit rights: ability to verify compliance with BAA terms,
Data residency: where PHI is processed and stored (US requirements for most covered entities).

Vendors with healthcare BAAs (as of 2026)

Major providers offering BAAs for AI services include:

AWS Bedrock (via AWS BAA),
Google Cloud Vertex AI (via Google Cloud BAA),
Azure OpenAI Service (via Microsoft BAA),
Anthropic (enterprise/API with BAA),
OpenAI (enterprise/API with BAA for eligible use cases).

Always verify current BAA terms directly—this list changes.

Section 3: Engineering Controls for AI + PHI

A BAA is necessary but not sufficient. Engineering must enforce PHI boundaries:

PHI detection and routing

Scan prompts for PHI before sending to external models (NER, regex, or classification),
Route PHI-containing requests to BAA-covered endpoints only,
Block PHI from reaching non-BAA APIs entirely.

Minimum necessary

Send only the PHI required for the specific AI task—not full patient records,
De-identify where possible before AI processing (remove names, dates, identifiers),
Use retrieval with access controls so the AI only sees PHI the requesting user is authorized to access.

Logging without PHI exposure

Metadata-first logging: request IDs, token counts, latency, model version—not prompt/response content,
Hash or tokenize identifiers in logs,
Separate audit trail for PHI access events (who accessed what, when).

Human-in-the-loop

Require human review before AI-generated content enters the clinical record,
Flag AI outputs as machine-generated in the medical record,
Escalation paths when AI confidence is low or PHI sensitivity is high.

Section 4: Preparing for OCR Guidance

While waiting for formal guidance, align with the direction of travel:

Inventory all AI systems that may process PHI,
Secure BAAs for every vendor in the inventory,
Implement PHI routing controls (technical, not just policy),
Document your AI risk assessment (which systems, what PHI, what controls),
Establish an AI governance committee (clinical + engineering + compliance),
Create an AI incident response runbook (model hallucination affecting patient care, PHI leak via prompt, unauthorized AI access).

When OCR guidance publishes, teams with this foundation adapt quickly. Teams starting from zero face a multi-year compliance gap.

Section 5: The Cost of Non-Compliance

HIPAA violations carry penalties of $100–$50,000 per violation, with annual maximums up to $1.5M per violation category. OCR investigations increasingly target:

AI systems processing PHI without BAAs,
Insufficient audit trails for AI-assisted clinical decisions,
PHI in AI training data without authorization.

The engineering cost of compliance (PHI routing, BAA-covered endpoints, metadata logging) is a fraction of the penalty exposure.

Conclusion

Do not wait for OCR guidance to act. If your HealthTech platform uses AI with PHI, you need BAAs today, PHI routing controls this month, and an AI governance framework this quarter.