SOC 2 and HIPAA for AI Teams: What Prompt-Testing Evidence US Auditors Actually Want
By Promptster Team · 2026-06-22
If you sell software to US enterprises or handle health data, the EU AI Act probably isn't your first compliance fire. SOC 2 and HIPAA are. And unlike the AI Act — which is genuinely about AI — SOC 2 and HIPAA predate the LLM era entirely. That means auditors aren't asking "is your model safe?" They're asking the same questions they always ask: who can access what, how do you know it changed, and where's the log?
This is the US companion to our EU AI Act prompt-testing evidence guide. Same approach: we're not making compliance claims, we're drawing an evidence-generation map from control requirements to the prompt-testing artifacts that satisfy them.
First: What These Frameworks Actually Are
- SOC 2 — an attestation (not a certification) against the AICPA Trust Services Criteria: Security (required), plus optionally Availability, Confidentiality, Processing Integrity, Privacy. A Type II report covers operating effectiveness over a window (typically 3–12 months), so you need evidence over time, not a snapshot.
- HIPAA — US law governing Protected Health Information (PHI). The Security Rule (technical, administrative, physical safeguards) is what touches your AI pipeline. If you send PHI to a model provider, you need a Business Associate Agreement (BAA) with that provider — and most consumer API tiers don't offer one.
Neither framework has an "AI clause." Your AI features are just another system that has to obey the same controls as your database.
The Four Control Areas That Touch Prompt Testing
1. Access control (who can touch prompts and keys)
SOC 2 CC6 and HIPAA §164.312(a) both demand that access to systems and sensitive data is restricted, role-based, and reviewed. For AI teams the highest-risk asset is provider API keys — a leaked key is a financial and data-exfiltration incident.
The auditor wants: keys are not in source control, are encrypted at rest, access is role-scoped, and rotation is possible. Our writeup on managing AI API keys securely with AES-256 is the technical baseline here — client-side encryption, server-side resolution, no keys in request bodies.
2. Change management (how prompt changes are reviewed and traceable)
SOC 2 CC8 is explicitly about change management: changes to production are authorized, tested, and documented. A prompt is a production change. If a "minor prompt tweak" doubles your hallucination rate, an auditor will ask: who approved it, what testing gated it, and can you roll back?
The artifact: prompt version history with diffs and approval trail. This is exactly what enterprise prompt management with tagging and versioning produces — versioned prompts, diffs between versions, and a reviewable trail.
3. Audit logging (the full record of what ran)
SOC 2 CC7 (monitoring) and HIPAA §164.312(b) (audit controls) require you to record activity and be able to reconstruct events. For AI: every model invocation — prompt, provider, model version, parameters, output, timestamp, user attribution — logged and retained.
4. Data handling (what leaves your boundary)
SOC 2 Confidentiality/Privacy criteria and HIPAA's PHI rules govern what data goes where. The critical AI-specific question: does PHI or confidential data get sent to a third-party model provider, and under what agreement? No BAA with the provider means PHI cannot go to that provider — full stop.
The Evidence Map
| Control | What the auditor wants | Prompt-testing artifact |
|---|---|---|
| SOC 2 CC6 / HIPAA §164.312(a) — access control | Keys restricted, encrypted, role-scoped | Encrypted key store + access logs; AES-256 key handling |
| SOC 2 CC8 — change management | Prompt changes authorized + tested | Prompt version diffs + eval-score delta per version, in a PR trail |
| SOC 2 CC7 / HIPAA §164.312(b) — audit logging | Reconstructable activity record | API request history (prompt, provider, model, params, output, timestamp) exported to a log warehouse |
| SOC 2 CC7.2 — monitoring | Ongoing quality/anomaly signal | Scheduled eval runs with drift detection |
| SOC 2 Confidentiality / HIPAA PHI | Sensitive data stays in-boundary or under BAA | Data-flow doc + redaction config showing what's sent to providers |
| HIPAA §164.308(a)(1) — risk analysis | Documented risk assessment of the AI pipeline | Test methodology + provider-comparison records as supporting evidence |
These map cleanly to the same primitives the EU AI Act post called out: version history, scheduled comparisons, exported history. The frameworks differ; the artifacts overlap heavily.
The HIPAA BAA Trap
This is the one teams get wrong most often. Sending PHI to a model provider's standard API tier is a HIPAA violation unless you have a signed BAA with that provider. Some providers offer BAAs on enterprise tiers; many don't on default tiers.
Your options, in order of preference:
- De-identify before the call — strip PHI per the Safe Harbor method (18 identifiers) so what you send isn't PHI at all. Pair this with the redaction discipline from building an eval dataset from production traffic.
- Use a provider tier with a BAA — and document which providers are BAA-covered in your data-flow map.
- Don't send it — for the highest-sensitivity workloads, keep inference in-boundary.
A multi-provider testing tool helps here in a specific way: it lets you document, per request, which provider a given prompt was routed to — so you can prove PHI never went to a non-BAA provider.
Common Misconceptions
"SOC 2 certifies our AI is safe." No. SOC 2 attests that your controls operate as described. It says nothing about model quality. An auditor never scores your model.
"We're SOC 2 compliant." SOC 2 isn't pass/fail compliance — it's an attestation report with a defined scope and period. You either have a clean report for a scope or you don't.
"HIPAA only applies to hospitals." HIPAA applies to covered entities and their business associates. If you process PHI on behalf of a covered entity, you're a business associate and on the hook.
"Encryption alone satisfies HIPAA." Encryption is one technical safeguard. The Security Rule also requires access control, audit controls, integrity controls, and administrative safeguards (risk analysis, training, incident response).
The Minimum Posture for an AI SaaS Pursuing SOC 2 Type II
- API keys encrypted at rest, never in source, access role-scoped and logged.
- Every prompt change goes through a reviewed PR with a version diff and eval results attached.
- Every model invocation is logged (prompt, provider, model, params, output, timestamp) to a retained store.
- Scheduled eval runs provide a continuous quality/monitoring signal over the audit window.
- A documented data-flow map showing what data reaches which provider, plus BAAs where PHI is involved.
That's the floor that turns "we test our prompts" into "we have evidence our prompts are managed under control."
The Real Lesson
US auditors don't care that you're using AI. They care that the AI is governed like every other production system: access-controlled, change-managed, logged, and bounded by your data-handling policy. The good news — and the through-line from our EU companion — is that the same prompt-testing artifacts (version diffs, request history, scheduled evals, encrypted keys) feed SOC 2, HIPAA, and the EU AI Act at once. Generate the evidence once; satisfy three frameworks. Just don't send PHI to a provider you don't have a BAA with. That one's not a mapping exercise — it's a hard line.
Research compiled from AICPA Trust Services Criteria and the HIPAA Security Rule (45 CFR Part 164) as of June 2026. Informational, not legal advice; engage a qualified auditor and counsel for compliance decisions.