AI Compliance Architecture: GDPR, HIPAA and FedRAMP Explained

Each new customer brings its own rules, and a first design usually breaks all of them at once. Data goes everywhere, stays forever, and nobody can say who saw it.

Picture a common setup. A small team in Bengaluru builds an app that listens to a doctor's visit, with the patient's permission, and drafts the clinical note. A clinical note is the doctor's written record of the visit: symptoms, findings and the plan. The setup is made up, but the problem is real.

The team now wants three new customers. One is a group of clinics in Germany. One is a 300-bed hospital in the US. The third is a US federal health agency.

Today's design is simple. Audio goes from the doctor's phone to one server in Mumbai. The full transcript goes to a model service: a company that runs the AI model and answers requests over the internet. The draft note comes back and is kept forever.

Each customer sends a security questionnaire, a long form that asks how a vendor protects data. The honest answers are bad. Debug logs print whole transcripts, and all 9 engineers can read the production database. The model service's standard terms let it keep prompts for 30 days. Nobody can find every copy of one patient's data.

AI compliance architecture means designing your system so the law's rules are kept by the system itself, not by people remembering a policy. Data goes only where the law and your contracts allow, stays only as long as needed, and leaves a record of who touched it.

For the note-taking app, that turns into four design jobs. Decide which data may reach the model. Put a deletion timer on every copy. Record who opened what. Sign the right contract with every vendor the data passes through. The rest of this guide shows where each rule set pushes on those four jobs.

You need five ideas, each with a plain meaning and an example from the note-taking app.

Personal data. Personal data is any information about a person who can be identified, directly or indirectly. Example: a patient's name, a phone number, or even "the 52-year-old teacher with diabetes from a small village near Munich". Health data is a sensitive kind of personal data, and the law protects it more strongly.

PHI. Protected health information (PHI) is the US term, from HIPAA, for health information that can be linked to a person. It covers such data held by hospitals, health plans and their vendors. Example: the US hospital's transcript of a visit, with the patient's name and date of birth in it.

Controller and processor. These are GDPR roles. The controller decides why and how personal data is used. The processor handles the data only on the controller's instructions. GDPR requires a binding contract between them, often called a data processing agreement. Example: each German clinic is a controller, and the team is its processor.

Covered entity and business associate. These are HIPAA roles. A covered entity is a health plan, a healthcare clearinghouse (a firm that converts billing data into standard formats), or a provider that sends standard transactions like insurance claims electronically. A business associate is a vendor that creates, receives, maintains or transmits PHI on a covered entity's behalf. Example: the US hospital bills insurers electronically, so it is a covered entity, and the team is its business associate.

Sub-processor. A sub-processor is your own vendor that touches your customer's data, such as your cloud host or the model service. Under GDPR, you need the customer's written permission before using one, and you must pass the same duties down. HIPAA calls the same vendor a subcontractor, and it counts as a business associate too.

The picture below shows the three deals as chains. Read each arrow as "hands data to", and notice the paper (a contract or a certificate) written on each one.

GDPR requires a legal reason to use personal data, and asks you to use as little as you need for as short a time as you need. It also gives people rights over their data and protects it when it leaves Europe. In an AI system, each rule becomes a design decision.

GDPR is the EU's General Data Protection Regulation. It covers the German clinics directly, because they are controllers based in the EU. The team in Bengaluru is bound at least through its processing agreement with them, which must carry GDPR's processor duties.

Lawful basis: why are you allowed to use this data?

A lawful basis is the legal reason for using personal data. GDPR lists six: consent, contract, legal obligation, vital interests, public task and legitimate interests. Health data gets extra protection. Using it is forbidden unless one of a short list of exceptions also applies, such as explicit consent or the provision of medical care.

Each clinic, as controller, picks the basis. The team's job is to use the data only for the purpose the clinic set: drafting notes. That rules out quietly reusing German transcripts to improve the team's own model. So the team adds a setting called training_allowed that is False for every customer.

Data minimisation and retention: how little, and for how long?

Data minimisation means collecting and sending only the data a task needs. Example: the model needs the conversation to draft the note. It does not need the patient's name, insurance number or past records, so the app removes those before the request leaves its system.

Retention is how long you keep each kind of data, and GDPR expects you to delete or anonymise data once you no longer need it. The team writes one number per data type (made up): audio is deleted 7 days after drafting, transcripts after 30. The signed note goes to the clinic, which keeps it under its own medical record rules.

Rights, transfers and automated decisions

People have rights over their data, including access, correction, erasure and objection. The clinic must answer within one month, extendable by two more months for complex requests, and the processor must help. So the team tags every stored item with a patient key. One search now finds a patient's audio, transcript, note and any copies.

GDPR does not define a transfer in one line. The European Data Protection Board, the body of EU data regulators, treats it as a GDPR-bound organisation making personal data available to one outside the European Economic Area. That area is the EU plus Iceland, Liechtenstein and Norway.

A transfer is allowed to a country the European Commission has found to protect data well enough, called an adequacy decision. Otherwise you need a safeguard. The most common is standard contractual clauses (SCCs): ready-made contract terms published by the Commission, plus a transfer impact assessment, a written check of the other country's laws.

India has no adequacy decision as of 19 October 2026. The Board says remote access from outside the area counts as a transfer, even if data is only shown on a screen. This holds when the viewer works for a separate organisation, like the team, not for the clinic itself. So the team plans for SCCs and asks a lawyer. Where the data physically lives is a bigger topic, covered in the separate data residency guide.

Finally, GDPR gives people the right not to be subject to decisions made only by a machine, when those decisions have legal or similarly serious effects on them. The app's notes are drafts that a doctor reads, edits and signs, so a person makes the final call. If the model alone decided which patients get an urgent appointment, that rule could apply.

HIPAA requires that PHI goes only to vendors who sign a business associate agreement. Every system holding electronic PHI also needs safeguards such as access control, audit records and protection in transit. In an AI system, that makes the model service part of your compliance chain, not just a service you call.

HIPAA is the US Health Insurance Portability and Accountability Act. Its rules protect PHI held by covered entities like the US hospital, and by their business associates like the team.

A business associate agreement (BAA) is a contract in which a vendor promises to protect PHI the way HIPAA requires. The US hospital will sign one with the team. The team in turn needs one with every subcontractor that creates, receives, maintains or transmits PHI for it. That includes the cloud host and the model service.

US health regulators say a cloud provider needs a BAA even if it stores only encrypted PHI and has no key. The narrow exception is a transmission-only carrier, whose access to PHI is only passing. If a model service will not sign a BAA, PHI cannot go there, however good the model is.

The HIPAA Security Rule covers PHI in electronic form. It asks for safeguards in three groups: administrative (policies, training, risk analysis), physical (locked rooms and devices) and technical. The technical ones map almost directly onto code.

• Access control: every person gets their own unique login. The team goes further than the rule and removes standing access (access that is always switched on) to production PHI.
• Audit controls: the system records activity on PHI, such as who opened or changed a note, and when. Someone must also examine those records.
• Integrity: PHI is protected from improper change or destruction.
• Person or entity authentication: you check that a person or system is who it claims to be before giving access.
• Transmission security: PHI is guarded while it travels between systems, usually with encryption.

Some HIPAA items are marked "addressable". US health regulators state that this does not mean optional. You must judge whether the measure is reasonable for you, and if it is not, write down why and what you do instead.

Encryption, both stored and in transit, is addressable today, but the US hospital's questionnaire asks for it anyway. So the team encrypts PHI everywhere it is stored and everywhere it travels. A proposed update would make encryption required with limited exceptions. As of 19 October 2026, it is still a proposal.

FedRAMP is the US government program that sets security rules for cloud services that hold federal information. For an AI product sold to an agency, it usually means a separate, certified copy of your service. It also means a careful account of every outside service under it, including the model service.

FedRAMP stands for Federal Risk and Authorization Management Program. Its Consolidated Rules for 2026 rename the approval from FedRAMP authorization to FedRAMP Certification, so you will see both names for a while. The agency still gives its own approval to use a service, called an authorization to operate (ATO).

Two ideas decide how hard this gets. An impact level is the agency's rating of how bad it would be if its data leaked, was changed or became unavailable: Low, Moderate or High. These levels come from a US government standard called FIPS 199.

A certification class (A, B, C or D) describes how much security assurance a cloud provider commits to give. Class C is presumed adequate for most Low or Moderate systems, and Class D for most systems at any level. FedRAMP warns that classes are not one-for-one replacements for impact levels.

The model service is the tricky part. On FedRAMP's newer path, called 20x, the 2026 rules make a provider document third-party resources that may affect federal customer data. It records each one's purpose and the controls that limit the risk. A model service that is already FedRAMP certified is the easiest one to justify to the agency.

So the practical answer is a separate stack for government customers. A stack here means a complete, separate copy of the system: its own cloud account on FedRAMP certified services, its own model service and its own access rules. The team's estimate is 12 to 18 months of work (made up).

If the agency itself provides or pays for care, it is often a HIPAA covered entity too. Then the government stack also needs BAAs, because FedRAMP adds to HIPAA and does not replace it.

The diagram shows how one codebase becomes three separate stacks. Look at what differs in each box, and at what is shared between them: only the code, never the data.

Yes, but for a note-drafting tool the duties are usually light, because the EU AI Act puts its heaviest rules on high-risk systems. That can change if the product starts acting like a medical device.

The EU AI Act is the EU's law about AI systems themselves. GDPR covers personal data; the AI Act covers what the AI does and how risky it is. It bans a few practices and sets heavy rules for high-risk systems. Some systems must tell people they are dealing with AI. The Commission says the Act introduces no rules for AI deemed minimal or no risk.

The dates moved in 2026. A change called the AI Omnibus entered into force on 27 July 2026. High-risk rules for the listed use cases now apply from 2 December 2027. For AI built into regulated products, such as some medical devices, they apply from 2 August 2028.

For the app, the lesson is to keep the doctor as the decision maker and write down what the product does and does not do. If the team adds diagnosis suggestions, it must check whether the app has become a medical device needing an outside assessment.

If it has, high-risk duties follow. The European Commission lists them as risk management, high-quality data, activity logging, detailed documentation, clear information for users, human oversight, and robustness, cybersecurity and accuracy.

GDPR protects people's personal data, HIPAA protects patients' health information in the US health system, and FedRAMP certifies cloud services for US federal agencies. The first two are laws about data. FedRAMP is a government program about the security of the service that holds the data.

You build it by mapping every flow of sensitive data, then shrinking each one. Send less, keep it for less time, let fewer people see it, and sign a contract with every vendor it reaches. Before the steps, pause and predict: how many places does one visit's data land?

1. 1

   Map the data flows

   A data flow is one path that data takes between two places. List every place a visit's data lands: phone, upload server, transcription, model service, database, logs, backups and analytics. The team guessed 4 places; the map found 11 (made up).

2. 2

   Record only after consent

   The app starts recording only after the doctor confirms the patient agreed, as each clinic's process requires. No confirmation, no audio.

3. 3

   Redact before the model

   A redaction step swaps names, phone numbers and ID numbers for placeholders like [PATIENT_1] before any text goes to the model service. The real values go back in after the model replies.

4. 4

   Set a timer on every data type

   Audio, transcripts and drafts each get a deletion period, enforced by a daily job. Set each timer when the data is first stored, so data from a refused request still expires. Backups expire on a matching schedule, or the timer means little.

5. 5

   Remove standing access

   Engineers no longer have everyday access to production data. Support access needs a ticket, ends automatically after a few hours, and is logged.

6. 6

   Choose vendors by their paper

   In each stack, use only cloud and model services with the right contract or certificate. Ask each model service how long it keeps prompts and whether it trains on them. Get the answers written into the contract.

Here is one visit's journey after the changes. Follow the main path from the doctor's phone to the clinic's record system, and watch the two side boxes: the audit log and the deletion job.

Keep the rules for each stack in one file

The team keeps the rules for each stack in one place, so a reviewer can read them in a minute. The values are made up for this guide.

STACK_RULES = {
    "eu": {"model_endpoint": "eu-notes-model", "paper": "processing agreement + SCCs",
           "audio_days": 7, "transcript_days": 30, "training_allowed": False},
    "us": {"model_endpoint": "us-notes-model", "paper": "BAA",
           "audio_days": 7, "transcript_days": 30, "training_allowed": False},
    "gov": {"model_endpoint": "gov-notes-model",
            "paper": "FedRAMP Certification + agency ATO (+ BAA if a covered entity)",
            "audio_days": 1, "transcript_days": 7, "training_allowed": False},
}

Each stack name maps to its own settings. The model_endpoint setting names the model service covered by that stack's paper, and the paper setting records which contract or certificate that is. The audio_days and transcript_days settings are the deletion timers, shorter in the government stack. The training_allowed setting is False everywhere.

Put every rule inside one function

Now the function that drafts a note. The helpers redact, ask_model, restore, audit_log and schedule_delete are made up for this guide, and each does what its name says. NOTE_PROMPT is a made-up constant holding the instructions for writing a note.

def draft_note(visit):
    rules = STACK_RULES[visit.stack]        # unknown stack: KeyError, nothing sent
    if not visit.consent_confirmed:
        raise PermissionError("No recording consent for this visit")
    audit_log(visit.id, user=visit.doctor_id, action="draft_note")
    # Timers first, so a failed model call still lets the data expire
    schedule_delete(visit.audio_id, after_days=rules["audio_days"])
    schedule_delete(visit.transcript_id, after_days=rules["transcript_days"])
    text, placeholders = redact(visit.transcript)    # names become [PATIENT_1]
    draft = ask_model(rules["model_endpoint"], NOTE_PROMPT, text)
    note = restore(draft, placeholders)     # real names return inside your system
    return note                             # still a draft until the doctor signs

What just happened? The first line looks up the rules for the visit's stack. An unknown stack name raises a KeyError, which is what you want: failing loudly beats sending data to the wrong place. The consent check stops the function before any data moves.

The audit record and the deletion timers come before the model call. If the model service times out, the attempt is still on record and the data still expires. In our test run with stub helpers, a timed-out call left 1 audit record and 2 timers. A refused call, for an unknown stack or missing consent, sets no timers here. So a default timer must be set as soon as the audio and transcript are stored, as the visit diagram shows at transcription.

The redact helper returns the cleaned text and a list of which placeholder stands for which real value. That list never leaves your own system, and only the cleaned text goes to the model. The restore helper puts the real names back. The return value is still a draft: the doctor signs it, not the code.

Log who did what, to which record and when, but not the sensitive text itself. Logs get copied, searched and kept for months, so a transcript in a debug log is a leak waiting to happen.

Think of the visitor register at the gate of an apartment society. The guard writes the visitor's name, the flat number and the time, but not what was said inside the flat. A good audit log works the same way. Where the picture breaks: a guard keeps one book, while your logs get copied into many tools.

The team now keeps four kinds of records, each with its own rules.

• Audit log: who opened, created, changed or deleted PHI, with the record ID and the time. It is protected from edits and kept for a fixed period, because it is evidence.
• Application log: errors, timings and request IDs for debugging, but never prompts, transcripts or notes.
• Metrics: counts and durations, such as notes drafted per hour and model response time. No names or IDs at all.
• Quality samples: full prompts and replies kept for checking quality, only for customers whose contract allows it. They stay inside that customer's stack and expire on the transcript timer.

A simple filter removes known sensitive fields before anything is written. Here it is, with a second line that shows what it misses.

SENSITIVE_KEYS = {"text", "prompt", "note", "transcript"}

def scrub(fields):
    return {k: "[removed]" if k in SENSITIVE_KEYS else v for k, v in fields.items()}

print(scrub({"request_id": "r-104", "ms": 812, "prompt": "Patient reports chest pain"}))
print(scrub({"request_id": "r-105", "error": "Could not parse: Patient reports chest pain"}))

We ran it on Python 3.13, and it prints two lines. The first is {'request_id': 'r-104', 'ms': 812, 'prompt': '[removed]'}. The second keeps the whole error text, patient words included, because the filter checks only field names.

So test the whole pipeline, not the filter. Create a fake patient in a test stack, with a realistic made-up name that carries a unique marker, such as CANARY-7731. A real-looking name also tests whether redaction catches names. Run one visit end to end. Then search every log store, trace tool and analytics table for CANARY-7731.

Expect 0 matches outside the audio, transcript and note stores, and quality samples if enabled. Any match is a leak to fix before the first real customer. The trap is often the model call itself, because some tracing and monitoring tools can record full prompts and replies.

Start from the customers and their data, and name each rule set and your role under it. Then show how data flows, deletion, logging and vendor contracts enforce the rules. Interviewers want to see rules turned into design, not a list of rule names.

1. 1

   Ask who the customers are

   Where are they, and what data will the system see? "Clinics in Germany, a US hospital and a US agency" gives you three different rule sets.

2. 2

   Name each rule set and your role

   Processor under GDPR, business associate under HIPAA, cloud service provider under FedRAMP.

3. 3

   Draw the data flow

   Mark every place sensitive data lands, including logs, backups and the model service.

4. 4

   Shrink each flow

   Redact before the model, set deletion timers, and remove standing human access.

5. 5

   Put paper on every arrow

   Processing agreements, SCCs, BAAs and certification, including for the model service.

6. 6

   Say how you would prove it

   A canary search for leaked text, an audit log sample and the per-stack rules file. Then say which legal readings you would confirm with a lawyer.