LLM Data Residency: Where Your AI Actually Runs

The note leaves your system, and you lose track of where it goes. Your database has one known location, but a call to a large language model sends the full text somewhere else. Copies can stay behind in places nobody listed.

Picture a common setup. A chain of six clinics in Kochi sees about 1,200 patients a day, and after each visit the doctor types a note: symptoms, test results, medicines and the plan. Staff turn each note into a short summary by hand, about 3 minutes each, or 60 hours of work a day. The setup is made up, but the problem is real.

The clinics' one engineer builds a tool that sends each note to an LLM: an AI program trained on huge amounts of text that can read and write language. A model service (a company that runs models and lets your code use them over the internet) does the work. Doctors check 200 summaries in a test week and approve them. The cost is about 40 paise per summary (made up).

Then a partner clinic in Germany wants the same summaries for video consultations with patients who live in the EU. It keeps those patients' records on servers in the EU. Its data protection officer (the person responsible for privacy rules) has three questions. Which country is our patient's note in when it reaches your AI, who can see it, and how long does it stay there?

The engineer knows where the database is, and nothing else. This guide answers those three questions, in order, and ends with the setup the team picks.

Six words carry the whole topic. Each gets a plain meaning and an example from the clinics.

• Personal data: any information about a person who can be identified from it. A note saying a named 34-year-old has dengue is personal data, and so is the test result inside it.
• Region: a named group of data centres in one area that a cloud platform or model service lets you pick, such as an India region or an EU region. Picking a region picks roughly which country your servers sit in.
• Data residency: the physical place where data is stored and processed. The clinics' notes are stored in an India region and summarised by a model in that same region.
• Data sovereignty: whose laws can reach the data. Usually that is the country where it sits, and sometimes also the country where the company holding it is based.
• Data Fiduciary and Data Processor: the roles India's DPDP Act uses. The fiduciary decides why and how data is processed; the processor processes it for the fiduciary. GDPR calls the same roles controller and processor.
• Retention: how long a copy is kept before it is deleted. A model service might keep prompts for 30 days, then delete them.

LLM data residency means knowing and controlling the countries where every piece of data your AI feature touches is processed and stored. That includes prompts, outputs, logs and caches, not just your database. Data sovereignty asks a different question: whose laws can reach that data, wherever it sits.

Think of a bank locker. Residency is which branch your gold sits in, say Kochi. Sovereignty is which rules the bank must obey. If the bank's head office is abroad, that country's rules may also decide when the locker is opened, even while the gold stays in Kochi.

• Gold maps to your data: the note and its summary.
• Branch maps to the region where the data is stored and processed.
• The bank's rules map to the laws that can reach the data, from where it sits and from where the company is based.
• The locker key maps to the encryption keys: who holds them affects who can be forced to open the data.

Where the picture breaks: gold sits still in one locker. A note is copied every time a model reads it, so an AI feature is like a locker that photocopies your gold into other rooms. That copying is why residency for AI is harder than for a database.

Here is a real example of legal reach. A US law, the CLOUD Act of 2018, covers communication, cloud storage and computing providers under US jurisdiction. Under a valid US order, they must disclose data they possess or control, even when it is stored abroad. So an EU region run by a US company fixes residency, but not every question of sovereignty.

Every call sends the whole prompt out of your system, and the output comes back in. Around that one call, extra copies can appear: the service's logs, its caches, your own logs and any monitoring tool you use.

Pause and predict: one note goes in and one summary comes out. Before you read the diagram, guess how many places could still hold a copy after the request finishes.

The answer is up to five: the cache, the service's logs, your logs, the monitoring tool, and the app itself, which now stores the summary. Here is each kind of data in plain words.

• Prompt: the instructions plus the full note. It is processed wherever the model runs, even if only for a second.
• Output: the summary. It holds the same health details in fewer words, so it is just as sensitive as the note.
• The service's logs: records a model service may keep, for example to check for abuse. How long, and where, depends on the service and your contract.
• Caches: saved copies that make repeat requests faster. As of October 2026, some services keep the processed start of a prompt for minutes. Some settings keep it for up to 24 hours, sometimes by default.
• Your own logs: lines your code writes, such as a print(prompt) left in for debugging. These often go to a logging tool hosted in another country.
• Monitoring tools: tools that record every prompt and answer so you can check quality. Useful, and one more copy in one more place.

What just happened? The model call is one copy you chose. The others appear as side effects nobody chose. So a residency plan starts as a list of every box in this diagram, with a country and a retention period beside each one.

An LLM can run in four common places: a model company's own service, your cloud platform's region, your own hosted open-weight model, or your own building. Each one moves the line between what you control and what someone else controls.

Option 1: a model company's own service

Your code sends the prompt over the internet to an API (the web address and rules your code uses to call another company's software). The company runs the model on its own servers. This is the quickest start. Whether you can pick a processing region, and what is kept, depends on the service, the plan and your contract.

Option 2: a model through your cloud platform's region

Many cloud platforms offer popular models in their own catalogue, billed through your cloud account and covered by the platform's terms plus each model's licence. Not every model is offered in every region.

Some platforms also offer cross-region routing that sends a request to another region with free capacity. As of 19 October 2026, one platform's global option can process a request in any supported commercial region worldwide. Check that your setting pins processing, not just storage.

Option 3: a self-hosted open-weight model

An open-weight model is one whose trained numbers, called weights, are published, so you can download and run it. You rent GPU servers (machines with chips built for AI maths) in the region you choose, and prompts never go to a model company. But you now run everything: updates, scaling and security. Each model also has a licence you must read.

Option 4: on premises

On premises means the model runs on servers in a building you own. You get the most control and the most work: hardware, power, cooling, backups and someone to fix it at night. For a six-clinic chain, this is rarely worth it unless a law or a partner demands it.

Cost changes shape too, using made-up numbers. A service call costs 40 paise per summary, while a self-hosted GPU server costs Rs 2 lakh (200,000 rupees) a month whatever it writes. At 36,000 summaries a month, that is about Rs 5.56 each. On these numbers, self-hosting breaks even only at 5 lakh (500,000) summaries a month, about 14 times today's volume, before counting staff time.

Ask in writing what they keep, how long, in which country, who can read it, and whether it trains models. A sales page is not enough. The answer that counts is in your contract or data processing agreement (the contract that says how a processor may handle your personal data).

Training use matters because research has shown that language models can repeat pieces of their training data word for word. One 2020 study extracted names, phone numbers and email addresses that a model had memorised from public web pages in its training text. The OWASP list of LLM risks recommends letting users opt out of having their data used for training.

The engineer sends this checklist to every model service. The logging and monitoring vendors get it too, because they hold copies as well.

• Do you keep prompts and outputs? For how long, and why?
• Do you use our data to train or improve models? Is that off by default, and does the contract say so?
• In which country are prompts processed, and where are saved copies kept? Does our region setting cover both?
• Can your staff read our prompts, for example in a safety review? From which countries?
• Do you cache prompts or outputs, and for how long?
• Which other companies (subprocessors) handle our data, and in which countries?
• How do we ask you to delete our data, and how fast is it done?

India's DPDP Act lets data go abroad unless the government restricts the destination country, while stricter Indian laws still apply. GDPR works the other way round: personal data leaving the EU needs a transfer tool first. Both rules are summarised here for engineers, not as legal advice.

India: the DPDP Act, 2023

Section 16 lets the central government restrict transfers to countries it names in a notification (an official order in the government gazette). A country not named is not blocked by this section. But Section 16 also keeps alive any other Indian law that gives more protection or limits transfers more, so that stricter law still applies.

Section 8 adds two duties that reach your model service. A fiduciary may use a processor only under a valid contract. When consent is withdrawn or the purpose is served, it must erase the data and make its processor erase it, unless a law requires keeping it. If the clinics process EU patients' notes in India under a contract with the German partner, Section 17(1)(d) switches off these duties and Section 16.

Timing matters. Both sections start applying on 13 May 2027, eighteen months after the gazette notification dated 13 November 2025. From then, Rule 15 also lets the government set conditions on making transferred data available to a foreign government or bodies it controls. Rule 13 lets it require larger or riskier organisations it names (Significant Data Fiduciaries) to keep certain data in India.

EU: GDPR transfer rules

A transfer happens when an organisation covered by GDPR sends personal data to, or makes it available to, another organisation in a country outside the European Economic Area (EEA). The EEA is the EU plus Norway, Iceland and Liechtenstein. A transfer needs a transfer tool, and the common ones are:

• Adequacy decision: the European Commission has found that a country protects data well enough, so data can flow there like it flows within the EU.
• Standard contractual clauses (SCCs): fixed contract text adopted by the Commission. Before signing, both sides must assess and document whether the destination's laws stop the clauses from working, called a transfer impact assessment.
• Binding corporate rules: approved internal rules for transfers inside one group of companies.
• Derogations: narrow exceptions, such as explicit consent for a specific transfer. EU regulators say they must be read restrictively so the exception does not become the rule.

As of 19 October 2026, the Commission's adequacy list has 17 entries, including Japan, the United Kingdom, Switzerland and Brazil, which was added in January 2026. India is not on it. So if an EU patient's note were summarised by a model in India, the parties would need another tool, such as SCCs.

Ask three questions in order: what data goes into the model, where the people it belongs to live, and how much running work your team can handle. Then pick the simplest option that satisfies all three.

Run the clinics through the questions. The data is health notes, so it is sensitive. The patients live in two places, India and the EU. The team is one engineer, so self-hosting or on premises is too much work for now.

The choice is option 2, used twice. Notes from patients in India go to a model in an India region, and notes from EU patients go to the same model in an EU region. The engineer first confirmed that this model is offered in both regions. Logs stay in the same region for 30 days, and the contract says no training on the clinics' data.

Remove what the model does not need before the prompt leaves. A summary does not need the patient's name or phone number. Here is the clinics' first version, which you can run as it is.

import re

PHONE = re.compile(r"\b\d{10}\b")    # exactly ten digits in a row

def prepare_note(note_text, patient_name):
    text = note_text.replace(patient_name, "[PATIENT]")
    return PHONE.sub("[PHONE]", text)

note = ("Test Patient, 34, fever for 3 days, dengue positive. "
        "Mobile 9876543210, home +919876500000. Advised Test to rest.")
print(prepare_note(note, "Test Patient"))

Expected output on Python 3.13: "[PATIENT], 34, fever for 3 days, dengue positive. Mobile [PHONE], home +919876500000. Advised Test to rest." The full name and the plain ten-digit number are gone.

What just happened? Two things slipped through. The pattern matches exactly ten digits standing alone, so the number written with +91 in front stays. The replace call matches only the exact full name, so the first name in "Advised Test to rest" stays.

This is the naive version, and it is wrong for production. A real system needs patterns for every phone and ID format it stores, and names taken from the patient record, not guessed. Test it with your own made-up notes before you trust it.

No, not by itself. An EU region fixes where the data sits, but access from outside the EEA can still be a transfer, and the provider's home country can still make demands.

EU regulators treat remote access from a country outside the EEA (a third country) by another organisation as a transfer, even when the data is only shown on a screen. Your own employee on a trip abroad is not a transfer, because they are part of your organisation. But if the model service's support staff in another country can open your prompts, that access needs a transfer tool. Ask where support and safety reviewers sit, not just where servers sit.

The same guidance covers foreign orders. If a processor in the EU hands data to a third country's authority because that country's law demands it, regulators count that disclosure as a transfer. This is where the CLOUD Act example from earlier meets GDPR.

The clinics face this in the human workflow too. Kochi doctors join some EU video consultations and read notes from India, so that remote access is a transfer the partners must cover with a transfer tool, such as SCCs. If the Kochi clinics are themselves covered by GDPR, the 2021 SCCs do not fit; as of 19 October 2026, the Commission is developing separate clauses. The AI step must not add a second, unplanned country.

Route on the patient's home region, stored on their record, and refuse anything you do not recognise. The app keeps each record in that patient's home region, and a copy of the router runs in each region. So an EU note is read, cleaned and summarised only inside the EU.

The router is a few lines. You can run this part on its own; the addresses use the reserved .example domain, and the code only picks an address, so nothing is sent.

ENDPOINTS = {
    "IN": "https://ai-india.clinic.example/summarise",
    "EU": "https://ai-eu.clinic.example/summarise",
}

def pick_endpoint(home_region):
    if home_region not in ENDPOINTS:
        raise ValueError(f"No approved model region for {home_region}")
    return ENDPOINTS[home_region]

print(pick_endpoint("EU"))
print(pick_endpoint("AE"))

Expected output: the EU address on the first line, then a traceback ending in "ValueError: No approved model region for AE". AE is the country code for the United Arab Emirates, standing in for any region you have not approved.

Look at what the code did. A patient with an unknown region stopped it instead of quietly falling back to India. Engineers call this failing closed: when in doubt, send nothing.

Checking that no copy leaks with a canary

Use a canary: a made-up marker that should only ever appear where your data map says. Then search for it everywhere else. A canary cannot see inside the model service's own logs and caches; the contract checklist covers those.

1. 1

   Create a canary note

   In a test environment, write a fake EU patient note containing a unique marker such as CANARY-7Q4X and a fake ten-digit number.

2. 2

   Run the whole pipeline

   Send it through the router, the model, your logging and your monitoring, exactly as a real note would go.

3. 3

   Search every store

   Search your log tool, monitoring tool, error tracker and any database in India for CANARY-7Q4X and for the fake number.

4. 4

   Compare with the data map

   Expect zero matches outside the EU stores on your map, and the number shown only as [PHONE]. Any other match is a leak to fix.

Start by tracing the data, not by naming a region. Interviewers want to see that you know a model call creates copies, and that you can control each one.

1. 1

   Ask about the data and the people

   Find out what goes into the model and where its owners live. Here: health notes, patients in India and the EU.

2. 2

   Trace every copy

   List the prompt, output, logs, caches and monitoring tools, and say where each one sits and for how long.

3. 3

   Pick where the model runs

   Compare the four options and choose the simplest one that meets the rules, such as one cloud region per patient group.

4. 4

   Lock down retention, training and access

   Say what the contract must state: retention, no training on your data, deletion, and which countries support staff work from.

5. 5

   Enforce it in code

   Remove identifiers before sending, route by home region, fail closed on unknown regions and prove it with a canary test.

6. 6

   Name the trade-offs

   Tighter residency can mean fewer model choices, more cost or more running work. Say which you would accept and why.