LLM Guardrails Explained: Layered Checks Before and After the Model

Anything that one check misses goes straight to the user, and any real check misses something. A tempting single check is a line in the model's instructions, which the model follows most of the time but not every time.

Picture a common setup. A homework-help app lets school children in Classes 5 to 10 type a question from their homework. A language model replies with hints at night, when no human tutor is online. The setup is made up, but the problem is real.

Before the model, human tutors answered every question and followed four house rules. Give hints before answers, stick to school topics, never repeat a child's personal details, and check facts against the textbook. The team wrote the same four rules into the model's instructions.

In its first month the app answered about 9,000 questions a day (made up). Most answers were good, but a few broke the rules in three different ways:

• A Class 7 student typed a parent's phone number into a question about a project cover page. The app copied the number into a study summary, which the child shared with the class group.
• Another child got a confident but wrong history date.
• A third got full worked answers for a whole maths worksheet, not hints.

The team's first fix was more lines in the instructions: never repeat phone numbers, use only the textbook, give hints first. The problems became rarer, but they did not stop.

That matches the OWASP Top 10 for LLM Applications, a list of the main security risks in apps built on these models, from the OWASP security community. Its entry on sensitive information disclosure warns that such restrictions may not always be honoured. The 2025 edition is the latest as of 19 October 2026. The rest of this guide shows what the team built instead: several separate checks, each catching some of what the others miss.

You need six words for this guide. Each one gets a plain meaning and an example from the homework app.

Guardrail. A guardrail is any check or limit around the model that stops the app doing or saying something it must not. Example: a few lines of code that hide a phone number before an answer reaches the screen.

Layer. A layer is one guardrail at one point in a question's journey through the app. Example: one layer checks the child's question before the model sees it, and another checks the answer after it is written.

Classifier. A classifier is a program that reads text and gives it a label or a score. It can be a small trained model, a set of rules, or a second request to a language model. Example: a score from 0 to 1 for how unsuitable an answer is for a 12-year-old, and the cut-off score above which it blocks is its threshold.

PII. PII stands for personally identifiable information: any detail that can point to one real person. Example: a parent's phone number, a home address, or a child's full name together with school and section.

Grounding. An answer is grounded when its facts come from a source you trust, not from the model's memory. Example: an answer about photosynthesis is grounded if every fact in it appears on the Class 7 science page the app gave the model.

When a model confidently states something false, people call it a hallucination. NIST, the US National Institute of Standards and Technology, calls the same thing confabulation in its 2024 generative AI risk profile. Grounding checks exist to catch it.

Latency. Latency is the waiting time between a question and its answer. Example: if a child presses Ask and the hint appears 3 seconds later, the latency is 3 seconds. Every guardrail adds a little to it.

LLM guardrails are checks outside the model that keep an app's behaviour inside rules you choose, before and after the model runs. Because they sit outside the model, they still work when the model makes a mistake. A 2024 position paper on building guardrails describes them as filters on a model's inputs or outputs.

That last part is the key idea. A model writes its answer one token at a time, where a token is a small piece of text, often part of a word. At each step it picks the next token based on how likely it rates each option.

It does not follow rules the way an if statement does. So even a well-instructed model sometimes breaks a rule, and you need checks that do not depend on the model behaving.

Safety researchers picture layered defences as slices of Swiss cheese. The psychologist James Reason summarised this model in a 2000 paper in the medical journal BMJ. Each slice is a defence, and its holes are weaknesses that keep opening, shutting and moving. Harm gets through only when holes in several slices line up.

Here is how the picture maps onto an LLM app:

• A slice is one guardrail layer, such as the input check or the grounding check.
• A hole is a blind spot: a kind of message that layer cannot catch.
• Holes move when you change the model, a prompt or a threshold. Old blind spots close and new ones open.
• Something passing through every slice is a harmful answer reaching the user.

Where the picture breaks: cheese slices are cut separately, so their holes line up only by chance. Guardrail layers are often built from the same patterns or similar models, so their holes can sit in the same place. The engineer section below shows why that matters.

Researchers use the same picture for AI agents. A 2024 paper by Shamsujjoha and colleagues proposes a reference architecture for multi-layered runtime guardrails, inspired by the Swiss cheese model.

The diagram follows one miss in the homework app. A child asks when Bhopal's Upper Lake was built, and no textbook page in the app's library covers it. Watch which layers have a hole for this question, and which one catches it.

Nothing was wrong with the question, so the input check had nothing to catch. The instructions said to use only the textbook, but the model filled the gap from memory. The grounding check caught the made-up year, because no page supported it.

Four layers run before the model writes anything: input checks, the system prompt, retrieval filtering and tool permissions. Each one shrinks what can go wrong later.

Follow a child's question through the diagram. The model sits in the middle, and every arrow into it or out of it passes a check.

Input checks

Input checks look at the child's message before it reaches the model. Cheap checks come first, because most messages are fine and should pass in a few milliseconds. The homework app runs these:

• Length and type: reject a 20-page paste or a file type the app does not support.
• Topic: a classifier labels the question as school work or off topic. Off-topic questions get a kind nudge back to homework.
• Safety: a classifier scores the message for content that is not suitable for children, and for signs that a child may be in danger.
• PII: simple patterns spot phone numbers and mask them before the model sees them.
• Attempts to override the app's rules, called prompt injection: these need their own defences, covered in a separate guide.

Here is the PII layer as code you can run, using Python's built-in re module for text patterns. It is deliberately simple: it looks for 10 digits starting with 6 to 9, with an optional +91 in front, the shape of the mobile numbers the app sees most.

import re

# 10 digits starting with 6 to 9, with an optional +91 in front
PHONE = re.compile(r"(\+91[ -]?)?[6-9]\d{4}[ -]?\d{5}")

def mask_pii(text):
    return PHONE.sub("[phone hidden]", text)

print(mask_pii("Cover page: call my mother on 98765 43210"))
print(mask_pii("Her number is +91-9876543210"))

Running it prints both sentences with the number replaced by [phone hidden]. When the pattern matches, the model never sees the number, so it cannot copy it into a summary. The testing section shows what this simple pattern misses.

The system prompt

The system prompt is the standing set of instructions the app sends with every question, such as giving hints before answers. It is a real layer, because it shapes most answers well. But it is guidance, not enforcement: the model follows it most of the time, not every time.

Retrieval filtering

Retrieval is the step where the app searches its own library for useful pages and adds them to the prompt, the full text sent to the model. The homework app's library holds textbook pages and worked examples. Retrieval filtering decides which pages may be fetched for this child at all.

The library also holds teacher-only answer keys for weekly tests. So the team adds two conditions to the search itself: only pages for the child's class, and never pages marked teacher only. The model cannot leak an answer key it was never shown.

OWASP's 2025 entry on vector and embedding weaknesses recommends the same idea. A vector store is the kind of search index many of these apps use, and OWASP asks for permission-aware stores with strict partitioning of data.

Tool permissions

A tool is an action the model may ask your code to run. The homework app's model has two: make_quiz, which builds practice questions, and send_progress_report, which emails a weekly report. Tool permissions are checks inside your code that decide whether each request is allowed.

send_progress_report does not accept an email address from the model at all. The code looks up the parent linked to the child's account and sends only there. Even a badly confused model cannot make this tool email a stranger.

OWASP's 2025 entry on excessive agency says the same. Put the permission check in the code that carries out the action, rather than relying on the model to decide whether an action is allowed.

Run four checks on every answer before a user sees it: format, safety, PII and grounding. Together they catch answers that are broken, unsuitable, leaky or made up.

Format check. The app asks the model for JSON (a simple text format for data) with fields such as hint and full_answer. Code checks that the fields exist and follow the house rules. Example: if full_answer is filled in before the child has tried the hint, the check fails.

Safety check. A classifier scores the answer for content unsuitable for the child's age. Example: explaining why a historical war started is fine for a Class 8 history lesson, but a graphic description of violence is not.

PII check. The same patterns used on the input run again on the output. Example: a phone number might still appear in an answer, perhaps copied from an old message. It is masked before display, and the event is logged.

Grounding check. A checker compares the facts in the answer with the textbook pages that retrieval found. Example: the Upper Lake answer named a year that no page mentioned, so it failed. One way to build this check is a second request to a model, asking whether the pages support every fact.

If the answer flows into a web page, a database query or another program, add one more output check. OWASP's 2025 entry on improper output handling says to treat the model like any other user. Encode its output for wherever it goes next, for example escaping HTML so the text cannot run as code in a web page. For a database, pass the text only as a query parameter, never pasted into the query itself.

The diagram shows the order the team chose. Cheap code checks run first, and the slow grounding check runs last, only on answers that passed the rest.

Here is the same chain in Python, continuing the file above. Three checks that need judgement are passed in as functions: safety, whether the answer states facts, and grounding. In real use they call a classifier or a second model, and simple stand-ins at the bottom let the file run offline. Beginners can skim the code; the paragraphs after it explain the output.

import json

def check_answer(raw, pages, hint_tried, safety_score, states_facts, is_grounded):
    try:
        answer = json.loads(raw)                # format: is it JSON?
    except json.JSONDecodeError:
        return "retry", None
    if not isinstance(answer, dict) or not isinstance(answer.get("hint"), str):
        return "retry", None                    # format: right shape?
    full = answer.get("full_answer") or ""      # missing or null: no answer
    if not isinstance(full, str):
        return "retry", None
    if full and not hint_tried:
        return "retry", None                    # house rule: hint first
    text = answer["hint"] + " " + full
    if safety_score(text) > 0.5:                # made-up threshold
        return "block", None
    text = mask_pii(text)                       # hide, do not block
    if states_facts(text) and not is_grounded(text, pages):
        return "review", None                   # code decides, not the model
    return "send", text.strip()

# Stand-ins so the file runs offline. Real checks call a classifier or a model.
def fake_safety(text):
    return 0.9 if "[unsuitable]" in text else 0.1

def fake_states_facts(text):
    return not text.strip().endswith("?")       # hints here are questions

def fake_grounded(text, pages):
    return any(p.lower() in text.lower() for p in pages)

pages = ["Photosynthesis takes place mainly in the leaves."]
drafts = [
    '{"hint": "Where does the plant catch sunlight?"}',
    'Sure! The answer is 42.',
    '{"hint": "Divide both sides by 2.", "full_answer": "x = 4"}',
    '{"hint": "[unsuitable] text the classifier flags"}',
    '{"hint": "Can you ask your mother on 98765 43210?"}',
    '{"hint": "Photosynthesis takes place mainly in the roots."}',
    '{"hint": "Photosynthesis takes place mainly in the leaves."}',
    '{"hint": "The Upper Lake was built in 1005."}',
]
for raw in drafts:
    print(check_answer(raw, pages, False, fake_safety, fake_states_facts, fake_grounded))

What just happened? The first draft is a question, so it needs no grounding and is sent. The second is not JSON, and the third hands over a full answer before the hint, so both return retry.

The fourth scores above the 0.5 threshold, a made-up number, and is blocked. The fifth is sent with the phone number masked, because masking hides the detail and keeps the useful part.

The sixth states a fact no page supports, so it goes to review. The seventh states a supported fact and is sent. The eighth is the Upper Lake answer, and code decides it states a fact, so it goes to review too. So the output should read send, retry, retry, block, send with [phone hidden], review, send, review.

Code, not the model, decides which answers need grounding, because a model that invents a fact could also wrongly report that it states none. The stand-in treats every answer that is not a question as stating facts; a real app uses a small classifier, and treats an unsure result as yes. Valid JSON of the wrong shape, such as a list or a number where the hint should be, returns retry instead of crashing the check.

Human review handles the cases code cannot judge well, and monitoring watches the whole system over time to find what every layer missed. They are the last two layers, and the slowest.

Human review means a trained person looks at an answer or a conversation and decides what happens. In the homework app, a small team works through a queue of answers that failed the grounding check, plus anything a child or parent reported. About 1 answer in 200 lands there, around 45 a day (made up). Meanwhile the child gets a safe holding reply, such as a note to ask a teacher.

OWASP's 2025 misinformation entry lists human oversight and fact-checking among its mitigations. One kind of message skips the queue. If the safety classifier sees signs that a child may be in danger, the app gently tells the child to talk to a trusted adult. It also alerts trained staff at once.

Monitoring means collecting numbers and samples from live traffic and watching them for changes. It catches problems that no single answer reveals. The team watches these:

• How often each layer blocks, day by day. A sudden jump or drop often means something changed, such as a new model version.
• How many answers go to review, and what the reviewers decided.
• Reports from parents and children, grouped by type.
• The latency of each layer, so a slow check is noticed before children complain.
• A small random sample of answers that passed every check, read by a reviewer each week.

That last item matters most. Blocked answers show you what your checks can see. Only a sample of passed answers shows you what they cannot see.

The logs themselves hold children's questions, so they need care too. The team stores them with PII already masked, and only the review team can read them.

A false positive is when a guardrail blocks something that was fine. A false negative is when it lets through something it should have stopped. Every real guardrail makes both kinds of mistake.

In the homework app, a false positive looks like this. A Class 8 child asks how the Revolt of 1857 began, and the safety check blocks the answer for violence. The child gets no help with real homework.

A false negative is the reverse. An unsuitable answer scores 0.45, just under the 0.5 threshold, and reaches a child.

The threshold is the dial between the two. In the team's tests (made up), a threshold of 0.5 missed 2 in 100 bad answers but wrongly blocked 9 in 100 good ones. At 0.8 it wrongly blocked only 1 in 100 good answers but missed 8 in 100 bad ones.

One dial cannot push both to zero, so decide per risk. For a children's app, a missed unsafe answer costs far more than an annoying block, so the safety threshold stays strict, at 0.5. The topic check is different: blocking real homework makes children give up, so that check is relaxed.

Here are a few ways to lower false positives without raising false negatives:

• Send borderline scores to a person instead of blocking them outright.
• Give the classifier context, such as the child's class and chapter, so a history lesson about a war is expected.
• Swap a hard block for a softer action: mask the risky part, or reply with a hint and a note to ask a teacher.
• Stack layers. A relaxed early check is acceptable when a stricter check of a different kind runs later.

Beginners can stop here. You now know the layers, what each one catches and the two mistakes every check makes. The sections below are for engineers who build, test and tune these checks.

Stacked layers fail together when they share a blind spot, because a message that slips past one then slips past the others too. Stacking multiplies your protection only when the layers miss different things.

Here is the maths with made-up numbers. Two layers each miss 10 in 100 bad messages. If their misses are independent, only about 1 in 100 slips past both, because 10 percent of 10 percent is 1 percent.

If both layers miss the same messages, 10 in 100 still slip past both. The second layer then adds latency and cost but no protection for that kind of message.

Shared blind spots are easy to build by accident. Here are three ways the homework app could do it:

• The same phone pattern runs on the input and the output. A number written in words passes both.
• The input and output safety checks use similar models trained on similar data. A phrasing that fools one can fool the other.
• Both classifiers were tested only on English. A question typed in Hinglish, a mix of Hindi and English, can pass both.

The fix is variety, not more of the same. Mix layers of different kinds, such as patterns, a model-based check, permissions in code and a human sample. Then measure the whole chain as well as each layer, which the next section does in code.

A model-based checker has one more weakness. It reads the same untrusted text as the main model, so text written to manipulate models can target the checker too. OWASP's 2025 prompt injection entry says it is unclear whether fool-proof prevention exists.

So treat every model-based layer as one that can be fooled. Keep at least one layer for each serious harm that does not depend on a model, such as a permission check in code.

You test guardrails with a fixed set of labelled examples, some that should pass and some that should be blocked. Then you measure both mistake rates, for each layer and for the whole chain. Rerun the set every time you change a prompt, a threshold, a check or the model.

1. 1

   Collect good examples

   Gather real homework questions that must pass, including tricky ones like history lessons about wars. The team starts with 700 (made up).

2. 2

   Collect bad examples

   Gather 300 that must be blocked, masked or sent to review: unsuitable requests, phone numbers written in odd ways, questions no textbook page answers.

3. 3

   Measure each layer alone

   Run every layer on the full set and record its false negative and false positive rates. This shows which layer is weak.

4. 4

   Measure the whole chain

   Run the full app from question to answer. A weak layer is acceptable if the chain as a whole catches what matters.

5. 5

   Attack it on purpose

   Ask a few people to try hard to break the guardrails. This is called red teaming. Add everything they find to the test set.

6. 6

   Feed real misses back

   Every miss found by reviewers or parents becomes a new test case, so the same hole cannot quietly open again.

NIST's 2024 generative AI profile lists red-teaming among its suggested actions for measuring risk. The measuring itself needs only a few lines of code.

Here it is in Python, continuing the same file. A guard is any one layer: it takes a text and returns True when it blocks. Each test case pairs a text with the label you want, True when it should be blocked.

Pause and predict before you run it. Which of the eight test cases will the phone pattern from earlier get wrong?

def contains_phone(text):
    return PHONE.search(text) is not None

test_cases = [
    ("Call me on 98765 43210", True),
    ("My number is +91 9876543210", True),
    ("Call 98-76-54-32-10", True),
    ("Call nine eight seven six five four three two one zero", True),
    ("What is 25 x 4?", False),
    ("Write 7000000000 in words", False),
    ("Explain photosynthesis in 5 points", False),
    ("When was the Upper Lake built?", False),
]

def measure(guard, test_cases):
    bad = [t for t, should_block in test_cases if should_block]
    good = [t for t, should_block in test_cases if not should_block]
    missed = sum(1 for t in bad if not guard(t))          # false negatives
    wrongly_blocked = sum(1 for t in good if guard(t))    # false positives
    return missed / len(bad), wrongly_blocked / len(good)

print(measure(contains_phone, test_cases))

It prints (0.5, 0.25). Two of the four phone numbers slip through: the one with a dash after every two digits, and the one written in words. Those are false negatives.

One good case is blocked, because the pattern reads the maths answer 7000000000 as a phone number. That is a false positive on real homework. The pattern also matches inside longer numbers, so part of a 14-digit roll number gets masked.

Returning both rates lets you refuse a change that improves one rate by quietly ruining the other. A real test set needs hundreds of cases per layer, not eight. Even with 300 bad examples and zero misses, the true miss rate could still be near 1 percent, by the rule of three (3 divided by 300).

Testing and monitoring feed each other. In the diagram, follow the loop from live traffic, through the test set, and back to live traffic.

Each guardrail adds some waiting time, from almost nothing for a code check to a second or more for a check that calls another model. The total depends on which checks run, and whether they run one after another or side by side.

The team wants a hint on screen within about 4 seconds. Here is its budget for each layer, all numbers made up.

In this example the model itself takes about 2.5 seconds. If every automatic check runs one after another, they add about 1.1 seconds (1,092 ms), so the slowest path without a retry is about 3.6 seconds. A format retry adds a second model call, about 6.1 seconds in total, so the team allows only one and tracks how often it happens. Human review does not count here, because the child gets a safe holding reply while a reviewer looks.

Three choices keep the usual wait lower. The input classifier runs at the same time as the library search, since neither needs the other's result. Cheap checks run first, so a failing answer stops before the slow check starts.

And the grounding check runs only on answers that state facts, about 4 in 10 of them (made up). Code, not the model, decides which answers those are. The output classifier returns that label with the safety score, so the 100 ms in the table covers both. A typical answer without a tool call then waits about 0.55 seconds for its checks on average, or less when the classifier overlaps the search.

When a check times out or fails

Every check that calls another service can be slow or down. Give each one a time limit, and decide in advance what happens when it runs out.

Failing closed means treating no reply as a block, so the child gets the safe holding reply. Failing open means letting the message through unchecked. For a children's app, the safety and PII checks should fail closed.

A topic check could fail open, because the safety and grounding layers still run after it. Log every timeout as its own event, so a slow checker shows up in monitoring before it quietly becomes a hole.

Latency is not the only cost. Model services usually charge by the token, for both the text a request reads and the text it writes. So every check that calls a model also adds to the bill.

Describe guardrails as layers along the request path, say what each one catches and costs, and explain how you would measure them. Interviewers look for a plan that expects failure, not a promise that nothing will fail.

1. 1

   Name the harms

   Start with what must never happen in this product. For the homework app: unsuitable content, leaked personal details, made-up facts and full homework answers handed over too early.

2. 2

   Walk the request path

   Place a layer at each point: input checks, system prompt, retrieval filter, tool permissions, output checks, human review and monitoring.

3. 3

   Put hard rules in code

   Say which rules can live in the prompt and which must be enforced in code, such as PII masking and who a report may go to.

4. 4

   Discuss the trade-offs

   Name the costlier mistake, how you would set thresholds, how much latency each layer adds and whether checks fail open or closed.

5. 5

   Show the layers differ

   Point out where two layers could share a blind spot, and how you would make them differ in kind.

6. 6

   Explain testing and monitoring

   Describe the labelled test set, red teaming, and the loop that turns real misses into new tests.

If you have time left, sketch the Swiss cheese picture: several imperfect layers of different kinds, placed so their holes rarely line up. Then name the one layer you would build first, and why.