Parent Document Retrieval: Search Small, Return Big Context

The search finds the right page but hands the model only a sliver of it. The model then answers from one line, without the steps, warnings or product details around it. Parent document retrieval fixes this by searching small and returning big.

Picture a common setup. A home-appliance brand runs a support search that answers customers from about 1,200 product manuals and 8,000 service notes. The setup is made up, but the problem is real.

The team built it as plain RAG. They cut every manual into chunks of about 100 words, turned each chunk into numbers for search, and searched those. Then the content team read 300 saved chats and found three repeating failures.

• A customer asks why the washing machine shows E21 and will not drain. The search returns one chunk: "E21: drain fault." The five cleaning steps sit in the next chunk, so the reply stops at the name of the error.
• A customer asks how to descale a coffee maker. The best chunk says "repeat step 3 twice" but step 3 lives two chunks earlier, so the model guesses.
• The team tries bigger chunks of 800 words. Now the right manual section often ranks below a loosely related one, because one big chunk mixes many topics into one set of numbers.

So the team is stuck between two bad choices. Small chunks match well but carry too little. Big chunks carry enough but match badly. The way out is to stop choosing: search one thing and return another.

Six words carry the rest of this guide. Each is explained once here and then used the same way throughout.

• Prompt: the text your program sends to a large language model, the AI that writes the answer.
• Chunk: one small piece of a document, here about 100 words.
• Embedding: a list of numbers that captures what a text means, so texts with similar meanings get similar numbers.
• Vector index: a database that stores embeddings and quickly finds the ones closest to the embedding of a question.
• Document store: any plain key-value store, like a table where you look up a row by its id.
• Token: a small piece of text, often a word or part of a word. Model services usually count limits and prices in tokens.

Parent document retrieval is a RAG technique where you search small child pieces but return the larger parent they belong to. The child is good for matching, and the parent is good for answering.

Child. A child is a small piece of text you search over, such as a 100-word chunk. Example: the line "E21: drain fault" is a child.

Parent. A parent is the bigger unit you actually give the model, such as a full manual section. Example: section 4.2 of the washing machine manual, with the error, the cause and all five steps.

Parent id. A parent id is the label that ties a child to its parent. Every child stores it, so finding a child tells you exactly which parent to fetch.

Think of a museum. Visitors scan the small labels beside each object, because a short label is quick to match against what they are curious about. Each label names its gallery, so a visitor who likes one walks there and sees the whole room.

Map it part by part. The label is the child, the gallery is the parent, and the gallery name printed on the label is the parent id. Where the picture breaks: search compares numbers, not meaning read by a person, so a key can match words that only look related.

The guide on RAG chunking strategies covers how to cut parents into children. This guide covers what happens next: what you index, how search walks from keys to parents, and how to add keys that are not chunks at all.

It works with two stores and one link. The vector index holds the small keys with their parent ids, and the document store holds the full parents, looked up by id. The parent never needs an embedding of its own.

Start at the section in the diagram and follow it two ways. It goes once, whole, into the parent store, and once, broken into keys, into the vector index.

The query side runs the other way. The code embeds the customer's question, searches the keys, and swaps the hits for their parents before anything reaches the model.

1. 1

   Embed the question

   The customer types "washing machine E21 not draining". The code turns it into an embedding with the same embedding model used for the keys.

2. 2

   Search the keys

   The index returns the 20 closest keys. Here, 9 of them point to section 4.2, 6 to section 4.3, and 5 to five other sections.

3. 3

   Group by parent

   The code collects the hits under their parent ids. Twenty hits shrink to seven distinct parents.

4. 4

   Rank the parents

   Each parent gets one score from its keys. Section 4.2 wins, because its best key matched the question most closely.

5. 5

   Fetch and fit

   The code fetches whole parents from the store, best first, until the token budget of 3,000 tokens is full. Then it builds the prompt.

Multi-representation retrieval gives each parent several different search keys, such as chunks, a summary and likely questions, all pointing to the same parent. A question can match whichever form of the parent it resembles most.

A representation is one way of writing the same content for search. The chunk is the raw text. The summary is the gist. A hypothetical question is how a customer might ask about it.

Back in the museum, one object can carry more than one label: a short line for children, a longer one for scholars, and a card of common visitor questions. Every label names the same gallery.

This fixes a gap that chunking cannot. Customers write symptoms: "water stays in the drum after the wash". Manuals write causes: "drain pump obstruction". A key written as a customer question usually shares more words and shape with the customer than the manual's chunks do.

The idea is older than modern RAG. A 2019 query-prediction paper trained a model to predict 10 likely queries per document and appended them to the document's text. The expanded text then went into a classic keyword index.

Multi-representation retrieval keeps each generated text as its own key instead. That way a summary or question can be scored, replaced or deleted on its own.

You group the matching children by parent id, give each parent one score, and return the best parents that fit your token budget. Without this step, one long section can fill the prompt three times over.

Three scoring rules are common. Two of them hide a trap that the toy program below makes visible.

• Best child wins. A parent's score is its highest child score. If your search returns hits best first, keeping each parent's first hit gives the same ranking.
• Count of hits. A parent with more matching children ranks higher. Long parents have more keys, so they get more chances to appear in the hits.
• Sum of scores. It adds up every child score of a parent, so it leans the same way, towards long parents.

The program runs with no installs on Python 3.11, 3.12 and 3.13, where we tested it. Save blocks A, B and C in one file, in that order. Its embed function is a toy that counts shared words, so real embedding scores will differ.

Block A builds the index. It stores three made-up manual sections as parents, cuts them into 12-word chunks, and adds one question key to section 4.2.

import math
import re
from collections import Counter, defaultdict


def embed(text):  # toy stand-in: word counts; a real system calls an embedding model
    vec = defaultdict(int)
    for word in re.findall(r"[a-z0-9]+", text.lower()):
        vec[word] += 1
    return vec


def similarity(a, b):  # cosine: 1.0 = same words in same mix, 0.0 = nothing shared
    dot = sum(n * b.get(w, 0) for w, n in a.items())
    size = math.sqrt(sum(n * n for n in a.values()) * sum(n * n for n in b.values()))
    return dot / size if size else 0.0


parent_store = {}  # parent id -> (version, full text); parents are never embedded
child_index = []   # keys: text, vector, parent id, version


def index_section(pid, text, version, extra_keys=()):
    child_index[:] = [k for k in child_index if k["parent_id"] != pid]  # old keys out first
    parent_store[pid] = (version, text)
    words = text.split()
    chunks = [" ".join(words[i:i + 12]) for i in range(0, len(words), 12)]
    for key in chunks + list(extra_keys):  # extra keys: summary, questions
        child_index.append({"text": key, "vector": embed(key),
                            "parent_id": pid, "version": version})


index_section("wm7-4.2", "Error E21 means the washer cannot drain. Switch off and "
              "unplug the machine. Open the small flap at the bottom front. Place a "
              "shallow tray and a towel under it. Turn the drain pump filter slowly to "
              "let water out. Remove lint and coins from the filter. Refit the filter "
              "and run a rinse cycle to test the drain.", version=1,
              extra_keys=["Why is water still in my washer after the wash?"])
index_section("wm7-4.3", "Fit the drain hose so its highest point sits between 60 "
              "and 90 cm above the floor. A hose that is too low can siphon water out "
              "of the drum. A kinked hose slows draining and can trigger a drain "
              "error.", version=1)
index_section("wm7-6.1", "General care. Wipe the door seal after each wash. Leave the "
              "door open so the drum can dry. Clean the detergent drawer every month. "
              "Check the water inlet hose for cracks. Run a hot drum clean cycle every "
              "month. Clean the drain pump filter every few months. Keep the machine "
              "level so it does not shake. Do not overload the drum. Empty all pockets "
              "before each wash. Use the right detergent for the water in your area. "
              "Unplug the machine if you will not use it for a long time.", version=1)

Block B is the query side. It searches the keys, keeps the best child per parent, and fills a budget, with word counts standing in for tokens.

def search(question, k):
    q = embed(question)
    hits = [(similarity(q, key["vector"]), key["parent_id"]) for key in child_index]
    return sorted(hits, reverse=True)[:k]


def find_parents(question, k=10, budget=120, max_parents=3):
    best = defaultdict(float)
    for score, pid in search(question, k):
        best[pid] = max(best[pid], score)  # best child wins; each parent kept once
    chosen, used = [], 0
    for pid in sorted(best, key=best.get, reverse=True):
        size = len(parent_store[pid][1].split())  # words stand in for tokens here
        if used + size > budget:
            continue  # this parent would overflow: skip it, try the next one
        chosen.append(pid)
        used += size
        if len(chosen) == max_parents:
            break
    return chosen, used


question = "washer shows E21 and will not drain"
print("keys:", len(child_index))
print("top 3 hits:", [(round(s, 2), p) for s, p in search(question, 3)])
print("whole section as one key:",
      round(similarity(embed(question), embed(parent_store["wm7-4.2"][1])), 2))
print("parents, words used:", find_parents(question))
symptom = "water stays in the drum after the wash"
print("symptom top hit:", [(round(s, 2), p) for s, p in search(symptom, 1)])
print("symptom hits per parent:", Counter(p for _, p in search(symptom, 10)))

Run blocks A and B together, and you should see exactly this:

keys: 18
top 3 hits: [(0.4, 'wm7-4.2'), (0.22, 'wm7-4.3'), (0.22, 'wm7-4.2')]
whole section as one key: 0.28
parents, words used: (['wm7-4.2', 'wm7-4.3'], 100)
symptom top hit: [(0.6, 'wm7-4.2')]
symptom hits per parent: Counter({'wm7-6.1': 5, 'wm7-4.2': 4, 'wm7-4.3': 1})

Now the trap. The symptom query's best single hit is the question key of section 4.2, at 0.60. But section 6.1, the longest section with 8 keys, owns 5 of the 10 hits. Counting hits would send the customer to general care instead of the drain fault.

In this toy, only best child wins plus a question key finds the drain fault section. To check the second column, empty the question list in block A so it reads extra_keys=[]) and run again. The top hit becomes (0.51, 'wm7-6.1').

Published work uses the same rule. The Dense X Retrieval paper, covered below, tested passage search this way. It fetched a slightly larger number of small units, mapped each to its source passage, and scored each passage by its best unit.

If you use a ready-made parent retriever from a library, read its source before trusting it. Check how it orders parents, whether it removes duplicates, and whether it caps tokens at all.

For support content, a good start is chunks plus hypothetical questions, then summaries for long sections. Each kind fixes a different failure, and each one costs something to make and store.

Summaries

A summary key is a few sentences that state what the whole parent is about. It helps with broad questions, like "what does the eco mode do", that no single chunk answers. A model writes it, so it can drop or bend details.

RAPTOR (2024) pushes this further. It recursively embeds, groups and summarises chunks into a tree, so search can match fine detail or the big picture. The RAPTOR guide covers it in depth.

Hypothetical questions

A hypothetical question key is a question the parent can answer, written ahead of time by a model. Start with three to five per section; this guide's table uses five. That is a starting point, not a standard. For section 4.2, one is the key in block A: "Why is water still in my washer after the wash?"

The bet is that a question shares shape and words with other questions more than with manual prose. The toy above shows it in miniature; measure it on your own questions before you trust it.

HyDE (2022) attacks the same gap from the query side. At search time a model writes a hypothetical document that answers the question, and the search embeds that instead. It costs a model call per question, while question keys cost calls only at indexing time.

Propositions

A proposition is one short, self-contained fact pulled out of the text. Example: "Error E21 on the WM-7 means the washer cannot drain." In the Dense X Retrieval paper's Wikipedia index, propositions averaged 11.2 words. Each passage held about 6.3 of them, and passages averaged 58.5 words.

Across five open-domain question sets, indexing propositions beat passages of up to 100 words on average for all four dense retrievers the authors tried. Recall@20 is the share of questions with an answer in the top 20 hits.

It rose by about 10 points on average for the two retrievers not trained for the task, and about 2.7 for the two trained ones. In their EntityQuestions analysis, most of the gain came from questions about rarer entities. The cost is size: that works out to about 10.8 propositions per 100 words, so one 900-word section becomes roughly 95 to 100 proposition keys.

Many vectors per document inside the retriever

Late-interaction retrievers go one level lower. A 2020 paper on late-interaction passage search stores one vector of 128 numbers per token of each passage, dropping punctuation. For each query token it takes the best match in the passage, then adds those matches up.

You do not design these keys by hand; the retriever makes them. Storage grows with every token, and you need a retriever built for it.

Make the parent the smallest unit that fully answers a typical question, which is usually one section, not one whole document. A parent that is too big wastes tokens and can bury the answer.

In the setup, the team first tried whole manuals as parents. Some ran to 40 pages, far more than a 3,000-token budget can hold, so one manual could not fit at all.

Long prompts carry a second risk. The Lost in the Middle study (2023) tested multi-document question answering and key-value lookups. The models tested often did best when the relevant text sat at the start or end of the input, and worse in the middle.

• Use the document's own structure first. Manual sections and service-note entries already group one topic each.
• Cap the parent size. If a section runs past your cap, split it at its subheadings into smaller parents.
• Try a window instead of a fixed parent. Return the matched chunk plus one or two chunks on each side.
• Measure on real questions. Compare answer quality and token use for each parent size before you settle.

The window option is handy when documents have no clean sections, like long service notes typed by engineers. Each chunk stores its position, and the code fetches its neighbours by number.

The main risks are stale keys, wasted tokens and permission leaks through parents. Each one comes from the gap between what you search and what you return.

• Stale keys. An editor changes a section, but old questions still point to it, or to a parent that no longer exists. Guard: rebuild all keys of a parent on every edit.
• Duplicate context. Two children of one parent both win, and the parent gets pasted twice. Guard: group by parent id before fetching.
• Budget blow-outs. One giant parent crowds out every other result. Guard: a size cap per parent and a budget check per prompt.
• Over-confident keys. A generated question matches strongly but the parent only half answers it. Guard: spot checks, and tests with known answers.
• Permission leaks. A child is allowed, but its parent holds a paragraph a customer must not see. Guard: check access on the parent, not only the child.

Stale keys are the quietest failure, so give updates their own path. Follow it from the editor's change to the nightly check at the end.

Block C runs that path on the toy index. Its stale_keys function is the nightly check: it counts keys whose parent is gone or whose version no longer matches the parent store.

def stale_keys():
    return sum(1 for k in child_index
               if k["parent_id"] not in parent_store
               or parent_store[k["parent_id"]][0] != k["version"])


print("stale before edit:", stale_keys())
new_text = parent_store["wm7-4.2"][1] + " If E21 returns, call service."
index_section("wm7-4.2", new_text, version=2,  # full rebuild, question key remade too
              extra_keys=["Why is water still in my washer after the wash?"])
print("stale after full rebuild:", stale_keys())
parent_store["wm7-4.3"] = (2, "New hose text.")  # job died before the keys were rebuilt
del parent_store["wm7-6.1"]                       # section deleted, keys left behind
print("stale after a half-done update:", stale_keys())

Run all three blocks, and the last three lines should read:

stale before edit: 0
stale after full rebuild: 0
stale after a half-done update: 12

In production, run the same count against your vector index's stored parent ids and versions. On a healthy day it is zero. Anything else means an update died halfway, and the job reruns those sections.

It costs model calls once at indexing time and more storage for as long as you keep the keys, but almost nothing extra per question. Search still runs one query, and the grouping step is plain code.

Take the setup's made-up numbers. About 20,000 sections need two model calls each, one for the summary and one for the questions. That is 40,000 calls for the first full build.

After that, only edited sections cost anything. If the team edits around 300 sections a week, the weekly bill is 600 calls. The index grows from 180,000 chunk keys to 300,000 keys, since each section adds one summary and five questions.

The real per-question cost lives in the prompt. Parents are bigger than chunks, so each answer reads more tokens. A budget of 3,000 tokens keeps that bounded, and it is the number to watch.

Skip it when your documents are already short, or when answers fit inside a single chunk. Then the parent adds tokens without adding meaning.

A list of error codes with one line each is a good example. The chunk is the whole answer, so returning the full table only buries it. Product FAQ pages, where every answer is two lines, are the same.

Also hold back on generated keys until plain chunks fail in a measurable way. The guide on naive, advanced and modular RAG covers how to add one improvement at a time.

These signs say parents and extra keys will help:

• Answers often span several chunks, like steps, tables or warnings.
• Customers describe symptoms, while documents describe parts and causes.
• Long sections lose out to short, loosely related chunks in search.
• You can afford a batch job that rebuilds keys when documents change.

Explain it as separating what you search from what you return. Then walk through indexing, querying and updates, in that order.

1. 1

   State the trade-off

   Small chunks match well but carry too little. Big chunks carry enough but match badly. You want both.

2. 2

   Draw two stores

   A vector index of small keys with parent ids, and a document store of full parents looked up by id.

3. 3

   Add representations

   Name chunks, summaries and hypothetical questions as keys. Tie each to a failure it fixes.

4. 4

   Walk the query

   Search wide, group by parent, score by best child, fill a token budget, then build the prompt.

5. 5

   Cover updates

   Version each parent, delete and rebuild all its keys on edit, and run a check for stale keys.

6. 6

   Close with measurement

   Compare answer quality and tokens per question against plain chunks on a fixed test set.