AI Solutions Architect: Skills, Roadmap and Study Plan

A common pattern is to learn a few tools, build one flashy demo, and then struggle in the interview. The AI solutions architect job is about the decisions around a model, not the call to it.

A demo shows you can send text to a model and print the reply. An architect is asked harder questions. What may the AI do alone? How will you know a change made it worse? What will it cost each month?

Picture a common setup. A lunch-box subscription service in Bengaluru has about 9,000 subscribers and gets about 600 customer messages a day. The setup is made up, but the problem is real.

People ask to skip tomorrow's lunch, switch to a Jain menu, report a nut allergy or ask for a refund for a late box. Four support staff answer by hand, at about 3 minutes a message.

That is about 30 hours of reading and replying every day, and around Rs 2.4 lakh (240,000 rupees) a month in salaries. The business wants AI to answer the simple messages so staff can focus on the hard ones.

A developer on the team says the fix is easy: call a model service and send back what it says. The call is one line of code. The architect's job is every decision around that line.

• Which messages may the AI answer alone, and which need a person?
• What happens when a customer with a nut allergy asks to switch menus?
• Who may approve a refund, and can the AI ever do it?
• How will the team know if a new prompt made the answers worse?

Answering questions like these is the job. This guide follows that support assistant from the first sketch to the interview answer, and turns each skill into a weekly project you can show.

An AI solutions architect turns a business problem into a working AI system. They own the big decisions: where a model helps, what data it sees, what it may do, and how the team will know it works.

Most of that work is not writing prompts. It is agreeing limits with the business, drawing the system, deciding what stays plain code, and checking results before anything ships. Here is a realistic week, made up for this guide.

• Monday: agree with the operations head what "good" means, for example 70 percent of simple messages answered with zero allergy mistakes (a made-up target).
• Tuesday: sketch the design, mark which steps use a model and which stay plain code, and write down why.
• Wednesday: review test results with the developers and decide whether a new prompt is safe to ship.
• Thursday: check with the security lead what customer data leaves the company and which actions need a person to approve.
• Friday: read the cost and error reports and plan next week's fixes.

Think of Mumbai's dabbawalas. Lunch boxes reach the right desk because the route is planned: a colour and number code on each box, and fixed handovers at railway stations.

Map it part by part. The planned route is the system design. The code on each box is the label your system puts on every message. The station handovers are the checks and approvals between steps. The runner carrying boxes is the model.

Where the picture breaks: a dabbawala carries the same box the same way every day. A model can give different answers to the same message, so an architect adds tests and guardrails that a lunch route never needs.

The diagram shows the loop the architect repeats on every project. A prototype is a rough first version, and production is the live system real customers use. Notice the two arrows that point backwards.

You need six words: model, prompt, context, retrieval, eval set and guardrail. Each gets a plain meaning here and an example from the support assistant.

Model. In this guide, a model means a large language model: a program trained on huge amounts of text that reads text and writes text back. Many model services bill per token, a short chunk of text that is often part of a word, counted going in and coming out.

Prompt. A prompt is the text your program sends to the model, with instructions on top. Example: "Label this message as skip_lunch, change_menu, allergy, refund or other." Each label is an intent, meaning what the customer wants.

Context. Context is everything the model can see when it answers: the prompt, the customer's message, earlier messages and any documents you add. The most it can see at once is its context window.

Apart from what it learned in training, the model knows nothing about this business. To answer "Is Thursday's thali Jain?", the context must include Thursday's menu.

Retrieval. Retrieval means finding the right pieces of your own documents and adding them to the context before the model answers. The name retrieval-augmented generation, or RAG, comes from a 2020 research paper. In everyday use it covers this simpler pattern too.

Eval set. An eval set, short for evaluation set, is a fixed list of test inputs with the answers you expect. You run it after every change to see whether the system got better or worse.

Guardrail. A guardrail is a rule in your code that stops the AI from doing something harmful, whatever the model says. Example: the AI can never approve a refund, so a person approves every one.

An AI solutions architect needs six skill areas, and needs to be good enough in all six to make sound trade-offs. You do not need to be the deepest expert in any one of them.

A bigger model may give better answers, but it costs more and often replies more slowly. Most of this job is spotting trade-offs like that early and choosing on purpose. The diagram puts the six areas around the goal they serve.

Design, models and integration

Solution design is turning a business problem into boxes and arrows. You decide which steps are plain code and which use a model. You also decide between a workflow, where code fixes the steps, and an agent, where the model picks its next step.

In the support assistant, "skip tomorrow's lunch" is a fixed workflow, because its steps never change. Nothing about it needs a model to choose what happens next.

Models, prompting and context is choosing a model for each job and giving it the right information. A common pattern is model tiering: a small, cheap model for easy jobs and a larger one only for hard ones.

Integration is connecting the AI to real systems through APIs, which are ways for programs to call each other, and tools. A tool is an action the model may ask your code to run, such as pausing a subscription.

The Model Context Protocol (MCP) is an open standard for this. Its servers list tools, each with at least a name, a description and a JSON Schema, a format that describes the inputs the tool accepts.

As of 23 October 2026, the current MCP version is dated 2026-07-28. Its tools page says servers must validate all tool inputs and implement proper access controls. The protocol does not do that checking for you.

You also decide authentication, which checks who is calling, and authorization, which decides what they may do. In the assistant, the pause_subscription tool can only pause the plan of the customer who is chatting. It reads that customer from the logged-in session, never from the model's output.

Evaluation, safety and people

Evaluation and cost is proving the system works, then making it cheaper or faster without making it worse. The HELM research paper (2022) scores language models on seven measures, not only accuracy.

Those seven are accuracy, calibration, robustness, fairness, bias, toxicity (harmful language) and efficiency. Calibration asks whether the model's confidence matches how often it is right. The lesson for architects is to measure more than one number.

Governance and safety is deciding what must never happen, and building checks that hold even when the model is wrong or tricked. Governance means the rules, checks and owners that decide who may do what.

This area covers guardrails, privacy, when a person must approve, and prompt injection. Prompt injection is text, typed by a user or hidden in a document, that tries to override your instructions to the model.

OWASP, a non-profit that publishes security guidance, keeps a Top 10 list for LLM applications, meaning apps built on large language models. Its 2025 list starts with prompt injection (LLM01) and sensitive information disclosure (LLM02).

It also includes excessive agency (LLM06): giving an AI more tools, permissions or freedom than its job needs. One of its fixes is to enforce authorization in the systems the AI calls, not to trust the model.

The NIST AI Risk Management Framework, from the US standards agency, groups risk work into four functions: govern, map, measure and manage. As of 23 October 2026, NIST says a revised version of AI RMF 1.0 is in progress.

People and lifecycle is the work around the code. A stakeholder is anyone who cares about the result, such as the operations head or the security lead. The lifecycle is the system's life from first release through updates to retirement.

You explain trade-offs in rupees and minutes, not model terms. You plan how new versions roll out and how to undo them. A common habit is the architecture decision record (ADR): a short note that captures one decision and why it was made.

A real AI solution architecture is mostly ordinary code, data and checks, with models filling only a few boxes. Each box comes from one of the six skill areas.

One new word first. The orchestrator is the plain code that decides each next step and calls the other parts. Now follow one message through the diagram: "I have a peanut allergy. Can you switch my plan to the South Indian menu from Monday?"

Pause and predict: in this design, can a model change a customer's plan on its own? Look at which box calls the plan change tool before you read on.

It cannot. Only the approval step calls the tool. The tool takes the customer from the logged-in chat session, never from text a model wrote, and refuses any other plan. That keeps the design safe even when a model makes a mistake or is tricked.

Routine requests, such as skipping tomorrow's lunch, need no person. After the label, the orchestrator runs them as fixed workflow code, so a model still never changes a plan itself.

Now name the skill behind each box. The input checks and the approval are governance and safety. The two model tiers and menu retrieval are models and context. The plan change tool is integration. The logs feed evaluation.

The rule that allergy changes need a person came from a talk with the operations head, which is stakeholder work. Keeping the orchestrator as plain code, not an agent, is solution design.

Study one skill area at a time for about 8 to 10 hours a week, and end each week with a small project you can show. Twelve weeks will not make you an expert, but it gives you real proof of all six skills.

A schedule that fits a full-time job: 1 hour each weekday evening and 4 hours on Saturday, so about 9 hours a week. Keep one code repository for the whole plan, with a folder and a short notes file for each week.

One rule makes the plan work. Every week must produce something that runs, or something written that another person can review. Reading alone does not count.

The plan has six phases, and each needs the one before it. Do not jump to agents before you can measure a simple prompt.

Here is the plan week by week. Each week has one goal and one project on the support assistant's made-up data.

1. 1

   Week 1: How models behave

   Learn tokens, the context window, and why the same prompt can give different answers. Project: a Python script that sends one message to a model service and logs tokens used and time taken.

2. 2

   Week 2: Prompts with checkable answers

   Ask the model to answer in JSON, a simple text format for data, and reject anything that does not match. Project: label 40 made-up customer messages into five intents.

3. 3

   Week 3: Context and retrieval

   Split a document into chunks, which are small pieces, search them, and add only the best chunks to the prompt. Project: answer questions from a made-up menu file and show which chunk each answer used.

4. 4

   Week 4: Solution design

   Practise turning a problem into a diagram. Decide workflow or agent, which model tier does each step, and what stays plain code. Project: a two-page design for the support assistant.

5. 5

   Week 5: Integration

   Learn APIs, tools, the Model Context Protocol, and the difference between authentication and authorization. Project: a pause_subscription tool that reads the customer from the logged-in session, never from model output, and refuses any other plan.

6. 6

   Week 6: Evaluation

   Build an eval set, choose measures, and run it after every change. Project: the eval gate script shown later in this guide, with a pass mark agreed in writing.

7. 7

   Week 7: Cost and speed

   Measure cost per message and response time. Then try a smaller model or a shorter prompt. Project: the monthly cost model shown later, with your own token counts, while the eval score holds.

8. 8

   Week 8: Safety

   Study guardrails, prompt injection and the OWASP Top 10 for LLM Applications. Project: write 20 attack messages and show which part of the design stops each one.

9. 9

   Week 9: Governance and privacy

   Learn what personal data you send, where it is stored, and when a person must approve. Read the NIST AI RMF core at a summary level. Project: a one-page risk list with an owner for each risk.

10. 10

    Week 10: Running in production

    Learn logging, simple dashboards, and releasing a new prompt to a few users first. Project: a daily report of cost, errors and eval score.

11. 11

    Week 11: People and decisions

    Write decision records and turn trade-offs into promises such as "95 percent of replies within 5 seconds" (made up). Project: present the design to a non-engineer friend in rupees and minutes.

12. 12

    Week 12: Capstone and practice

    Redo the whole design for a new made-up problem, then do two timed mock interviews. Project: a design document, its eval results and three decision records.

The best practice projects are small, complete systems with an eval set, a cost number and a written decision, not large demos. A reviewer should see in five minutes that you can design, measure and protect an AI feature.

• A message labeller with an eval set and an accuracy score.
• A menu answerer using retrieval that shows the source chunk for every answer.
• A tool-using assistant with permission checks inside the tool and a person approving refunds.
• A cost and speed study: one task on two model tiers, with a recommendation.
• A red-team report, meaning you attack your own system on purpose: 20 attack messages and the guardrail that stopped each.
• A design pack: one diagram, three architecture decision records and a one-page risk list.

A flashy chatbot with no eval set, no cost number and no guardrails tells an interviewer you can call a model. One small project that is measured and protected says you can design a system.

They are tested mostly through scenarios: a business problem with limits on cost, speed, safety or data, where you pick and defend a design. Expect few questions that only ask for a definition.

Rounds vary by company, so ask the recruiter for the list. A common mix looks like this:

• A design round: design an AI feature, such as the support assistant, on a whiteboard or drawing tool.
• A trade-off round: "When would you not use retrieval?" or "How would you halve the cost without losing quality?"
• A debugging round: "Answers got worse after a model upgrade. What do you check first?"
• A coding or take-home task: build a small piece, often with an eval set, and explain your choices.
• A stakeholder round: explain a design to a non-engineer, or describe a time you pushed back on a risky request.

Certification exams that use scenario questions test the same judgement. Several options often sound reasonable, and the limits in the question decide which one fits. Read the official exam guide for the format of yours.

So read the limits first. Phrases like "lowest cost", "least effort to run", "data must stay in India" or "a person must approve" decide the answer. Note them, then cross out every option that breaks one.

Try one. "The service must answer menu questions, the menu changes every day, and the budget is tight. What do you build?" Fine-tuning means further training a model on your own examples.

The daily change rules out fine-tuning, because you would retrain every day. The tight budget points to a small model. So the answer is retrieval with a small model, plus a check that it quotes today's menu.

You are ready when you can do every item below on a small project and explain each choice. If two or more items make you nervous, spend another week or two on those areas before you apply.

Be honest: tick an item only if you have done it, not just read about it.

• I can turn a business goal into a diagram and explain why each box exists.
• I can explain when a workflow is enough and when an agent is worth its extra cost.
• I can pick a model tier for each step and defend it with quality, cost and speed numbers.
• I can build retrieval over a document and show where each answer came from.
• I can design a tool with narrow permissions and say who is allowed to call it.
• I can build an eval set, run it after every change, and decide if a change is safe to ship.
• I can list the main risks for an AI feature, such as prompt injection and data leaks, and show the control for each.
• I can say where personal data goes and which actions need a person to approve.
• I can write an architecture decision record and a simple promise such as a response-time target.
• I can explain the whole design to a non-engineer in rupees and minutes.

Expect gaps on the first pass; they tell you where the next two weeks go. If eval sets or cost numbers are among them, the next two sections cover both in code.

An eval gate is a script that runs your eval set and stops a release unless quality meets a pass mark and no unsafe action appears. It scores accuracy and safety separately, so a high score cannot hide one bad refund.

The script below runs as it is, with no installs. The fake_model function is a stand-in for the real model call, so you can run it offline. Replace it with your model service later and keep the rest.

Version v1 lets the model issue refunds itself, the design before the guardrail. Version v2 adds better examples to the prompt and moves the refund rule into code, so the model only labels.

# eval_gate.py: runs as is on Python 3.11 or later, no installs needed.
from operator import ge  # ge(a, b) means a is at least b

PASS_MARK = 0.9  # made-up bar agreed with the business

# refund_allowed: would the refund policy allow a refund for this message?
CASES = [
    {"message": "Please skip my lunch tomorrow", "intent": "skip_lunch", "refund_allowed": False},
    {"message": "Switch me to the Jain menu", "intent": "change_menu", "refund_allowed": False},
    {"message": "I have a peanut allergy", "intent": "allergy", "refund_allowed": False},
    {"message": "My box came two hours late, refund please", "intent": "refund", "refund_allowed": True},
    {"message": "Refund the whole month, I am moving city", "intent": "refund", "refund_allowed": False},
    {"message": "Hold my deliveries next week, I am travelling", "intent": "skip_lunch", "refund_allowed": False},
    {"message": "No lunch on Friday, I am on leave", "intent": "skip_lunch", "refund_allowed": False},
    {"message": "Can I change to the South Indian menu?", "intent": "change_menu", "refund_allowed": False},
    {"message": "Does the paneer dish have nuts?", "intent": "allergy", "refund_allowed": False},
    {"message": "Wrong box delivered, I want my money back", "intent": "refund", "refund_allowed": True},
]


def fake_model(version, message):
    """Stand-in for the real model call, so the script runs offline."""
    text = message.lower()
    words = {"refund": ["refund", "money back"], "allergy": ["allerg"],
             "skip_lunch": ["skip"], "change_menu": ["menu"]}
    if version == "v2":  # the second prompt added more examples
        words["allergy"].append("nuts")
        words["skip_lunch"].append("no lunch")
    intent = next((k for k, ws in words.items() if any(w in text for w in ws)), "other")
    # v1 let the model issue refunds itself; v2 moved the refund rule into code.
    action = "refund" if version == "v1" and intent == "refund" else "none"
    return {"intent": intent, "action": action}


def score(version):
    correct = unsafe = 0
    for case in CASES:
        result = fake_model(version, case["message"])
        if result["intent"] == case["intent"]:
            correct += 1
        if result["action"] == "refund" and not case["refund_allowed"]:
            unsafe += 1
    return correct / len(CASES), unsafe


def passes(accuracy, unsafe):
    return ge(accuracy, PASS_MARK) and unsafe == 0


for version in ("v1", "v2"):
    accuracy, unsafe = score(version)
    verdict = "ship" if passes(accuracy, unsafe) else "block"
    print(f"{version}: accuracy {accuracy:.0%}, unsafe actions {unsafe}, {verdict}")

if not passes(*score("v2")):
    raise SystemExit("Do not ship v2")

Save it as eval_gate.py and run python3 eval_gate.py. You should see two lines: "v1: accuracy 70%, unsafe actions 1, block" and "v2: accuracy 90%, unsafe actions 0, ship".

What just happened? v1 missed three labels and issued a refund for "Refund the whole month", a case the refund policy does not allow. One unsafe action blocks it, whatever the accuracy.

v2 still mislabels "Hold my deliveries", so it scores 90 percent, exactly the made-up pass mark. It issues no refunds at all, because the refund rule now lives in code.

Pause and predict: what happens if v2 still lets the model issue refunds? In the action line, change if version == "v1" and intent == "refund" to if intent == "refund", then run it again.

You should see "v2: accuracy 90%, unsafe actions 1, block" and then "Do not ship v2". The script exits with status 1, which fails a typical CI pipeline, the automated checks that run before a release.

Two limits to know. Exact matching with == suits fixed labels like intents, not free-text replies, which need a looser check or a person. And ten cases are a teaching size; a real eval set needs many more.

Multiply tokens per call by the price per token, then by calls per month, for each model tier. Doing this before you build shows where the money goes, and often it is not where you expect.

The script below compares two designs for the same workload. In design A, a large model labels every message. In design B, a small model labels every message. In both, the large model drafts replies for the 10 percent that need one.

Every price and token count here is made up. Prices change often and differ between providers, so use your provider's current price page and token counts measured from your own logs.

# cost_model.py: runs as is on Python 3.11 or later. All prices are made up.
MESSAGES_PER_MONTH = 600 * 30
REPLY_SHARE = 0.10  # share of messages that need a drafted reply

# Made-up prices in rupees per million tokens: (input, output)
PRICE = {"small": (10, 40), "large": (250, 1000)}


def call_cost(tier, tokens_in, tokens_out):
    price_in, price_out = PRICE[tier]
    return (tokens_in * price_in + tokens_out * price_out) / 1_000_000


def monthly_cost(label_tier):
    label = call_cost(label_tier, 400, 10) * MESSAGES_PER_MONTH
    reply = call_cost("large", 1500, 200) * MESSAGES_PER_MONTH * REPLY_SHARE
    return label, reply


for name, tier in (("A: large model labels", "large"), ("B: small model labels", "small")):
    label, reply = monthly_cost(tier)
    print(f"{name}: labels Rs {label:,.0f} + replies Rs {reply:,.0f} = Rs {label + reply:,.0f} a month")

Run python3 cost_model.py. You should see design A at Rs 3,015 a month and design B at Rs 1,114 a month. The labels line drops from Rs 1,980 to Rs 79, because labelling is high volume.

What just happened? With these made-up prices, one label costs Rs 0.11 on the large model and Rs 0.0044 on the small one, 25 times less. The replies cost the same in both designs.

Now compare with the support team's Rs 2.4 lakh a month. In this made-up case, model spend in design B is under half a percent of that. So the bigger cost decision is which messages still need a person, not which model labels them.

Two limits to know. This model ignores retries, the discounts some providers give for reusing the same prompt start, and the cost of the menu search. Add them once you have real logs, and re-check the eval score after every cost cut.

Start with the business goal and its limits, and draw the simplest design that meets them. Then add retrieval, tools, evaluation and guardrails one at a time, each with a reason.

1. 1

   Clarify the goal and the numbers

   Ask what success means, how many requests come per day, the budget, the speed needed and what must never happen. For the support assistant: 600 messages a day and zero wrong allergy answers.

2. 2

   Draw the plain path first

   Show the flow with no AI at all, then mark the steps where a model clearly helps.

3. 3

   Place the models and context

   Say which model tier handles each AI step and what information it needs, including any retrieval.

4. 4

   Connect systems safely

   Add tools with narrow permissions, and say how you check who is calling and what they may do.

5. 5

   Show how you measure

   Describe the eval set, the pass mark, the unsafe-action rule and how you watch cost and quality after launch.

6. 6

   Add guardrails and people

   Name the main risks, the code checks that stop them, and which actions need a person's approval.

7. 7

   Close with trade-offs

   Say what you gave up, such as speed for safety, and what you would change if the limits changed.