Catalog My learning Blog
LLM System Design Course: RAG, Vector Databases, Cost · Phase 5 · Lesson 25 · 9 min read

AI Agents and Tool Calling Architecture

On this page
  1. Learning Objective
  2. Concept Explanation
  3. Worked Example: A Bounded Agent Loop
  4. Real-World Examples
  5. Review Questions & Answers

Learning Objective

Describe how tool use works mechanically, explain what distinguishes an agent from a pipeline and why that difference introduces unbounded cost and compounding error, and apply the structural controls - step limits, budgets, deterministic orchestration, and typed tool contracts - that make agentic systems operable.

Concept Explanation

Tool use is mechanically simpler than its reputation suggests. You supply the model with typed descriptions of the functions available. Instead of prose, it may emit a structured request to call one of them with arguments. Your code, not the model, validates and executes that call, and returns the result into the conversation, and the model continues with the result available. The model never executes anything; it emits an intention that your code chooses to honor. Understanding that division precisely is what makes the security posture clear, because every authorization, validation, and rate limiting decision belongs on your side of that line, and the ReAct pattern published in 2022, interleaving reasoning with acting, is simply this loop run more than once.

An agent is that loop with the number of iterations decided by the model rather than by your code. A pipeline has a fixed shape: retrieve, rerank, generate, validate, return. An agent decides at each step what to do next, which tool to call, and when it is finished. That flexibility is genuinely valuable for tasks whose required steps cannot be enumerated in advance, such as debugging, multi-source research, or open-ended data work. It also converts every property from the first lesson of this course from bounded to unbounded. Cost is no longer a known prompt size but the sum over an unknown number of iterations, each carrying the accumulated conversation. Latency is the sum of a variable number of sequential model calls. And correctness compounds multiplicatively: a step that succeeds ninety-five percent of the time succeeds about sixty percent of the time over ten steps, which is why agent demonstrations are impressive and agent deployments are difficult.

The single most useful design principle in this area is to keep as much of the orchestration deterministic as the task allows. If the sequence of steps is known, write it as code and use the model for the individual judgments inside each step. Reserve genuine agency for the parts that truly cannot be enumerated, and prefer a constrained state machine, where each state permits only a small set of tools and transitions, over a free-form loop with twenty tools available at every step. This reduces the search space the model must navigate, makes behavior far easier to test and trace, cuts cost substantially, and is almost always what the eventual production version of an ambitious agent converges to after the prototype meets real traffic.

Tool design is where most agent reliability is won or lost, and the guidance is unglamorous. Few tools rather than many, because a large tool surface degrades selection accuracy. Names and descriptions written for the model as the reader, since they are the only documentation it gets. Strict typed schemas with enumerations and bounded ranges rather than free-form strings. Errors returned as structured, actionable messages the model can respond to, because a tool that returns a stack trace produces a confused retry while one that returns "no customer found with that id; use customer.search by email first" produces a correct next step. Idempotency wherever possible, since retries are certain. And granularity chosen so each tool does one thing that maps to a real capability, rather than a general-purpose tool with a mode parameter that the model will select incorrectly.

The operational controls are non-negotiable and each corresponds to a documented failure. A hard cap on iterations prevents the infinite loop that the first wave of autonomous agents in 2023 became famous for. A token and cost budget per task, checked between steps, prevents an expensive spiral. Loop detection, noticing the same tool called with the same arguments repeatedly, catches the most common degenerate pattern early. Every tool call carries the requesting user's permissions rather than a service account, per the previous lesson. Irreversible actions require human confirmation. And the entire trajectory - every step, tool call, argument, and result - is traced, because debugging a fifteen-step agent run without the trajectory is not possible. Evaluation also changes shape: a pipeline is evaluated on its output, an agent must be evaluated on its trajectory as well, since arriving at a correct answer through eleven wasted tool calls is a different system from arriving at it in two.

PropertyPipelineAgentControl
Step countFixed and knownDecided by the modelHard iteration cap with a defined terminal response
Cost per requestPredictable from prompt sizeUnbounded and accumulatingPer-task token budget checked between steps
LatencySum of a known set of callsSum of an unknown number of sequential callsWall-clock deadline for the whole task
Failure modeA stage fails visiblyError compounds across steps; loops silentlyLoop detection and per-step validation
DebuggabilityStage-level traceRequires the full trajectoryTrace every step, argument, and result
EvaluationScored on the outputScored on the output and the trajectoryMeasure step count and tool-call efficiency too

Worked Example: A Bounded Agent Loop

Every guard in this loop maps to a specific failure that unguarded agents produce reliably. The loop is short; the controls are the substance.

Python
async def run_agent(task, principal, max_steps=8, budget_usd=0.50, deadline_s=60):
    messages = [system_prompt(), user(task)]
    # Tools carry the USER's permissions, never a service account.
    tools = build_toolset(for_principal=principal)
    spent, started, seen = 0.0, now(), []

    for step in range(max_steps):                    # HARD CAP on iterations
        if spent > budget_usd:
            return Partial(messages, reason='budget_exhausted')
        if now() - started > deadline_s:
            return Partial(messages, reason='deadline_exceeded')

        out = await llm.complete(messages, tools=tools, max_tokens=1000)
        spent += cost_of(out)
        trace.step(step, out)                        # the whole TRAJECTORY

        if not out.tool_call:
            return Final(out.text, steps=step + 1, cost=spent)

        call = out.tool_call

        # LOOP DETECTION - the most common degenerate pattern.
        sig = (call.name, stable_hash(call.args))
        if seen.count(sig) >= 2:
            messages.append(tool_error(call, 'This exact call was already made '
                                             'twice and returned the same result. '
                                             'Try a different approach or stop.'))
            continue
        seen.append(sig)

        # VALIDATION happens in CODE, on our side of the line.
        if call.name not in tools:
            messages.append(tool_error(call, f'Unknown tool. Available: {list(tools)}'))
            continue
        if not tools[call.name].schema.validates(call.args):
            messages.append(tool_error(call, schema_error_message(call)))
            continue
        if tools[call.name].irreversible:
            return RequiresConfirmation(call, messages)   # human sees real args

        try:
            result = await tools[call.name].run(call.args, as_principal=principal)
        except ToolError as e:
            # STRUCTURED, ACTIONABLE errors - not a stack trace.
            result = {'error': e.code, 'hint': e.recovery_hint}
        messages.append(tool_result(call, result))

    return Partial(messages, reason='max_steps_reached')   # defined terminal state

# Evaluate the TRAJECTORY, not only the answer:
#   task_success_rate | mean steps | mean cost | redundant_call_rate
#   | hit_max_steps_rate | required_confirmation_rate
A bounded agent loop in which the model proposes a tool call, application code validates authorization and arguments, executes with the requesting user's permissions, and returns a structured result, wrapped by an iteration cap, a cost budget, a wall-clock deadline, and loop detection.
A bounded agent loop in which the model proposes a tool call, application code validates authorization and arguments, executes with the requesting user's permissions, and returns a structured result, wrapped by an iteration cap, a cost budget, a wall-clock deadline, and loop detection.

Real-World Examples

  • AutoGPT went viral in 2023 as one of the first widely used autonomous agent frameworks, and the failure pattern users reported most consistently was that it would loop, repeat the same actions, pursue tangents, and consume substantial API spend without completing the assigned task. That very public experience is the direct origin of the now-standard controls of hard step limits, cost budgets, and loop detection in every serious agent framework since.
  • Anthropic published the Model Context Protocol as an open standard in late 2024 for connecting models to tools and data sources through a common interface, and it was adopted by a wide range of vendors and open-source projects within a year. Its significance for architecture is that it pushes tool definitions, authentication, and capability boundaries into a standardized layer outside the model, which is exactly where authorization and validation decisions belong.
Common PitfallReach for a free-form agent loop only when the required steps genuinely cannot be enumerated in advance. Error compounds multiplicatively across steps, so a ninety-five percent per-step success rate is roughly sixty percent over ten steps, and cost and latency become unbounded at the same time. Most ambitious agent prototypes converge on a constrained state machine with deterministic orchestration and model judgment inside each step, and starting there saves the detour.

Review Questions & Answers

The model only ever emits a structured request to call a named tool with arguments; it executes nothing. Your code receives that request, checks that the tool exists and is permitted for the requesting principal, validates the arguments against a typed schema, executes the call under that principal's authority, and returns a result into the conversation. The boundary matters because it locates every security decision on the deterministic side: authorization, argument validation, rate limiting, idempotency, and confirmation for irreversible actions are all code, not model behavior, which is what makes the system defensible even when the model is manipulated.

Because the steps are sequential dependencies, so the probability of the whole task succeeding is roughly the per-step rate raised to the number of steps. Ninety-five percent per step is about sixty percent over ten steps and about thirty-six percent over twenty. Agents also compound in a second way that a pipeline does not: a wrong intermediate result is fed back into the context and shapes every subsequent decision, so an early error is not merely one failure but a corrupted premise for the rest of the trajectory. Reducing the number of genuinely agentic steps is therefore the most effective reliability lever available.

A small tool surface so selection accuracy stays high, names and descriptions written for the model as the reader since they are its only documentation, strict typed schemas with enumerations and bounded ranges instead of free-form strings, and one tool per real capability rather than a general tool with a mode parameter. Error messages matter because they are the model's only feedback channel for recovering from a mistake. A stack trace conveys nothing actionable and typically produces a confused repeat of the same call, while a structured message stating what was wrong and what to try instead gives the model exactly what it needs to take a correct next step.

Because the same answer can be produced by wildly different processes with wildly different cost, latency, and risk. An agent reaching the right result in two steps and one reaching it after eleven steps, six redundant calls, and three failed tool invocations are not the same system, even though an output-only evaluation scores them identically. Trajectory metrics such as mean step count, redundant call rate, rate of hitting the step cap, and cost per completed task expose degradation that output scoring hides, and they also reveal which tools are being misused or misselected, which is where most reliability improvement actually comes from.