Catalog My learning Blog
Build AI Agents with MCP: 30-Day Course · Phase 2 · Lesson 6 · 5 min read

LLM Workflow Patterns: Prompt Chaining and Routing

On this page
  1. Learning Objective
  2. Concept Explanation
  3. Worked Example: Routing with a Cheap Classifier and a Parallel Review
  4. Real-World Examples
  5. Review Questions & Answers

Learning Objective

Learn the three foundational LLM workflow patterns, prompt chaining, routing and parallelization, and know when each beats a free-running agent. Implement each one in a few lines and add programmatic checks between steps.

Concept Explanation

Before reaching for a fully autonomous agent, most production systems are better served by a workflow: a fixed path through a few model calls that you design. Anthropic's "Building effective agents" catalogued the patterns that teams kept rebuilding, and three of them cover most needs. They are worth knowing well because even inside an agent, individual steps are often best implemented as one of these workflows.

Prompt chaining splits a task into a sequence of steps, where each call works on the output of the previous one. Writing marketing copy, then translating it, is a chain. So is extracting the facts from a contract, then checking them against a policy, then drafting a summary. The benefit is accuracy: each call has one focused job, which is easier than one giant prompt doing everything. You can also insert gates between steps, ordinary code that checks the intermediate output and stops or retries if it fails, for example verifying that an extracted date parses or that a draft stays under a length limit.

Routing classifies an input and sends it to a specialized handler. Customer questions about refunds, technical issues and sales each get their own prompt, tools, or even model. Routing lets you optimize each path separately instead of writing one prompt that must handle everything and does each case slightly worse. It is also a cost lever: simple queries can go to a small, fast model while hard ones go to the most capable model. The classifier can be a model call, a cheap embedding similarity check, or plain rules when the categories are obvious.

Parallelization runs several model calls at the same time and combines the results. It has two variants. Sectioning splits a task into independent parts handled concurrently, such as reviewing a pull request for security, performance and style in three separate calls. Voting runs the same task several times and aggregates, for example asking three times whether content violates a policy and flagging it if two of three say yes. Sectioning improves focus and latency; voting improves confidence on decisions where a single sample is noisy.

These patterns compose. A router can send an input to a chain, one step of which fans out in parallel. The test of a good design is that each step is simple enough to evaluate on its own and that you can say exactly what happens for any input. That property disappears the moment you hand control to an agent, which is why workflows remain the default for well-understood tasks.

PatternShapeMain benefitExample
Prompt chainingStep 1 then step 2 then step 3, with gatesAccuracy through focused stepsExtract facts, validate, then draft
RoutingClassify, then dispatch to a specialistSpecialized prompts, cost controlRefund, technical or sales handler
Parallel sectioningIndependent subtasks run concurrentlyFocus and lower latencySecurity, performance, style reviews
Parallel votingSame task several times, aggregateConfidence on noisy judgmentsTwo of three moderation votes

Worked Example: Routing with a Cheap Classifier and a Parallel Review

A router uses a small model to classify, then a chain handles the code review path with three parallel specialist reviews and a final merge step.

Code
import asyncio
import anthropic

client = anthropic.AsyncAnthropic()

async def ask(prompt: str, model: str = "claude-opus-5-5", system: str = "") -> str:
    msg = await client.messages.create(model=model, max_tokens=4000, system=system,
                                       messages=[{"role": "user", "content": prompt}])
    return "".join(b.text for b in msg.content if b.type == "text")

async def route(request: str) -> str:
    label = (await ask(f"Reply with one word, REVIEW, QUESTION or OTHER:\n{request}",
                       model="claude-haiku-4-5")).strip().upper()
    if label == "REVIEW":
        return await review_pipeline(request)
    if label == "QUESTION":
        return await ask(request, system="Answer engineering questions concisely.")
    return "Sorry, I can only review code or answer engineering questions."

async def review_pipeline(diff: str) -> str:
    focuses = {
        "security": "Find injection, auth and secret-handling problems only.",
        "performance": "Find N+1 queries, needless allocations and blocking I/O only.",
        "readability": "Find naming, structure and missing-test problems only.",
    }
    # Parallel sectioning: three focused reviews at once.
    reviews = await asyncio.gather(*[ask(diff, system=f"You review diffs. {rule}") for rule in focuses.values()])
    findings = "\n\n".join(f"## {name}\n{text}" for name, text in zip(focuses, reviews))
    # Gate: if no reviewer found anything, skip the merge call entirely.
    if all("no issues" in r.lower() for r in reviews):
        return "No issues found."
    # Chain step: merge and deduplicate.
    return await ask(f"Merge these reviews into one prioritized list, removing duplicates:\n{findings}")

print(asyncio.run(route("Please review: def get(id): return db.execute(f'SELECT * FROM t WHERE id={id}')")))

Every path is visible in code, so you can write a test for each label and each gate. The cheap Haiku classification costs a fraction of the main calls, and the three reviews take about as long as one because they run concurrently.

A cheap model routes each request. The review path fans out to three focused reviews in parallel, a code gate skips the merge when nothing was found, and a final chained call merges the findings.
A cheap model routes each request. The review path fans out to three focused reviews in parallel, a code gate skips the merge when nothing was found, and a final chained call merges the findings.

Real-World Examples

  • Anthropic's "Building effective agents" (December 2024) describes prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer as the building blocks it saw across dozens of customer deployments, and recommends composing these before adopting autonomous agents.
  • GitHub's code review features and many internal review bots use the sectioning idea: separate passes or prompts for security, correctness and style, because a single prompt asked to check everything tends to miss categories.
Key TakeawaysMost LLM features are best built as workflows. Prompt chaining improves accuracy by giving each call one job, with code gates between steps. Routing sends inputs to specialized prompts and lets cheap models handle easy cases. Parallelization either splits independent work for speed or votes for confidence. These patterns compose and stay testable because the path is defined in code.

Review Questions & Answers

A gate is ordinary code between chain steps that checks the intermediate output, for example that a date parses or a draft is under a length limit. It catches errors early and stops or retries before a bad intermediate result flows into later steps.

Each category gets a prompt optimized for it, which improves quality, and simple categories can be sent to a smaller, cheaper and faster model, reserving the most capable model for the hard cases.

Sectioning splits a task into different independent subtasks that run in parallel, such as separate security and performance reviews. Voting runs the same task several times and aggregates the answers to increase confidence in a noisy judgment.

The sequence of steps is fixed in code, so each step can be tested in isolation and you can predict exactly which calls happen for a given input. In an agent the model chooses the path at run time, so behaviour varies between runs.