Agentic Workflow

Also called agent workflow, agentic workflows, agentic AI workflow

Quick answer

An agentic workflow is a process in which a large language model finishes a task through several steps instead of one answer. Along the way the model can make a plan, call tools such as web search or code execution, check its own output, and revise it. The goal is a better result than a single prompt gives.

~20 sec read

011 min

The problem an agentic workflow solves

A single prompt gives a language model one pass. The model reads the request and writes the answer from start to finish. It cannot look anything up, test what it wrote, or go back and change it.

A person doing the same job would not work that way. A developer writes code, runs it, reads the error and fixes it. A researcher searches, reads, and searches again.

The one-pass approach fails in predictable ways:

  • The model cannot check a fact it does not remember, so it may state a wrong fact with full confidence. This failure is called hallucination.
  • It cannot run its own code, so a bug stays in the answer.
  • It cannot notice that its first draft left out part of the request.

An agentic workflow gives the model the missing moves. Andrew Ng described the idea in The Batch, the weekly newsletter from DeepLearning.AI. With an agent workflow, he wrote, a developer can ask the model to iterate over a document many times instead of writing it in one pass. A typical run plans an outline, decides which searches are needed, writes a draft, reads the draft critically and revises it.

Each pass starts from the output of the previous pass. The answer can improve instead of being fixed at the first attempt.

021 min

What counts as an agentic workflow

The term is used in two ways, and the difference changes what you build.

In the broad sense, which this entry follows, it means any multi-step process where a model plans, uses tools, checks or revises its work. Ng uses it this way.

In the narrow sense, Anthropic separates two kinds of system in its December 2024 guide to building agents. Workflows are systems where models and tools are orchestrated through predefined code paths. Agents are systems where models dynamically direct their own processes and tool use. In that vocabulary the programmer fixes the order of steps in a workflow, and the model chooses the order of steps in an agent.

Anthropic calls both kinds "agentic systems". Most real products sit between the two: the code fixes some steps and leaves others to the model. This entry covers the whole range and says which end it means when the difference matters.

032 min

How an agentic workflow works

The running example is a product manager who asks a system: "Summarise how three competitors price their plans." A one-pass model would answer from memory, and its prices could be out of date. An agentic workflow handles the same request in five steps.

  1. Plan. The model writes a short list of steps: find each pricing page, copy the plan names and prices, put them in one table.
  2. Act. The model asks to use a tool, which is a function the surrounding program can run on its behalf, such as web search. The program runs the search and returns the results to the model as new input.
  3. Observe. The model reads the returned pages and fills in the table.
  4. Check. The model compares its table with the pages. It finds that one competitor lists a yearly price while the other two list monthly prices.
  5. Revise or stop. The model converts the price so the three match, then checks again. When the check passes, or when a step limit is reached, the program stops the loop.

The interesting step is the fourth. The model's own output goes back in as input, so an error in the first draft can be found and removed. In a one-pass answer there is no such step.

The loop itself is ordinary code. The program calls the model, runs any tool the model asked for, adds the result to the conversation and calls the model again. The model never runs anything by itself. It only asks, and the program decides whether to do it. This is also where limits are enforced: a maximum number of steps, a budget, and a list of tools the model may use.

042 min

The patterns agentic workflows are built from

Two public catalogues name the common building blocks. They overlap, and a real system often combines several.

Ng's letter names four patterns:

  • Reflection: the model examines its own work to find ways to improve it.
  • Tool use: the model is given tools such as web search or code execution.
  • Planning: the model comes up with a multistep plan and carries it out.
  • Multi-agent collaboration: more than one agent works together, splitting up tasks and debating ideas.

Anthropic's guide describes five workflow shapes, each with a different job:

PatternWhat it doesUse it when
Prompt chainingEach model call processes the output of the previous oneThe task splits cleanly into fixed steps
RoutingClassifies an input and sends it to a specialised next stepInputs differ in kind and need different handling
ParallelizationRuns several model calls at once and combines the outputs in codeParts are independent, or you want several attempts
Orchestrator-workersA central model breaks the task into parts and hands them to worker modelsYou cannot predict the subtasks in advance
Evaluator-optimizerOne model call writes, another critiques, in a loopYou have clear criteria and revision helps

Anthropic says prompt chaining is ideal when a task can be cleanly decomposed into fixed subtasks, and that its main goal is to trade off latency for higher accuracy by making each call an easier task. The orchestrator-workers shape is the most flexible, because the central model decides at run time what the subtasks are.

051 min

A concrete example: HumanEval

The clearest published numbers come from coding. In Ng's letter, the results were collected from several research teams on HumanEval, a widely used coding benchmark. The numbers are not from one controlled experiment, and the letter compares different models and setups.

Ng's analysis put GPT-3.5 at 48.1% correct with one zero-shot prompt, and GPT-4 at 67.0%. "Zero-shot" means the prompt contains no worked examples. Wrapped in an agent loop, GPT-3.5 reached up to 95.1% in the results he collected.

The comparison shows the size of the effect. Moving from GPT-3.5 to GPT-4 raised the score by about 19 points. Adding the loop to the older model raised it by about 47 points. Ng wrote that the improvement from the loop dwarfs the improvement from the newer model.

A research paper reports a similar result on the same benchmark. In Reflexion (Shinn and others, 2023), an agent writes code, receives feedback from tests, then writes a short note about what went wrong and tries again. The authors report 91% pass@1 on HumanEval, against 80% for GPT-4. Pass@1 is the share of tasks that a single submitted answer solves.

Both sources have a limit worth stating. HumanEval tasks come with tests that tell the loop whether it succeeded. That clear signal is what makes the loop work here, and most business tasks do not have one.

061 min

A second case: ReAct and live lookups

Coding has tests. Question answering does not, so the loop needs a different kind of feedback. The ReAct paper (Yao and others, 2022) shows what that looks like.

ReAct prompts a model to write a line of reasoning, then an action, then to read what the action returned, and to repeat. The reasoning helps the model update its plan and handle exceptions. The actions let it reach outside sources for facts it does not hold.

On question answering (HotpotQA) and fact checking (Fever), the model could query a simple Wikipedia interface. The authors report that this addressed hallucination and error propagation, two problems that appear when a model reasons step by step with no outside check. On two interactive benchmarks, ALFWorld (a text-based household task game) and WebShop (a simulated online store), ReAct beat imitation and reinforcement learning methods by an absolute success rate of 34% and 10%, respectively.

The one variable that differs from the coding case is the feedback. In HumanEval the feedback is a test result, which is exact. In ReAct it is a retrieved page, which the model must read and judge. The coding loop checks against a rule. The lookup loop checks against evidence that may itself be wrong or incomplete. That is why the lookup case needs more care in how results are read.

072 min

What this means for your work

Each role makes a different decision.

  • A product manager decides what "done" means for the task. A loop without a stopping rule runs until it hits a limit. Write the pass condition in words first, such as "every price is traced to a page opened in this run".
  • An engineer decides where the loop ends and what it records. Set a maximum number of steps and a cost budget. Log every model call and every tool result, because the cause of a wrong answer is usually in step three, not step six.
  • A designer decides what the user sees while the loop runs. A user waiting 90 seconds with a blank screen assumes failure. Showing the current step, and allowing the user to stop or correct the run, turns waiting into supervision.
  • A founder decides how to price and promise the feature. One request now costs several model calls, so cost per task, not cost per call, is the number to track.

Limit what the model is allowed to do. A model that can only read web pages cannot delete a database. Guardrails are checks placed in the code around the model, such as refusing a tool call outside an approved list. Put them in code, not only in the prompt.

Start with the simplest version that could work. One call with good context first, then a fixed chain, then a loop where the model picks the steps.

081 min

What an agentic workflow costs

Anthropic states the trade directly: agentic systems often trade latency and cost for better task performance, and you should consider when that trade makes sense.

The arithmetic is simple. A five-step chain makes at least five model calls instead of one. A loop that retries adds more. Each call also carries the growing conversation, so later calls read more text than earlier ones. Time adds up the same way, because each step waits for the one before it.

Anthropic also names a quality cost for the more autonomous kind: the autonomous nature of agents means higher costs and the potential for compounding errors. A small error in an early step is read as fact by every later step.

Two controls keep this bounded. Cap the number of steps. Use a small, cheap model for simple steps such as routing, and keep the larger model for the steps that need it.

092 min

What an agentic workflow does not solve

Tasks that one call already handles. Sorting incoming support emails into five labels needs one well-written prompt. Adding a loop makes it slower and more expensive with no gain. Anthropic's advice is to find the simplest solution possible and increase complexity only when needed. That might mean not building an agentic system at all.

Long chains of small errors. Suppose each step is correct 90% of the time. Over ten steps, the chance that every step is correct is 0.9 multiplied by itself ten times, which is about 35%. This is an illustration, not a measurement of any system. It shows why a ten-step run can fail even when each step looks reliable.

Loops with nothing to check against. The model can only improve its work when something tells it what is wrong. Tests, a database lookup, and a schema validator all tell it. Its own opinion of its draft is weaker evidence. If you cannot name the check, the loop may add cost without adding quality.

Runs that never stop. A model can keep retrying a step that cannot succeed, for example calling a tool that returns an error each time. A step limit is a requirement, not an option.

A common overclaim is that a loop removes hallucination. It reduces it where the loop retrieves or tests something. The model can still read a retrieved page wrongly, or accept a wrong page.

101 min

Agentic workflow vs. nearby concepts

The deciding fact is who chooses the next step. If the program's code chooses it, the system is a workflow. If the model chooses it, the system is an agent. Most of the confusion comes from using one word for both.

ConceptWho picks the next stepHow it differs
Agentic workflowCode, the model, or a mixThe whole family of multi-step designs
AgentThe modelOne end of the range: the model directs its own process and tool use
Prompt chainingCodeA fixed sequence of calls; one pattern inside an agentic workflow
Multi-agent systemThe agents, in conversationSeveral models with separate roles, which may or may not be inside a workflow

When the answer is "use both", the usual order is to build a fixed chain first and let the model choose steps only inside the parts where the path cannot be predicted ahead of time.

111 min

Where the evidence is contested

Self-correction without outside feedback. Huang and others (2023, published at ICLR 2024) tested whether a model can fix its own reasoning errors with no external signal. They report that models struggle to self-correct reasoning without external feedback, and that performance sometimes gets worse after the attempt. Their objection applies to reflection loops where the only check is the model's own opinion. It does not apply to loops that run tests or retrieve sources, which is the situation in the HumanEval and ReAct cases above.

How much the headline numbers say. Ng's HumanEval comparison gathers results from different teams, models and setups. It supports the point that loops help on code. It does not give a fair measure of how much a loop adds on any other task. The 95.1% is also an upper result, written "up to".

The word itself. Ng uses "agentic" for a loop and treats it as a matter of degree. Anthropic reserves "agent" for systems where the model directs its own process and calls the fixed-path kind a workflow. Both are published and widely read, so you will meet both. State which one you mean in specifications and reviews.

While the evidence is unsettled, a practical rule follows: add a loop where you can name the check, and measure the result against the single-call version on your own tasks before you ship it.

121 min

How agentic workflows changed since

The pieces came first, in research papers. ReAct (2022) combined reasoning with actions. Reflexion and Self-Refine (2023) added feedback and revision. Self-Refine reports that, across seven tasks, outputs made with its method improved by about 20% absolute on average over one-step generation, using the same model as writer and critic.

AutoGen (Wu and others, 2023) packaged the multi-agent idea as an open-source framework in which multiple agents converse to complete tasks.

The name arrived next. In March 2024, Ng's series of letters on agentic design patterns gave practitioners the words reflection, tool use, planning and multi-agent collaboration, and moved the topic from papers to product teams.

The most recent change is a split in meaning. In December 2024, Anthropic's guide separated workflows with predefined paths from agents that direct themselves, and advised teams to start with the simpler kind. Current practice follows that advice more than the early enthusiasm for fully autonomous agents. Details of what models and tools can do change quickly, so check dates on any claim about current capability.

?6 questions

Questions people ask

What is an agentic workflow in simple terms?

It is a way of using a language model in several steps instead of one. The model can plan, use tools, check its work and revise it before giving the final answer. The steps are repeated until a check passes or a limit is reached.

Is an agentic workflow the same as an AI agent?

No. An AI agent is a system where the model decides its own next steps. An agentic workflow is the wider family. It includes fixed chains where code sets the order, and agents where the model sets it.

Do I need several agents to build an agentic workflow?

No. One model in a loop with a tool and a check is already an agentic workflow. Multiple agents help when the task has separate roles or parts that can run side by side.

How much does an agentic workflow cost to run?

It costs at least one model call per step, so a five-step run costs at least five calls. Later calls also read the growing conversation. Track cost per finished task, not per call.

How do I stop an agentic workflow from looping forever?

Put the limit in code. Set a maximum number of steps and a cost budget, and end the run when either is reached. Do not rely on the model to decide to stop.

Does an agentic workflow remove hallucinations?

No. It reduces them where the loop looks facts up or runs tests. The model can still misread a source or accept a wrong one, so check the sources it used.

▶3 videos

Watch

  • Understanding Agentic Workflow

  • Agentic AI Frameworks Explained: Workflows, Multi-Agent, & Production

    IBM Technology

  • Agentic AI: Workflows vs. agents

    Google Cloud Tech

§7 sources

Sources

Show all 7 sources
  1. Andrew Ng, "Agentic Design Patterns Part 1: Four AI agent strategies that improve GPT-4 and GPT-3.5 performance", The Batch, DeepLearning.AI, 20 March 2024.

  2. "Building effective agents", Anthropic Engineering, December 2024.

  3. Shunyu Yao and others, "ReAct: Synergizing Reasoning and Acting in Language Models", 2022.

  4. Noah Shinn and others, "Reflexion: Language Agents with Verbal Reinforcement Learning", 2023.

  5. Aman Madaan and others, "Self-Refine: Iterative Refinement with Self-Feedback", 2023.

  6. Jie Huang and others, "Large Language Models Cannot Self-Correct Reasoning Yet", ICLR 2024.

  7. Qingyun Wu and others, "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation", 2023.

Keep reading

More from AI & Engineering

All of AI & Engineering
All of AI & Engineering

Missing a concept?

Tell us what to write next. We’ll email you when it’s live.

Several? Separate them with commas.

Only to tell you when it’s live.

How we use your email: privacy.