ReAct: interleaving reasoning and actions in language models

TL;DR

ReAct prompts a language model to produce reasoning traces and task actions in alternation. Reasoning lets the model plan, keep track of progress and handle surprises; actions let it fetch information from outside — in the paper, a simple Wikipedia API or a simulated environment — so its reasoning rests on what it actually observed. The authors report that on question answering and fact verification it avoided the hallucination and error-propagation problems of chain-of-thought reasoning, and that on two interactive decision-making benchmarks it beat imitation- and reinforcement-learning methods while being prompted with only one or two examples.

The problem

In 2022 two lines of work were progressing largely separately:

• Reasoning-only prompting, such as chain of thought (our breakdown), writes out intermediate steps but relies only on what is inside the model. It cannot look anything up or correct itself against the world, so an invented fact early in the chain propagates to the answer.
• Acting-only approaches use language models to pick actions in an environment, but without explicit reasoning about goals, progress or what went wrong.

The authors asked what happens when one model does both, interleaved in a single trajectory.

The method

ReAct keeps the task's normal actions and adds one more kind of step: a free-form thought. A thought does not change the environment and produces no observation; it only adds to the context the model conditions on next. A trajectory therefore alternates like this (schematic, not copied from the paper):

Thought:     what I need to find out, and how
Action:      search[some entity]
Observation: text returned by the tool
Thought:     what that tells me; what to do next
Action:      lookup[some phrase]
Observation: ...
Action:      finish[answer]

How the paper applies it:

• Frozen model, few-shot prompts. The main experiments prompt a large frozen language model (PaLM) with a few human-written trajectories of thoughts, actions and observations. The prompting results involve no training.
• Knowledge tasks. For HotpotQA (multi-hop question answering) and FEVER (fact verification), the action space is deliberately minimal: search for a Wikipedia page by name, look up a phrase within the current page, and finish with an answer. The authors say plainly that this is much weaker than modern retrievers — the point is to make the model retrieve through explicit reasoning.
• Decision-making tasks. For ALFWorld (a text-based household game) and WebShop (a simulated online shop), thoughts appear only sparsely: the model decides when to reason, typically to break down the goal, track subgoals and use common sense about where objects are likely to be.
• Controlled baselines. "Act" uses the same example trajectories with the thoughts removed, which isolates the contribution of reasoning. Standard prompting and chain-of-thought prompting are the other baselines.
• Hybrids with chain of thought. On the knowledge tasks the authors also switch between ReAct and self-consistent chain-of-thought: fall back to chain of thought when ReAct does not answer within a step budget, or fall back to ReAct when sampled chains disagree.

What the paper reports

The headline results, as reported by the authors in the abstract:

These are the authors' own measurements. Two points of context from the body of the paper. First, the decision-making comparison pits a prompted model against baselines trained with imitation or reinforcement learning — impressive for sample efficiency, but not a like-for-like comparison of the models. Second, on the knowledge tasks ReAct alone was not uniformly better: it slightly trailed chain-of-thought prompting on HotpotQA while beating it on FEVER, and the combined ReAct/chain-of-thought methods gave the best prompting results. All prompting methods remained well below supervised task-specific systems on those two benchmarks.

Why it mattered

In our reading, ReAct's contribution is a simple, inspectable loop — think, act, observe, repeat — that later agent work could build on. Two verifiable examples:

• Reflexion (Shinn et al., 2023 (external site)) adds written self-reflection between attempts at a task. It uses ReAct as the action generator in its decision-making experiments, and a ReAct agent is one of the setups in its question-answering experiments.
• The authors' own code repository (external site) points readers to LangChain's ReAct agent for applying the method to more tasks.

Limitations and open questions

Most of these come from the paper's own error analysis and discussion:

• Search quality dominates. On HotpotQA, uninformative search results were a major source of failure, and once derailed the model struggled to recover.
• Loops. A failure mode specific to ReAct was repeating earlier thoughts and actions without making progress. The authors suspect greedy decoding contributes.
• Less flexible reasoning. Interleaving actions constrains how reasoning is structured; in the authors' analysis ReAct made more reasoning errors than chain of thought, even though it hallucinated less.
• Prompt length limits. Complex tasks with large action spaces need more demonstrations than fit in a prompt. Fine-tuning on trajectories showed promise in the paper's experiments on HotpotQA.
• Reproducibility. The main experiments used PaLM, which was not openly accessible; the authors published all prompts and added GPT-3 results to compensate.
• Acting has risks. Connecting a model to real environments can mean retrieving private information or taking harmful actions; the paper limited its experiments to Wikipedia and WebShop.

Reading notes for practitioners

• Log every step. The thought–action–observation trace is the most useful debugging artifact an agent produces. Store it with each run.
• Bound the loop. Set a step budget and detect repeated actions — the paper's own failure analysis shows loops happen.
• Tool quality is agent quality. If search returns poor results, better prompting will not rescue the agent. Evaluate your tools on their own.
• Keep a fallback. The best prompting results came from switching between ReAct and chain of thought with simple rules. A fallback path is cheap insurance.
• Grant the fewest permissions possible. Give the agent only the actions it needs, especially anything that writes, sends or spends.