Google DeepMind’s Dream-RSI helps AI agents learn from past searches to cut compute
Researchers at Google and DeepMind have introduced Dream-RSI, a method that helps AI agents tackle hard search tasks more efficiently by replaying past runs and testing new strategies without repeating costly computations. The approach improved performance on coding and math benchmarks while cutting the number of attempts needed, and the code is available on GitHub.
The technique reuses data from completed searches, letting the agent simulate what would have happened if it had picked a different path. Because the results already exist, thousands of alternative strategies can be tested without calling the model or evaluator again, a process the researchers call dreaming.
Why exploration is the bottleneck for self-improving AI
Self-improving AI agents are designed to discover new algorithms, solutions to math problems, or faster code on their own. They propose a solution, evaluate the result, learn from it, and try again. Over thousands of attempts, they gradually work toward a good outcome.
For complex tasks, the search space grows enormous. The agent must decide which promising approaches to pursue, which to try in parallel, and which to abandon. That process, called exploration, can determine whether the search succeeds or wastes compute chasing the wrong ideas.
Existing approaches handle exploration in two ways. A fixed search strategy cannot learn from experience, so the agent may hit the same dead ends repeatedly. Adapting the strategy during a search avoids that rigidity but is expensive, because it takes many attempts to find out whether a new strategy works, and testing countless alternatives would mean repeating long runs.
How Dream-RSI replays recorded search histories
Dream-RSI changes how the agent searches, not the underlying AI model. The researchers propose reusing data from a completed search to test alternative strategies within the space the agent has already explored.
The agent records its attempts and their results as it searches, building a search tree of decisions and outcomes. The researchers compare this to navigating an unfamiliar area. On a first visit, you hit dead ends, double back, and struggle to find a route. Once you have a mental map, you can plan another route without visiting every spot again.
Dream-RSI applies that principle to recorded search histories. Rather than testing a new strategy in a live run, the agent runs it against stored results. This lets it check what would have happened if it had pursued other approaches first or abandoned some earlier. The system does not invent entirely new solutions during replay; it tests different decisions within the recorded search tree.
Because all results are stored in the search tree, thousands of alternative strategies can be tested without calling the model or evaluator again. The agent then plays through thousands of variations and selects the best one before putting it to work in a live search. The process repeats in a loop. After each search, the agent uses the recorded results to test better strategies, then applies the improved version to its next live run. Throughout this cycle, only the search strategy changes; the model generating the solutions remains untouched.
What Dream-RSI found on coding and math benchmarks
The researchers tested Dream-RSI with Gemini 3.1 Pro and Gemini 3.7 Flash on eight tasks across three areas. Each comparison used a baseline with the same starting conditions but a fixed search strategy.
One task asked the system to write the fastest possible program for a statistical calculation commonly used in genomics and finance. Dream-RSI’s program ran faster than the established libraries sklearn and glmnet on all six test datasets. With Gemini 3.1 Pro, average runtime fell from 3,587 to 2,931 milliseconds, while the number of attempts dropped from 550 to 317.
Dream-RSI also outperformed a competing system called SimpleTES, which needed 51,200 runs, compared with Dream-RSI’s 317 attempts on the same task. The same pattern held for math optimization tasks and efforts to write efficient GPU kernels, with comparable or better results at much lower computational cost.
On two GPU tasks, Dream-RSI matched performance while cutting the number of runs by a factor of up to 2.43. On two others, it delivered up to 2.09 times the performance within the same budget.
The researchers note that the learned strategy initially cuts the number of attempts, then increases it when progress stalls.
Where explicit instructions can hurt exploration
In a follow-up analysis, the researchers tested another way to use search histories. Instead of replaying them to test strategies, they condensed them into instructions telling the agent where to search. On one GPU task, the version with these instructions performed worse than the version without them.
The researchers suggest that overly specific directions can narrow the search space too much, preventing the agent from exploring a broader range of approaches. The same analysis showed how the learned strategy adjusted its effort. As performance improved, it initially reduced the number of attempts. When progress stalled, it increased the search effort again, which coincided with further gains.
How Dream-RSI fits into the broader push toward recursive self-improvement
Recursive self-improvement has drawn growing attention recently as a path toward agents that can discover better algorithms on their own. Google DeepMind introduced AlphaEvolve in 2025, using the same basic principle. Gemini Flash generates code proposals, Gemini Pro analyzes them, and an evolutionary algorithm selects the best versions. Dream-RSI works one level above that process by optimizing the search strategy itself.
AutoTTS takes a related approach, using a coding agent to search for algorithms in a simulated environment. Those algorithms decide when a language model should start, expand, or abandon reasoning paths. Google Research recently presented a different way to reuse past runs with WikiSkill, a system that records failures and successes in a wiki and turns them into reusable instructions for the agent. Dream-RSI’s follow-up analysis suggests that explicit instructions like these can restrict exploration on open-ended search tasks.
FAQ
What is Dream-RSI?
Dream-RSI is a method from Google and DeepMind that helps AI agents improve their search strategy by replaying past search histories. It tests alternative strategies against stored results instead of running new live searches, a process the researchers call dreaming.
How much compute does Dream-RSI save?
On a program-synthesis task, Dream-RSI cut attempts from 550 to 317 while improving runtime. On GPU kernel tasks, it matched the same performance with up to 2.43x fewer generations, or achieved up to 2.09x higher performance within the same budget. It also beat the competing system SimpleTES, which needed 51,200 runs on one task versus Dream-RSI’s 317.
Why do explicit instructions hurt some search tasks?
In a follow-up analysis, condensing recorded histories into direct instructions led to worse performance on one GPU task. The researchers attribute this to over-constrained exploration, where narrow directions prevent the agent from sampling a broader range of approaches.
Related coverage
- Google Home opens its doors to third-party AI agents through MCP
- How To Measure The Google Searches Your Social Posts Cause, Using Branded Queries In Search Console
This article summarizes reporting from the-decoder.com.