Noam Brown – Agent swarms, alignment, & recursive self-improvement
New episode with Noam Brown.
We talk about multi-agent, Navier-Stokes, and what the current explosion of maths progress tells us about what happens once you automate AI research.
And we also discuss how we will know if the models are actually aligned before we kick off RSI.
Watch on YouTube; listen on Apple Podcasts or Spotify.
Sponsors
- Jane Street has been interested in AI for a lot longer than you’d think, and not just for trading. In 2011, a full year before AlexNet and over a decade before ChatGPT launched, they hosted the first FOOM Debate between Eliezer Yudkowsky and Robin Hanson on whether AI would lead to an intelligence explosion. Now Jane Street is revisiting the question with a new panel: Daniel Kokotajlo, Ege Erdil, Ryan Greenblatt, and Jaime Sevilla, hosted by Ron Minsky in San Francisco this October. I expect it to be a truly excellent conversation. Register athttps://janestreet.com/dwarkeshjanestreet.com/dwarkesh
- Grok Bot has made handing off work super easy. It runs on its own cloud computer, where it installs the tools it needs to handle tasks end-to-end. For the podcast, we use Grok Bot to help produce our videos. You may have noticed that our ads feature animations of real websites. Getting these pixel-perfect used to mean running a convoluted, multi-step workflow ourselves. Now we just let Grok Bot handle it. Best of all, Grok Bot has learned all of our specs and preferences, so we don’t have to redescribe the task each time! Try Grok Bot for yourself athttps://x.ai/botx.ai/bot
- Antithesis gives you the confidence of a giant test suite without actually having to write one. Say you’re doing a major backend refactor: building enough tests to trust it could take weeks. Antithesis solves this by running your software through countless simulated worlds, injecting faults and hunting for failures. On any PR, you can turn a dial to decide exactly how much testing you want. And because every run is fully deterministic, agents can branch off the moment a bug appears, rewind it, inspect memory, and replay it, all while the original test keeps running. Learn more athttps://antithesis.com/dwarkeshantithesis.com/dwarkesh
Timestamps
(00:00:00) – Multi-agent and Navier-Stokes
(00:15:28) – How will AI firms work?
(00:22:02) – What math progress tells us about recursive self improvement
(00:40:22) – Hugging Face and alignment
(01:01:18) – The internal/external model gap
(01:08:34) – Chain of thought is degrading
(01:14:12) – How will we know when alignment is solved?
Transcript
00:00:00 – Multi-agent and Navier-Stokes
Dwarkesh Patel
Today, I’m chatting with Noam Brown, who is a researcher at OpenAI. He was one of the foundational contributors to what became o1 and the reasoning models. Now he’s working on multi-agent systems. Speaking of which, you guys announced last week that you solved one of the Millennium Prize Problems with a system of 10,000 different AI agents that spent 130 billion tokens over 88 hours.
One of the reasons I’m interested in talking to you is that you were among the first people, maybe two or three years ago, who were thinking about how the reasoning models would allow us to see into the future. Because if you scale up inference compute, you
