Futurists have argued for years about whether the development of AGI will look more like a breakthrough within a small group (“fast takeoff”), or a continuous acceleration distributed a…

In 2008, thousands of blog readers - including yours truly, who had discovered the rationality community just a few months before - watched  Robin Hanson debate Eliezer Yudkowsky  on the future of AI. Robin thought the AI revolution would be a gradual affair, like the Agricultural or Industrial Revolutions. Various people invent and improve various technologies over the course of decades or centuries. Each new technology provides another jumping-off point for people to use when inventing other technologies: mechanical gears → steam engine → railroad and so on. Over the course of a few decades, you’ve invented lots of stuff and the world is changed, but there’s no single moment when “industrialization happened”. Eliezer thought it would be lightning-fast. Once researchers started building human-like AIs, some combination of adding more compute, and the new capabilities provided by the AIs themselves, would quickly catapult AI to unimaginably superintelligent levels. The whole process could take between a few hours and a few years, depending on what point you measured from, but it wouldn’t take decades. You can imagine the graph above as being GDP over time, except that Eliezer thinks AI will probably destroy the world, which might be bad for GDP in some sense. If you come up with some way to measure (in dollars) whatever kind of crazy technologies AIs create for their own purposes after wiping out humanity, then the GDP framing will probably work fine. Crossposted from the Astral Codex Ten Podcast .

Followup to:  Life’s Story Continues, Surprised by Brains, Cascades, Cycles, Insight, Recursion, Magic, Engelbart: Insufficiently Recursive, Total Na…

Continuation of:  Recursive Self-Improvement …

As part of my work for Open Philanthropy I’ve written a draft report on AI takeoff speeds, the question of how quickly AI capabilities might improve as we approach and surpass human-level AI. Will human-level AI be a bolt from the blue, or will we have AI that is nearly as capable many years earlier? Most of the analysis is from the perspective of a compute-centric framework, inspired by that used in the Bio Anchors report, in which AI capabilities increase continuously with more training compute and work to develop better AI algorithms. This post doesn’t summarise the report. Instead I want to explain some of the high-level takeaways from the research which I think apply even if you don’t buy the compute-centric framework. The framework h/t Dan Kokotajlo for writing most of this section This report accompanies and explains (h/t Epoch for building this!), a user-friendly quantitative model [...]

We aren’t convinced by any of the arguments we’ve seen to expect large discontinuity in AI progress above the extremely low base rate for all technologies. However this topic is controversial, and many thinkers on the topic disagree with us, so we consider this an open question. Details Definitions We say a technological discontinuity has...

This post is part 4 in our sequence on Modeling Transformative AI Risk. We are building a model to understand debates around existential risks from a…

Back in 2008 my ex-co-blogger Eliezer Yudkowsky and I discussed his “AI foom” concept, a discussion that we recently spun off into a book. I’ve heard for a while that Nick Bostrom was working on a book elaborating related ideas, and this week his

Once a lab trains AI that can fully replace its human employees, it will be able to multiply its workforce 100,000x. If these AIs do AI research, they could develop vastly superhuman systems in under a year:

Since 1900, the global economy has grown by about 3% each year, meaning that it doubles in size every 20–30 years. I’ve written a report assessing whether significantly faster growth might occur this century. Specifically, I ask whether growth could be ten times faster, with the global economy growing by 30% each year. This would […]

In terms of the depth and range of topics, this episode is the best I’ve done. No part of my worldview is the same after talking with Carl Shulman. He’s the most interesting intellectual you’ve never heard of. We ended up talking for 8 hours, so I’m splitting this episode into 2 parts. This part is about Carl’s model of an intelligence explosion, which integrates everything from: * how fast algorithmic progress & hardware improvements in AI are happening, * what primate evolution suggests about the scaling hypothesis, * how soon before AIs could do large parts of AI research themselves, and whether there would be faster and faster doublings of AI researchers, * how quickly robots produced from existing factories could take over the economy. We also discuss the odds of a takeover based on whether the AI is aligned before the intelligence explosion happens, and Carl explains why he’s more optimistic than Eliezer. The next part, which I’ll release next week, is about all the specific mechanisms of an AI takeover, plus a whole bunch of other galaxy brain stuff. Maybe 3 people in the world have thought as rigorously as Carl about so many interesting topics. This was a huge pleasure. Watch on  YouTube . Listen on  Apple Podcasts ,  Spotify , or any other podcast platform. Read the full transcript  here . Follow me on Twitter for updates on future episodes. Timestamps ( 00:00:00 ) - Intro ( 00:01:32 ) - Intelligence Explosion ( 00:18:03 ) - Can AIs do AI research? ( 00:39:00 ) - Primate evolution ( 01:03:30 ) - Forecasting AI progress ( 01:34:20 ) - After human-level AGI ( 02:08:39 ) - AI takeover scenarios This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkeshpatel.com

It’s easy to dismiss alarming AI-related predictions when you don’t know where the numbers came from. For example: what if we told you that within 15 years, it’s likely that we’ll see a 1,000x improvement in AI capabilities in a single year? And what if we then told you that those improvements would lead to explosive economic growth unlike anything humanity has seen before? You might think, “Congratulations, you said a big number — but this kind of stuff seems crazy, so I’m going to keep scrolling through Twitter.” But this 1,000x yearly improvement is a prediction based on *real economic models* created by today’s guest Tom Davidson, Senior Research Analyst at Open Philanthropy. By the end of the episode, you’ll either be able to point out specific flaws in his step-by-step reasoning, or have to at least consider the idea that the world is about to get — at a minimum — incredibly weird. Links to learn more, summary and full transcript. As a teaser, consider the following: Developing artificial general intelligence (AGI) — AI that can do 100% of cognitive tasks at least as well as the best humans can — could very easily lead us to an unrecognisable world. You might think having to train AI systems individually to do every conceivable cognitive task — one for diagnosing diseases, one for doing your taxes, one for teaching your kids, etc. — sounds implausible, or at least like it’ll take decades. But Tom thinks we might not need to train AI to do every single job — we might just need to train it to do one: AI research. And building AI capable of doing research and development might be a much easier task — especially given that the researchers training the AI are AI researchers themselves. And once an AI system is as good at accelerating future AI progress as the best humans are today — and we can run billions of copies of it round the clock — it’s hard to make the case that we won’t achieve AGI very quickly. To give you some perspective: 17 years ago we saw the launch of Twitter, the release of Al Gore’s *An Inconvenient Truth*, and your first chance to play the Nintendo Wii. Tom thinks that if we have AI that significantly accelerates AI R&D, then it’s hard to imagine not having AGI 17 years from now. Wild. Host Luisa Rodriguez gets Tom to walk us through his careful reports on the topic, and how he came up with these numbers, across a terrifying but fascinating three hours. Luisa and Tom also discuss: • How we might go from GPT-4 to AI disaster • Tom’s journey from finding AI risk to be kind of scary to really scary • Whether international cooperation or an anti-AI social movement can slow AI progress down • Why it might take just a few years to go from pretty good AI to superhuman AI • How quickly the number and quality of computer chips we’ve been using for AI have been increasing • The pace of algorithmic progress • What ants can teach us about AI • And much more Chapters: Rob’s intro (00:00:00) The interview begins (00:04:53) How we might go from GPT-4 to disaster (00:13:50) Explosive economic growth (00:24:15) Are there any limits for AI scientists? (00:33:17) This seems really crazy (00:44:16) How is this going to go for humanity? (00:50:49) Why AI won’t go the way of nuclear power (01:00:13) Can we definitely not come up with an international treaty? (01:05:24) How quickly we should expect AI to “take off” (01:08:41) Tom’s report on AI takeoff speeds (01:22:28) How quickly will we go from 20% to 100% of tasks being automated by AI systems? (01:28:34) What percent of cognitive tasks AI can currently perform (01:34:27) Compute (01:39:48) Using effective compute to predict AI takeoff speeds (01:48:01) How quickly effective compute might increase (02:00:59) How quickly chips and algorithms might improve (02:12:31) How to check whether large AI models have dangerous capabilities (02:21:22) Reasons AI takeoff might take longer (02:28:39) Why AI takeoff might be very fast (02:31:52) Fast AI takeoff speeds probably means shorter AI timelines (02:34:44) Going from human-level AI to superhuman AI (02:41:34) Going from AGI to AI deployment (02:46:59) Were these arguments ever far-fetched to Tom? (02:49:54) What ants can teach us about AI (02:52:45) Rob’s outro (03:00:32) Producer: Keiran Harris Audio mastering: Simon Monsour and Ben Cordell Transcriptions: Katy Moore

“Do you remember seeing these photographs of generally women sitting in front of these huge panels and connecting calls, plugging different calls between different numbers? The automated version of that was invented in 1892. However, the number of human manual operators peaked in 1920 -- 30 years after this. At which point, AT&T is the monopoly provider of this, and they are the largest single employer in America, 30 years after they’ve invented the complete automation of this thing that they’re employing people to do. And the last person who is a manual switcher does not lose their job, as it were: that job doesn’t stop existing until I think like 1980. So it takes 90 years from the invention of full automation to the full adoption of it in a single company that’s a monopoly provider. It can do what it wants, basically. And so the question perhaps you might have is why?” — Michael Webb In today’s episode, host Luisa Rodriguez interviews economist Michael Webb of DeepMind, the British Government, and Stanford about how AI progress is going to affect people’s jobs and the labour market. Links to learn more, summary and full transcript. They cover: The jobs most and least exposed to AI Whether we’ll we see mass unemployment in the short term  How long it took other technologies like electricity and computers to have economy-wide effects Whether AI will increase or decrease inequality Whether AI will lead to explosive economic growth What we can we learn from history, and reasons to think this time is different Career advice for a world of LLMs Why Michael is starting a new org to relieve talent bottlenecks through accelerated learning, and how you can get involved Michael’s take as a musician on AI-generated music And plenty more If you’d like to work with Michael on his new org to radically accelerate how quickly people acquire expertise in critical cause areas, he’s now hiring! Check out Quantum Leap’s website . Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type ‘80,000 Hours’ into your podcasting app. Or read the transcript. Producer and editor: Keiran Harris Audio Engineering Lead: Ben Cordell Technical editing: Milo McGuire and Dominic Armstrong Additional content editing: Katy Moore and Luisa Rodriguez Transcriptions: Katy Moore

Or: Why AI Takeover Might Happen Before GDP Accelerates, and Other Thoughts On What Matters for Timelines and Takeoff Speeds I think world GDP (and economic growth more generally) is overrated as a metric for AI timelines and takeoff speeds. Here are some uses of GDP that I disagree with, or at least think should be accompanied by cautionary notes: Timelines: Ajeya Cotra thinks of transformative AI as “software which causes a tenfold acceleration in the rate of growth of the world economy (assuming that it is used everywhere that it would be economically profitable to use it).” I don’t mean to single her out in particular; this seems like the standard definition now. Takeoff Speeds: Paul Christiano argues for […]