AI skeptics and AI boosters are both wrong

"We’re in an intelligence explosion already and have been for decades," Andrej Karpathy said.

Earlier this month I attended an AI conference called The Curve in Berkeley. A lot of people there were “AGI-pilled.”

For example, I participated in a role-playing exercise organized by Daniel Kokotajlo, a co-author of the AI 2027 report. That report argues that AI systems will soon achieve human-level intelligence. Then they’ll rapidly improve themselves, leading to superhuman AI capabilities and an extreme acceleration of scientific discovery and economic growth.

I also attended a talk by another AGI-pilled writer, Nate Soares, who believes that superintelligent AI will kill everyone.

At the opposite end of the spectrum are skeptics who believe AI is not just overhyped but practically useless. This perspective wasn’t as well represented at the conference, but I appeared on a panel with perennial AI skeptic Gary Marcus. He laid out his case in a New York Times op-ed a couple of weeks ago:

These systems have always been prone to hallucinations and errors. Those obstacles may be one reason generative AI hasn’t led to the skyrocketing profits and productivity that many in the tech industry predicted. A recent study run by MIT’s NANDA initiative found that 95% of companies that did AI pilot studies found little or no return on their investment. A recent financial analysis projects an estimated shortfall of $800 billion in revenue for AI companies by the end of 2030.

Marcus argued that companies should “stop focusing so heavily on these one-size-fits-all tools and instead concentrate on narrow, specialized AI tools engineered for particular problems”—tools like AlphaFold, the Google DeepMind model for predicting protein structures.

Another skeptic is Ed Zitron, who has built a following making the case that OpenAI will never turn a profit because LLMs can’t generate value commensurate with their high inference costs. Zitron expects OpenAI to collapse in the next few years, and suggests that this could set off a tech industry contagion analogous to the failure of Lehman Brothers in 2008.

My view is between these extremes. I think today’s AI has genuinely impressive capabilities that are likely to improve further in the coming months and years. I think the AI industry is likely to be profitable in the long run, and that OpenAI’s basic business model is perfectly reasonable.

But I don’t think we’re very close to human-level intelligence. And I don’t think AI is about to drive the kind of massive social and economic changes that AGI-pilled folks expect.

So I recently found myself nodding along as Andrej Karpathy was interviewed on Dwarkesh Patel’s podcast. Karpathy co-founded OpenAI in 2015 before joining Tesla to lead its self-driving team. Since departing Tesla in 2022, the 39-year-old has become something of an elder statesman in the AI industry. People credit him with coining the phrase vibe coding back in February.

A key theme throughout the interview was a palpable frustration with both extremes in the AI debate.

“When I go on my Twitter timeline, I see all this stuff that makes no sense to me,” Karpathy said. He believes that fundraising needs have pushed AI leaders to make unrealistic promises about the pace of progress. At the same time, he said he was “overall very bullish on technology.”

If you have time, I encourage you to listen to the full two-hour conversation; it was packed with deep insights about the state of AI and its likely impact on the broader economy. But for those who don’t have two hours, I’ll highlight the bits I found most interesting.

Then I’ll discuss that MIT study finding that 95% of enterprise AI projects fail. Skeptics like Marcus love to cite it as evidence that AI is useless. But the study’s actual findings were more interesting than that — and they don’t really support the views of either AI skeptics or the AGI-pilled.