On Inherent AI Risk: “Extinction-Level Capitalism”
Matthew Butterick is a lawyer, programmer, writer, and designer. He’s written a long, interesting piece about the inherent risks of AI called Extinction-Level Capitalism. It is well-worth a read; I’ve excerpted several passages here but urge you read the whole thing.
In practice, certain people in a capitalist liberal democracy tend to get increasingly rich. Absent countermeasures, the wealthy gain control of the political apparatus, thwarting liberal-democratic norms. This tension between capital and politics is a long-considered topic. A key early work was, of course, Karl Marx’s Capital (about which more later). In the current era, Mancur Olson’s book The Rise and Decline of Nations set out how small groups with a shared interest (which could include capital concentration) can effectively undermine stable societies. More recently, economists Robert Reich (“How Capitalism is Killing Democracy”), James Galbraith (The Predator State), and Yanis Varoufakis (Technofeudalism: What Killed Capitalism) are among those who have studied the escalating political consequences of rising wealth inequality. The synthesis might be: as more wealth becomes concentrated in the hands of fewer citizens, liberal democracy weakens, because whichever citizens are losing economic relevance will also lose political relevance. A nation sending many of its citizens toward economic irrelevance risks becoming politically illiberal.
Sci-fi plots are optimized for cinematic impact. So as a metaphor for AI risk, they can lead to faulty intuitions. Among realistic AI risks, we can expect that most will be boring, slow, and depend on minimal extra technology. Whether AI will cause literal human extinction is esoteric—a lightning strike. But AI could easily induce future economic and political conditions that most Americans today would consider intolerable—a cancer that extinguishes a certain way of life. Nobody’s going to make a movie about boring AI risks. But they comprise the majority of worrisome AI outcomes.
Marx’s observation has a subtler implication too. New technology often holds itself out as the starting point of a narrative: from now on, everything is different. When we consider the technology alone, that narrative dominates. But when we zoom out and consider the historical context, the new technology becomes the current endpoint of a much longer political narrative.
What would Marx say to AI critics—social, legal, economic, political—that have arisen so far? Maybe that we’re missing the bigger picture. That as a human invention, AI may be the starting point of a new technological narrative. But as an affront to human workers, it continues a long tradition of capitalist technologies, beginning with the Industrial Revolution (if not earlier).
When we think about AI risk, we’re necessarily making guesses about the future. But when we frame AI in the narrow sense of new technology, we’re primarily considering a timeline that starts now. Whereas when we shift to thinking of AI as a capitalist instrument, we’re considering a timeline that starts centuries ago and has evolved continuously into the present. We can and should study those existing economic and political trends, because those will likely shape the future trajectory. Put differently: AI may be new. But it’s not immune to history.
“Technology always makes certain jobs obsolete; new ones will arise.” AI’s predicted labor replacement is unprecedented in three ways: the diversity of tasks replaced; its outsize effect on highly educated workers; and the backdrop of 50 years of wage stagnation. Automation-driven transitions aren’t necessarily easy, even when they’re narrow and the economy can absorb the workers. Those who handwave over the details should study historical examples. When you tell a large group of workers that their skills no longer have economic value, you risk a political and social tinderbox. Recall Carl Benedikt Frey’s comment: “the short run can be a lifetime”.
Along these lines, I expect that to succeed financially, Big AI will likely need to demolish a significant number of existing tech companies and grab their revenue for itself. By the process described above: Big AI essentially uses its tech customers as an R&D facility. Big AI licenses models to these companies. Tech companies compete to adapt their businesses to AI. Once a concept is proven, Big AI directly takes over that market. The labor-replacement story will grow into a company-replacement story. Many of those tech companies—and their shareholders in the public markets—may also find that AI is a poisoned chalice.
The value of the concentrated resource creates what Jeffrey Frankel calls “a political contest to capture ownership”, which in turn encourages the emergence of autocratic or oligarchic institutions captured by an economic elite who seek to retain control of the resource. The process is self-reinforcing in two ways. First: the economic elite uses its wealth to repress political opponents. Second: as the government derives more income from the concentrated resource, it relies less on taxation of citizens, which weakens democratic accountability.
I could have easily excerpted the whole thing.
Socials & More