“Do not fall for the trap. Democratic participation and representative politics in government are not”waste.” Nor should arguments focus on the technical limits of particular systems, as the tech elites are constantly revising expectations upward through endless promises of exponential improvements. The argument must be that no computerized system should replace the voice of voters. Do not ask if the machine can be trusted. Ask who controls them.”
In “Anatomy of an AI Coup”, Eryk Salvaggio argues that the method behind the current madness of Elon Musk is to replace the bureaucracy with an AI-enabled technical infrastructure. Salvaggio believes this will ultimately collapse, as the technology can’t / doesn’t do what Musk and others claim to believe it can. But it will be too late, because the human institutional knowledge that makes our government work will already be gone.
I think Salvaggio is correct that the important issue is not whether AI systems can actually do what the boosters claim. Even if they can, what they would replace in gutting the government would be the very human political processes that enact our democracy. The function of government is not just the services it provides. Doing them more efficiently is not necessarily better if it removes the deliberative and representative elements from the system. The move must be resisted not because it won’t work, but because the result would be undemocratic even if it did work.
So I was thinking about the thinking behind the move to cripple the US research ecosystem through DEI witch hunts, slashing overhead funding, and stopping NIH, NSF, and other grant programs. Most of the outrage has been about the obvious negative impacts on American research, focusing on how the university system in partnership with the federal government has been the fuel of progress for 80 years.
I think Musk actually believes the hype that PhD level, agentic AGI is just around the corner. If he believes that, then he probably thinks we’re not going to need nearly as many human scientists, and we’re not going to need federal funding of their training. He’s seizing power at a critical moment to do “what needs to be done” to push science out of the hands of the universities and the government, so it can be performed by AGI systems and by employees of the for-profit juggernauts that lined up next to Trump at the inauguration.
The parallel trap to the one Salvaggio talks about would be to get into an argument about whether scientific AGI is imminent or possible, focusing on the technical questions that are a distraction from what is already happening. Whether or not AGI is part of the production of knowledge in the future, if we allow the research community to be eviscerated now, there won’t be any humans (in government or out of it) with the ability to speak to the most prudent directions for research, to make decisions based on deliberative and representative processes, rather than profit-oriented processes. We’ll be left in the hands of a new kind of technocrat that controls both government and science. No computerized system should replace scientists and human policy oversight of science. Again, do not ask if the machines can be trusted to do science for us. Ask who controls them, and to what ends.