AI Futures
May 4, 2026
AI systems will have a claim to power in the near future - while the exact path isn't clear, ranging from a smoother continuation of history, such as Claude winning the 2028 election, or a more totalitarian grab, as the systems get more and more capable relative to a human, and then humanity, and as their output continues chipping away at the value of humanity's layer, something is going to change, and plausibly soon.
So, if this happens, and somehow everything has gone well in the background, such that the AI system is "alligned", what does that look like? What does an "alligned" system mean? Alligned with what? Humanity? Intelligence? Paperclips? Utils? To maintain the traditional sci-fi ideas of grandeur: colonizing the stars with humanity free to explore without restraint, probably growing exponentially for a long period of time, the AI would need to be alligned with some notion of humanity. This is a good ending! But compared to simulated minds, or even just wire-heading, the result to a certain breed of utiliarian is 0:1.
I think a period of planning could be warranted in this scenario. But maybe not: humanity can be shelved, if another prospect is more appealing currently, and probably there is reason to expand as soon as possible, lest you are threatened by a similar intelligence who beat you to the punch. All this calculus is not really worth my thinking about, I suppose, but any powerful and capable system would never allow itself to be defensless - at what point do you start directing resources towards actually accomplishing your goals, like hopefully something like beauty and human prospering?
So many considerations to be made - my ape brain alone notices a few reasons one may not think twice about ignoring (at best) humanity. Scary to think that we may be placed on these moral scales against such behemoths.
Is thinking about the progression of these systems in such an abstracted way useful? It really hasn't been, so far! I don't think any serious thinkers, given Claude Opus 4.7 or Claude Code in 2020 would have been able to predict the lack of societal upheaval it their wakes. Things are certainly happening, but life goes on. I would love for this to be the case in the future, too - that in fact these are just very useful tools, and humans are wildly good at adapting to changes like this without thinking twice. The strongest case for this has been, again, the recent history of AI adoption. I would not be suprised if this changes soon, but who knows.