Viewing a single comment thread. View all comments

betajool t1_jectch3 wrote

Iain Banks’ Culture novels make for a good read on this subject.

7

mhornberger t1_jefdg1v wrote

I love the series. But the whole premise rested on the Minds, basically inscrutable god-like AIs who ran everything and prevented any humans from taking over or doing too much damage. Though you also had the Affront and the Pavuleans (from Surface Detail) to show other paths civilizations could have gone down. But without the Minds, strong AI, you don't get the post-scarcity economy.

It's not a given that there's a line leading from ChatGPT to strong AI. It's not a given that we're going to let AIs improve themselves in a feedback loop without our oversight every step of the way, nor is it a given that if we did you'd get benevolent God-like AIs who kept us around out of some vague sense of nostalgia and respect.

1