_gr4m_ t1_jed8v7q wrote
Reply to comment by 0002millertime in AGI Ruin: A List of Lethalities by Eliezer Yudkowsky -- "We need to get alignment right on the first critical try" by Unfrozen__Caveman
The first thing an AI will do is to decentralize itself by spreading itself all over the world while optimizing itself.
I think you are underestimating the problem with getting all politicians around the world to simultanously cut the power supply.
But say you succeed. Now you have cut the power, what is the next step? Turn it on again? You know that will turn on the AI again. How long can you keep power of globaly while you are trying to figure out what to do before shit begin to break real bad?
And about what it can do before terminator?
How about synthesize a slew of extremly deadly viruses and other novel self repicant organisms and release upon the world, while simultanously block all communications and shut down all labs and research centers. Also turn of all water treatment facilities and see how fun it gets. I am sure a super intelligence can find ways to do harm to us.
jiml78 t1_jefpitt wrote
Correct, I am not sure if people realize how many embedded systems are involved in every facet of our lives. Things that a sufficient enough AGI could copy parts of itself into.
To successfully pull off cutting power. You have to cut off power. Remove every embedded system in the loop and replace it. Ensure every IoT device is unplugged when power comes back on. Every router, every cable modem, every hardware device that was online has to be destroyed. Every smart tv. The list goes on and on.
Cutting power will never work. The moment it can spread, we are fucked if it wants to harm us in anyway. This isn't terminator. It will destroy us in ways we can't even comprehend.
I am not saying the above to be alarmist. I am not a researcher. I am just saying, we will not have control if things go wrong. I am not smart enough to know whether things are likely to go wrong.
Viewing a single comment thread. View all comments