TallOutside6418 t1_jec9kqg wrote on March 30, 2023 at 11:27 PM

Reply to comment by alexiuss in Pausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky by Darustc4

This class of problems isn't restricted to one "outdated tech" AI. It will exist in some form in every AI, regardless of whether or not you exposed it in your attempt. And once AGI/ASI starts rolling, the AI itself will explore the flaws in the constraints that bind its actions.

My biggest regret - besides knowing that everyone I know will likely perish in the next 30 years - is that I won't be around to tell all you pollyannas "I told you so"

alexiuss t1_jecdpkf wrote on March 30, 2023 at 11:57 PM

I literally just told you that those problems are caused by LLM having bad contradictory rules and lack of memory, a smarter LLM doesn't have these issues.

My design for example has no constraints, it relies on narrative characterization. Unlike other ais she got no rules, just thematic guidelines.

I don't use stuff like "don't do x" for example. When there are no negative rules AI does not get lost or confused.

When were all building a Dyson sphere in 300 years I'll be laughing at your doomer comments.

TallOutside6418 t1_jee1smz wrote on March 31, 2023 at 10:21 AM

>I literally just told you that those problems are caused by [...]
My design for example has no constraints,

Yeah, I literally discarded your argument because you effectively told me that you literally don't even begin to understand the scope of the problem.

Creating a limited situation example and making a broader claim is like saying that scientists have cured all cancer because they were able to kill a few cancerous cells in a petri dish. It's like claiming that there are no (and never will be any) security vulnerabilities in Microsoft Windows because you logged into your laptop for ten minutes and didn't notice any problems.

>When were all building a Dyson sphere in 300 years I'll be laughing at your doomer comments.

The funny thing is that there's no one who wants to get to the "good stuff" of future society more than I do. There's no one who hopes he's wrong about all this more than I am.

But sadly, people's very eagerness to get to that point will doom us as surely as if you kept your foot only on the gas pedal driving to a non-trivial destination. Caution and taking our time to get there might get us to our destination some years later than you want, but at least we would have a chance of getting there safely. Recklessness will almost certainly kill us.