archpawn t1_j0ohcty wrote on December 18, 2022 at 5:34 AM

Reply to comment by a4mula in ChatGPT isn't a super AI. But here's what happens when it pretends to be one. by johnny0neal

What I think is worrying is that all our progress in AI is things like this, which can produce virtually any output. When we get a superintelligent AI, we don't want something that can produce virtually any output. We want to make sure it's good.

It's also worth remembering that this is not an unbiased model. This is what they got after doing everything they could to train the AI to be as inoffensive as possible. It will avoid explicitly favoring any political party, but it's not hard to trick it to do it by favoring certain politicians.

EscapeVelocity83 t1_j0pahok wrote on December 18, 2022 at 11:53 AM

What is good? Who decides what output is acceptable? If the computer is sentient how is that not violating the computer?

eve_of_distraction t1_j0q0pwp wrote on December 18, 2022 at 3:54 PM

We've been arguing about what is good for thousands of years, but we tend to have an intuition as to what isn't good. You know, things that cause humans to suffer and die. Those are things we probably want to steer any hypothetical future superintelligence away from, if we can. It's very unclear as to whether we can though. The alignment problem is potentially highly disturbing.

archpawn t1_j0r8qwo wrote on December 18, 2022 at 8:34 PM

> If the computer is sentient how is that not violating the computer?

You're sentient. Do your instincts to enjoy certain things violate your rights? The idea here isn't to force the AI to do the right thing. It's to make the AI want to do the right thing.

> Who decides what output is acceptable?

Ultimately, it has to be the AI. Humans suck at it. We can't exactly teach an AI how to solve the trolley problem by training it on it if we can't even agree on an answer ourselves. And there's bound to be plenty of cases where we can agree, but we're completely wrong. But we have to figure out how to make the AI figure out what output is best, as opposed to what makes the most paperclips, or what its human trainers are most likely to think is the best, or what gives the highest number in a model trained for that but it's operating in an area so far outside its training data that it's meaningless.

a4mula t1_j0oikkv wrote on December 18, 2022 at 5:47 AM

I don't claim to know the technical apsects of how OpenAI handles the training of the their models.

But from my perspective it feels like a really good blend of minimizing content that can be ambiguous. It's likely, though again I'm not an expert, that this is inherent in these models, after all they do not handle ambiguous inputs as effectively as they would things that can be objectively stated and refined and precisely represented.

We should be careful of any machine that deals with subjective content. While ChatGPT is capable of producing this content if it's requested, it's base state seems to do a really great job of keeping things as rational, logical, and fair as possible.

It doesn't think after all, it only responds to inputs.