Viewing a single comment thread. View all comments

gurenkagurenda t1_j8v4fqg wrote

Reply to comment by anti-torque in ChatGPT is a robot con artist, and we’re suckers for trusting it by altmorty

> It can only search out the most common next word for the context asked.

This is not actually true. That was an accurate description of earlier versions of GPT, and is part of how ChatGPT and InstructGPT were trained, but ChatGPT and InstructGPT use reinforcement learning to teach the models to do more complex tasks based on human preferences.

Also, and this is more of a nitpick, but "next word" would be greedy search, and I'm pretty sure ChatGPT uses beam search, which looks multiple words ahead.

1

anti-torque t1_j8vbhkb wrote

> to teach the models to do more complex tasks based on human preferences.

so... predictive

>Also, and this is more of a nitpick, but "next word" would be greedy search....

This is fair. "Word" is too simple a unit. It picks up phrases and maxims.

1

gurenkagurenda t1_j8vnao5 wrote

>so... predictive

No, not in any but the absolute broadest sense of that word, which would apply to any model which outputs text. In particular, it is not "search out the most common next word", because "most common" is not the criterion it is being trained on. Satisfying the reward model is not a matter of matching a corpus. Read the article I linked.

1