[deleted]

I recently complied and went through a reading / watching list, going from basic NLP to ChatGPT:

- NLP Demystified to learn NLP, especially transformers

- Medium article nicely summarizing the main points of GPT-1, 2 and 3

- GPT-1 lecture and GPT-1 paper to learn about general idea of GPT-like models

- GPT-2 lecture and GPT-2 paper to learn about large scale self-supervised pretraining that fuels GPT training

- GPT-3 lecture 1 and GPT-3 lecture 2 and GPT-3 paper to learn about GPT-3

- InstructGPT page and InstructGPT paper to learn about InstructGPT, the sibling model of ChatGPT; as far as I understand, this is the same as "GPT-3.5"

- ChatGPT page to learn about differences between InstructGPT and ChatGPT, which are relatively small as far as I understand; it is also sometimes called "fine-tuned GPT-3.5", AFAIK

Bonus reading (heavy math warning, experience with RL required!):

- the main difference between GPT-3 and InstructGPT/ChatGPT is reinforcement learning with human feedback (RLHF)

- RLHF is based on Proximal Policy Optimization algorithm

- PPO page and PPO paper

> I’d say that even calling it “AI” is misleading because it’s not intelligent.

I’d say it’s misleading for a different reason. We don’t know what intelligence is. Every time a computer can perform a task, that task is no longer considered a test of “intelligence”. Well, if every task is reducible to something unintelligent then perhaps intelligence was really a mirage in the first place.

If an A.I accomplishing a task means it's not actually intelligence then A.I is an impossible term. Let's pack our bags and call it a day lol

I guess there will always be room for interpretation on what intelligence is since after all is just a label we put on things. What I was thinking though was something like intelligence in animals and how that's considered intelligence without necessarily comparing animals to human beings.

Thank you, I'll go through that. If I may ask, how did you get there? I only seem to get clickbait articles and videos no matter what keywords I google. Is there any kind of special "prompt" you've to google to get the results you want like with Stable Diffusion? :D

And in your example, continuing the pattern, just because a computer can’t do that it’s not intelligent? That’s an extremely narrow view of intelligence.

Animals evolved to be good at some very specific things to fill an ecological niche. Humans evolved to be good at different things. I mainly see people discounting computer capabilities by measuring them against humans. Things that are easy for us are hard for computers and vice versa. But it’s highly unlikely that computers would be at all similar to people, since we’ve been specializing for millions of years.

Computers are machines, merely calculators. I'd not say a calculator is smart because it can do advanced math operations faster than any human being ever could.

I wasn't talking about AI in general, only about ChatGPT. While the definition of what intelligence is can be subjective, my frustration was more about the focus ChatGPT gets on things that are beyond its capabilities, like giving correct information or doing math. That happens because people see how good it is at some very complicated things but it can't do extremely basic things.

Maybe I should have used better words to express myself, but what I meant is that people seem to expect ChatGPT to be AGI, which is not.

I would say something that can do calculations perfectly and faster than any human is pretty smart. If a human could do that we’d call them a genius.

I think there will always be a discussion on where to set the bar on what's considered intelligence, but the bar has to be set somewhere, because if anything that is alive is to be considered intelligent then there'd be no point in talking about what intelligence is. Even plants have learned through evolution to point the leaves towards the sun. Should we consider that intelligence too?

And I’m saying that intelligence is just a word we use to point to ourselves. It doesn’t have an objective meaning which is why there is no test people can agree on.

That will always happen on anything that we decide pretty much arbitrarily. Where does the color blue start and end in the electromagnetic spectrum? No one can objectively say so, but it's still useful referring to things as blue or whatever color they are.

Yes, and imagine how annoying it would be for people to keep saying “that’s not really blue”.

It could start a new debate about the dress on twitter.