ingeniare t1_j03tzp3 wrote on December 13, 2022 at 9:43 PM

Reply to comment by beezlebub33 in Is it just me or does it feel like GPT-4 will basically be game over for the existing world order? by Practical-Mix-4332

It's just a rumour and I think Sam Altman basically denied that this was the case. Another, perhaps more plausible, rumour is that GPT-4 will have a very different architecture where the parameter count between it an GPT-3 doesn't say much because it's no longer just about bruteforce scaling.

beezlebub33 t1_j04k5c9 wrote on December 14, 2022 at 12:43 AM

That would, IMHO, be a big win. Even if the scaling hypothesis is correct, why would you want to solve the problem that way, when there are probably far better ways to solve it.

Sure, we could fly an interstellar spacecraft to another solar system, but it would be a bad idea to do it, because in the time that it would take to get there, some other ways of getting there would be invented. IF you left for the stars now, people would be waiting for you when you got there.

In the same way, simply scaling compute and data may get you to a certain amount of intelligence. But the costs and effort would be huge. It would probably be better to spend that time and effort (and money) on making the underlying ideas better. And even if it turns out that, yes, we have to scale, waiting until computational costs come down further is probably a good idea.