troll_khan t1_j2e6w40 wrote on December 31, 2022 at 4:33 PM

Let's see how GPT-4 performs in a few months.

HeinrichTheWolf_17 t1_j2eafk0 wrote on December 31, 2022 at 4:57 PM

I’m really interested to see how many parameters it winds up having.

Analog_AI t1_j2ed2s8 wrote on December 31, 2022 at 5:15 PM

They were saying it won’t be larger than GPT-3. They want to focus on finessing it and squeezing more out or customizing and pre training before they increase size any further. It will be much better than it’s predecessor anyway. If this approach yields good harvest they won’t increase the size of GPT-5 either. They will only do so if they hit a wall.

Borrowedshorts t1_j2ffld8 wrote on December 31, 2022 at 9:44 PM

It seems like most AI companies have been doing this for now. I wonder if they're optimizing a local maxima instead of a global and that the global can only be achieved through further scale.

Analog_AI t1_j2fjzjc wrote on December 31, 2022 at 10:16 PM

They will not stop increasing size. More like taking a breather and squeezing progress from maximizing pre training Also waiting for more cost reduction for scaling. Just a breather. And they also work on tech for beyond the GPT models. More integration of vision recognition and other things will come. Some form of weak AGI will be here in the next 5 years.