Submitted by hducug t3_yanf0j in singularity
It’s been 2 years now since the release of gpt-3. Gpt-4 could be released any time now. I remember someone at open ai saying that gpt-4 would likely be released this year in July/august. 2 months later and there is no sign of gpt-4.
U-PaLM and Flan PaLM come to mind.
The first one shows we were noising the data incorrectly, different kind of noising has major benefits. Second shows that training on thousands of tasks boosts the model capability to follow instructions and also get better score. So maybe OpenAI had to change their plans midway.
It's also possible that they don't want to scale up even further because it's very impractical. Too expensive to use, not just to train. And recent models like Flan get GPT-3 scores on many (not all) tasks with just 3B parameters.
There's also a question about training data - where can they get 10x or 100x more? I bet they transcribe videos, probably all videos that can be accessed. Another approach is to use raw audio instead of text, works well. I bet they have a large team just for dataset building. BLOOM was managed by an organisation with 1000 people and a lot of their effort was into the dataset sourcing, trying to reduce biases.
>recent breakthroughs might've meant that they've restarted from scratch once or twice
That is almost certainly not happening given the cost of training. Probably more a case of "Measure thrice, check twice, cut once"
I wonder about the Graphcores "good computer" set for 2024 .
Graphcore had some business setbacks recently.
As far as GPT-4 being released in July/August, it's not possible to exactly predict how long it will take to have software ready because writing software requires solving problems for which you won't know the complexity of the solutions until after you've solved them.
But here's some info on Open Ai's progress and direction, for those who are interested.
Open Ai past timeline:
GPT-1 - 2018
GPT-2 - 2019
GPT-3 - 2020
Open Ai Codex - 2021 (GPT-3 but fine tuned to produce code)
Dalle-E - 2021 (Images from text)
Dalle-E 2 - 2022
Open Ai wants to be the first to build general artificial intelligence. Dalle-E helped them understand vision systems, and gpt helped them understand language models.
They are in a race to produce AGI, against companies such as deep mind.
A friendly race since they share so much of their findings periodically.
Friendly, just like Mohammad Ali's friendly public demonstrations of how to be the best boxer ever, followed by whooping you.
I can't find the link anymore but there were rumours it would be between November and February.
But in reality no one knows
Yeah I also heard that it would be released in December of 2022 but I doubt it! They'll probably release DallE 3
The end of this year or next year sometime. Looks like we are set up for another large leap forward in AI next year as a whole as well.
These are exciting times, really can’t wait to see the progress this enables over the next decade in medicine, science, gaming, VR, art, etc.
I'm wondering, whether Microsofts CTO Kevin Scott is talking about GPT-4 here. GPT-4 could be released later this year.
Interesting, how is sure that multi-modal models will be released later this year if he's not the one releasing them? Multi-modal means texts, pictures, touch and stuff like that combined into one?
In the first quarter of 2023 i think.
Can you guys imagine if you somehow directly linked a gpt-3 or 4 with a stable diffusion ML type platform. That would create kinda it's very own metaverse.
Within a year IMHO.
Just imagine what 2023 will bring us in AI advancements. Feels a lot like the semiconductor shrinking progress but further on the exponential curve. I wouldn’t be surprised if gpt-4 is delayed so they can incorporate all the new ways to train. I think the next step will be big+efficient and see if we can crack into those remaining cognition tests that Ai still falls short on
We are already living in GPT-4.
I am100% sure gpt-4 is comming out next year...and i xan even bet on it...im that sure...(source:trust me bro ;)
Does it matter? We have Bloom model
You will have better luck getting to know that by asking Sam Altman on Twitter than here. This is asked so many times that my answer now is, we'll get there when we get there.
It will present itself to the world when it's ready 🙂
My median is 2 months, and 80% is 6 months. I heard such estimates from 2 independent sources with alleged knowledge of matter a couple of months ago.
In Elon time, right? I'm not sure it will come out as standalone service as Microsoft is now practically owner of OpenAi, so OpenAi not open source anymore.
Microsoft owns GitHub as well, Satya is pretty big on open source recently
was that about supporting open source or buying a large dataset with the excuse of supporting open source.
Supporting open source without being honest about their reasons is still supporting open source
Open source is open, there is nothing to buy.
look two comments up where the context of this is Microsoft now owning Github.
zero_for_effort t1_itbx41r wrote
I'm certainly in no position to provide any insight as a complete outsider to tech and AI, but I do wonder if all the recent breakthroughs might've meant that they've restarted from scratch once or twice. What do you do if you realise your model will be obsolete before it's even fully trained?