Viewing a single comment thread. View all comments

redlow0992 t1_j7elzrw wrote

Reply to comment by CKtalon in [D] Yann Lecun seems to be very petty against ChatGPT by supersoldierboy94

Are we only talking in the context of LLMs and language? If not, your statement is simply incorrect. In past two years FAIR published a number of high-quality self-superviser learning frameworks that come with open source implementations. On top of my head, MoCo (and its versions), Barlow Twins, VicReg, Swav all came from FAIR. They are the one that showed that SSL for computer vision does not need to be contrastive only. Some of these papers have some 5K citations in the span of 3 years and are used by many researchers on a daily basis.

But yeah, tell me how they are chasing corporate KPIs and are publishing junk.

12

red-necked_crake t1_j7exuvy wrote

not to mention being a company that is willing to put out huge ass models AND training logs which is infinitely more useful to our community than three vague blogposts and 1000 retweets by ex web3 grifters on twitter claiming GPT-4 will quite literally have 100 trillion parameters and worshipping Sam Altman as God LOL.

People keep claiming that others dismiss engineering effort that went into ChatGPT, GPT3, and turn a blind eye to relative opaqueness on techniques and tricks that went into making these models happen (not even a dataset available). Other than showing a proof of concept (which is SIGNIFICANT but not sufficient for SCIENCE), how exactly do we, as a community of ML, benefit from OpenAI getting all the hype and Satya's money? (Whisper is a weird counterpoint to my arguments though.)

11

supersoldierboy94 OP t1_j7f26hq wrote

He said for production. Meta hasnt produced fully baked production-ready products from their research for public consumption.

That is the point of the post and Yann's reaction as a Meta employee reeks pettiness.

He first told everyone that ChatGPT is not revolutionary at all. May be a fair point. That's debatable. Then proceeds to post a chart about Meta and Google big tech as producers of research that others just consume. Then when asked about what research has they put into production, he claims that it's not that we CANT, it's that we WONT. Then proceeds to bring out what happened to Meta's first trial to do it -- Galactica that embarassingly failed. So all in all, he seems to be criticizing why these companies just consume established knowledge by sprinkling something on top from what they have published.

I'd honestly expect Google and META to be quite cautious now on how they publish stuff since OpenAI's moves build on top of the established research that they do.

No one also said they are publishing junk. That's a strawman. The point is that he's being overly critical to startups like OpenAI who consumes established knowledge that they voluntarily opened to the public and has started to profit from it, while they have failed to produce something profitable or usable for public consumption.

0