[D] Yann Lecun seems to be very petty against ChatGPT

Submitted by supersoldierboy94 t3_10uw974 in MachineLearning

I get that he is one of the godfathers of AI. Mostly on the research side which immediately puts him very hostile against engineers. But I guess it is understandable given the fact that he works on Meta and Meta has faced a lot of backlash (for good and bad reasons), most especially with Galactica where their first rollout got so bad they had to close it immediately. It's also particularly funny given his political leaning that he is very spiteful of a company that uses open-source knowledge and builds on top of it.

Lately, his social media and statements are barrages against ChatGPT and LLM's. Sure, he may have a point here and there but his statements look very petty. Here are some examples

"By releasing public demos that, as impressive & useful as they may be, have major flaws, established companies have less to gain & more to lose than cash-hungry startups. If Google & Meta haven't released chatGPT-like things, it's not because they can't*. It's because they won't."*

> Except that anyone in the IT industry knows that big tech companies cant release something very fast because of politicking and bureaucracy in the system. It takes years to release something into public in big tech compared to startups.

"Data on the intellectual contribution to AI from various research organizations. Some of organizations publish knowledge and open-source code for the entire world to use. Others just consume it."

> Then adds a graph where the big tech is obviously at the top of the race for most number of AI-related research papers (without normalizing it to the number of researchers per org)

"It's nothing revolutionary, although that's the way it's perceived in the public," the computer scientist said. "It's just that, you know, it's well put together, it's nicely done."

> Except that it is indeed revolutionary in terms of the applied research framework -- adding on top of open-source, state-of-the-art research and quickly putting it into production for people to use.

"my point is that even the engineering work isn't particularly difficult. I bet that there will be half a dozen similar similar systems within 6 months. If that happens, it's because the underlying science has been around for a while, and the engineering is pretty straightforward."

"I'm trying to correct a *perception* by the public & the media who see chatGPT as this incredibly new, innovative, & unique technological breakthrough that is far ahead of everyone else. It's just not."

"One can regurgitate Python code without any understanding of reality."

"No one is saying LLMs are not useful. I have forcefully said so myself, following the short-lived release of FAIR's Galactica*. People crucified it because it could generate nonsense.* ChatGPT does the same thing. But again, that doesn't mean they are not useful."

He also seems to undermine the rapid engineering work and MLOps that come with ChatGPT which is funny because Meta hasn't released any substantial product from their research that has seen the light of the day for a week. Also, GPT3 to ChatGPT in itself in a research perspective is a jump. Maybe not as incremental as what Lecun does every paper, but compared to an average paper in the field, it is.

To say that LLMs are not intelligent and it just regurgitates Python code probably haven't used CoPilot, for example.

It's a classic case of a researcher-engineer beef. And that a startup can profit from derivatives of research that big tech has published. OpenAI broke their perspective on the profit from research. Big tech tried to produce revolutionary research papers on a surplus but never puts them into production thinking that they are the only companies that could if they want to. Then once one company created a derivative of a large research work and profited from it, it baffled them. Although people could argue that Stable Diffusion did this first in the Generative Image Space.

It's one thing to correct misconceptions in the public. It's also one thing not to be petty about the overnight success of a product and an immediate rise of a company that got embraced warmly by tech and non-tech people. It's petty to gatekeep. At the end of the day, ML is not just about research, it's applied research. It's useless until it reaches the end of the tunnel. 99% of research papers out there are just tiny updates over the state of the art which has been a pointless race for about a year or two, with no reproducible code or published data.

Inventing combustion engine is just as important as putting it in the car.

18

Comments

You must log in or register to comment.

danjlwex t1_j7ed72r wrote

My take is that you seem quite intent on painting him as petty. His statements seem quite reasonable and rational, especially in the face of the over exuberant reactions we mostly see about chatGPT.

> Mostly on the research side which immediately puts him very hostile against engineers... It's a classic case of a researcher-engineer beef

Seems like you have had some bad experiences that led to these feelings. There is no built in animosity between these groups. Just different goals.

97

fallweathercamping t1_j7edw3y wrote

This. Majority of reactions are irrationally exuberant and often pablum for the vacuous “content” creation cycle. It’s as if, if one doesn’t affirm the super positive, life altering results surely to come, you may get left behind. Let’s see what actual problems ChatGPT solves.

24

supersoldierboy94 OP t1_j7edvto wrote

Fair point. But you can be correct and petty at the same time. Remember that he blamed the people using Galactica casually as the reason it got paused. Then wonders and asks people why ChatGPT hasn't faced the same backlash given that "it spouts sh-t*.

Although one could argue that usable LLMs in production are quite revolutionary. NVIDIA'S GauGan or GAN based txt to image models, the base diffusion models have been there for a year or two but hasn't received the same publicity and profits as Stable Diffusion or Midjorney. It's basically the same line of framework.

It's narrow-minded thinking to brush the architecture upgrades and the engineering work that made it possible -- which has always been his statements. But that is a fair point considering he is mainly a researcher not an engineer.

−5

supersoldierboy94 OP t1_j7ef7rn wrote

> some bad experiences thst led to these feelings

I work as an Applied Researcher so I do both research and engineering. No beef on it. It's bad to say it as beef. It's like "dev-QA" relationship. Researchers would want the largest models possible yielding the best metrics, Engineers want the easiest to deploy and monitor. The former also undermines what engineers do as just packaging it up. Yann just said it above.

−10

danjlwex t1_j7ejdhf wrote

I have no clue why your are being down voted.

−7

supersoldierboy94 OP t1_j7ejgh8 wrote

Lecun's fanbois for sure.

Or either side of the research or engineering perspective that has no clue what the other side does.

−5

danjlwex t1_j7ejxtd wrote

You have a lot of angst to work through, my friend. Really, you have built up some divide between research and engineering that simply does not exist.

17

supersoldierboy94 OP t1_j7elvss wrote

The beef does not exist. But the divide between research and engineering exist. It's one of the fundamental reasons why some startups fail -- they dont know how to balance which and do not know how to construct a team. There's a "divide" between data science and data engineering and folks who work on that know that there is.

−2

danjlwex t1_j7empve wrote

In my 35 years of working with both engineers, corporate researchers and academics, I have not experienced this divide you describe. Research isn't something that happens at startups. There is no revenue to support research in a startup. The entire focus is on product.

16

supersoldierboy94 OP t1_j7enajd wrote

> research isnt something that happens at startups

Entirely depends on the startup and the product. R&D happens on many startups. Unless someone has a limited exposure on AI and ML-oriented startups, this is far from truth. OpenAI is an applied research company. They produce research papers and puts it into production. In the electronics department, OnePlus has risen as a great R&D startup capable of producing rapid R&D-based products. Grammarly puts a ton of money on its R&D to create a more domain-specific GPT model because it is vital to their product.

> The divide you describe

One does not need to probe deeper into this. Ask an experienced Data Engineer, a Data Scientist, and a DevOps. There is a clear DISTINCTION of what they do and how they balance each other. The divide isnt hostile. It's more of "we want this, you cant have all of this type of relationship, besides the usual difference of who works with what.

5

etesian_dusk t1_j7gp3f0 wrote

>Lecun's fanbois for sure.

The fact that you have an unpopular, and in my opinion shallow, view of current NLP, isn't an argument for calling everyone else 'fanboys'

3

_poisonedrationality t1_j7e7rz6 wrote

Doesn't sound petty at all to me. Sounds like he's dispelling misconceptions about the progress ChatGPT represents.

53

yaosio t1_j7ebfxa wrote

If I listened to critics I would think zero progress has been made at all. Every time new software comes out that does something that couldn't be done before it's handwaved away as easy, or obvious, or something else. If it was so easy then it would have already been done. Well with ChatGPT...it has. https://beta.character.ai/ beat ChatGPT by a few months and has a bit more power because it's easier to make the chat bot answer as you want. I don't think it's as good as ChatGPT though.

5

supersoldierboy94 OP t1_j7ebug1 wrote

You know its just being petty when he isnt even talking about it in the Generative Image space. ChatGPT is very much like Midjourney and Stable Diffusion where these models are small incremental updates over the main papers. But has put the proper applied research and MLOps work to bring these into production and profit from it.

−6

MrTacobeans t1_j7ekqqr wrote

But why is that bad? If the researchers wanted moola they should have made a business or published/ran the models they created from their own research. If you don't want to get stepped on by someone else talented enough to piece it together don't release your ideas.

Don't get butt hurt when a primarily publicity or capitalist based company implements your idea and makes it into a product.

0

supersoldierboy94 OP t1_j7e8jbe wrote

You can be factually correct and be petty at the same time. You can read more about his conversations with people who argue with him or all the the time he brings up Galactica's failed rollout comparing it to ChatGPT and wondering why it hasn't been paused as well given that, a quote from him, "that Galactica even produces less BS".

He also seems to undermine the rapid engineering work and MLOps that come with ChatGPT which is funny because Meta hasn't released any substantial product from their research that has seen the light of the day for a week. Also, GPT3 to ChatGPT in itself in a research perspective is a jump. Maybe not as incremental as what Lecun does every paper, but compared to an average paper in the field, it is.

You may have a toxic aunt. But if you always talk about it in the dinner table, that's petty.

−1

VeritaSimulacra t1_j7ehytu wrote

I am also very petty, so it’s good to see I have stuff in common with an ML great.

52

rafgro t1_j7eq9ek wrote

Nah, it's not engineering vs science or OS vs closed. It's much simpler:

>FAIR's Galactica. People crucified it because it could generate nonsense. ChatGPT does the same thing.

YLC threw a fit over the whole Galactica debacle. He had lovely aggressive tweets such as "Galactica demo is off line for now. It’s no longer possible to have some fun by casually misusing it. Happy?" or describing people who disliked Galactica as "easily scared of new technology". To see the success of ChatGPT just a few weeks later must have been really painful.

42

supersoldierboy94 OP t1_j7ervit wrote

Exactly. He was blaming the users for the Galactica debacle and wondering why OpenAI's ChatGPT is getting adoption when "it spews the same bS" as per his words. And also proceeds to tell that it is just because people had been drstroying Meta's reputation overall.

12

visarga t1_j7kgxrq wrote

FB was too scared of the bad PR. OpenAI wasn't. People tried to trash chatGPT millions of times, Galactica just a few times. I think chatGPT handled the adversarial attacks pretty well.

Google is another scared company, their models haven't seen any attacks yet, so they are unknown. I don't care how nice their screenshots look, what I want to see is how people hack it. Then I can form an opinion. People are the true test set.

3

whiskey_bud t1_j7eit5o wrote

>I get that he is one of the godfathers of AI. Mostly on the research side which immediately puts him very hostile against engineers

I find it odd that you seem to expect / want a serious conversation, but then start with some weird ad-hominem against the man. You talk about "fanbois" in your first sentence, but then expose yourself as nothing better, to be honest. The rest of your post isn't much better TBH - trying in infer intentionality and make false equivalencies.

23

supersoldierboy94 OP t1_j7ej5p9 wrote

Please point out the 'ad hominem' against him instead of generalities when I just literally quoted all the things he said and gave my own take on it.

> infer intentionality

Point it out. You can conclude intentionality based on his line of reasoning, conversation trails, and position.

1

whiskey_bud t1_j7ekfua wrote

>Please point out the 'ad hominem' against him

I literally quoted it.

9

supersoldierboy94 OP t1_j7em0v4 wrote

That's not an ad hominem. An ad hominem attacks the subject as basis of its argument. Telling that this person is X based on Y is not ad hominem. It's a conclusion of the quotes I laid down.

5

beezlebub33 t1_j7fbjii wrote

"Henry Ford did nothing revolutionary, the engineering work in making a car isn't particularly difficult, it's just perceived that way by the public. There will be a half dozen other car manufacturers in 6 months."

LeCun is going too far the opposite way. I would not be surprised if he has access to systems at FAIR that could do something similar, so dismisses the whole thing or misses the main point. But, like Ford, what OpenAI has done with Dalle2 and ChatGPT is make AI useable and available to us benighted common folk.

It doesn't matter whether Google and Meta not releasing something like this is due to a can't or a won't. It's all the same to the rest of humanity who can't use it in either case.

13

[deleted] t1_j7kik6w wrote

[deleted]

1

OneMillionSnakes t1_j7f0auj wrote

I agree with most of those statements. I don't think he's being petty he's just being honest about what ChatGPT represents to him.

Now I am biased as on a personal level I'm kinda sick of ChatGPT. It's good at carrying on a brief chat and it's very well polished. But it's quite mundane and people are already talking about using it or some variant to make marketing and web pages in a web that's already full of AI generated articles and targeted ads. It should be used perhaps for chats when trained on a corpus including some support docs or something. Not much more than that.

I do think there could be some negative ramifications in the worst case. I have a friend whose a graphic designer at a major company whose been told by her employers this the future of ads. Higher ups say stuff like this all the time and it doesn't wind up coming true so it hopefully won't become a real problem. Still it's a bit concerning that people on the oustside of these fields are perhaps overvaluing ChatGPT so much.

9

supersoldierboy94 OP t1_j7f1die wrote

> some variant of it Just the other day, some researchers already released BioGPT which is trained on biomedical text. It's particularly good. Sitll needs some time to test its accuracy against real medical professionals

I'd respectfully disagree on the usage. While it has been shown to generate weird sequences, with the right usage, you can guide it to create particularly effective articles and stories. It's summarization tool is also good. Grammar is particularly good as well.

> What chatGPT represents to him

It can be true and petty at the same time. When asked, he will revert to complaining why Galactica was shut down blaming the people using it and pointing as to why ChatGPT does more mistakes but is still standing. Why would someone also suddenly post a paper contribution chart saying that others just 'consume' the research?

−1

OneMillionSnakes t1_j7h28eq wrote

Yeah I mean these seem grounded and not that petty to me personally but that's fairly subjective. His criticisms seem fairly mild. I don't think they're worth getting worked up over.

2

CKtalon t1_j7eba6d wrote

Whatever Meta has put out in the past year has been fairly disappointing compared to what's already available—OPT, NLLB, Galactica. It probably advanced the field with the knowledge gleaned from producing these models, but for production, they all feel half-baked and lack polish. It was like they were just rushing out something to meet some KPI.

So yes, I find Lecun being petty that his team can't seem to produce something 'good' to the general public.

7

redlow0992 t1_j7elzrw wrote

Are we only talking in the context of LLMs and language? If not, your statement is simply incorrect. In past two years FAIR published a number of high-quality self-superviser learning frameworks that come with open source implementations. On top of my head, MoCo (and its versions), Barlow Twins, VicReg, Swav all came from FAIR. They are the one that showed that SSL for computer vision does not need to be contrastive only. Some of these papers have some 5K citations in the span of 3 years and are used by many researchers on a daily basis.

But yeah, tell me how they are chasing corporate KPIs and are publishing junk.

12

red-necked_crake t1_j7exuvy wrote

not to mention being a company that is willing to put out huge ass models AND training logs which is infinitely more useful to our community than three vague blogposts and 1000 retweets by ex web3 grifters on twitter claiming GPT-4 will quite literally have 100 trillion parameters and worshipping Sam Altman as God LOL.

People keep claiming that others dismiss engineering effort that went into ChatGPT, GPT3, and turn a blind eye to relative opaqueness on techniques and tricks that went into making these models happen (not even a dataset available). Other than showing a proof of concept (which is SIGNIFICANT but not sufficient for SCIENCE), how exactly do we, as a community of ML, benefit from OpenAI getting all the hype and Satya's money? (Whisper is a weird counterpoint to my arguments though.)

11

supersoldierboy94 OP t1_j7f26hq wrote

He said for production. Meta hasnt produced fully baked production-ready products from their research for public consumption.

That is the point of the post and Yann's reaction as a Meta employee reeks pettiness.

He first told everyone that ChatGPT is not revolutionary at all. May be a fair point. That's debatable. Then proceeds to post a chart about Meta and Google big tech as producers of research that others just consume. Then when asked about what research has they put into production, he claims that it's not that we CANT, it's that we WONT. Then proceeds to bring out what happened to Meta's first trial to do it -- Galactica that embarassingly failed. So all in all, he seems to be criticizing why these companies just consume established knowledge by sprinkling something on top from what they have published.

I'd honestly expect Google and META to be quite cautious now on how they publish stuff since OpenAI's moves build on top of the established research that they do.

No one also said they are publishing junk. That's a strawman. The point is that he's being overly critical to startups like OpenAI who consumes established knowledge that they voluntarily opened to the public and has started to profit from it, while they have failed to produce something profitable or usable for public consumption.

0

supersoldierboy94 OP t1_j7ebeiw wrote

> to meet some KPI

Big tech in a nutshell

Or to close some JIRA tickets perhaps

−4

dataslacker t1_j7gfa6c wrote

There’s probably some resentment that google and meta could have released something similar over a year ago but chose not to because they didn’t think it would be responsible. Now the company that was founded on being “responsible” released it to the world it a way that hasn’t satisfied a lot of researchers.

5

du_dt t1_j7eohva wrote

MetaAI released their galactica chatbot a month before chatgpt, but it was heavily criticized for “dangerous AI generated pseudoscience nonsense” and shutdown a few days lter. Now OpenAI does the same and everyone praises them - well, I get why Yann is being saulty about it.

4

ok531441 t1_j7eu88c wrote

Galactica was doomed to fail because it was specifically marketed as a science tool which puts very high expectations on factual and mathematical correctness. ChatGPT on the other hand is marketed as chat.

19

visarga t1_j7kkc5b wrote

Maybe they come to their senses and put it back. I wanted to use it to find references for my random ideas, see what results they have.

1

supersoldierboy94 OP t1_j7esbbj wrote

Fair point. But why is he blaming the people instead of his whole company going as far as "it's just people destroying Meta's reputation"?

I have high respects for him as a researcher, and in fact I've read his books and papers. He's great when he speaks as a researcher. It's different when he's speaks as a Meta employee vested with the companies interest. That's why I take his Meta-driven statements for/against companies with a grain of salt.

I wont be even surprised if the big tech companies are behind the Stable Diffusion/Midjourney lawsuit since it would do them good. Considering the fact that Meta partnered with Shutterstock to produce their own.

3

PredictorX1 t1_j7ffj8j wrote

>I get that he is one of the godfathers of AI.

What does that even mean? Very many people have contributed to this field.

2

choHZ t1_j7hnrtc wrote

I get that he is annoyed that people believe ChatGPT is such a milestone breakthrough unique to OpenAI. It is not, since most big players already have or capable of having LLM tuned to similar capabilities. Yet from the InstructGPT paper, the way they label their data is nothing that any big players can't handle. I also get that he is pissed when people praise OpenAI for its "openness" — OpenAI is absolutely not a fan for the whole open source movement, though maybe reasonably so.

My question is why don't the big players give their bots similar exposure? I find it hard to believe that ethics and some internet critics to be the only reasons.

2

reditum t1_j7il5mj wrote

Facebook: move fast and b r e a k things

Also Facebook/Meta/Zuckerbronium when they actually need to do something different that doesn't involve buying companies to form a monopoly: look guys we're working really hard on these legs for your avatars

1

DeepGamingAI t1_j7jq9jk wrote

To me all AI debate these days are just a regurgitation of "glass half full or half empty" discussions. Yes, LLMs are far more intelligent than anyone anticipated them to be by this point in time, and no they aren't general intelligence. The constant back and forth between these two groups can essentially be replayed year after year and not much has changed in terms of arguments.

1

f10101 t1_j7kea8m wrote

I always find it curious that lot of these "godfathers of AI" seem to be a bit like this. It gets draining to listen to them, as they have a tendency to reframe any debate or definition just so they can be right.

1

FLQuant t1_j7mobsc wrote

The technical name is butt hurt.

Yeah, he has some important and relevant points, butknow he spent his whole day on Twitter complaining about ChatGPT.

Most of the complaints are that it isn't the first, the most advanced nor the best. I find it very curious coming from some who works at Meta since this kind of critic work for almost all Meta products.

1

_Arsenie_Boca_ t1_j8a4q2t wrote

His position as rival makes his statements look petty, and they might be. But still, I agree with most of his statements you quoted here.

1

DrHaz0r t1_j7fmzlx wrote

I think you are all missing the point that Schmidhuber has basically invented Chat-GPT in the 90s already. Just with smaller networks, smaller datasets and less compute power. Although even Schmidhuber basically just stole from Gauss, but no one talks about that.

0

clueless1245 t1_j7ipckn wrote

Lol at your previous posts. https://www.reddit.com/r/LateStageCapitalism/comments/zdeix8/ai_art_is_very_dystopian/j044ec2/

You are obviously disturbed and just latching on to arguing for thousands of words online as an outlet.

Cannot wait for next month when Ukraine or COVID-19 is back on the news cycle and you move on to /r/worldnews.

0

supersoldierboy94 OP t1_j7iqnqj wrote

If you've read the comments, you will know the answer. You already went my profile for some stalking reasons yet you ignored that lol

1

luckymethod t1_j7j41yx wrote

He's right and you're full of it.

0

supersoldierboy94 OP t1_j7j471i wrote

Thanks for adding contribution to this discussion like your contribution to the field. Salute.

1

Rohit901 t1_j7ek7ix wrote

Lol I kinda agree with you here, and Lecun reminds me of Sheldon from Big Bang theory who is constantly berating and insulting engineers (Howard)

−1

MonsieurBlunt t1_j7eeg25 wrote

Yea looks like Meta is making him say this stuff.

I assumed he jerks off to chat GPT responses when he is alone. I am continuing to assume that tbh

−3

supersoldierboy94 OP t1_j7eemd5 wrote

tbf, he has screenshots where he talks to it, then posts it in his Twitter thread to say, I told you so

1

bacon_boat t1_j7fbjcg wrote

I think his view reflects his disappointment as a researcher that it's not novel ideas and algorithms that lead to success. It's scale + engineering.

But anyone with a broader view sees that ChatGPT represents a massive milestone for AI.
Who really cares how novel the algorithms are, openAI built a killer product, and deserve the recognition.

Lecun is maybe also salty because Deepmind / OpenAI are perceived as leaders, and Meta isn't.

−4

supersoldierboy94 OP t1_j7fcbh3 wrote

Meta is a leader in the research community alongside Google as top contributors. The funny thing is that he started posting that graph of AI related paper contributions to show supremacy and to undermine OpenAI and DeepMind as merely consumers of research. But Meta hasnt provided any product from their research that has reached the public. When they tried, they immediately shut it down.

He also kinda blames the public perception as to why Meta cannot publish products without scrutiny pointing the thing that people are still overly criticizing Facebook/Meta for obviously great reasons in the past.

It is indeed a massive milestone maybe a bit above Stable Diffusion. I'd still argue that Github Copilot was bigger but since its mainly for devs, it didnt get the publicity that it wanted. It's a massive milestone because common folks pondered the idea of AI takeover which have shifted every one else's perspective on the domain. It's the culmination of decades of R&D that the public can interact to -- a gateway to AI and its complexities.

Common folks and the public do not really care about sophisticated algos that never see the light of day.

−1