Submitted by adversarial_sheep t3_1274w45 in MachineLearning
Yan LeCun posted some lecture slides which, among other things, make a number of recommendations:
- abandon generative models
- in favor of joint-embedding architectures
- abandon auto-regressive generation
- abandon probabilistic model
- in favor of energy based models
- abandon contrastive methods
- in favor of regularized methods
- abandon RL
- in favor of model-predictive control
- use RL only when planning doesnt yield the predicted outcome, to adjust the word model or the critic
I'm curious what everyones thoughts are on these recommendations. I'm also curious what others think about the arguments/justifications made in the other slides (e.g. slide 9, LeCun states that AR-LLMs are doomed as they are exponentially diverging diffusion processes).