It’s honestly not that difficult to keep up with the significant improvements in deep learning from a fundamental research perspective. AlphaTensor, amongst many others, is a cool application of reinforcement learning. It’s implications are huge by any standards but it’s not the discovery of a new model. Although diffusion models are also not new outside of ml, they recently started to gain attention as a new model type to train. For me, this is what’s interesting: new types of models, new ways to use SGD in a DL setup. CNNs, RNNs, Att,… are fundamental computation models. Extensions, improvements and upscaling is what I see in 99.99% of papers. GAN is a new model, VAE is a new model, actor critic is a new mode, etc. when it comes to fundamental dl research there are not many of the big ones imho.