My PhD advisor gave me his two cents on this the other day. A lot of research has focused on inference. That's why we have seen gains in models that do better at classification tasks and improved language models. There is less work done on knowledge discovery. That is, how can we leverage machine learning to explore science better? Some good recent work on this is on models related to drug discovery, but I'm sure you could find more examples of this, especially in the field of causal inference.

I'm hopeful we'll see a paradigm shift, where machine learning will help us to solve hard problems in the natural sciences. My research is on applying ML to understand how we can change our environment to reduce climate risks, and this is a popular domain with tons of opportunities IMO.