Comments

You must log in or register to comment.

logicbloke_ t1_j1iswtn wrote

You typically won't get to know about impactful projects until they produce impressive results. Most research work is speculative, you won't get to know if it's going to work or the impact it has till experiments are completed, and researchers will not disclose results till they publish the work in some form.

27

RuairiSpain t1_j1j2qhg wrote

Also, think a lot of ML research is now highly dependent on GPU parallel computation, which is expensive. My guess is that it will be the academics that collaborate the GCP, AWS, Azure to get near free usage of GPU clusters. Coming up with a paradigm shift means a fair bit of trial and error experimentation. For the time being Cloud providers have been happy to promote ML pipelines to academics. But that may change with the tightening of costs with the layoffs and recession.

The transformer and self-attention progress has been an interesting achievement. I foresee us stuck in this trend until most ML groups have fully explored the avenues of research on self-attention. Without the advances in GPUs I don't think we'd be where we are now.

What's next? I'd love to see more progress on recommender systems and sparse training data.

I feel there are more gains to be had out of the more boring stuff in ML: data rangling design patterns to help non-data scientists choose the best model and customisation to answer their hypothesis questions. Also, the mechanics and infrastructure around ML at scale in an enterprise is not mature.

There are a lot of pain points for ML and big data teams in Enterprise, they need to skill up on a variety of hardcore DevOps tasks. Once it becomes trivial to spin up a ML pipeline with a Cloud infrastructure team supporting that work, then we'll see more commercial successes and collaborations between academia and Tech companies.

ML is at a strange evolutionary stage, there are lots of Tech companies relying on ML models to give them a USP over their competitors. So their willingness to share their breakthroughs is small. Once the barriers to entry are reduced for enterprise scale ML modelling, that's when we'll see more adoption of ML systems across whole sectors. Right now ML commercial research is too expensive for small players to get involved

3

Dylan_TMB t1_j1jew6k wrote

The easiest steps are this:

  1. Have domain knowledge in a very specific area that is niche.

  2. Find where AI hasn't been applied yet in that niche.

  3. Apply AI in that specific domain of the niche.

14

[deleted] OP t1_j1ksica wrote

[deleted]

1

Dylan_TMB t1_j1l7g1o wrote

That's kind of the secret sauce you don't know what you don't know. If you could just know what that niche is then people would just be doing it. You often have to already have domain knowledge in some areas.

An example may be GIS stuff (spatial imaging and mapping data). If you know a lot about environmental sciences and geology then maybe there is some interesting problem to be solved in that field and you can be the first to do it. But it requires you to know the problem.

There is no such thing as an easy to find problem that will also give quick results. If it exists it will be done already. If you don't have domain knowledge then you're out of luck and will have to put the work into getting more cross discipline knowledge or innovating on the archetecure side of things.

4

[deleted] OP t1_j1l7z2n wrote

[deleted]

2

Dylan_TMB t1_j1l8o93 wrote

Maybe but I think these problems fall into a unique category. Those that know of the problems but don't know ML won't know how to ask or how to articulate them as ML problems. Those with ML knowledge but don't know if the problems obviously don't know they exist. Those that know the problem and ML are solving the problem themselves most of the time.

I think a list of problems is a nice idea but again I don't think those with knowledge of problems know what is a good ML problem and what isn't.

I will say I think a good blueprint for looking for these problems is to find a problem where the data for the problem is similar to a well understood problem.

For example if a problem in a niche can be framed as a seq2seq problem you can use translation models to try and solve it.

Another good one is trying to find problems that can be framed as a game. Reframing problems as games to use reinforcement learning is a good project.

2

111llI0__-__0Ill111 t1_j1zbyu6 wrote

Would you say lack of domain knowledge is what makes stats and CS not always the best majors for ML/AI? It seems like problem formulation is critical for applied modeling and this stuff is not easy to learn if all you learn is tools

1

fun-n-games123 t1_j1jk01f wrote

My PhD advisor gave me his two cents on this the other day. A lot of research has focused on inference. That's why we have seen gains in models that do better at classification tasks and improved language models. There is less work done on knowledge discovery. That is, how can we leverage machine learning to explore science better? Some good recent work on this is on models related to drug discovery, but I'm sure you could find more examples of this, especially in the field of causal inference.

I'm hopeful we'll see a paradigm shift, where machine learning will help us to solve hard problems in the natural sciences. My research is on applying ML to understand how we can change our environment to reduce climate risks, and this is a popular domain with tons of opportunities IMO.

7

Intrepid-Learner t1_j1kx19e wrote

My hope stays with continual learning, specifically class level continual learning. Being more energy efficient is always nice, while at the same time emulate how humans learn.

Need more hopium for a chance at better approach than backprop to change the paradigm altogether for neural networks.

These are pretty general theoretical concepts, but can be applied better in many domains. Aiding discoveries in natural science seems most interesting.

4

[deleted] OP t1_j1l89ay wrote

[deleted]

1

alphabet_order_bot t1_j1l89th wrote

Would you look at that, all of the words in your comment are in alphabetical order.

I have checked 1,249,038,142 comments, and only 243,123 of them were in alphabetical order.

−2

bernhard-lehner t1_j1lresi wrote

Uncertainty estimation/Anomaly and Novelty Detection is far from being solved for real-world problems, despite numeruous papers presenting impressive, but mostly cherry-picked results.

4

mlresearchoor t1_j1y8ijq wrote

Impressive applied ML results will come in healthcare, multimedia (e.g., video summarization), sustainability, efficient ML (e.g., TinyML), robotics (e.g., vision-language navigation), human-machine interaction, and more. It's important for our community to value research that uses smaller domain-specific datasets, as well as massive datasets.

But many of the greatest breakthroughs in the next decade will probably come from collaborations between those academic ML researchers and large industry labs.

2