haljm t1_ivwwvj0 wrote on November 11, 2022 at 5:18 AM

Find people to talk to! I definitely agree that the problem with learning by doing in ML is that it would probably be several full time jobs.

What I would suggest (and how I try to get by): find someone -- or several someones -- to discuss ideas with. Instead of learning by doing, learn by discussing how you would hypothetically do something, e.g. solve a certain problem or extend a method to do something else. Your advisor is great for this. If your advisor doesn't have time, try other professors, postdocs, more senior PhD students, or basically anyone who will talk to you.

As for keeping track of the field, my opinion is that there's not actually that much truly new stuff in ML, and everything basically builds on the same themes in different ways. Once you're sufficiently familiar with the general themes, the high level is enough to understand new stuff. I basically never read papers beyond the high level, except if I'm considering using it for my research.

LightGreenSquash OP t1_ivxgmef wrote on November 11, 2022 at 9:40 AM

I think I mostly agree on keeping track of the field, the only thing that's not clear to me is what should be considered "foundational" in an area where most of the exciting things have happened in the last ten years or so.

But don't you think that just discussing ideas ends up hiding a significant part of the complexity of actually getting them to do something? It's true that learning by doing seems rather time-consuming, but wouldn't we consider it strange if someone said that he'd learn about algorithms without trying to implement/use/benchmark them, theorems without solving problems with them, or even coding techniques by simply reading about them?

Then again, I guess you'll inevitably end up actually having to do things in your PhD/job/whatever, but I'm concerned that a lack of "foundational" knowledge and experience can greatly hamper you at some time-critical point of this process.