Submitted by Light991 t3_xvol6v in MachineLearning
Who are the most successful PhD researchers? Or more precisely people who achieved the most (in terms of citations, impact etc.) while being PhD students? If you can give examples in last say 50 years but also more recent ones say last 5 years.
Someone I noticed: Chelsea Finn
Sure but papers she published back then are now very influential.
Any concrete papers you have in mind?
Just sort her papers by citations and look at the years…
It's nice to meet you, CHELSEA FINN!
(Non-ml) Gerard 't Hooft who’s got a Nobel prize for his phd work?
Yes, MAML is on top. But I don't think it has been very impactful, neither has the whole field of meta-learning really been.
I met him during a summer school this summer, super nice guy!
Not to rag on Chelsea Finn, who I am certain is brilliant but,
But ADAM paper was wrong, so. It is no better than cooking up an equation, which I guess is impressive, but if you know the right people then the overall contribution is very low. Like ADAM was literally 1 or 2 steps away from whatever Hinton was doing, and Hinton was literally the co-author's (forgot his name) supervisor or something.
I keep on trying to find one thing that she published that's considered to be successful and I find myself having to very narrowly define success. This was when I was doing a general survey on RL techniques.
Yeah but those are also the most well connected people in the ML world. It is easy to be impactful when you are having tea with Hinton, LeCun and Bengio every afternoon.
Are you american?
Hundreds of Phd students were in their positions. Few made such an impact.
It's not easy to be impactful, even with a good supervisor. In Krizhevsky's case, one could even argue he had a big impact, despite having Hinton as a supervisor. Alexnet was kind of built behind Hinton's back as he didn't approve of the research direction. Hinton did turn around later and recognize the importance though.
It did have an error in their convergence proof (which was later rectified by other people). But
Regarding "whatever Hinton was doing": Are you talking about RMSprop? Sure, it's another momentum optimizer. There are many of them.
What has been impactful according to you? What makes you say meta learning hasn't been impactful?
(I have published myself in the meta-learning field, and worked a lot on robotics)
I see no applications of meta learning appearing, outside of self-citations within the field. The SOTA in supervised learning doesn't use any meta-learning. The SOTA in RL neither. The promise of learning to learn never really came true...
... until large supervised language models seemed to suddenly meta-learn as an emergent property.
So not only did nothing in the meta-learning field really take off and had some impact outside of computer science research papers, its original reason of being has been subsumed by a completely different line of research.
Meta-learning is no longer a goal, it's understood to be a side-effect of sufficiently large models.
Are the SOTA in RL for few-shot learning not meta-learning based?
I am not sure what you would consider SOTA in few-shot RL. The benchmarks I know are quite ad-hoc and don't actually impact much outside of computer science research papers.
The people that work on applying RL for actual applications don't seem to use meta-RL.
PeedLearning t1_ir29bk8 wrote
Chelsea Finn? I knew very little people who were using MAML during her Phd, and even fewer after.
I reckon e.g. Ian Goodfellow had a lot of impact during his Phd. Alex Krizhevsky is another name with big impact.