Submitted by RandomProjections t3_yxzaz3 in MachineLearning
So I was talking to my advisor on the topic of implicit regularization and he/she said told me, convergence of an algorithm to a minimum norm solution has been one of the most well-studied problem since the 70s, with hundreds of papers already published before ML people started talking about this so-called "implicit regularization phenomenon".
And then he/she said "machine learning researchers are like children, always re-discovering things that are already known and make a big deal out of it."
"the only mystery with implicit regularization is why these researchers are not digging into the literature."
Do you agree/disagree?