Decadz OP t1_j41udg4 wrote
Reply to comment by Optimal-Asshole in [D] Are there any papers on optimization-based approaches which combine learned parameter initializations with learned optimisers? by Decadz
Thanks for the recommendation! I was unaware of this follow up work, which naturally extends Baydin et al. original work [1]. Categorically, I would consider this paper to be more about meta-optimization (theory), similar to [2, 3]. I was looking for more applied meta-optimization work.
[1] Baydin, A. G., et al. (2017). Online learning rate adaptation with hypergradient descent.
[2] Maclaurin, D., et al. (2015). Gradient-based hyperparameter optimization through reversible learning. ICML.
[3] Lorraine, J., et al (2020). Optimizing millions of hyperparameters by implicit differentiation. AISTATS
Viewing a single comment thread. View all comments