currentscurrents t1_j8qa46s wrote
Reply to comment by Red-Portal in [D] Lion , An Optimizer That Outperforms Adam - Symbolic Discovery of Optimization Algorithms by ExponentialCookie
This is a hand-designed optimizer. By definition, learned optimizer researchers would rather we learn an optimizer than hand-design one.
Learned optimizers are probably the future, but the compute budget required to create one is prohibitive.
Viewing a single comment thread. View all comments