Decadz OP t1_j41ry5u wrote
Reply to comment by Jumbofive in [D] Are there any papers on optimization-based approaches which combine learned parameter initializations with learned optimisers? by Decadz
Thanks for reminding me of this work! This is an interesting paper, however; as far as I am aware their method only considers the problem of meta-learning an optimizer and not the parameter intialization.
Jumbofive t1_j41thj4 wrote
Is there anything to be gained from doing both at once instead of two techniques within the same system? I guess that's what you are trying to find out lol
Decadz OP t1_j41wx24 wrote
Yes, as you said that's what im trying to find out! Will be interesting to know whether you can combine the two approaches into one technique, or have two seperate approaches being used in one system.
Was just enquiring to make sure i'm not going to spend time reinventing the wheel haha. Will also be interesting to have some insight into how the two approach interact, and whether the benefits can stack or if they overlap.
Viewing a single comment thread. View all comments