Viewing a single comment thread. View all comments

Decadz OP t1_j41ry5u wrote

Reply to comment by Jumbofive in [D] Are there any papers on optimization-based approaches which combine learned parameter initializations with learned optimisers? by Decadz

Thanks for reminding me of this work! This is an interesting paper, however; as far as I am aware their method only considers the problem of meta-learning an optimizer and not the parameter intialization.

3

Jumbofive t1_j41thj4 wrote

Is there anything to be gained from doing both at once instead of two techniques within the same system? I guess that's what you are trying to find out lol

3

Decadz OP t1_j41wx24 wrote

Yes, as you said that's what im trying to find out! Will be interesting to know whether you can combine the two approaches into one technique, or have two seperate approaches being used in one system.

Was just enquiring to make sure i'm not going to spend time reinventing the wheel haha. Will also be interesting to have some insight into how the two approach interact, and whether the benefits can stack or if they overlap.

2