realjunkman t1_j2a54lt wrote
There was a paper recently about finding parameter regions that are unused and only updating those on fine tuned data. Can't remember the name but that was an interesting approach.
derpderp3200 OP t1_j2avw24 wrote
Interesting! I thought about something similar, a "no parameter is left unused" during training, but using unused regions for fine-tuning sounds like a much more clever application of the principle.
realjunkman t1_j2cc4nm wrote
It was a presentation I saw at EMNLP this past year. I’ll try and look for it, but if I don’t report back… it was a presentation during day 3!
Viewing a single comment thread. View all comments