realjunkman
realjunkman t1_j2a54lt wrote
Reply to [D] Has any research been done to counteract the fact that each training datapoint "pulls the model in a different direction", partly undoing learning until shared features emerge? by derpderp3200
There was a paper recently about finding parameter regions that are unused and only updating those on fine tuned data. Can't remember the name but that was an interesting approach.
realjunkman t1_j2cc4nm wrote
Reply to comment by derpderp3200 in [D] Has any research been done to counteract the fact that each training datapoint "pulls the model in a different direction", partly undoing learning until shared features emerge? by derpderp3200
It was a presentation I saw at EMNLP this past year. I’ll try and look for it, but if I don’t report back… it was a presentation during day 3!