Submitted by SimonJDPrince t3_10jlq1q in MachineLearning
arsenyinfo t1_j5m33oc wrote
As a practitioner, I am surprised to see no chapter on finetuning
SimonJDPrince OP t1_j5o0orv wrote
Can you give me an example of a review article or chapter in another book that covers roughly what you expect to see?
NoRexTreX t1_j5o9hd8 wrote
I can't give an example material outside of just huggingface documentation but it's the big thing right now to leverage pre trained models so if your book doesn't mention it then it's missing the hipest thing. And also adapterhub.
arsenyinfo t1_j5r4b5l wrote
Random ideas from the top of my head:
- intro why transfer learning works;
- old but good https://cs231n.github.io/transfer-learning/;
- a concept of catastrophic forgetting;
- some intuition on answering empirical questions like what layers should be frozen, how to adapt LR etc.
SimonJDPrince OP t1_j5taba8 wrote
Thanks. This is useful.
Apprehensive-Grade81 t1_j5oijh0 wrote
Definitely would like something like this. Maybe SOTA benchmarks as well.
new_name_who_dis_ t1_j5oix1c wrote
Fine tuning isn’t any different than just training…? You just don’t start with random network, but fine tuning doesn’t really have anything different besides that and the size of the dataset
SimonJDPrince OP t1_j5olux3 wrote
That was kind of my impression. And I do discuss this in the chapters on transformers and regularization. Was wondering if there is more to it.
Viewing a single comment thread. View all comments