xenotecc
xenotecc t1_j3v3gr0 wrote
Reply to comment by suflaj in [D] Have you ever used Knowledge Distillation in practice? by fredlafrite
How small do you make the student, when a teacher is let's say ResNet101? How do you find a good student/teacher size ratio?
Are there any tricks to knowledge distillation? Or just standard vanilla procedure?
xenotecc t1_j3vp75c wrote
Reply to comment by suflaj in [D] Have you ever used Knowledge Distillation in practice? by fredlafrite
Thank you for the reply!