Submitted by ackbladder_ t3_zrpsfm in MachineLearning
svantana t1_j14jwo4 wrote
Reply to comment by ackbladder_ in Reduce paramter count in an NN without sacrificing performance [P] by ackbladder_
Yeah, "distillation" is a key term here. Also, paperswithcode has joint data on performance and parameter counts, which gives a nice overview of the current pareto front. rwightman's repos is another nice resource.
Viewing a single comment thread. View all comments