svantana t1_j14jwo4 wrote on December 21, 2022 at 4:53 PM Reply to comment by ackbladder_ in Reduce paramter count in an NN without sacrificing performance [P] by ackbladder_ Yeah, "distillation" is a key term here. Also, paperswithcode has joint data on performance and parameter counts, which gives a nice overview of the current pareto front. rwightman's repos is another nice resource. Permalink Parent 4
svantana t1_j14jwo4 wrote
Reply to comment by ackbladder_ in Reduce paramter count in an NN without sacrificing performance [P] by ackbladder_
Yeah, "distillation" is a key term here. Also, paperswithcode has joint data on performance and parameter counts, which gives a nice overview of the current pareto front. rwightman's repos is another nice resource.