LeanderKu t1_j6hd6hp wrote on January 30, 2023 at 9:35 AM

Reply to comment by JamesBaxter_Horse in [R] Train CIFAR10 in under 10 seconds on an A100 (new world record!) by tysam_and_co

Well, this is also an assumption. It would be interesting which lessons do translate and which won’t. I wouldn’t dismiss it so quickly. Also, It’s a fun game to play and interesting in its own!

tysam_and_co OP t1_j6hfdaj wrote on January 30, 2023 at 10:06 AM

And, safe to say, there's stuff I'm not sharing here (yet?) that I've found as a result of that. Some hyperparameters are more network-specific, some are dataset specific. And some behave in ways just weirdly enough that you might get an entirely new adaptive method out of it... ;))))

I hadn't thought about that in the exact words you'd put it for a long while, but I think you're right very much so! It is quite a fun game, and very interesting to play in its own right. There's very much this bizzare, high-dimensional "messy zen" to it all. ;D

Thanks again for your comment, it warmed my heart and made me smile seeing it. Have a good evening/night/etc! :D :))) <3 <3 :D