MadScientist-1214 t1_j8ox26g wrote on February 15, 2023 at 10:09 PM

Better than AdamW if (a) the model is a transformer, (b) not a lot of augmentations are used. Otherwise, the improvements are not that large. I doubt this optimizer works well with regular CNNs like efficientnet or convnext.

CoderHD t1_j989j2g wrote on February 20, 2023 at 12:33 AM

In my limited testing on a UNet like CNN, it doesnt even come close to the performance of adam sadly. With that said, i might be doing something wrong.