deep_alichemist t1_jd2mzoh wrote on March 21, 2023 at 12:11 PM Reply to [D]: Vanishing Gradients and Resnets by Blutorangensaft Use any kind of normalization additionally to skip connections. ResNet alone is not enough, except if you carefully tune everything (eg. https://arxiv.org/abs/1901.09321). Permalink 2
deep_alichemist t1_jd2mzoh wrote
Reply to [D]: Vanishing Gradients and Resnets by Blutorangensaft
Use any kind of normalization additionally to skip connections. ResNet alone is not enough, except if you carefully tune everything (eg. https://arxiv.org/abs/1901.09321).