Submitted by hardmaru t3_ys36do in MachineLearning
VinnyVeritas t1_iw9ajwe wrote
Reply to comment by master3243 in [R] ZerO Initialization: Initializing Neural Networks with only Zeros and Ones by hardmaru
The performance is not better: the results are the same within the margin of error for standard (not super-deep networks). Here I copied from their table:
Cifar10
ZerO Init 5.13 ± 0.08
Kaiming Init 5.15 ± 0.13
Imagenet
ZerO Init 23.43 ± 0.04
Kaiming Init 23.46 ± 0.07
Viewing a single comment thread. View all comments