bluevase1029 t1_iw1khv8 wrote on November 12, 2022 at 5:55 AM Reply to comment by master3243 in [R] ZerO Initialization: Initializing Neural Networks with only Zeros and Ones by hardmaru I believe it's still difficult to be absolutely certain you have same initialisation across multiple machines, versions of pytorch etc. I could be wrong though. Permalink Parent 4
bluevase1029 t1_iw1khv8 wrote
Reply to comment by master3243 in [R] ZerO Initialization: Initializing Neural Networks with only Zeros and Ones by hardmaru
I believe it's still difficult to be absolutely certain you have same initialisation across multiple machines, versions of pytorch etc. I could be wrong though.