shingekichan1996
shingekichan1996 OP t1_j5u40dx wrote
Reply to comment by mgwizdala in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996
exactly the paper I need to read! Thanks!
shingekichan1996 OP t1_j5u22zn wrote
Reply to comment by mgwizdala in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996
Curious about this, I have not read any paper related. What is its effect on the performance (accuracy, etc) ?
shingekichan1996 OP t1_j5tjy44 wrote
Reply to comment by IntelArtiGen in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996
I think single GPU for SSL contrastive learning is a research direction to pursue, I'm not sure if anyone published papers on it, but if there's none, I'm surprised.
shingekichan1996 OP t1_j5tiupg wrote
Reply to comment by IntelArtiGen in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996
For 224x224 images, sure. But for images with large sizes, for example satellite images, it is hard to get 200+ batch size for a single gpu.
shingekichan1996 OP t1_j5wlavz wrote
Reply to comment by melgor89 in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996
I saw an implementation of that paper here: https://github.com/raminnakhli/Decoupled-Contrastive-Learning
And I saw also that the same paper is rejected at NeurIPS'21 becuase of its similar impact on other methods like Barlow Twins, SimSiam, BYOL, etc.
However, at first glance at the re-implemented results, it works great on small batch-size indeed.