shingekichan1996 OP t1_j5wlavz wrote on January 26, 2023 at 1:42 AM Reply to comment by melgor89 in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996 I saw an implementation of that paper here: https://github.com/raminnakhli/Decoupled-Contrastive-Learning And I saw also that the same paper is rejected at NeurIPS'21 becuase of its similar impact on other methods like Barlow Twins, SimSiam, BYOL, etc. However, at first glance at the re-implemented results, it works great on small batch-size indeed. Permalink Parent 3
shingekichan1996 OP t1_j5u40dx wrote on January 25, 2023 at 4:09 PM Reply to comment by mgwizdala in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996 exactly the paper I need to read! Thanks! Permalink Parent 1
shingekichan1996 OP t1_j5u22zn wrote on January 25, 2023 at 3:57 PM Reply to comment by mgwizdala in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996 Curious about this, I have not read any paper related. What is its effect on the performance (accuracy, etc) ? Permalink Parent 1
shingekichan1996 OP t1_j5tjy44 wrote on January 25, 2023 at 1:53 PM Reply to comment by IntelArtiGen in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996 I think single GPU for SSL contrastive learning is a research direction to pursue, I'm not sure if anyone published papers on it, but if there's none, I'm surprised. Permalink Parent 1
shingekichan1996 OP t1_j5tiupg wrote on January 25, 2023 at 1:44 PM Reply to comment by IntelArtiGen in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996 For 224x224 images, sure. But for images with large sizes, for example satellite images, it is hard to get 200+ batch size for a single gpu. Permalink Parent 1
[D] Self-Supervised Contrastive Approaches that don’t use large batch size. Submitted by shingekichan1996 t3_10ky2oh on January 25, 2023 at 1:00 PM in MachineLearning 28 comments 73
shingekichan1996 OP t1_j5wlavz wrote
Reply to comment by melgor89 in [D] Self-Supervised Contrastive Approaches that don’t use large batch size. by shingekichan1996
I saw an implementation of that paper here: https://github.com/raminnakhli/Decoupled-Contrastive-Learning
And I saw also that the same paper is rejected at NeurIPS'21 becuase of its similar impact on other methods like Barlow Twins, SimSiam, BYOL, etc.
However, at first glance at the re-implemented results, it works great on small batch-size indeed.