Viewing a single comment thread. View all comments

netw0rkf10w OP t1_j0gcgxy wrote

Reply to comment by TimDarcet in [D] What are the strongest plain baselines for Vision Transformers on ImageNet? by netw0rkf10w

Thanks. DeiT is actually a very nice paper from which one can learn a lot of things. But the training regimes that they used seem a bit long to me: 300 to 800 epochs. The authors of MAE managed to achieve 82.3% for ViT-B after only 100 epochs, so I'm wondering if anyone in the literature has ever been able to match that.

1

TimDarcet t1_j1w6ifs wrote

I think the supervised training they report in MAE is 300 epochs, they used a different recipe compared to finetuning (appendix, page 12, table 11)

2

netw0rkf10w OP t1_j2939o2 wrote

You are right, indeed. Not sure why I missed that. I guess one can conclude that DeiT 3 is currently SoTA for training from scratch.

1