levand t1_j9qe7ev wrote on February 23, 2023 at 8:45 PM

Reply to comment by suflaj in Why bigger transformer models are better learners? by begooboi

> These models are too small to truly overfit on their datasets.

I thought we were talking about 175 billion parameters, literally some of the biggest models in existence? Although it is true that at some point models get big enough that they become less prone to overfitting (and it's not clear why): https://openai.com/blog/deep-double-descent/

levand t1_j9pmtz7 wrote on February 23, 2023 at 5:56 PM

Reply to comment by Appropriate_Ant_4629 in Why bigger transformer models are better learners? by begooboi

Well that’s not the whole story, a bigger model is also more prone to over fitting depending on the training data.

levand t1_j7o5zeb wrote on February 8, 2023 at 5:03 AM

Reply to Is there any AI-distinguishing models? by Such_Share8197

This is inherently a super hard problem, because (to oversimplify) the loss function of any AI generating NN is to minimize the difference between a human generated and AI generated images. So the state of the art for detection & generation is always going to be pretty close.