Viewing a single comment thread. View all comments

More-Horse-3281 t1_jadyg0x wrote

I have no experience with GCP AutoML, but I have experienced heavy overfitting when using FLAML and auto-sklearn. Did you experience the same? (I.e. AutoML outperforming the open source algos on training data?) I have the feeling that a lot of AutoML solutions „cherry-pick“ models that just happened to shine on the training data.

12

fedegarzar OP t1_jaevmj5 wrote

I agree. Overfitting is a common problem in AutoML solutions. A proper validation strategy should improve the performance in unseen data, but in our experience, most of the AutoML solutions lack this feature.

9