Comments

You must log in or register to comment.

Latter_Security9389 t1_izirxj3 wrote

You are missing a test split. It's common to pick the best validation checkpoint but you still want a test split (that's completely unseen during training/model picking) to test your model.

You also need to be careful with the metrics you look at to test your model because your classes are very imbalanced.

3

CrazyCrab OP t1_izisk28 wrote

I think with this few images I can't afford having a test set. Also, I thought that since I have approximately 50 million pixels to classify in the validation dataset, and given that computer vision practicioners often don't have a test split, I don't really need a test split. Now I'm not sure.

0

Nameless1995 t1_izit9gd wrote

Perhaps you can try cross-validation.

1

CrazyCrab OP t1_iziuhq4 wrote

Do you suggest doing cross validation with the training stopping mechanism "train for precisely the same number of steps I did in this run" or with "train using checkpointing and choosing the best checkpoint as I did in this run"?

0

plocco-tocco t1_izj4iy8 wrote

I would take the best checkpoints (aka when the validation loss starts diverging from the training loss). Not the same number of steps because it can happen that the networks don't converge to a minima at the same time, some may be stuck somewhere for longer.

1