Do you suggest doing cross validation with the training stopping mechanism "train for precisely the same number of steps I did in this run" or with "train using checkpointing and choosing the best checkpoint as I did in this run"?

I think with this few images I can't afford having a test set. Also, I thought that since I have approximately 50 million pixels to classify in the validation dataset, and given that computer vision practicioners often don't have a test split, I don't really need a test split. Now I'm not sure.

Ok, so my annotated data consists of about 50 images of size 10000x5000 pixels on average. The task is binary segmentation. Positives constitute approximately 8% of all pixels. 38 images are in the training part, 12 images are in the test part (I divided them randomly).

The batch cross entropy plot and the validation cross entropy plot were crazy unstable during training. After a little bit of training there mostly wasn't any stable trend in either going up or down. However, as the time went on, the best validation cross entropy over all checkpoints went down and went down...

So I think my checkpoint-selecting method gave me a model overfit to the validation dataset. I.e., I expect that on future samples the performance will be more like on the training dataset than on the validation dataset. The only other likely explanation I can think of is that I got unlucky and my validation dataset turned out to be significantly easier than my training dataset.

I don't have a test set. It's not specific to a task.

Recently, I have overfit to the validation dataset by doing this. The task is semantic segmentation. I trained for a very long time and I took the model with the best validation loss. Well, I got 0.02 nats/pixel cross entropy on val and 0.04 on train, 14% iou on val vs 24% on train.

The US has a long term visa, I forgot its name. The idea is that it's for exceptional people, including exceptional scientists. And to prove that you're exceptional, there's a list of like 9 items, of which you need to have 3 or more or something like that. One of them is "has published an article in a serious publication venue". Another is "has been a reviewer of the works of others at a serious venue" or something like that. I wonder if other countries have this.