There are of course pre-trained models for CNN (most of the time it is pre-trained on ImageNet in a supervised manner). If you talk about self-supervised pre-training for CNN specifically have a look at the recent papers Convnext 2 or SparK (BERT-style pretraining for convnets).