If you have tabular data, the solution is to use XGBoost. Resnets are pretrained on imagenet, meaning if you need it pretrained for any other task, you'll have to do it yourself. I do not see how the task would benefit from a ResNet.

While convolution is a bit funky with tabular data (what locality are you exploiting?), I think that attention is a mechanism that might make sense in the deep learning context for tabular data. For example, take a look at recent work such as https://openreview.net/forum?id=i_Q1yrOegLY (code and PDF linked from there).

I did not want to use resnet as is, I'm not requiring the convolutional layer itself. I'm looking for general purpose residual blocks with skip connections

Resnet is just the architecture. It has nothing to do with pertaining data. You can have skip connections with linear layers as well.

You need to understand why you want to use resnet architecture. The key reason for using resnet is that as NNs get deeper the inputs to each successive layers become smaller and smaller. This can be circumvented by adding the input data to the output of some layers later.

So in the case of tabular data you need to see why you want a NN and if it is the only thing that works for you. Next question is whether you want to have a very deep NN necessarily. What if you transformed the inputs? If yes, then you should be able to build a skip connections trivially.

Yes, which is why I think you’ll find that link of particular interest since they comment on it (and attention).

Any particular reason why resnet only and looking for general purpose residual blocks

Friends don’t let friends use deep learning on tabular data

I know, however try finding a resnet pretrained on something other than a CV dataset.