What everyone is saying here is valid, but I have an add from a love language perspective too, but as a woman.

I'm going to hazzard a guess that physical touch is low on his love language list. When I test out, it's below 10% for me - and I have extremely high sex drive. But I DO NOT crave the small touches. So I like them, sure. But I don't need or crave them so I have to be reminded to give them to another. Sounds like your husband could be like me, but doesn't seem to care to adapt to your language for some reason. There's no compromise happening. You've got to find a way to meet in the middle. You need more PT to be "shown" love he needs another way.