It might be worth getting some more breathable earpads for this purpose (something like velour or perforated leather, as opposed to sealed leather). I'm in the same boat - I have the Audio-Technica ATH-M50xBT2, and the stock pleather pads get way too hot for me when doing any sort of activity with them on, so I'm looking for a replacement that will stay cooler.

Note that any alternate earpads will change the sound signature, and also noise cancellation performance in case of active noise cancelling headphones like the XM4.