You have a misunderstanding of temperature. Temperature is different than humidity and that is also different from heat loss.

To start backwards, when you are sitting at a window and the weather is cold outside, when you feel , "the cold" at the window you are actually feeling not the cold air coming in, you are experiencing heat loss from your body.

When you stick a thermometer outside it senses the air temperature and it reaches a state when the liquid inside the bulb is the same as the outside air temperature. There is no heat exchange taking place as they are in equilibrium. Unlike your body which generates heat and you will continue to feel the heat loss just like at the window.

Humidity is a measurement of the air to carry water vapor without it returning to a liquid state. It is expressed as a percentage where the current water load of the air is compared to the maximum capacity.

Capacity goes up as a function of temperature. That is why the cold months seem dryer.

I hope that helps you to understand.