Update: I had someone accompany me outside to test the fumes out with a bic lighter around 1pm. Figured I'd go ahead and test worst case scenario, either I'd be fine or suddenly it wouldn't be my problem anymore.

So all went fine there. I turned on the light and shut the door 5 hours ago and been fine since.

It's currently 17 degrees Fahrenheit outside the well house and 54 degrees inside of it.