Could be cheap headphones, maybe your sound card doesn’t mute fully- could be a combo. In terms of fixing it, the most cost-effective solution is probably better headphones. If you can stretch even to $20 there will be a world of difference.