The root of your problem is that you are stuck assigning 100% confidence to the prediction that your horse is a cat or a dog. It’s perfectly rational that you might have p(cat)=1e-5 and p(dog)=1e-7 for a horse picture, right?

So when you normalize those, you get basically p(cat)=1.

Try binary classification of cat vs not cat and dog vs not dog. Don’t make them sum to one.