It's a reasonable boundary to have. If I were you, I'd not encourage him to hang out with that friend but I'd also not try to prevent him from it. 'Sure' instead of 'yes, please do go out with her'.

However, personally, if my partner expressed a wish that I join them/interact with a person I've wanted to avoid, I would do it, because they're my partner, I love them and one potentially uncomfortable afternoon/evening/night is a tiny sacrifice to make. If I proceeded to dislike the person after that, my partner should respect that.

Dying on hills of principles, especially when they're so low stakes, is never a good sign in a relationship.

You should never open a relationship if both partners aren't enthusiastically for it. You did, despite not feeling comfortable with that. Now your options are the same as they were before, you either speak to your partner and go back to being monogamous or you speak to your partner and break up. Unfortunately you may simply be incompatible and as sad as it is, trying to hang onto your relationship as it is now is harming your physical and mental health. There is no trick to it, you are with the wrong person. Love just isn't enough for a healthy, fulfilling relationship.