If you feel comfortable removing one person from the group go for it.

If it didn't feel off to you, you wouldn't be asking.

I think it's in bad taste but it doesn't seem to be to you and your fi