At that point I would pay for him to stay in a hotel. It's not fair, but having to let him stay in your house because of family pressure isn't fair either. So either tell them flat out no, or pay for a nights stay in a hotel.

But obviously you know the situation best.