I say be straight forward with them. "I don't think that's a question I need to answer" If they ask why: "I don't owe you an explanation for keeping my personal life personal. But I'd be willing to discuss anything related to our business" or alternatively "you first". And then you still don't have to reply.