You made the right call. Her telling you that was an attempt to try to get you chasing again. That's a race you'll never win in the long term.

Prepare yourself in advance for "Yes I know, he told me, and I agree with him".

Seems to be a lot of information you're leaving out of this. I really hope your brother every once in a while dropping a rude comment isn't enough for you to bail on your family, so I assume there's hopefully a lot more.

More info would help towards answering your questions.