Leaving someone who makes you feel like shit all the time is always a good move. He is trying to isolate you. He's beating down your confidence to have control over you. If you could trade places with anyone in the world who would you feel comfortable subjecting to your situation? If you flinched and went no one leave immediately.