Yes. Definitely drop him too. What a shallow and rude thing to say. He's more concerned with the number than how you look even, which is just stupid.

I can understand people having/wanting a certain lifestyle and wanting a partner with similar goals. But being willing to leave you over the number on the scale while you're in the process of working on your fitness after an injury is just....ridiculous. You can do, and deserve, better.