Maybe sitting down with him and having an honest conversation about everything would be helpful. Like telling him how much his words hurt you and have damaged your self esteem. Even if he already apologized it does not mean those feelings he caused just magically disappear.

But on the other hand you need to work on yourself aswell, if you are able to get professional help to work on your self esteem, it sounds like you could really benefit from building yourself up more, maybe get a hobby, were you can express yourself and gain more confidence.