He can believe whatever he wants about drinking alone, he doesn't have to agree with your feelings on it, but he also shouldn't get to tell you what you are or are not "allowed" to do. He's not your owner.