Then divorce him, but you can't have it both ways. Words are tricky, because regardless of what they were said, they obviously can't be taken back. But if you can't move on, then move on from the relationship, you both deserve it, and he sounds like a ninkapoop.