I'd say something, and I'd definitely say something if he brings up the evening or suggests you come back over again.

"I enjoy your company and we have a good time together [other words here], but I have to tell you - I'd rather see you elsewhere or at my place. Your apartment was [be specific] and very uninviting. If you want to entertain women at your apartment, you might consider [whatever he needs to do]. Anyway just needed to say that. So meanwhile, [SUBJECT CHANGE]."