When the image or 'form' of the 'thing' in your mind associates with the word in a new language rather than having to actually process the translation from your original language to the new one - that's when you 'know' a language, IME. And its when more than 80% of the words coming out of someone's mouth have that image association in your head despite accent, speed, age, colloquialisms, etc.

I became fluent in Fijian (for the most part, its a non-written language) in the 90's when I lived there, and learning a language by experience (vs study/books) really reinforces my point above. The sound associates with the image in your brain--that's when you 'know' a language.