One thing to watch for with these language models is they can be made to say anything. Especially with a service that has broken with it's original purpose such as OpenAI, is that these language models can be trained on generated text from a similar AI language model. What they can do (and already do in my opinion) is generate bunches of text saying something to the same effect, such as "the sky is yellow", in many different variations. Then when the language model is trained on this generated text it will learn to say that the sky is yellow.

The fact is these things don't actually "know" human language, they use a lot of math to come up with their responses. The math is what's fine-tuned during the training.