There are approaches to combine multiple stages of language modelling and retrieval. Demonstrate Search Predict: Composing retrieval and language models for knowledge intensive NLP.

This paper is very interesting. They don't create or fine-tune new models. Instead they create sophisticated pipelines of language models and retrieval models. They even publish a new library and show this way of working with LMs.

Practically, by combining retrieval with language modelling it is possible to verify against references. The ability to freely combine these transformations opens up the path to consistency verification. A LM could check itself for contradictions.