I don't see the problem. The language model can have feedback from code execution. If it is about facts, it could have access to a search engine. But the end effect is that it will be much more correct. A search engine provides grounding and has fresh data. As long as you can fit the data/code execution results in the prompt, all is ok.

But if we save the correctly executed tasks and problems we could make a new dataset to be used in fine-tuning the model. So it could learn as well.