I think what you're looking for is the ML field called "continual learning", where an agent is put into practice and learns in the real world how to improve as it acts in it (from my understanding). afaik its one of the more experimental ones