I think genetic algorithms may have a new role to play in problems involving inference / text generation / prompting with language models, even if they aren't used to train the models themselves.

For example, in our recent work on natural-language prompting, we use a genetic algorithm to generate prompts that are semantically coherent -- the genetic algorithm lets us make use of suggestions by a language model, for which gradients would be hard to obtain.