Cheers

You just need a notion of a fitness function and then you can apply permutations to the tokens.

The latest version is here: https://2022.ecmlpkdd.org/wp-content/uploads/2022/09/sub_1229.pdf

Of course we require no encoding model, so the notion of a latent space only holds up until closer inspection.

Hey, this is it actually! We are optimizing a discrete variational autoencoder with no gumbel-softmax trick.

We recently published an evolutionary method to sample from the latent space of a variational autoencoder. It is still alive and well. Just a bit niche.

Indeed, the generative model produces data points, and the discriminative one classifies them together with the real data. I think that for your purposes it is easier to refer to your algorithm as "adversarial in nature". You are using games where the algorithms are expected to reach a Nash equilibrium, but also there is no gradient (presumably) from one agent to another.

It's what the author of those papers wanted as a name for GANs. Arguably it is more intuitive in a RL context, although I cannot speak about the equivalence as I am not super familiar with GANs.

Inverse predictability minimisation might be a term if you can get around the controversy. As a bonus, you might make a certain German very happy.

Maybe you are interested in Tishby's rate distortion. E.g. in this paper they do an analysis of the behaviour of mutual information in the hidden layers as a neural network is trained to convergence.

Nobody else has mentioned resnets, yet. They have something like higher order weights with f(x) = σ(W1σ(W0x+b0)+b1) + σ(W0x+b0). Highway networks take it a step further with f(x) = σ(W0x+b0)σ(W1x+b1) + xσ(W2x+b2). However, both are done to resolve gradient issues.