[D] What is the correct term for a non-GAN system where two or more networks compete as part of training?

Submitted by mosquitoLad t3_11au29r in MachineLearning

As the title.

I believed adversarial training was a catch-all term describing systems where two or more networks--with similar or distinct, but always mutually exclusive, goals--compete in zero-sum games and improve over time, but I'm finding that adversarial training relates to security, while generative adversarial networks specifically describe generation and detection.

edits:

It doesn't capture the spirit of what I'm looking for, but a broader term is Multi-Agent Reinforcement Learning

8

Comments

You must log in or register to comment.

Optimal-Asshole t1_j9ugfhx wrote

Adversarial is still the right term, it just has another definition in the context of security. You could also try min max or zero sum game

19

mosquitoLad OP t1_j9uhqag wrote

Thanks, that helps. I need to give a brief educational presentation about the subject, so I didn't want to throw out the wrong terminology.

1

filipposML t1_j9uhfac wrote

Inverse predictability minimisation might be a term if you can get around the controversy. As a bonus, you might make a certain German very happy.

2

mosquitoLad OP t1_j9uiffv wrote

Are you referring to this? "Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)"

https://arxiv.org/abs/1906.04493

Looking up IPM verbatim turned up a reddit post linking to that.

3

filipposML t1_j9uokeu wrote

It's what the author of those papers wanted as a name for GANs. Arguably it is more intuitive in a RL context, although I cannot speak about the equivalence as I am not super familiar with GANs.

2

mosquitoLad OP t1_j9vazbs wrote

My loose understanding of GANs is that one agent creates assets i.e. images and audio, while another agent attempts to differentiate assets based on if they were or weren't created by an agent. The results create automatically labeled data that can be used in subsequent training cycles, optimally leading to higher quality asset output.

I'm mixed about the IPM label. Predictability Minimization seems okay by itself; Inverse seems tacked on. Maybe something like Counter Predictability Exploitation?

1

filipposML t1_j9vc6dw wrote

Indeed, the generative model produces data points, and the discriminative one classifies them together with the real data. I think that for your purposes it is easier to refer to your algorithm as "adversarial in nature". You are using games where the algorithms are expected to reach a Nash equilibrium, but also there is no gradient (presumably) from one agent to another.

1

mosquitoLad OP t1_j9vq86y wrote

Nash Equilibrium is a new term for me; and you are right, that seems like a logical end state. I do not understand what a gradient is in this context; would this terminology apply when information is being processed by a series of agents, each having a direct influence on the quality of the output of other agents?

1

[deleted] t1_j9vqstv wrote

[deleted]

2

mosquitoLad OP t1_j9vrjsb wrote

Completely fair

1

[deleted] t1_j9vse6h wrote

[deleted]

1

mosquitoLad OP t1_j9vtp3b wrote

Thanks. It's not so big as a seminar. I'm in a public speaking course where each primary speech falls into a certain criteria, this one being Educational. I'm a senior CS major, the majority are freshman non-CS, so I've to make sure whatever I say is both accurate and explained in simpler terms (less 3Blue1Brown, more Code Bullet I guess).

1

clueless1245 t1_j9wzcbn wrote

Idk what he means specifically by the "gradient being passed between" two agents but in a GAN (part of) the loss function of the generator is the inverse of (part of) the loss function of the discriminator, so the gradients calculated at generator output and discriminator output are linked.

A less formal way of saying it: The generator's gradient depends on the discriminator's loss.

This should be true for any adversarial game, I would think?

2

mosquitoLad OP t1_j9x2guv wrote

The less formal way conveys the concept better; and it makes sense, the worse the discriminator performs (whether it is overly sensitive or less sensitive when attempting to sus out the validity of assets), the worse the generator performs, at least with regard to the quality of the output for human purposes. If I'm understanding the use of gradient correctly, the generator become trapped in a local minimum because it discovers how to consistently exploit the weaknesses of the discriminator.

I don't know for sure if it always applies; you could apply an evolutionary algorithm where two or more competing populations are tackling the same problem from opposing sides, and have relatively infrequent breeding between members of the populations, motivating avoidance of bottlenecking while enabling the development of unique solutions; over several generations, any short term loss should serve to be a long term gain. But, I guess they'd still be dependent on how the scoring system works (equivalent to loss function?).

2

clueless1245 t1_j9x3dlc wrote

Its also an issue for generator training though if the discriminator gets 100% all the time, if I remember correctly. Theres various stuff you can look up to make training more stable which I don't have on hand rn.

2