I mean yeah I agree, enforcing something like this is going to be very very difficult. But there are several clear examples of something like DallE generating images very similar to or nearly identical to copyrighted IP.

IANAL, but I presume a claimant could be able to establish some reasonable certainty to a court that licensed works were used in a way that breaks the license, at which point OpenAI (or really any AI company) would be responsible for defending their practice or non-use of those licensed works

Just because something is free to access doesn't mean you have the right to do whatever you want with it, especially with regards to making derivative works without attribution or otherwise breaking license terms. This is what licenses and copyrights are for.

For example, if OpenAI scraped a code repository that uses a Creative Commons NonCommercial license and is using that code for monetary gain without the owner's consent, they're breaking that license. It'd have to be argued whether the fact that OpenAI used that code to train their models which may generate code to similar likeness counts as distributing the source, and whether having a user use that model under a paid service counts as a commercial violation of those terms.

The algorithm is IP, yes. But GPT-X is part model part training data.

relevant xkcd