Right, hence why its relevant to large models trained on huge datasets. If the model can reconstruct data such that it is substantially similar to the original, then we have a problem. Whether from the viewpoint of copyright infringement or privacy law (gdpr).

You can make the same argument about lossy compression. Am I really infringing on copyright if I record an episode of House, re-encode it and redistribute it? It's not the 'original' episode, but a lossy copy of it. What if I compress it in a zip file and distribute that? In that case, I am only sharing something that can imperfectly recreate the original. The zip file itself does not resemble a video at all.

> (...) a new state of the art performance of 50.3% accuracy on the MedQA biomedical question answering task.

Oof, the fact that the accuracy is only 50% does not inspire confidence.