>It is astonishingly unpopular to actually do the count, and notice that something like Stable Diffusion contains the gist of all of art history and the personal styles of basically all famous artists, thousands of celebrities, the appearance of all sorts of objects, etc., in a model that in a synapse-for-parameter count matches the brain of a cockroach.

I want to call that out for being wrong. SD's phase space contains loads of jibberish and how good an image model is is dictated by how little bad images it's phase space contains, not by how many good ones it does. If your argument was right then RNG would be the best generative image model because the phase space of it's outputs containts every good image.