qrayons t1_je06jmp wrote

I read Chollet's article since I have a lot of respect for him and read his book on machine learning in python several years ago.

His main argument seems to be that intelligence is dependent on its environment. That makes sense, but the environment for an AI is already way different than it is for humans. If I lived 80 years and read a book every day from the day I was born to the day I died, I'd have read less than 30k books. Compare that to GPT models which are able to read millions of books and even more text. And now that they're becoming multimodal, they'll be able to see more than we'll ever see in our lifetimes. I would say that's a drastically different environment, and one that could lead to an explosion in intelligence.

I'll grant that eventually even a self-improving AI could hit a limit, which would make the exponential curve to look more sigmoidal (and even Chollet mentioned near the end that improvement is often sigmoidal). However, we could still end up riding the steep part of the sigmoidal curve up until our knowledge has increased 1000 fold. I'd still call that a singularity event.


qrayons t1_jcglmnc wrote

Yeah right now most people barely have the hardware that you would need to generate the environments/actors, let alone experience it in VR at the same time. I think what we'll see as an intermediate step is an explosion of content created for VR ahead of time, and then you'll be able to step in and experience it. It'll be a while before we can simultaneously generate and experience.


qrayons t1_jcbi3sk wrote

I mean, Karpathy was kind of right. This is from his original post.

> I’ve seen some arguments that all we need is lots more data from images, video, maybe text and run some clever learning algorithm: maybe a better objective function, run SGD, maybe anneal the step size, use adagrad, or slap an L1 here and there and everything will just pop out. If we only had a few more tricks up our sleeves! But to me, examples like this illustrate that we are missing many crucial pieces of the puzzle and that a central problem will be as much about obtaining the right training data in the right form to support these inferences as it will be about making them.

One of the crucial pieces we were missing was attention. So much of the advancement we are seeing now is because of transformers.


qrayons t1_j9ouqi9 wrote

I feel like there's a fine line. In general, I recognize the creativity that goes not only into crafting the prompt, but also using artistic vision to adapt the prompt based on the outputs and select the best images. Not to mention the editing that can occur in the outputted images.

However, if there are absolutely zero restrictions on copyrighting AI generated art, then someone like Disney could write a program that generates millions of outputs of unique cartoon characters and they would all be copyrighted and then if someone recreated a similar character (by pure chance), that person would be violating Disney's copyright, and that doesn't feel right.


qrayons t1_j9few1h wrote

> Blake's biggest mistake was that he didn't release the full, unedited transcripts . When I learned that the transcripts were edited he lost all credibility with me, and I assumed the worst.

That was my reaction as well. Is there any other information that lends credibility to what he was saying? I stopped paying attention when I saw that he edited the transcripts.

Also interesting, I remember when reading the transcripts that I had a list of questions that I knew lambda would fail at and it would demonstrate how basic a lot of these language models still are. Then when I got access to chatGPT I asked those questions and it passed with flying colors and I've had to rethink a bunch of things since then.


qrayons t1_j6i1e3w wrote

In medical, there's more of a gap between the innovation and the impact in the market (need time for safety testing, etc.). One of the biggest breaks in medicine within the last several years was the solution for protein folding. We still haven't experience the impact of this yet (a tidal wave of new and more effective pharmacological treatments).