damn i wish i could read

Ayo just 10 more of these

Would you look at that, all of the words in your comment are in alphabetical order.

I have checked 1,197,950,303 comments, and only 233,667 of them were in alphabetical order.

Okay, so what?

Pretty dang cool. They apparently up the efficiency of AIs by 2x by using different masks (noises) on the training data, and get smarter AI from it. They only need to do this as a small boost to existing models

They then do prompt engineering, but put it in the training data (under the hood, so to speak), which they call fine tuning and this lets the model follow instructions easier

So it's not just about qty of neurons, but how they get trained too

They also seem to further prove the empirical scaling law we've seen as models get larger

To quote Two Minute Papers: "What a time to be alive!" I'm convinced nearly all jobs will be done by AI in 15-20 years now, and not just done, but done better

Speculation on my part, but the fine tuning they do could also be done with ethical instructions, and may prevent some of the bad stuff AI does when trained on iffy data

lea michele is that you

I’m all for advanced AI politicians.

> AIs by 2x

it is by 3% on their graph lol.

>I'm convinced nearly all jobs will be done by AI in 15-20 years now, and not just done, but done better

language models already exceeded human performance in many benchmarks, but they struggle to replace humans at any work, why is that, what do you think?

Just based on language model progress and Kurzweil's predictions. I could easily be wrong, I'm just a random internet person, but I do have a Google TV and it can understand and search pretty dang well from voice alone. It queues up what I like as if I had a personal DJ. Ten years ago my cell phone couldn't even browse the net! Every ten years sees a new, impressive tech and AI looks like the next wave. Robotics is lagging, but some kitchen robots have started up (Flippy) as have a lot of warehouse robotics. Things are changing faster and faster. (Which makes sense as there are now 8 billion people figuring things out. It seemed like 7 billion yesterday!)

> it can understand and search pretty dang well from voice alone.

there is component where some model translates your voice to text, but searching part contains tons of human hand-crafted code.

So, current language models are good for some narrow tasks (translation is the main one), but still not on the level of abstract thinking human posses. My bet they won't be able, unless some large advancement.

I was impressed by the Google Say-Can project. I think we'll see an AI model that can do science pretty soon, it's the goal of Deepmind, and they've done amazing things. And, I think Google moved to a Transformer AI for search two years ago...

I did read a recent paper saying LLM were just doing fancy statistics and hardly thinking at all, but if it's sorta doing it now in only a few years it'll surpass Human level.

If I were to bet, I would say an evolutionary algo + interacting LLM based robots will be beyond anything we can imagine. First in a MMO like setting, then the real world. But we'd need super duper computers to do it, which we might not get because Moore's law may be done. Well... hmm... if the AI MMO was text based, the LLMs could run wild and evolve away without costing too much energy

I dunno... just dreaming! ^_^;;;

>First in a MMO like setting, then the real world.

nop, this transition is very hard because of following reason:

current wave of AI can approximate giant datasets, that's something it is doing very well. So, all your examples is: they throw terrabites of data on neural network, and it learns patterns. But this kind AI can't generalize and do abstract thinking which means it can't learn from very few examples.

Meaning yes, they can ask AI to play MMO 100 millions times and it will learn from its own mistakes, but you would need to do the same 100 million times in real world, which is not very feasible.

Another issue is that: MMO has much smaller level of freedom than real world, which makes MMO is not a good benchmark.

I’m looking forward to when I can say }%^# that’s annoying and the algo says ‘would you like me to elaborate on that and post it on reddit?’ And when I say yes, it composes some epic essay of complaint for me that doesn’t include the }%^#.