Viewing a single comment thread. View all comments

footurist t1_j37hsvn wrote

Reply to comment by Cryptizard in We need more small groups and individuals trying to build AGI by Scarlet_pot2

It's the underestimation. The thing is, for some reason AGI seems like an approachable problem on first sight. There's something about it that makes you think there has to be some simple, yet surprisingly undiscovered way of building it.

But if and when you actually try to build something, no matter how naive or small, you very quickly recognize the incredible hidden complexity.

I've tried it too, I admit. You go from "I think it's doable" to "hell no, this isn't ever gonna work" in a couple of hours, lol.

7

Scarlet_pot2 OP t1_j398xn5 wrote

The expectations should be tampered. the foundations of AGI aren't going to be made in a couple of hours, but just as "guess the next word" was found out and led to LLMs, I'm sure there are many of simple small discoveries waiting to be found. And many diverse groups trying different things and sharing their results could lead to some of those. It may not be you that builds the million dollar model, but you could make the first simple program that shows promise and ends up being the base idea for large models a few years down the line, by someone.

2

footurist t1_j399rxd wrote

Between the lines I read the assumption that "guess the next word" is definitely agreed upon as being part of or precessor of future AGI, when that's actually highly unclear. Right now they're standing in front of the brick wall of lack of actual reasoning and therefore highly inconsistent emulated reasoning. And it's not clear that's susceptible to a fix or workaround. It could actually be a fundamental limitation of the architecture.

1

Scarlet_pot2 OP t1_j39b08w wrote

IMO, guess the next word isn't going to lead to AGI alone, but it most likely will play a part. Let's assume "guess the next word" fills the part of the brain for prediction down the line for when AGI is developed. Maybe a small group develops the first thing that will later on fit another part of the brain, like how to make memory work. or how to develop reasoning. or any other parts.

The goal should be to make discoveries that could lead to parts of AGI when extrapolated out.. and at least some of those can be found by small groups trying new approaches. John Carmack said that all the code of AGI would be able to fit on a USB. the goal should be to find parts of that code.

1

visarga t1_j39x1lv wrote

The code, yes, but the dataset will be the entire internet and loads of generated data. We have the people, what is necessary is to give them access to compute.

1

Mental-Swordfish7129 t1_j3bbbpn wrote

>I've tried it too, I admit. You go from "I think it's doable" to "hell no, this isn't ever gonna work" in a couple of hours, lol.

I've been at it for around 12 years in my little free time and I've made fairly steady progress excluding a few setbacks. I think I must have gotten very lucky many times. I know that when I look at my approach back then, that I was wayyy off and very ignorant and ridiculous.

2