Viewing a single comment thread. View all comments

Scarlet_pot2 OP t1_j398xn5 wrote

Reply to comment by footurist in We need more small groups and individuals trying to build AGI by Scarlet_pot2

The expectations should be tampered. the foundations of AGI aren't going to be made in a couple of hours, but just as "guess the next word" was found out and led to LLMs, I'm sure there are many of simple small discoveries waiting to be found. And many diverse groups trying different things and sharing their results could lead to some of those. It may not be you that builds the million dollar model, but you could make the first simple program that shows promise and ends up being the base idea for large models a few years down the line, by someone.

2

footurist t1_j399rxd wrote

Between the lines I read the assumption that "guess the next word" is definitely agreed upon as being part of or precessor of future AGI, when that's actually highly unclear. Right now they're standing in front of the brick wall of lack of actual reasoning and therefore highly inconsistent emulated reasoning. And it's not clear that's susceptible to a fix or workaround. It could actually be a fundamental limitation of the architecture.

1

Scarlet_pot2 OP t1_j39b08w wrote

IMO, guess the next word isn't going to lead to AGI alone, but it most likely will play a part. Let's assume "guess the next word" fills the part of the brain for prediction down the line for when AGI is developed. Maybe a small group develops the first thing that will later on fit another part of the brain, like how to make memory work. or how to develop reasoning. or any other parts.

The goal should be to make discoveries that could lead to parts of AGI when extrapolated out.. and at least some of those can be found by small groups trying new approaches. John Carmack said that all the code of AGI would be able to fit on a USB. the goal should be to find parts of that code.

1

visarga t1_j39x1lv wrote

The code, yes, but the dataset will be the entire internet and loads of generated data. We have the people, what is necessary is to give them access to compute.

1