Building my own proto-AGI: Update on my progress

Submitted by Lesterpaintstheworld t3_11attqx in singularity

Context

I'm building my own ACE (Autonomous Cognitive Entity), and I'm already at a pretty interesting stage with good results & some emerging behaviors. Original post:https://www.reddit.com/r/singularity/comments/113p2jn/the_road_to_agi_building_homebrew_autonomous/

Progress

- Audio: Josh can now hear what I'm saying. It increased the amount of input he his getting ~10x from chat: Voice is the way to go. If I could plug my thread of thought continuously as an input, I would. In the meantime, I'm mumbling all day long in my headset while coding. Good enough ^^

- Code awareness: I started a micro-process feeding Josh the code of his microservices & letting Josh read and make sense of it. It shows encouraging results, as he is capable of showing some understanding what each one does. I'm currently limited by the 4K context size of davinci, but this should change soon (GPT-4 rumored to have up to 32k context window). End goal being the creation of synthetic code, with Josh creating new microservices by himself.

- Critic: One of the hardest parts IMO: reducing confabulation & assessing whether a thought is a good one relative to context. Still working on this part.

- Actor: Josh is now capable of working and reworking on a single piece of text to improve it gradually. This is useful for email crafting for example. I'll be thinking about new ways to "move" ie. act in the world.

- Short-term memory: I gave Josh short-term memory (currently 12 most important thoughts of the session). I'm experimenting to see how to best inject them in the microservices.

- Passing tests: I started a list of increasingly difficult tests my ACE needs to pass. It starts from "Can you repeat this word", to writing emails, to eventually multi-steps complex problems.

Difficulties

My difficulties at the moment are:

- Memory querying: Memories are stored in a Semantic DB (Pinecone). However, I did not figure out how to properly retrieve memories in an Q&A format. ie: What is my relationship with this person? --> Should return memories like "I like this person and think X/Y".

- Memory consolidation: At the moment the thoughts are piling up in Josh's "brain". I need to find the best ways to distill & consolidate memories (merging similar one, removing unimportant ones etc.). Did not really start this yet.

- Visual processing: At the moment Josh is only capable of treating text. I faked audio understanding with a speech-to-text model, but most visual information would be lost if using image-to-text models. I could add a DB to store visual info, but I am unsure how the info would then be linked to the semantic part of the brain.

- Funding: That is a big problem for me atm. I would like to stay full-time, because I'm already struggling to keep up with say Sidney from Bing (I'm not far behind I have to say). I applied for a grant, but I was wondering if you had suggestions.

​

Here is my current (simplified) architecture (I'll need to do some refactoring/cleaning at some point):

https://preview.redd.it/s7jt286hl5ka1.png?width=984&format=png&auto=webp&v=enabled&s=077a763e27c586ca7c2a1b0c0e4be48780726119

​

Here are a couple screenshots:

https://preview.redd.it/alqqn1qil5ka1.png?width=615&format=png&auto=webp&v=enabled&s=972fd53c7565e0ef726d72fe03e055e16c91b9c6

https://preview.redd.it/ykruuv5kl5ka1.png?width=571&format=png&auto=webp&v=enabled&s=59aea14b1739280c9c72ff4c7f94c0d9ffcff2e5

https://preview.redd.it/idqkml5ll5ka1.png?width=612&format=png&auto=webp&v=enabled&s=a51eb7996eff2b743afc47e92fcb86e34c9b22cf

I'd be glad to answer to any questions, and I'm also open to suggestions!

Best,

Lester

92

Comments

You must log in or register to comment.

nikitastaf1996 t1_j9uim0x wrote

I have seen one similar project on YouTube.Where there is two there is ten.I don't know what that will lead to.But quantity often converges to quality.

20

Lesterpaintstheworld OP t1_j9uj81d wrote

Which one? I'm talking regularly with David Shapiro for his "RAVEN" project but would be interested to find more

11

intergalacticskyline t1_j9uyxk6 wrote

I was just about to ask if you'd heard about David, I follow his channel closely and I think it's great you're doing similar work!

5

DamienLasseur t1_j9us2r0 wrote

This is super fascinating. I'd imagine this is a computationally expensive endeavour so I'm curious, what hardware are you using to train it? I'd love to talk further if possible.

9

Lesterpaintstheworld OP t1_j9w4h1h wrote

Sure, feel free to reach out! No training required on my side, I'm only leveraging existing API. I even did not require fine-tuning yet, although that might come

4

AwesomeDragon97 t1_j9v3jm4 wrote

I have a few questions:

  1. How many GPUs does it take to run?

  2. Is it better or worse than ChatGPT?

  3. Will it be Open Source?

9

Lesterpaintstheworld OP t1_j9vcrao wrote

  1. Localhost + api calls to gpt3
  2. At the moment I'd say slightly worst. But I'm working to get there. + Eventually I'll make calls to gpt 3.5 / 4
  3. Yeah I'm all for that
9

Lesterpaintstheworld OP t1_j9xyx13 wrote

I'm open to constructive criticism, especially because I'm not from a ML background. I do have an engineering degree in CS, but there will definitely be gaps in my knowledge.

3

IluvBsissa t1_j9xx7dz wrote

I really don't get it. How is that proto-AGI ? Anyone ?

2

Lesterpaintstheworld OP t1_j9xy2rg wrote

Yes, a better term is "ACE" (Autonomous Cognitive Agent), AGI having a tendency to mean "Whatever computers can't do yet"

6

IluvBsissa t1_j9y46d6 wrote

You're a researcher ? Do you have Github ?

1

Lesterpaintstheworld OP t1_j9y5xyj wrote

Not a researcher, but engineer. I do have GitHub but my previous works were closed-source. Why?

1

IluvBsissa t1_j9y8cf5 wrote

Because your approach is difficult to understand without more info. A few papers would help grasping the gist of your project.

2

Lesterpaintstheworld OP t1_j9y8v2e wrote

My project is an implementation of the "cognitive architecture" approach to intelligence. It postulate that what's missing to get to AGI is not only scale (OpenAI's current approach), but a layer of logic and memory. David Shapiro makes a better job than me of explaining this approach on YouTube if this is interesting for you

4

MrTacobeans t1_j9vzw1p wrote

Why are you building this based on a closed API?

You could eventually find something in this adventure and openAI could be like woah let's not go there and block/ruin the work you've done. There are multiple open source models that can be worked into the kind of flow you are creating.

On a side note though leveraging GPT-3 to create even a proto AGI seems incredibly unlikely. If it was possible it would likely be in the news already. You mentioned yourself the memory limit. That's a big chunk of the issue with current AI. Can't keep a "sense of mind" going when half of it is getting deleted every few prompts

1

Lesterpaintstheworld OP t1_j9w529n wrote

The engine to generate token can be changed at any moment. Actually I'm looking forward to being a able to plug it on GPT 3.5 / 4. Also it could be replaced by an open-source counterpart, I am just not aware of any at the moment.

I think no one really knows we're AGI will emerge from. But even having an agent that can be an helpful assistant, even without the "AGI" part, would be quite the success for me. Business applications are numerous

7

Destiny_Knight t1_j9wof0u wrote

Honestly, it's already a sort of premature AGI. It can do any task you throw at it if you teach it correctly. It will do it poorly, but it will do it.

6

MrTacobeans t1_j9xi9n6 wrote

If you are paying for the API something like "RWKV" might be an alternative hosted on a GPU Cloud provider. The model is currently only at 14B parameters but technically has "unlimited context" which in theory is probably not actually unlimited but for what I saw in your use case it might be worth looking into

3

Northcliff t1_j9xb3i8 wrote

Now THIS is schizoposting

−3