User1539 t1_j5gxrwt wrote on January 22, 2023 at 10:50 PM

Reply to comment by TinyBurbz in People are already working on a ChatGPT + Wolfram Alpha hybrid to create the ultimate AI assistant (things are moving pretty fast it seems) by lambolifeofficial

Honestly, what we need is something to translate between what an LLM can 'understand' needs to be done and the physical world.

Right now, we can ask an LLM what the process of, say, changing the oil in a car is.

We can also program an industrial robot to do that task, basically blind.

To automate jobs, we need an LLM style understanding of the task, and the steps required, coupled to the ability to take each of those steps and communicate it to a 'body', checking as it goes that the process is following correctly.

So, if an LLM could, say, break the problem into steps, taking into account the situation around it, it could probably do the job.

Imagine typing into Chat GPT a prompt like 'You are programming a robot arm. You need to pick up a glass. Write the code to pick up the glass in front of you. '

Then automatically send that to a camera/arm, and have the image processing describing back 'The arm is to the left of the glass by 2 inches', please program the arm to grab the glass.

'The glass has been knocked over to the left, and is now on its side, 4 inches in front of the hand. please program the arm to grab the glass'

Ultimately it would be more complicated than that, but I think that's the basic idea of what many researchers are working on moving forward.

With a feedback loop of video being able to 'describe' to the LLM what is happening, and the LLM adjusting to meet its task, you could have a very useful android.

TinyBurbz t1_j5hp5hl wrote on January 23, 2023 at 1:58 AM

>With a feedback loop of video being able to 'describe' to the LLM what is happening, and the LLM adjusting to meet its task, you could have a very useful android

Thats GAN/GameAI territory and is already out there. The algorithm is given an outcome like "win this match" or "pour water into this cup" and works out how to do so on it's own. It's how a lot of self-driving models work, and how OpenAI helped deliver AI that is indistinguishable from real players to Dota2 (they even rage if they cant follow their normal routine.)

What I foresee ultimately is tools we already used super-enhanced by AI. For example, a Wolfram-GPT macro for VisualStudio that generates the menial part of code; leaving the coder to figure-out harder logic themselves which the macro can then pick up on and offer complete code for.

Or, let's say someone is writing a story, but doesn't want to write out a full conversation between two characters, or perhaps they need help crafting a lore without also having to write a prequel.

While I know art AI's make beautiful renderings, to me, their potential is squandered on the lazy. Getting more into this, AI art could be so much more if used as a tool. It could do amazing things like generating real-world textures allowing every tree in a game to be unique. But as it stands people seem so much more interested in letting AI do the work for them, instead of letting AI enhance the work they already have done.

I know this sub has a hard one for letting AI do all this shit on it's own as if it's alive, but to me, that really stifles these tools. As it stands right now, AI is a viral app fad that will fade into the background to deliver nothing but ads and more disturbing YouTube Kids content. I know how badly people want self-aware machines, and mistake these tools for something living. Everyone arguing about letting AI have no limits is missing the point of what the creators of these tools want from them.

RabidHexley t1_j5lthcl wrote on January 23, 2023 at 10:31 PM

> While I know art AI's make beautiful renderings, to me, their potential is squandered on the lazy. Getting more into this, AI art could be so much more if used as a tool. It could do amazing things like generating real-world textures allowing every tree in a game to be unique. But as it stands people seem so much more interested in letting AI do the work for them, instead of letting AI enhance the work they already have done.

This is the main thing that sticks out to me about the AI art revolution, in terms of how it'll really change the game. People are laser-focused on the idea of AI creating bespoke art pieces in their entirety. But a lot of art; be it illustration, animation, game design, comics, etc. contains a lot of tedious, repetitive "art <space> work" that is only tangential to the artists' creative vision and could automated by tech like this.

Another example would be something like a comic book, manga, or animated series. Where the artist designs the world and art style, draws out the characters and their unique looks etc. But then is able to use AI to rapidly generate back-drops or background characters that fit into their specific style. Allowing them to focus on the more specific, key, creative segments of the work.

This could drop the cost and massively increase the accessibility for mediums that currently require numerous tedious hours to produce an incredibly small amount of content, or huge teams of creatives made to do grunt work.