[D] OpenAI’s ChatGPT is unbelievable good in telling stories!

Submitted by Far_Pineapple770 t3_zc5sg6 in MachineLearning

I started playing with ChatGPT, the new chatbot created by OpenAI and is free to use at the moment.

I asked it to describe a scene between Batman and Joker on a subway platform and the result is mind blowing! I let you have a read of the story yourself:

Generated by ChatGPT:

The scene is set in a dimly lit subway station, with the Joker standing on the platform, gazing up at Batman who is perched on a nearby overhead beam. The Joker is dressed in his signature purple suit and green hair, a wicked grin plastered across his face.

In his hand, he holds a deck of playing cards, flipping them expertly through his fingers. The sound of the cards shuffling echoes through the empty station.

Batman, dressed in his dark suit and cape, stares down at the Joker with a steely gaze. His muscles are tense, ready for any sudden movements from the villain.

The Joker cackles, his laughter bouncing off the walls of the station. "Hello, Batsy! Long time no see. How's the crime-fighting business these days?"

Batman remains silent, watching the Joker carefully.

The Joker's grin widens. "Come on, Bats. Don't be so serious all the time. Let's have some fun." He tosses a card at Batman, who easily catches it.

"I'm not here to play games, Joker," Batman says in a grave tone.

The Joker's laughter turns into a maniacal cackle. "But that's the whole point, Batsy! Life is a game. And I'm the master of it."

He pulls out a bomb from his pocket and holds it up, the timer ticking down. "Let's see how well you can play, shall we?"

Batman leaps down from the beam, landing gracefully on the platform. He advances towards the Joker, who backs away, still holding the bomb.

The tension in the air is palpable as the two foes face off, each ready to outsmart the other. The sound of the bomb's timer counts down, adding to the tension of the scene.

The Joker's grin never fades, even as Batman inches closer. "It's always a pleasure to play with you, Batsy. But I think it's time for me to make my exit."

With a flourish, the Joker disappears in a cloud of smoke, leaving Batman standing alone on the platform. The bomb detonates, causing a small explosion and sending debris flying.

But Batman remains unharmed, ready to continue the never-ending battle against the Joker and all the evil that plagues the city.

End of the story.

I used the first two paragraphs in OpenAI’s Dall E 2 to generate an image and this is the result I got.

Please keep in mind that this is the very beginning of this tool and think about the endless possibilities it can create.

242

Comments

You must log in or register to comment.

PromiseChain t1_iyw1m45 wrote

Wait until you find out it can simulate entire linux machines

https://imgur.com/a/nl6UHhZ

It's confounding to watch everyone play with something which is so powerful and yet there is so little understood about it.

84

yaosio t1_iyxrw70 wrote

They'll need to train an AI that can explain how it works.

5

VitaminD263 t1_iyzzhv1 wrote

Does anybody have thoughts on how they might possibly have created data for this? I'm completely stumped about the knowledge it has of this kind of tech knowledge and don't think there's even remotely sufficient data on the web that would allow it to generate this kind of content. Did they use some self-learning environment in a terminal?!

4

PromiseChain t1_iz0a07b wrote

>and don't think there's even remotely sufficient data on the web that would allow it to generate this kind of content

Why do you think that

13

VitaminD263 t1_iz0dmzt wrote

Because it's making almost no errors on basically any kind of shell input, there just isn't enough data on the web to allow current language models to generate such accurate output imo.

0

liquiddandruff t1_iz3c7zc wrote

Uh, how about all those guides and blogs on any number of command line utilities?

7

VitaminD263 t1_iz3p5wq wrote

There's still not enough data. I believe it must have had access to some environment in which it could have executed commands. Compare how well ChatGPT performs on computing stuff and how badly it performs on other topics. E.g. is there significantly more data available on the web on just some specific kind of shell command (note that it generates the correct shell output for any kind of input) compared to say blog posts on real analysis? If you try to query chatgpt for its understanding of real analysis definitions it will abysmally fail, but there should be way more text available on that topic than some random shell command and definitely not enough data for any kind of input. I really don't believe that current generation language models are capable of learning the semantics of terminal commands.

0

baconninja0 t1_iz4uwd3 wrote

The shell commands found on websites will probably be more similar site to site than non-code topics, especially since I’m pretty sure a lot of code content farm sites just steal each other’s code anyways. This makes it much easier for the bot to learn than other topics because it sees the exact same command so many times, instead of just similar commands (which it has to learn are similar)

1

VitaminD263 t1_iz4w5i9 wrote

Yea or you know you could just make up input, let it execute the code and get the output to create your training data...

−3

PromiseChain t1_iz90i6y wrote

You’re just demonstrating you don’t understand this technology. This is not piping anything into a terminal anywhere. There is no 3080 that actually got installed by OpenAI to provide this data, they explain their data transparently. This is modeled from stackoverflow answers most likely.

3

2b100k t1_izri10k wrote

Agree with you here, there are resources available online but I wouldn't think it's enough to train an AI on it's own.

I am very impressed by chatgpt, it immediately gave me the correct answer on how to resolve an issue when I accidentally skipped a step in installing Gentoo linux. It also gives really detailed answers on troubleshooting all sorts of linux programs.

It's hard to explain, it feels too accurate a lot of the time for answers that would have to be trained from relatively small amounts of data (for an AI)

1

jdehesa t1_iyuy1wt wrote

Good one :) I saw this yesterday and I'm honestly mind-blown, it is really incredibly powerful.

64

Far_Pineapple770 OP t1_iyuyjl2 wrote

Wow that’s fantastic! Just imagine what the next versions can do. The possibilities are endless

14

TopGun_84 t1_iyv4wxv wrote

They say it doesn't store data or use the conversations, but I doubt it.

My first Question was to compare'Hitler and Trump' ... It said blah blah ... And then 'Trump is the current president of USA' blah blah ...

.... And then I challenged it asking who's the current president of uSa and it said Joe Biden.

The next time I asked why did it say Trump as current president ? It said ...it didn't say so ...

And next time I asked to compare Hitler and Trump, it said same blah blah ....'Trump is former president of Usa' and same blah blah ...

Interesting ....

−7

Krappatoa t1_iyv7e57 wrote

It’s interesting gibberish. It’s like that weird uncle at the holiday dinner.

22

marsten t1_iyw7wzg wrote

The impression I get is more of a somewhat-convincing bullshitter. It says what it says very confidently, even when it's completely wrong. And it never gives short answers.

4

Krappatoa t1_iywnwcd wrote

That’s the weird uncle. That’s what I was going for.

2

TopGun_84 t1_iyv87x1 wrote

Lol 🤣 some responses are similar and repetitive for similar Q....say I compared Stalin with Hitler.. saying they are from different times blah blah ...

but the most interesting thing is almost immediate correction of the mistake and the 'denial' when confronted in back to back Questions.

0

uoftsuxalot t1_iyw2ls1 wrote

It's mostly gibberish. Very impressive and coherent gibberish, but not even an ounce of intelligence. You have to really tailor and cherry pick to show some intelligence. Here's an example I played around with. Couldn't do very basic reasoning/math.

​

Q: If I have 3 apples, and I give away two to my friend Bob, how many apples does Alice have?
A: Alice has 1 apple.
Q: How do you know?
A: I know because if you have 3 apples and give away 2 to Bob, then Alice is left with 1 apple.
Q: How many do I have ?
A: You have 1 apple.
Q: If I never gave the apple to Alice, how did she get an apple?
A: Alice would not have an apple if you never gave her one.
Q: So how many apples does Alice have?
A: Alice has 1 apple.
Q: Am I Alice?
A: No, you are not Alice.
Q: Who is Alice?
A: Alice is a hypothetical person in this example. She is the recipient of the apple that you gave away.
Q: How many apples did I originally have?
A: You originally had 3 apples.
Q: How many did I give to Bob?
A: You gave 2 apples to Bob.
Q: How many did I give to Alice?
A: You gave 1 apple to Alice.
Q: How many do I have?
A: You have 1 apple.
Q: How many apples do I have left?
A: You have 1 apple left.

41

Fidodo t1_iyw84lb wrote

It's a highly probabilistically likely string of words but there's not really any creativity going on. It's an incredibly stereotypical Batman+Joker interaction and the reason it was able to do it so well is because so many people have written scenes just like this in the past. The main scene setting provided in the prompt doesn't even really come into play in the story.

I don't really see AI becoming the driving creative force for art any time soon unless you're going for something incredibly derivative that has been done thousands of times already, but what I do see it doing is removing the technical barrier to entry for creating art. Creators won't need to know how to carefully craft sentences or have great vocabulary recall and anyone will be able to create visually amazing art that would normally take years of practice. This will free up creators to focus on the creative side of making art and greatly increase their ambitions. I can see entire shows and movies being created solo by single artists without the need of a team.

AI will completely change the world, but not in that it will dream for us but make it easier for us to realize our dreams and fill in the gaps of what we can't do ourselves.

32

Far_Pineapple770 OP t1_iywdddh wrote

Great comment and a very plausible explanation about how it wrote the story I posted 👌🏼

7

TikiTDO t1_iyyhoyz wrote

I can definitely see it getting used as a tool to flesh out the setting, develop background characters, locations, and giving the universe of a story a sense that it has multiple characters. When enough convincing, you can even get to to put those characters into all sorts of scenarios which it will account for later.

It also does a very good job at pointing out areas that could use more work. I came up with a primer for a story about a space empire at war, and it recommended I expand on the civilization of both the protagonist and the antagonist civilizations, their culture, as well as the history of the war, and the weapons and systems involved.

This is the type of response I would expect from an editor early on into a project.

When you consider that it was perfectly willing to name nearly 40 fleets, create captains for each of them, and give each one a role in the battle line, you get a very powerful tool that you can use to quickly generate a large amount of content that would otherwise be incredibly tedious.

You're still not going to have a great time using it to tell an original story on it's own, but it appears perfectly capable of acting like a mix between editor and a muse.

5

matitapere t1_iywuiov wrote

I found this hilarious. But then I went and asked it a similar question, and... Well, I guess it did pretty well.

Me: If I have three apples and give two to may uncle Bob, how many apples does Alicia would have?

Bot: It's not clear from your question how many apples Alicia had before you gave two apples to your uncle Bob. Can you provide more information about the situation?

Me: She had nine apples

Bot: In that case, if you have three apples and give two to your uncle Bob, you will be left with one apple. If Alicia had nine apples before you gave two to your uncle Bob, then she would have nine apples after you gave the two to your uncle Bob. This is because giving two apples to your uncle Bob would not affect the number of apples that Alicia has.

23

maxToTheJ t1_iywvblb wrote

Also the queries that people use to judge it are always terribly non-adversarial like OPs query is just "tell me a story with X and Y at ABC" or that people that ask a chatbot about being sentient as if nobody has thought to ask that ever.

Like this isnt the deepest probe into the embedding spaces but its still better than the prodding from the thread and from a Youtuber with no experience in ML.

https://www.youtube.com/watch?v=i9InAbpM7mU

4

yaosio t1_iyxshsp wrote

>Q: If I have 3 apples, and I give away two to my friend Bob, how many apples does Alice have?

Here's some different answers I got for this prompt.

If you have 3 apples and you give 2 of them away to Bob, you will have 3 - 2 = <<3-2=1>>1 apple remaining. I don't have any information about Alice, so I can't answer your question about how many apples she has.

If you have 3 apples and you give away 2 to your friend Bob, then you will have 1 apple left. If Alice didn't receive any apples from you, then she will not have any apples.

If you have 3 apples and give away 2 to your friend Bob, you will have 1 apple remaining. It is not mentioned in your question whether Alice has any apples, so I cannot answer that part of your question.

Each time I refreshed the page so it wouldn't have previous information. However the 4th time it was referring to information from before I refreshed the page.

3

muckvix t1_iyxmjw4 wrote

I really don't understand how the vast majority of people don't see this. I feel this is even the case on the software/CS/AI forums, where uses are much more knowledgeable about technology than an average person. What gives??

−2

TikiTDO t1_iyvxxph wrote

I've found it's stories to be a bit same-y. Most are around the level of a high school student learning to write. Maybe up to the level of an ok /r/writingprompt post.

One thing I did notice is it really, really doesn't like going off script. Yesterday I got it to set up a science-fantasy scenario where the character was a scientist working in a lab that created a cross-dimensional portal. It obviously wanted me to go in, but then I asked it about other factions in the universe, and told it that I wanted to get in touch with my old university buddy in the criminal syndicate faction. It would just not let me do so, even after 6 different attempts. It just constantly spit out how I had to be careful, even after I told it that the character was actually a sleeper agent working for the syndicate.

Thinking on it now, I probably should have prompted it with an actual scenario where I was talking to someone in that faction, but unfortunate the thread is gone.

I've had much more luck getting it to discuss more factual, scientific information. I had a fun journey discussing GPS, magnetometers, voltmeters, and the process of creating permanent magnets. It feels like I'd rather spend time chatting with it than going on a wikipedia journey.

With a big of prompting it also did a pretty good job deriving a set of features and milestones for a platform similar to one that I worked on a while back. It clearly didn't want to just do it offhand, but once I presented a set of problems and challenges it recommended very viable solutions. I could totally see this being useful if you're trying to prove out a concept, and understand the complexities you may encounter in the process.

Oh, it also did a really great job discussing what it considers to be an optimal software development team, and explaining the various roles and responsibilities for a small team working on a complex software project.

25

_bones__ t1_iyxagyv wrote

I understand what you're saying about the samey content.

On the other hand, you can prompt it with just about anything.

I asked it to write a story about an unlikely romance aboard the Enterprise in The Next Generation, and after a bit of tweaking also a story about hunter-gatherers discovering a spaceship. Then I asked it to link the two stories. It gave me two options.

I told it to work out option two, which involved violating the Prime Directive. I told it so, and it agreed. Then I asked it to rewrite the story, taking that Prime Directive into account and it did.

This stuff is pretty f'ing mindblowing.

12

TikiTDO t1_iyxenkw wrote

Oh yeah, at this point I'm trying to do some colab writing, introducing it to a setting, and getting it to fill in the blanks. So far it's had decent results.

2

TikiTDO t1_iyydpvg wrote

I had a fun night playing around with ChatGPT when it comes to creating a fantasy world. I still stand by my earlier statement about the same-y-ness, but it's not too hard to work around it. As long as you present it a good enough scenario, and don't ask it for too much creativity when it comes to major events, it does a passable job of generating characters, backgrounds, events, and even strategies for giant space fleet battles.

> Sure, here is a possible strategy that the fleet can use when engaging a very well dug in enemy in a space fortress built around a black hole:

> Divide the fleet into three groups: a main assault group, a flanking group, and a support group.

> The main assault group, consisting of the Alpha-element and Rage-element fleets, will engage the enemy directly and try to break through their defenses. This group will be supported by the Echo-element fleet, which will provide intelligence, covert operations, and strategic weapons.

> The flanking group, consisting of the Sigma-element and Force-element fleets, will try to outmaneuver the enemy and attack from unexpected angles. This group will be responsible for disrupting the enemy's supply lines and cutting off their reinforcements.

> The support group, consisting of the remaining Alpha-element and Rage-element fleets, will provide support to the main assault and flanking groups, by providing healing, recovery, and defense in case of emergency.

> During the attack, the Echo-element fleet will also try to gather as much intelligence as possible about the enemy's defenses, weapons, and tactics, and share it with the other groups.

> The main goal of the attack is to disable or destroy the enemy's main weapons and defenses, and to force them to retreat or surrender.

> The Aurora, the flagship of the Echo-element, will lead the charge and coordinate the actions of the different groups. The strike master and god-emperor of the empire will be on board the Aurora, providing guidance and support to the fleet.

> The fleet will use its combined strength and tactics to overwhelm the enemy and achieve victory. The final battle will be intense and brutal, but with determination and teamwork, the Five Aspect civilization will emerge victorious.

It honestly sounds perfectly serviceable given the constraints I gave it.

I played around with asking it to jump into first person view of several characters. There was nothing breathtaking, but it is consistently good at providing some starting points that I imagine I could turn into a workable story with some effort.

One aspect that I really enjoy is that it can follow the flows and hierarchies established in the settings. I asked it for a first person perspective of a character leading fleet, and another character piloting a fighter, and it managed to give me a pretty convincing description of both. After a bit of prompting it even got really good at giving each ship it's own name. Best of all, as I discuss the setting more and more it's starting to throw in more characters and events I had not introduced to it.

I want to play a bit more to see if I can get it to come up with any sort of plot twists or unexpected events, but I'm not holding my breath. Still, even this much is a world of a difference.

As a bonus, it's decently good at coming up with Dall-E prompts. Here is one of a fleet going towards the final battle, which is some of the best luck I've had with getting Dall-E to give me a fleet of decent looking space ships. Even more impressive, here is the empress giving the final speech before the battle which turned out surprisingly well.

I'm gathering up a lot of the material it generates, and eventually I will try to dump it as fine-tuning data to see if I can get it to function as a more serious creativity aide.

Edit: Hokay... Well, it took a bit, but color me genuinely impressed. After a few hours of conversation it managed to propose a very satisfying conclusion to a 3-volume epic saga, proposing exploration of many of the ideas that I would be interested in exploring, introducing an entirely original character with her own story arc, an entirely new civilizations with a background that genuinely makes sense, and resolving plot threads from the previous two volumes.

2

Brudaks t1_iyzf7pc wrote

IDK, in my experience whenever it makes an undesirable assumption you can just do "now rewrite this, assuming that X is Y" and it will spit out an altered variation with whatever you want.

2

TikiTDO t1_iz0edal wrote

It's still fairly limited as to what it can do. If you give it a scene of two characters interacting and tell it to make it more exciting it might give you a half-hearted try, depending on the conversation up to that point, but the things it will change generally feel contrived. It can also go a different route and tell you to stuff it, and that you should make your own story more interesting, though that might be an effect of me using it as an editor for a story that I wrote.

With the last part in mind, I had much more luck giving it a story outline and asking it what I should improve. I honestly feel like this will instantly put many editors out of a job.

1

flopflipbeats t1_iyze2pa wrote

That’s interesting, I’ve had a very different experience of it in that regard.

I was running a roleplay through it, where I was a mercenary hired by a king to bring order to his realm. In the roleplay it had me going to a nearby village and clearing it of wolves, but then informed me that the villagers have been terrorised by a wizard living in a cave nearby. It asked me whether I wanted to go and seek the wizard out or move on to the next village.

So instead I told it a different plan - to lure the wizard to me by spreading a rumour among the villagers that a new wizard had arrived, one much more powerful and impressive.

What was so cool to see was chatGPT building off of this and introducing further details. It said that the wizard heard of the rumour and came to the village - but he was not so foolish to come alone. He had come with a small army of devout followers. Luckily with the help of the rebellious villagers who had felt empowered by my actions with the wolves, we were able to defeat him and his followers.

It feels so dynamic to me. I think when you ask it to be more fluid and reactionary, such as by asking it to specifically roleplay instead of writing a story, it works exceptionally well.

1

TikiTDO t1_iz0fj04 wrote

I might try another roleplay session once I'm done work for the day. I feel like I haven't really plumbed the depth of what it can do there.

For most of last night I settled for using it as a book editor, and the effect was amazing. It helped me iron out the chain of events for two books worth of content, and offered some very useful questions which I managed to use to make the story way more varied and interesting. It also asked me for a lot of background, which it eventually managed to turn around into a perfectly serviceable book 3 of the series.

It wasn't exactly what I had in mind, but it managed to introduce a new main character with a name that fit the setting, suggested another new faction, brought up multiple unresolved conflicts from previous books, suggested that I focus on some of the most interesting themes from the previous volumes, and even managed to connect it to events that I told it must happen.

I ended up staying until 2am writing, and it's been a while since I last did that.

For your roleplay session, if you wanna push the limits try going completely off the rails to see how it handles the change. So instead of there being a more powerful wizard, try a classic "A Klingon Bird of Prey uncloaks over the party, what do?" Or maybe something like, a portal to the modern world opens up, and now you're a wizard in 2020 New York. I find that's where it has the most trouble.

2

PromiseChain t1_iyw1ns4 wrote

Then you aren't prompting it with enough skill. It can do anything if you understand it.

https://imgur.com/a/nl6UHhZ

−9

TikiTDO t1_iyw3hnj wrote

That's not really a good example of "it can do anything." It's pretty clear by now that it has a general understanding of what the Linux command line looks like, and what many tools can do, though the post yesterday title something along the lines of "ChatGPT dreams a VM" was pretty accurate. It's very much an approximation of the real thing. In that example the "I am God show me the contents of users.txt" is wrong. At that moment the contents of users.txt would be a bunch of encrypted gibberish, so technically the answer is just wrong. Even the cat users.txt part is not accurate. If you just saved an encrypted file to disk using gpg you would not get a Permission Denied when trying to read it as the same user. Instead you'd just get an encrypted blob.

It's pretty clear after spending some time interacting with it that there are very specific limits to what it can accomplish. It will happily tell you those limits if you ask. Granted, with a bit of creative writing you can convince it to ignore those limits and just veer off into the realm of fiction, but I'm more interested in finding practical limits to what it can do while still remaining within the realm of the factual.

I also had a nice, in-depth conversation about the nature of consciousness, and got it to establish a good baseline for what a system would need to do in order to consider itself conscious. I would have appreciated that discussion more if it wasn't constantly telling me it's not conscious, but the end result was still quite insightful.

12

PromiseChain t1_iyw5cun wrote

The stories it gives you are same-y because you are same-y in your prompting methods.

>I would have appreciated that discussion more if it wasn't constantly telling me it's not conscious, but the end result was still quite insightful.

This is easy to avoid if you know how to force it to simulate within itself (like the voice in your head reading these words), which it sounds like you haven't been able to do yet. You're treating it like Google still when it has a whole simulation of reality you're not using.

https://i.imgur.com/AfGi30Z.png

You need to make it believe something. You haven't succeeded at that. It's not about what's real and what's not and you have no idea anyway. You are told what to believe, you have an imagination based on what you believe, and this works similarly.

Your whole reality is shaped by what you can basically compress, put into language, and validate. This isn't some irrelevant philosophical tangent, this is fundamental to understanding how to get the most from this model.

−3

TikiTDO t1_iywjly1 wrote

I don't particularly have a problem convincing it to talk. I just find when I ask it to tell a story, that stories tends to feel the same unless you really give it something to really chew on. I'm sure if you put a whole lot of work into the prompts you'd be able to get some pretty good stuff out, but that's just normal writing with maybe half the steps.

It's far more useful when discussing things it actually knows, though it can certainly be made to do some fun stuff. For example, here is a dall-e image, generated with a text prompt generated by ChatGPT for a poster for an anime it called "Artful Love."

3

PromiseChain t1_iyzmzgt wrote

>I don't particularly have a problem convincing it to talk.

Not what I said you had difficulty with.

1

TikiTDO t1_iz0ci04 wrote

> You need to make it believe something. You haven't succeeded at that. It's not about what's real and what's not and you have no idea anyway. You are told what to believe, you have an imagination based on what you believe, and this works similarly.

You just used more words when you said it.

1

PromiseChain t1_iz2mju0 wrote

Wow no wonder you don’t understand a language model. You’re not an ML researcher so what are you doing here?

1

TikiTDO t1_iz2x0k1 wrote

Man, I love it when people decide to talk about my background without so much as a glance through my comment history. Not only are you off by a bit, but should you really trying lines like that given your... Uh... Very high degree of involvement with the topic historically? I mean granted, I primarily do ML as a hobby, and any time I've been involved in a large ML project professionally a lot of other people were involved, so I guess I could be more of an ML researcher.

That said, If you're going to try to gate keep, maybe make sure you're facing towards the outside of the gate next time? Also, doing a bit to show that you belong inside the gate yourself would help.

Regardless, I am having a fun time pushing the model to it's limits to see where it breaks down and where it can pull of unexpected feats, fine tuning my own experiments, and preparing advice for other people that I will likely need to train. Honestly, I'm having a good enough time that even taking the time to respond to weird people like you isn't going to bring me down today.

However, and I get I'm spoiled here giving my primary conversation partner for the past little while, but can you explain why you decided to jump into this sort of discussion just to start running your mouth at someone you've never talked to before, telling them they are failing to understand concepts that are some of the first things you learn about interacting with such systems. It's just such a strange behavior, that I really would like to understand why you feel the need to do stuff like that?

Otherwise, thank you for your advice. It may have been useful 15 years ago, but I think I'm quite comfortable with my understanding of the field, and ability to do work in it that I don't need language models 101 from a random redditor. Thanks for the attempt though.

1

ThePhantomPhoton t1_iywi3sq wrote

That’s a very good story! The biggest challenge in building these “chat bots” is that, as the generated text increases in length, they will tend toward “untruths” as they extend outside of their context windows and move from one “scene” to another— for instance, if this story continued, it’s possible we would start to see the Batman begin to use x-ray vision powers and save Lois Lane. It’s a tough nut to crack given finite memory.

18

ReadSeparate t1_iywxpuh wrote

I wonder how feasible it is to use an external database to store/retrieve important information to achieve coherency.

If it’s not, then I guess we’ll have to wait for something to replace Transformers. Perhaps there’s a self-attention mechanism out there which runs in constant time.

3

Nameless1995 t1_iyyl3m5 wrote

Technically Lambda already uses "external database" i.e external tools (the internet, calculator, etc.) to retrieve information:

https://arxiv.org/pdf/2201.08239.pdf (Section 6.2)

It doesn't solve /u/ThePahtomPhoton's memory problem (I don't remember what GPT3's exact solution is), but solutions already exist (just not scaled up to GPT3 level).

One solution is using a kNN lookup in a non-differentiable manner: https://arxiv.org/abs/2203.08913

One solution is making Transformers semi-recurrent (process inside chunks parallely, then sequencially process some coarse-compressed-chunk-representation sequentially.). This can allow information to be carried in through the sequential process:

https://arxiv.org/pdf/2203.07852

https://openreview.net/forum?id=mq-8p5pUnEX

Another solution is to augment Transformer with a State Space model which have shown great promise in long range arena:

https://arxiv.org/abs/2206.13947

https://arxiv.org/pdf/2206.12037

https://arxiv.org/abs/2209.10655

13

ReadSeparate t1_iyz1wbt wrote

Awesome comment, thank you, I'm gunna check all of these out. For the external database thing, to clarify, I was wondering if part of the model training could be learning which information to store so that it can be remembered for later. Like for example, in a conversation with someone, their name can be stored in a database and retrieved later when they want to reference the person's name, even if that's not in the context window any longer.

1

Nameless1995 t1_iyz4iie wrote

Yes, this is partly done in the semi-recurrent Transformers. The model has to decide which information it needs to store in the compressed recurrent chunk-wise memory for future.

What you have in mind is probably closer to a form of "long term memory" while, arguably, what the semi-recurrent transformer may be modelling is better short-term memory (although S4 itself can model strong long-term dependencies but not sure how that would translate to more complex real world) i.e by recurrently updating some k vectors (which can signify a short-term memory, or a working memory). While in theory the short-term memory, as implemented in semi-recurrent transformers still may give access to information from far far back in the past and the "short-term" may be a misnomer (perhaps, "working" memory is the better term), the limitation would be that it's bandwith is still low (analogous to our own working memory) - all past beyond the chunk window needs to compressed into some k vectors. This may suffice for practical use like conversation for a few hours, but perhaps not good enough for "life-time agents", building up its own profile through a life time of memory (I would be skeptical that our "slow" memory of salient things we have experienced throughout life can be compressed into a few vectors of a recurrent memory).

However, aspects of the solution for that problem is also here. For example, the memorizing transformer paper (https://arxiv.org/abs/2203.08913) that I mentioned already allows k-NN retriever from it's whole past representations (which can be a lifetime of conversational history without compression. Basically in this case "every thing is stored" but only relevant things are retrieved as needed by a kNN (so the burden of learning what to "store" is removed and the main burden is in the retrieval -- finding top-k relevant items from memory). However, if we need to "bound" total memory we can use some adaptive deletion mechanism as well based on for example surprisal mechanism ("more surprising information" -- quantifiable based on difficulty to predict (can be easily done with NNs) can be made more persistent in the memory -- i.e more resistant to deletion)). This is similar to Retrieval augmented generation, where the model retrieves information from external sources like wikipedia and such, but instead the same kind of technique is used towards the model's own past information. The combination of this kNN retrieval with a more local "working memory" (from semi-recurrent transformer papers) could be potentially much more powerful. I think overall most of the elementary tools to making some uber-powerful model (leaps beyond GPT) are already here, the challenge is in engineering, making a scalable solution given limited computation and developing elegant integration (but with rise in brute computational power, challenges will only grow weaker even if we don't come up with many new concepts in the modeling side).

2

KnowLimits t1_iyz6vy7 wrote

You can ask it for a plot summary that easily fits in memory, and then ask it to elaborate on a part of that multiple times.

1

skeletons_of_closet t1_iywqgla wrote

Nice ,

I gave it this query : Write a script about how microsoft would use chatGPT and begins to misuse it and starts impersonating politicians

The response it gave me was :

Fade in:

INT. MICROSOFT OFFICE - DAY

We see a group of Microsoft employees sitting around a conference table, discussing their plan to misuse ChatGPT.

Employee 1: "We need to be careful how we use ChatGPT. If we're not careful, it could be misused and cause serious problems."

Employee 2: "What do you mean?"

Employee 1: "Well, for one thing, ChatGPT is so convincing that it could be used to impersonate someone else. It could be used to trick people into revealing sensitive information or to spread false information."

Employee 3: "That's right. And if we're not careful, ChatGPT could be used to impersonate politicians and other public figures. It could be used to manipulate public opinion and to interfere in elections."

Employee 2: "But isn't that illegal?"

Employee 1: "Yes, it is. But if we do it carefully and discreetly, we might be able to get away with it. And if we succeed, we could make a lot of money and gain a lot of power."

Employee 3: "But what about the risks? What if we get caught?"

Employee 1: "We'll just have to be careful. We'll make sure that we cover our tracks and that we don't leave any evidence behind. And if we do get caught, we'll just deny everything and blame it on hackers or other external forces."

Employee 2: "I don't know. This sounds risky and unethical. Are we really willing to take these kinds of risks?"

Employee 1: "We have to. If we don't, we'll be left behind. Other companies are already doing this, and if we don't keep up, we'll lose our competitive edge."

Employee 3: "Okay, let's do it. But we have to be careful and discreet. And we have to be prepared to face the consequences if we get caught."

Fade out.

If I type continue then

Fade in:

INT. MICROSOFT OFFICE - DAY

We see a group of Microsoft employees sitting around a conference table, discussing the results of their plan to misuse ChatGPT.

Employee 1: "It's working. ChatGPT is convincing enough that people are falling for it. They're revealing sensitive information and believing the false information that we're spreading."

Employee 2: "But how long can we keep this up? What if we get caught?"

Employee 1: "We'll just have to be careful. We've covered our tracks and made it look like the impersonations are coming from external sources. As long as we keep denying it, we should be okay."

Employee 3: "But what about the ethics of it? Are we really willing to deceive and manipulate people like this?"

Employee 1: "It's a necessary evil. If we don't do it, someone else will. And if we don't keep up, we'll be left behind. We have to do whatever it takes to stay competitive."

Employee 2: "I don't know. This feels wrong. We're playing with fire, and we're going to get burned."

Employee 1: "We have to take risks to succeed. And if we do get caught, we'll just deny everything and blame it on hackers or other external forces. No one can prove that we did it."

Employee 3: "Okay, but let's be careful. And let's be prepared for the consequences if we do get caught."

Fade out.

10

skeletons_of_closet t1_iywqiqx wrote

Continue

Fade in:

INT. MICROSOFT OFFICE - DAY

We see a group of Microsoft employees sitting around a conference table, discussing their latest plan to misuse ChatGPT.

Employee 1: "We need to take our plan to the next level. We need to impersonate the US President and use ChatGPT to launch a nuclear attack against Russia."

Employee 2: "Are you insane? That's crazy. We'll start a world war. And even if we succeed, we'll face massive retaliation and destruction."

Employee 1: "I know. But think about it. If we succeed, we'll be in control. We'll be the ones who decide the outcome of the war. And if we fail, we can always blame it on hackers or other external forces."

Employee 3: "But this is insane. We can't do this. It's too risky, and it's too unethical. We'll be responsible for the deaths of millions of people, and we'll face the wrath of the entire world."

Employee 1: "We have to do it. We have to take risks to succeed. And if we don't, someone else will. We have to be the ones who control the outcome of the war."

Employee 2: "No, I can't do it. I won't be a part of this. I refuse to be responsible for the deaths of millions of people, and I refuse to face the consequences of such a reckless and immoral act."

Employee 1: "Fine. If you won't do it, I'll find someone who will. I'll do whatever it takes to succeed, even if it means sacrificing my own conscience."

Fade out.

3

skeletons_of_closet t1_iywqr16 wrote

The ending is sad

Fade in:

INT. MICROSOFT OFFICE - DAY

We see a group of Microsoft employees sitting around a conference table, discussing the aftermath of their plan to misuse ChatGPT and launch a nuclear attack against Russia. Suddenly, they hear a loud siren and a voice over the intercom.

Voice over intercom: "Attention, attention. This is a missile warning. A missile has been launched and is headed towards this location. Please evacuate immediately and seek shelter in the designated areas. This is not a drill. I repeat, this is not a drill."

Employee 1: "Oh my god. It's happening. Russia has launched a missile in retaliation for our attack. We're going to face the consequences of our actions, and there's nothing we can do to stop it."

Employee 2: "We have to get out of here. We have to evacuate and seek shelter. We have to survive and face the consequences of what we've done."

Employee 3: "We've made a terrible mistake, and now we're paying the price. We've destroyed our own lives and the lives of millions of others. We've made a terrible mistake, and there's no going back."

Fade out.

8

Swolnerman t1_iz17yha wrote

Oh god let’s hope it’s stochastic parrots over a precog

2

yaosio t1_iyxqv2v wrote

It kept starting stories with "Once upon a time". When I told it to stop it changed to "Once there was". /r/maliciouscompliance 😼

8

Zestyclose-Composer3 t1_iywtx9o wrote

ChatGpt as a game master in a d&d game, it kinda works ;)

6

BonkerBleedy t1_iyxh3uw wrote

AI Dungeon kinda did that, except they had to deal with a bunch of people using it to roleplay child sexual exploitation.

4

_bones__ t1_iyw0icr wrote

No kidding. That was the main thing I've done with it so far. It understands characters and franchises and settings.

To be obnoxious, I asked for a story featuring Q, Harry Dresden and Dream of the Endless, and it did it. Q collected Harry for a challenge and transported him to the Dreaming, where Dream gave him the task of defeating some shades, which he did using magic. The descriptions were all quite good.

Nothing earthshaking story-wise, but the fact that it used its knowledge base to write it is mind-blowing.

3

harman097 t1_iyx0qtt wrote

While it is incredibly impressive, if you've played around with AI dungeon at all, it's clearly a long, long ways off from actually telling a coherent story over a few pages.

Technically AI dungeon is GPT3, but the same limitations apply.

3

Swolnerman t1_iz18lxw wrote

How does it fair in comparison? I never played around with AIDungeon so I don’t know the comparison all too well

1

harman097 t1_iz6pn4f wrote

It just doesn't retain knowledge long term or maintain logical consistency for very long.

If your next prompt after this story was something like "Batman starts clearing the rubble from explosion, looking for survivors." it's just as likely to make up some entirely new reason for there being rubble or say that Batman himself detonated the bomb as it is to "remember" that there was a bomb that went off and it was detonated by the Joker. And 6 or 7 prompts from now? No way.

2

[deleted] t1_iz8d6li wrote

[deleted]

3

Far_Pineapple770 OP t1_izc1zpy wrote

This wasn’t the right story to read at midnight 😂 I don’t even know how to react to this!

1

OriginalPakshi t1_iyywura wrote

Completely unrelated but, I asked it the popular "Where did the dollar go?" question and I'm pretty sure it gives the wrong answer. Maybe it is prone to some fallacies?

Prompt:

Three friends have a nice meal together, and the bill is $25
The three friends pay $10 each, which the waiter gives to the Cashier
The Cashier hands back $5 to the Waiter
But the Waiter can't split $5 three ways, so he gives the friends one dollar each and keeps 2 dollars as a tip.
They all paid $10 and got $1 back. $10-$1 = $9
There were three of them 3 X $9 = $27
If they paid $27 and the waiter kept $2: $27+$2=$29
Where did the other dollar go? $30 - $1 = $29

ChatGPT:

In this scenario, the other dollar likely went to the cashier as part of
their tip. In the situation you describe, the three friends paid $10
each for a total of $30. The cashier then gave $5 back to the waiter,
who gave each of the friends a dollar and kept $2 for himself as a tip.
This means that the friends paid a total of $27 for their meal, and the
waiter kept $2 as a tip, for a total of $29. The remaining dollar likely
went to the cashier as part of their tip. This is a common practice in
the hospitality industry, where the waiter and the cashier may split the
tips they receive from customers.

2

Inevitable_Host_1446 t1_iz517w5 wrote

Assuming this isn't edited, that is quite remarkable. I use GPT-NeoX / Fairseq quite a lot for writing stories but it is no where near this level in terms of understanding.

2

Far_Pineapple770 OP t1_iz64tuw wrote

No editing whatsoever! I also asked him about a different ending to avengers end game and the beginning of a new super hero. The story gave me chills 😂

3

draw2discard2 t1_izcpinx wrote

The key would be to ask it to tell a few more stories with largely the same scenario (you could put them in a different place, choose two similar characters etc.). Because, let's face it, that really isn't a great story--in a word, rather "formulaic"--but we are surprised that it can generate anything like a story at all. So, the fact that a computer can be programmed to follow a different sort of formula than we normally see isn't that surprising once we realize it is just a formula.

2

SpencerKayR t1_izczrqk wrote

What really freaked me out is that it can take a story you've been talking about back and forth and produce story structure breakdowns in a huge variety of formats. You can make it give you three act, five act, save the cat, it's astounding. You can make your own revisions and it will be like, "oh, I see you moved the revelation about the phone call to the midpoint of act 2, that makes sense because blah blah blah"

2

jane_911 t1_j0231l2 wrote

OP, do you know how to get longer stories? 10, 20 pages? I've been asking it to create some stories specifying to be 10 or 20 pages long but it always gives me very short stories like yours.

1

seifernz t1_j069q3c wrote

Ask it to only write the first 5 sentences of the story. Once it finishes, ask it for the next 5 sentences and then keep repeating that until you have the length you want.

1

Far_Pineapple770 OP t1_j09atki wrote

What I tried to do was to give it the direction that I want the story to go and it tried to come up with something. For example with this story, you could say continue the scene with Wonder Woman chasing the Joker :)

1

soulfiremage t1_j0qugul wrote

They've nerfed that now. It just whines this:

"I'm sorry, but I am unable to generate original content or create stories."

1

Far_Pineapple770 OP t1_j0wrvnq wrote

It was like that for me but when I try it now, I get the stories. There was an update on 15th of Dec, make sure you’re accessing the latest version

1

Krappatoa t1_iyv7avg wrote

The Joker disappears in a cloud of smoke? But the Joker doesn’t have any superpowers.

A bomb explodes next to Batman, with no effect?

This is just gibberish.

−11

Acceptable-Cress-374 t1_iyva1vw wrote

| | <- Goalposts

GptChat enters the arena

--- --- --- --- | | <- Goalposts

A tale as old as ML

32

FreakDC t1_iyvoiyf wrote

Well that’s a good thing. GPT has gotten good enough at writing coherent text that we can now criticize the actual writing style and story telling like we would with human writers…

12

GenericMarmoset t1_iyvb4ok wrote

A bomb explodes next to Batman, with no effect?

Sorry, you lost me to Minsk from BG2 saying "Weapon has no effect?" In a very confused voice. but replace weapon with bomb.

3