Viewing a single comment thread. View all comments

sideways t1_j4n69gn wrote

For what it's worth, I experimented a little with ChatGPT to write a 7000 word hard-boiled detective story. My main rule was that I couldn't write any original content myself - I was exclusively the editor.

I started with eliciting the stages of a detective story and then got ChatGPT to detail a plot based on those. Then I asked it to write individual scenes for each plot beat. Then I had it revise to correct inconsistencies and get the hard-boiled style down.

The process took maybe two or three hours. Compared to a human writer the result was mediocre and full of cliches. But... it was a recognizable story with plot and characters and it was mostly coherent. It was also a lot of fun to coach into existence.

From that experience, well, we're definitely not at the point where something like ChatGPT can write fiction coherently on its own for longer than a few pages and even with significant coaching, the results are "low human." But from this point to super-human? Who knows?

Might not take that long...

(Turns out that the full story was 4438 words.)

71

sumane12 t1_j4ngvw5 wrote

This is the way.

I've said it before, and I'll say it again. ChatGPT reminds me of a 13 year old who can never accept when they are wrong, has full access to the internet by thought and is very forgetful.

29

red75prime t1_j4pnhjh wrote

> and is very forgetful

To be more precise it has no long-term or procedural memory at all and it can't learn (from your interactions).

3

SoylentRox t1_j4nckq8 wrote

So here's the feature I think you need to make the tool work:

right now, the machine works by:

<current symbol buffer> + neural network -> <current symbol buffer> + 1 symbol

It needs to become

f(all previous sessions symbols) = salient context

<salient context> + <current symbol buffer> + neural network -> <current symbol buffer> + 1 symbol + <updated salient context>

"Salient context" is whatever the machine needs to continue generating text to match to something like a detective story. So it needs to remember the instructions, the main character's names, and so on. It does not need to remember every last word previously in the story.

To make it really good it needs to be aware of metrics of quality. Amazon/royalroad number of reviews and review ratings. Number of copies sold of the novel on the market. Etc. This way it can weight what it learns from text by how much humans liked that particular structure of text.

After that you'll need the AI to generate many stories, get user feedback, and iterate. I think eventually they will be good, and at some point past that it may discover ways to make them REALLY good that humans have not.

9

graham_fyffe t1_j4ndc3j wrote

You can ask chatGPT to write a summary of the story first, then the chapter names and chapter summaries, then each chapter one at a time. Try it! This hierarchical method can already achieve some of what you’re talking about.

6

SoylentRox t1_j4ndg65 wrote

So essentially it would just do something similar automatically.

3

FoxOwliegirl t1_j4vnsly wrote

That is also used by human writers, it is called the snowflake method.

1

graham_fyffe t1_j4ne8yf wrote

Oh and by the way, using human ratings of the model output is exactly how ChatGPT is trained. Human-in-the-loop reinforcement learning.

4

SoylentRox t1_j4neivp wrote

Correct but this was done at a small scale by chatGPT employees. I am saying we look at every novel that has data on its sales, every story on a site that has metrics of views or other measurements of quality and popularity, etc.

This might give the machine more information on what elements work that people like. Maybe enough to construct good stories.

3

red75prime t1_j4pogqn wrote

> <salient context> + <current symbol buffer> + neural network

That's RNN (recurrent neural network). As far as I know LSTM is still state of the art for them. And it struggles with long-term dependencies.

[He checks papers]

It looks like combination of transformer and LSTM does provide some benefits, but nothing groundbreaking yet.

1

threefriend t1_j4o2oez wrote

That's the same way I've used it to architect and write out larger programming projects. When you're stuck with a limited context window, a top-down approach is king.

4

ebawnix t1_j4r47x9 wrote

Same approach I’ve taken with moderate success. After awhile though, you can definitely tell it’s lost some of the context.

1

Educational_Grab_473 OP t1_j4ngseb wrote

That's interesting! I also played a bit with ChatGPT, trying to write a long story. Although even with me trying to correct and guide it, the story quickly became inconsistent and almost everytime the AI tried to end the story, finishing with the cliche ending of a fairy tale.

3

Baturinsky t1_j4nbojz wrote

What if as it writes, it compresses the previous chapters to a summary with only facts that are key for the following chapters?
I think it works kinda like that in AI Dungeon, but you have to fill in those "memos" yourself.

2

SoylentRox t1_j4ndmol wrote

Did you story have the poor and down on his luck detective who is overqualified for the job, his ratty run down office, and the clients as suspiciously attractive women?

2

sideways t1_j4nm082 wrote

That was definitely the default but I asked for something a little different and I got a dedicated female investigative journalist trying to track down a serial killer. One interesting twist is that the killer turned out to be a police officer himself. The climax of the story was the killer literally tackling the reporter as she burst into the police station with proof of his crimes!

5

SoylentRox t1_j4nmug9 wrote

And then what happened?! She gets arrested for resisting arrest and assaulting an officer? All the evidence she had on her disappears?

She spends time in jail until her defense attorney presents a cloud backup of her data to the DA? The charges get dropped but no one is punished for their actions but the serial killer?

3

sideways t1_j4noy81 wrote

Looks like I overestimated the word-count. The full story is 4438 words and here is the last 469 words:

>...As she approached the police station, she heard the sound of footsteps behind her. She turned to see the killer closing in on her, a cold glint in his eye.
>
>Kate's heart raced as she took off running, the killer hot on her heels. She weaved through the crowded streets, her breath coming in short gasps as she tried to outrun him. She knew that she couldn't let him catch her, not with the evidence she had against him.Finally, she spotted the police station in the distance. She redoubled her efforts, her legs pumping as she sprinted towards safety. She could hear the killer's footsteps growing closer and closer, and she knew that she was running out of time.
>
>Just as she reached the doors of the police station, the killer tackled her from behind. She went down hard, her head hitting the pavement with a sickening thud. She struggled to get up, but the killer was too strong.
>
>She heard the sound of sirens in the distance and knew that help was on the way. She just had to hold on a little longer. She summoned all of her strength and managed to break free from the killer's grasp. She scrambled to her feet and took off running, her heart pounding in her chest.
>
>She burst through the doors of the police station, her breath coming in ragged gasps. The officers on duty rushed to her aid, their faces grim as they took in the sight of the killer hot on her heels.
>
>Kate collapsed onto the floor, her body shaking as the officers apprehended the killer. She looked up at the chief of police, her eyes filled with tears of relief. The sight of the grim-faced officers and the smell of stale coffee in the station added to the tense atmosphere.
>
>###
>
>Kate presented the evidence to the chief of police, who was shocked and outraged by the revelation. The chief of police sat at his desk, his face grim as he listened to the recording. "This is serious," he said, his voice laced with disgust. "A police officer using his badge to commit murder and cover it up. It's an outrage.
>
>"What are we going to do?" Kate asked, her voice trembling.
>
>"We're going to do our jobs," the chief said, his voice firm. "We're going to arrest this killer and make sure that he pays for his crimes. And we're going to do everything in our power to restore trust in the department and ensure that something like this never happens again."
>
>Kate nodded, her heart heavy. She knew that this was a blow to the department, but she also knew that they couldn't let the actions of one corrupt officer tarnish the reputation of the many good officers who served with honor and integrity.The chief stood up, his face determined. "Let's go bring this killer to justice," he said, his voice filled with conviction.
>
>The killer was arrested and Kate's story was front-page news. The sound of her own relieved sigh filled the room as the chief congratulated her on a job well done.
>
>Kate's career was revitalized and the community was safer because of her bravery and determination. Despite the personal sacrifices she had made, Kate was proud of what she had accomplished. She had brought a killer to justice and exposed corruption in the police department. As she walked out of the police station, the warm sun on her skin and the sound of the city's hustle and bustle were a welcome reminder of the life she had fought to protect.

6

SoylentRox t1_j4nwbl3 wrote

Unrealistic but I mean, if you compare the AI to a high school student this is pretty good.

10

sideways t1_j4nwxt0 wrote

Exactly - and the fact that it's this good is pretty amazing. There's a real danger of goalpost moving.

8

SoylentRox t1_j4nxuwz wrote

Yeah. I would generalize to "most educated adults can't do better in their lifetime". I can maybe write a slightly better story since I've read a lotta stories, but not by a huge amount.

6

VictoryObvious6612 t1_j4p8g3k wrote

This is crap by high school standards, let alone college.

2

Denny_Hayes t1_j5bwq8o wrote

Agreed, this is more like middle school standard in regards to narrative, although with perfect grammar and spelling.

1

MassiveIndependence8 t1_j4xy4xq wrote

Can we see it? Is it posted anywhere?

2

sideways t1_j4xyh2c wrote

No - it was just for fun! But you can see the last few hundred words below somewhere in a separate comment.

1

VictoryObvious6612 t1_j4o1h6p wrote

AI will never surpass human authors.

−4

sideways t1_j4o2feq wrote

People said the same thing about Go.

But when it comes to art, what does "surpass" even mean? Did Steinbeck surpass Tolkien? Did Picasso surpass Klimt?

It's a meaningless question. What matters is whether AI will be able to create works that are meaningful and moving to humans - and I believe that it will.

10

VictoryObvious6612 t1_j4p8kq5 wrote

Go is solvable algorithmically.

Creating characters, plots and themes isn't. This will never happen.

0

MassiveIndependence8 t1_j4xyn0h wrote

That’s what the artist said

1

VictoryObvious6612 t1_j4y7zzx wrote

Kind of proves my point. AI art programs are good at making pretty renders. It's on par with calendar art. They aren't doing anything with any kind of message or thematic depth and almost certainly never will. You can get away with that in visual media, but not if you're trying to write a whole fucking novel.

1

MassiveIndependence8 t1_j4y8cww wrote

Lmao, sure bro RemindMe! 2 years

1