How long until an AI is able to write a book?

Submitted by Educational_Grab_473 t3_10dsyd0 in singularity

When I say 'book', I'm not talking about a regular book with 100 pages or more, I'm talking about something more like a long story between 5-10 pages. Right now, GPT-3, the most advanced AI that the public can get their hands on, can't write a story longer than 2000 words. And the short stories it does write are straight up generic and cliche as hell. Do you think GPT-4 will be any better? And when do you think an AI will be able to write a good book with at least 100 pages, only after we achieve AGI?

52

Comments

You must log in or register to comment.

sideways t1_j4n69gn wrote

For what it's worth, I experimented a little with ChatGPT to write a 7000 word hard-boiled detective story. My main rule was that I couldn't write any original content myself - I was exclusively the editor.

I started with eliciting the stages of a detective story and then got ChatGPT to detail a plot based on those. Then I asked it to write individual scenes for each plot beat. Then I had it revise to correct inconsistencies and get the hard-boiled style down.

The process took maybe two or three hours. Compared to a human writer the result was mediocre and full of cliches. But... it was a recognizable story with plot and characters and it was mostly coherent. It was also a lot of fun to coach into existence.

From that experience, well, we're definitely not at the point where something like ChatGPT can write fiction coherently on its own for longer than a few pages and even with significant coaching, the results are "low human." But from this point to super-human? Who knows?

Might not take that long...

(Turns out that the full story was 4438 words.)

71

sumane12 t1_j4ngvw5 wrote

This is the way.

I've said it before, and I'll say it again. ChatGPT reminds me of a 13 year old who can never accept when they are wrong, has full access to the internet by thought and is very forgetful.

29

red75prime t1_j4pnhjh wrote

> and is very forgetful

To be more precise it has no long-term or procedural memory at all and it can't learn (from your interactions).

3

SoylentRox t1_j4nckq8 wrote

So here's the feature I think you need to make the tool work:

right now, the machine works by:

<current symbol buffer> + neural network -> <current symbol buffer> + 1 symbol

It needs to become

f(all previous sessions symbols) = salient context

<salient context> + <current symbol buffer> + neural network -> <current symbol buffer> + 1 symbol + <updated salient context>

"Salient context" is whatever the machine needs to continue generating text to match to something like a detective story. So it needs to remember the instructions, the main character's names, and so on. It does not need to remember every last word previously in the story.

To make it really good it needs to be aware of metrics of quality. Amazon/royalroad number of reviews and review ratings. Number of copies sold of the novel on the market. Etc. This way it can weight what it learns from text by how much humans liked that particular structure of text.

After that you'll need the AI to generate many stories, get user feedback, and iterate. I think eventually they will be good, and at some point past that it may discover ways to make them REALLY good that humans have not.

9

graham_fyffe t1_j4ndc3j wrote

You can ask chatGPT to write a summary of the story first, then the chapter names and chapter summaries, then each chapter one at a time. Try it! This hierarchical method can already achieve some of what you’re talking about.

6

SoylentRox t1_j4ndg65 wrote

So essentially it would just do something similar automatically.

3

FoxOwliegirl t1_j4vnsly wrote

That is also used by human writers, it is called the snowflake method.

1

graham_fyffe t1_j4ne8yf wrote

Oh and by the way, using human ratings of the model output is exactly how ChatGPT is trained. Human-in-the-loop reinforcement learning.

4

SoylentRox t1_j4neivp wrote

Correct but this was done at a small scale by chatGPT employees. I am saying we look at every novel that has data on its sales, every story on a site that has metrics of views or other measurements of quality and popularity, etc.

This might give the machine more information on what elements work that people like. Maybe enough to construct good stories.

3

red75prime t1_j4pogqn wrote

> <salient context> + <current symbol buffer> + neural network

That's RNN (recurrent neural network). As far as I know LSTM is still state of the art for them. And it struggles with long-term dependencies.

[He checks papers]

It looks like combination of transformer and LSTM does provide some benefits, but nothing groundbreaking yet.

1

threefriend t1_j4o2oez wrote

That's the same way I've used it to architect and write out larger programming projects. When you're stuck with a limited context window, a top-down approach is king.

4

ebawnix t1_j4r47x9 wrote

Same approach I’ve taken with moderate success. After awhile though, you can definitely tell it’s lost some of the context.

1

Educational_Grab_473 OP t1_j4ngseb wrote

That's interesting! I also played a bit with ChatGPT, trying to write a long story. Although even with me trying to correct and guide it, the story quickly became inconsistent and almost everytime the AI tried to end the story, finishing with the cliche ending of a fairy tale.

3

Baturinsky t1_j4nbojz wrote

What if as it writes, it compresses the previous chapters to a summary with only facts that are key for the following chapters?
I think it works kinda like that in AI Dungeon, but you have to fill in those "memos" yourself.

2

SoylentRox t1_j4ndmol wrote

Did you story have the poor and down on his luck detective who is overqualified for the job, his ratty run down office, and the clients as suspiciously attractive women?

2

sideways t1_j4nm082 wrote

That was definitely the default but I asked for something a little different and I got a dedicated female investigative journalist trying to track down a serial killer. One interesting twist is that the killer turned out to be a police officer himself. The climax of the story was the killer literally tackling the reporter as she burst into the police station with proof of his crimes!

5

SoylentRox t1_j4nmug9 wrote

And then what happened?! She gets arrested for resisting arrest and assaulting an officer? All the evidence she had on her disappears?

She spends time in jail until her defense attorney presents a cloud backup of her data to the DA? The charges get dropped but no one is punished for their actions but the serial killer?

3

sideways t1_j4noy81 wrote

Looks like I overestimated the word-count. The full story is 4438 words and here is the last 469 words:

>...As she approached the police station, she heard the sound of footsteps behind her. She turned to see the killer closing in on her, a cold glint in his eye.
>
>Kate's heart raced as she took off running, the killer hot on her heels. She weaved through the crowded streets, her breath coming in short gasps as she tried to outrun him. She knew that she couldn't let him catch her, not with the evidence she had against him.Finally, she spotted the police station in the distance. She redoubled her efforts, her legs pumping as she sprinted towards safety. She could hear the killer's footsteps growing closer and closer, and she knew that she was running out of time.
>
>Just as she reached the doors of the police station, the killer tackled her from behind. She went down hard, her head hitting the pavement with a sickening thud. She struggled to get up, but the killer was too strong.
>
>She heard the sound of sirens in the distance and knew that help was on the way. She just had to hold on a little longer. She summoned all of her strength and managed to break free from the killer's grasp. She scrambled to her feet and took off running, her heart pounding in her chest.
>
>She burst through the doors of the police station, her breath coming in ragged gasps. The officers on duty rushed to her aid, their faces grim as they took in the sight of the killer hot on her heels.
>
>Kate collapsed onto the floor, her body shaking as the officers apprehended the killer. She looked up at the chief of police, her eyes filled with tears of relief. The sight of the grim-faced officers and the smell of stale coffee in the station added to the tense atmosphere.
>
>###
>
>Kate presented the evidence to the chief of police, who was shocked and outraged by the revelation. The chief of police sat at his desk, his face grim as he listened to the recording. "This is serious," he said, his voice laced with disgust. "A police officer using his badge to commit murder and cover it up. It's an outrage.
>
>"What are we going to do?" Kate asked, her voice trembling.
>
>"We're going to do our jobs," the chief said, his voice firm. "We're going to arrest this killer and make sure that he pays for his crimes. And we're going to do everything in our power to restore trust in the department and ensure that something like this never happens again."
>
>Kate nodded, her heart heavy. She knew that this was a blow to the department, but she also knew that they couldn't let the actions of one corrupt officer tarnish the reputation of the many good officers who served with honor and integrity.The chief stood up, his face determined. "Let's go bring this killer to justice," he said, his voice filled with conviction.
>
>The killer was arrested and Kate's story was front-page news. The sound of her own relieved sigh filled the room as the chief congratulated her on a job well done.
>
>Kate's career was revitalized and the community was safer because of her bravery and determination. Despite the personal sacrifices she had made, Kate was proud of what she had accomplished. She had brought a killer to justice and exposed corruption in the police department. As she walked out of the police station, the warm sun on her skin and the sound of the city's hustle and bustle were a welcome reminder of the life she had fought to protect.

6

SoylentRox t1_j4nwbl3 wrote

Unrealistic but I mean, if you compare the AI to a high school student this is pretty good.

10

sideways t1_j4nwxt0 wrote

Exactly - and the fact that it's this good is pretty amazing. There's a real danger of goalpost moving.

8

SoylentRox t1_j4nxuwz wrote

Yeah. I would generalize to "most educated adults can't do better in their lifetime". I can maybe write a slightly better story since I've read a lotta stories, but not by a huge amount.

6

VictoryObvious6612 t1_j4p8g3k wrote

This is crap by high school standards, let alone college.

2

Denny_Hayes t1_j5bwq8o wrote

Agreed, this is more like middle school standard in regards to narrative, although with perfect grammar and spelling.

1

MassiveIndependence8 t1_j4xy4xq wrote

Can we see it? Is it posted anywhere?

2

sideways t1_j4xyh2c wrote

No - it was just for fun! But you can see the last few hundred words below somewhere in a separate comment.

1

VictoryObvious6612 t1_j4o1h6p wrote

AI will never surpass human authors.

−4

sideways t1_j4o2feq wrote

People said the same thing about Go.

But when it comes to art, what does "surpass" even mean? Did Steinbeck surpass Tolkien? Did Picasso surpass Klimt?

It's a meaningless question. What matters is whether AI will be able to create works that are meaningful and moving to humans - and I believe that it will.

10

VictoryObvious6612 t1_j4p8kq5 wrote

Go is solvable algorithmically.

Creating characters, plots and themes isn't. This will never happen.

0

MassiveIndependence8 t1_j4xyn0h wrote

That’s what the artist said

1

VictoryObvious6612 t1_j4y7zzx wrote

Kind of proves my point. AI art programs are good at making pretty renders. It's on par with calendar art. They aren't doing anything with any kind of message or thematic depth and almost certainly never will. You can get away with that in visual media, but not if you're trying to write a whole fucking novel.

1

MassiveIndependence8 t1_j4y8cww wrote

Lmao, sure bro RemindMe! 2 years

1

alexiuss t1_j4nkim3 wrote

NovelAI can write a book now. It's not very good, somewhere on badly written fanfiction porn level, but it's a book alright.

The real question is - when can AI write a best seller book?

11

bubster15 t1_j4p9ugl wrote

I think the novelty alone might clinch it even if it kinda sucks tho

Can you imagine how wealthy someone would get if they created an AI author that makes a themed series of books while imitating a certain human persona and maintaining a social media presence?

1

Shamwowz21 t1_j50nb8l wrote

Perhaps, or one could use the same AI to have any kind of book you wish. I fail to see the use of the ‘real’ author, since I doubt they could dictate the direction I’d want a book better than an AI that adjusts in real-time my preferences whenever I open up my magic book full of anything I want it to be. You’re correct, so long as we don’t have the freedom to make the book ourselves. In fact, it may be seen as limited to even buy a book except for historical or novel purposes (pun intended).

1

VictoryObvious6612 t1_j4o1npg wrote

Literally never

−12

One_andMany t1_j4otdk5 wrote

Denial

3

VictoryObvious6612 t1_j4p8btv wrote

Realism

−2

bubster15 t1_j4p9j7a wrote

After messing with it, my impression was it does feels far off, but it may honestly just be 1 or 2 more orders of magnitude away for computing power and it could happen fast. At the end of the day, I was extremely impressed.

I don’t think it’s unrealistic to imagine AI becoming 100x more powerful than today, more a question of when not if. Just my opinion!

1

VictoryObvious6612 t1_j4pacux wrote

Computing power isn't the barrier.

The barrier is that really good writing relies on experiences and emotions. I don't doubt an algorithm could churn out a passable Hardy Boys mystery by 2050. A procedural that sticks to a basic template with basic characters and no themes to speak of.

But an AI writing something along the lines of Blood Meridian, I am quite confident saying this will literally never happen. Not unless we get fully sentient robots like Data from Star Trek.

−2

GreatBigJerk t1_j4qm9ko wrote

Creative writing is a learned skill. It's not some magic thing that's unachievable by AI.

Deniers act like ChatGPT is where AI development ends, as if what we have right now is proof that we are incapable of making something more complex. This is just a step to something bigger.

1

Emory_C t1_j4nfrmc wrote

Quite a long time, maybe. One of the key limitations of GPT-4, as well as other language models in its class, is the context window.

In the case of GPT-3, the context window is approximately 2,048 tokens. This means that when generating text, GPT-3 can only consider the 2,048 tokens immediately preceding the point at which it is generating text. This can make it difficult for the model to maintain coherence and consistency when generating longer texts, such as a book.

When GPT-4 is developed, it'll likely still be limited by the context window . Researchers would need to develop new architectures that are able to take into account a larger context window. This is complex as hell, and it is not clear if it'll be possible with our current hardware limitations.

7

Stranfort t1_j4n6s1w wrote

We are in 2023, Id say close to 2025. It only need a few more evolutions before I can write a good book with good human complexity style of writing.

5

Cointransients t1_j4n809m wrote

I’d say 3-5 years before it’s writing genuinely impressive original stuff on the level of pro novelists. But even next year it should be much better.

5

__Maximum__ t1_j4nqvzw wrote

I think what you are asking is how long until researchers find out a way of expanding the input size of the model? Because there ways you can get it write a book right now. For example, you can tell it to write the first page, then a summary of it. Then feed the summary and let it write another page. And so on.

As to increasing the input size, if no new way comes around, it will get bigger as the transformers get more efficient, which they will. Look up flash transformers for example.

5

Eledridan t1_j4omvem wrote

It can be done already, but the quality of the book will always be in question. You just break down what ChatGPT is doing into small consumable tasks. You don’t say, “Write me a book.” You say, “Please give me an outline for a book and list 12 chapters with titles relevant to the plot. Then you ask it to write each chapter. I’d recommend you also edit as you go. ChatGPT is a tool to save labor, but you have to give it reasonable, consumable inputs versus your expectations.

5

sumane12 t1_j4ngfsy wrote

Arguably it can be done now, you just need to take the architecture of gpt3 plus davinci and allow it to process enough tokens to write a full book without forgetting things from the start. I actually worked with it to create a concept for a game idea I had but after a certain amount of words, it gets confused. I was able to adjust for this by giving it summaries of where the story was upto and reminding it of points that we had already incorporated.

So tldr, 1) allow it to process more tokens, or 2) compensate for this with human guidance.

3

spacedrace t1_j4nc1r2 wrote

Does novelai not count?

2

Educational_Grab_473 OP t1_j4nfij0 wrote

NovelAI is a great tool, I have already used it a lot in the past, but I meant more like a book made out of a prompt, with little to no human interaction to create something good. And sadly NovelAI is still a bit behind of GPT-3, even with their GPT-Neox model

5

CrispinMK t1_j4o243f wrote

It's pretty stunning what the GPT models can / will be able to do with language, but we're still a long way from competing with professional writers. There's a big difference between coherent prose (which anyone with a bit of experience can write) and creating compelling original characters or tapping into the human condition. Great novels accomplish a lot more than just "good writing".

That being said, there is a lot of trash that gets published anyway! The AI may not be coming for Stephen King, but it will be coming for a lot of the mediocre stuff on Amazon.

2

nutidizen t1_j4n71ab wrote

It can already to that. It's just not available to the public as a free tool.

1

LambdaAU t1_j4pdovm wrote

ChatGPT has improved on context and memory quite a bit over GPT-3 and I assume it will be the same for GPT-4. You can have quite a long conversation with it remaining consistent but plotholes still appear after 1-2 pages of words (and the story’s are dull). Increasing the coherency of a response is definitely a focus at the moment so I wouldn’t be surprised if GPT-4 has the capability to write 5-10 pages which form a believable story. So it all depends on when GPT-4 is going to release which I’m guessing will be Q3 this year.

1

mskogly t1_j4pey5b wrote

Should be possible at some point. But since GPT is so general, I think you have to coax it a little first, to get in the "mood" to be an author, or at least to make it be consistent in its style. Perhaps retrain or feed it with crime novels from a particular author to better understand what constitues a GOOD story written in a certain style.
Sort of like the tricks people use to make ChatGPT write code. ChatGPT isn´t (wasn´t) willing to write code, you had to sort of get it to accept certain rules first, a sort of framework, but when that was done it could output code.

1

Rickywalls137 t1_j4pf90j wrote

AI is based on what it was trained on. The majority of stories are pretty dull and cliche. I’m guessing if you train an AI model on great stories it could generate great stories too.

Imagine an AI writing a story with Tolstoy, Kafka, Le Carre influences. Bonkers.

1

AncientGreekHistory t1_j4pfagq wrote

That's already happened. The question is how long it'll take for them to be able to write one that's any good. The hard thing here is figuring out how to teach it to do something that very few people really have an understanding of.

1

No_Ninja3309_NoNoYes t1_j4prtce wrote

Writing is editing. AI is not good at line editing or structural editing. It beats humans on quantity, but it hasn't been taught the basics: show not tell, avoid adverbs, avoid long sentences, try not to repeat yourself, keep the story consistent.

AI has been trained to not be biased. This is good of course, but when you write a story, you have to choose a side. You need to choose a coherent setting and cast of characters. So in the absence of personal preferences and history, AI can only choose the most prevalent patterns in the training data or random stuff. This means clichés or something incoherent. Also the training data is not up to date with modern ideas AFAIK. It leans toward books written before 1920.

So I think the question should be when can AI edit properly? When and how can it build a personal style and history? I don't think that throwing money and data at this is enough. Knowledge of psychology, neuromorphic hardware, spiking neural networks, extreme learning machines, or something else entirely is required. So a decade is probably insufficient.

1

ramonnl t1_j4q4paf wrote

Would love where we can read the book and give the a.i tasks of removing or changing things we don´t like.

1

Narrow-Editor2463 t1_j4s9c3a wrote

Nice, then you can have like a fully impenetrable bubble to live in... or to force others to live in. Fun times.

1

gavlang t1_j4qva4h wrote

It can already do that. What you might want to be curious about is when will it formulate its will to write that book.

1

Dumitru-Cozac t1_j4r54md wrote

Currently, AI can generate text that is coherent and reasonably well-written, but it is not yet capable of writing a book that is comparable in quality to one written by a human. GPT-3, the most advanced AI language model available to the public, can generate text that is relatively coherent and well-written, but it is still limited in its ability to create truly original and engaging stories.

It is difficult to predict exactly when an AI will be able to write a book that is comparable in quality to one written by a human. The development of AI and its capabilities is a rapidly evolving field, and it will likely depend on advances in natural language processing, machine learning, and other related technologies.

It's also important to note that while AI can generate text that is coherent and relatively well-written, it still lacks creativity and originality that are essential elements of a good book. It is unlikely that an AI will be able to write a truly great book without human-like creativity and understanding of human emotions.

And what else can I tell you honestly, with the utmost sincerity!!! Don't use articles written by AI... no matter how good the program is. Google prints you and penalizes you. And if you use them, use them only as ideas, nothing more. Professional advice. Google is smart as hell.

1

HeinrichTheWolf_17 t1_j4ra1pl wrote

It can basically do that right now, it’s just not very good at it yet.

1

ertgbnm t1_j4rcli6 wrote

Currently you can co-author with chatGPT and get a book of arbitrary length with enough revising, re-generations, and trial and error. The book will be ok, albeit cliche-ridden and surface level in a lot of areas but it'd be readable and worse books would certainly exist. Will we ever get to the point that we can do it with the click of a button? Probably. But that alone is super human. If you were told to write a book about a topic it would take you a lot of revising, rewriting, and trial/error too.

1

Smellz_Of_Elderberry t1_j697l1z wrote

I think by this time next year it will be capable of writing one. But we'll see.

1

One_andMany t1_j4otcgd wrote

It literally can already write very large books this is nothing new

0