Submitted by nacrosian t3_100jwxj in singularity

There has been a lot of concern about students using ChatGPT and similar AI tools to essentially complete their schoolwork automatically (Note: I don't encourage anyone to cheat). However, it turns out that there are tools that detect AI-generated text, and it's very likely that teachers and professors are going to be using them on all student work going forward. The main AI detector that I've seen discussed is https://huggingface.co/openai-detector/, and it is extremely good at detecting ChatGPT's work. As an experiment, I generated an essay using the prompt:

> Write an essay on any topic in the style of a college student.

The response:

> In the 21st century, social media has become a ubiquitous presence in the lives of people around the world. It has changed the way we communicate, the way we access information, and even the way we form and maintain relationships. While social media has brought many benefits, it has also had a number of negative impacts on modern society. > > One of the most significant effects of social media is the way it has transformed the way we communicate. In the past, people had to rely on phone calls, letters, or face-to-face conversations to stay in touch with friends and family. Today, social media allows us to connect with people instantly and from anywhere in the world. We can send messages, share photos and videos, and even hold virtual meetings all with the click of a button. > > However, this constant connectivity has also led to a decline in face-to-face communication skills. Many people are so accustomed to communicating through screens that they struggle to have in-person conversations or express their emotions through body language. This has led to a rise in social anxiety and a decline in emotional intelligence. > > Another impact of social media is the way it has changed the way we access and consume information. In the past, people had to rely on traditional media sources like newspapers, television, and radio to stay informed. Today, social media allows us to access a virtually limitless amount of information at any time. However, this has also led to the proliferation of fake news and misinformation. It is now easier than ever for people to spread false or misleading information, which can have serious consequences. > > Finally, social media has also had a major impact on the way we form and maintain relationships. In the past, people had to put in effort to stay in touch with friends and family. Today, social media allows us to stay connected with a large network of people without ever having to leave the comfort of our own homes. However, this constant virtual connection can also lead to a decrease in genuine human interaction and a rise in virtual relationships. > > In conclusion, social media has had a significant impact on modern society. While it has brought many benefits, it has also had a number of negative effects on the way we communicate, access information, and form and maintain relationships. It is important that we continue to critically evaluate the role of social media in our lives and strive to find a balance between virtual and in-person connections.

This is pretty good work. Unfortunately, the AI detector flags it as having a 99.98% chance of being AI generated! You are most likely getting a zero if you hand this in.

But it turns out that there's a relatively easy method to bypass this tool: https://quillbot.com/ is a website that lets you paraphrase text, and doesn't even require an account. The trick is to run your essay through it multiple times; after 4-5 paraphrases, all of ChatGPT's words have been changed completely beyond recognition. Here's the result:

> In the twenty-first century, social media has a significant impact on how individuals conduct their lives. It has changed how we communicate with one another, learn, and even establish and maintain relationships. Undoubtedly, social media has numerous benefits, but it has also had some negative impacts on modern culture. > > One of social media's most significant side effects is how it has altered the way we interact. In the past, calling, writing, or paying a visit in person were the only ways to communicate with friends and family. We can now instantly connect with people anywhere in the world thanks to social media. We may communicate with one another, share photos and videos, and even organise online get-togethers by just pressing a button. > > A loss of social communication abilities is a consequence of this frequent contact. Many people find it difficult to communicate in person or through body language since they are so accustomed to interacting online. As a result, social anxiety has increased and emotional intelligence has decreased. > > Our techniques of acquiring and consuming information have altered as a result of social media. Traditional media outlets like newspapers, television, and radio had to be used in the past to stay informed. Social media has made it possible for us to instantaneously access a seemingly limitless amount of knowledge. But as a result, there are now more instances of false information and fake news. More quickly than before, people could disseminate incorrect or inaccurate information, which could have negative outcomes. > > Not to mention, social media has had a significant influence on how we establish and retain friendships. Contacting friends and relatives in the past required work. Social media enables us to communicate with a variety of individuals without ever leaving the comfort of our homes. Continuous internet use, however, can encourage more virtual interaction rather than face-to-face interaction. > > Finally, social media has a significant influence on contemporary culture. Because of the positive and negative effects, our ways of communicating, learning, and establishing and keeping relationships have all altered. If we want to maintain a healthy balance between online and offline contacts, we must continually assess how social media impacts our lives.

Now, https://huggingface.co/openai-detector/ thinks that this has a 99.97% chance of being "real". However, the essay has clearly gotten a lot worse: all of ChatGPT's clean writing is gone, having been turned into a heap of synonyms and awkward rephrasings. Essentially, we've gone from a solid B to barely scraping by with a C.

So, what should we be taking away from these results? The main thing to understand is that English assignments are not obsolete—any student trying to pass off ChatGPT's work will be easily caught, and I'm sure that institutions will be designing harsh concequences for this sort of thing.

Therefore, I believe that the main usecase for ChatGPT for students is to create a rough draft or an outline for their work. In this case, the paraphrased essay was pretty shoddy, but it would only take a few minutes to revise it and polish it up into something decent. Using any of ChatGPT's work directly, even if you plan to revise it, is likely to become very risky as AI-detection tools improve further.

So, what are your thoughts on AI text detection? Do you think that future iterations of GPT are going to be so good that it will be impossible to distinguish their work from that created by a professional writer? Or will we be seeing an arms race between teachers and students?

12

Comments

You must log in or register to comment.

MrEloi t1_j2i6j54 wrote

  • chatGPT can detect AI generated text - just paste it in and ask!
  • Tools already exist to rewrite AI text to 'normal' text .. and will improve.
  • Soon the AI text will be 'watermarked' to flag its origins.

I think that soon we will all be EXPECTED to use AI tools in all aspects of our lives, at least in part.

So simply add a note 'Text partially edited/created by AI'.

If your school/college is unhappy with this, then they are living in the past and are therefore providing you with an outdated education.

12

Starshot84 t1_j2j7mlp wrote

Spell-check has become very much encouraged since it's debut

2

Key_Scene7806 t1_j2i6oy3 wrote

I honestly think that teachers and professors can't take the risk of using those AI-detection tools as they are too unreliable in their current state. As more and more advanced and personalized text-transformers emerge, it's just going to get harder from here on, if not impossible, to detect. With the right prompts, you can create several stories that will be assessed as 99% human by the detection tool. But you can also try to imitate the writing of GPT, which will be assessed as AI generated.

11

StringNut t1_j2i8c4t wrote

Step one:

ChatGPT, write me an essay about topic X.

Step two:

ChatGPT, write me a python code to obfuscate a body of text with samples from this exemplar work (a body of the students own writing)

8

p0rty-Boi t1_j2iagzc wrote

Can you imagine a teacher running this test on (30 kids to a classroom X 5 classes) 150 essays? It’s more likely teachers will use chat GPT to grade papers than detect fake essays IMO. What’s needed is some sort of assignment hub with these AI checking tools built in. At that point having AI grade the papers automatically for grammar and poor sentence structure would be easy, leaving the analysis of thesis to the human professor.

7

ButterflyWatch t1_j2ijh6x wrote

Just read through the essay and rephrase it yourself? Still way more efficient than writing it yourself.

Generally I don't think there's a way for educational institutions to keep AI out of students' schoolwork. As other commenters have said, it's more likely that AI use becomes expected.

4

TacomaKMart t1_j2ijkww wrote

This moral panic about ChatGPT cheating is pointless. The teachers who are freaking out think that their job is to teach students to write essays, but it never was. Their job is to teach students to effectively express themselves in writing. Very few people write essays for a living, but lots of people write emails or letters.

If ChatGPT helps them do that - like Grammarly on steroids - what's the problem? It just means the "I teach essays" teacher needs to modify their ancient lesson plans.

And I say this as a teacher currently marking a stack of essays.

4

camdoodlebop t1_j2icq7i wrote

run the text through japanese translation and then run that through an english translation to scrub the AI markers from the text

3

BrushNo8178 t1_j2naox2 wrote

The problem with that approach is that words might be back-translated to the original.

A better way might be to generate the text in Language1 and use a script to split the sentences into two arrays: Array1 containing sentences 1, 3 , 5.. and Array2 containing the sentences 2, 4, 6... . Array1 could then be machine translated from Language1 to Language2 and then to the target language, while Array2 is translated from Language1 to Language3 and then to the target language. These two arrays can be combined to produce the final output text.

2

dracsakosrosa t1_j2kj9tx wrote

I'd like to think that ChatGPT 4 will be completely undetectable as a machine-generated text. We saw incredible advancements in natural language processing and machine learning in 2022, and it's not a stretch to say that ChatGPT 4 will be able to produce human-like speech that is indistinguishable from a real person. Kids are 100% already using this for essays and coursework and there's no way that educational institutions are going to be able to keep up with it. By the time they implement any stable standardised method of detection we'll be two versions ahead haha

3

greenbeanbbg t1_j2ij5yb wrote

the solution is to change education to be more holistic rather than putting 25% of someone’s grade into a final paper 😭 we have ai so we can use our teaching time so much more wisely

2

AvgAIbot t1_j2if44r wrote

Couldn’t you just ask ChatGPT to rephrase the essay and make it unique?

1

nacrosian OP t1_j2iga0e wrote

I did try that but it was still often detected as AI generated.

0

nebson10 t1_j2l9tz0 wrote

Using ChatGPT should not be considered cheating. The curriculum must change to account for new tools.

1