GPT_scraper: A way to store ChatGPT conversations through hidden backend api via requests or selenium (no APIKEY needed!)

github.com

Submitted by Rodolflying t3_121vgdm in coolgithubprojects

Dont waste your api credits! 🤖

Using the backend hidden api from chat gpt, Maximize your ChatGPT experience scrap9kg your history with GPT_Scraper - the tool that makes scraping a breeze!

This is the github repo:

https://github.com/rodolflying/GPT_scraper

And medium article also: https://link.medium.com/XtNWuJIFsyb

Three Main tools:

  1. save all your chatgpt history using backend api from chatgpt website

  2. do the same but web scraping with selenium

  3. start and finish a new conversation and store it

4 min read

#ChatGPT #NaturalLanguageProcessing #PythonProgramming #DataScraping #AItools

29

Comments

You must log in or register to comment.

sprremix t1_jdnr9gx wrote

> Dont waste your api credits

chat.openai.com request also count toward your account usage/credits: https://platform.openai.com/account/usage

5

Rodolflying OP t1_jdnuazp wrote

It doesnt... check it out ! I still have all credits for this month 👌🏻 could you check again ? Note that im using the hidden api behind the frontend page and the other tool its a simple web scraping!

Try it yourself 😉

5

sprremix t1_jdolgwe wrote

Hm, I didn't check with your tool. Merely checked by entering something in chat.openai.com and seeing it popped up in my usage. If it doesn't show up using your tool it's pretty nifty!

4

Trolann t1_jdqphlq wrote

Maybe they're doing something like that for GPT4, but I doubt it and know they're not for GPT-3.

From your link (FAQ):

> Is the ChatGPT API included in the ChatGPT Plus subscription?

> No, the ChatGPT API and ChatGPT Plus subscription are billed separately. The API has its own pricing, which can be found at https://openai.com/pricing. The ChatGPT Plus subscription covers usage on chat.openai.com only and costs $20/month.

2

mikeyj777 t1_jdpiip4 wrote

This is neat, but chatGPT is free, and it stores all of the sessions, so I can go back to me trained-ish ai bots and work on different projects as needed.

2

Trolann t1_jdqpmtb wrote

Yes, but what OP posted is a way to do it programmatically, like via the API. You're doing it manually (and your sessions aren't trained or trained-ish either)

5

mikeyj777 t1_jdqx28p wrote

Well, whatever the right term is, I can select a previously stored chat. It recalls all previous prompts in that. So, if I've spent a long time trying to teach it how to generate the perfect image prompts, that isn't lost. I just call that trained-ish. Let me know the better term for it.

2

Trolann t1_jdqzp9l wrote

It doesn't recall all previous prompts is my point. It has a max of 4000 tokens of memory. If your sessions are longer than that, it isn't perfectly remembering what you said in the beginning.

5

Rodolflying OP t1_jdva8f9 wrote

Yes !! Sometimes its better to open a new conversations. Maybe with gpt4 the chatbit has a better 'memory', but its still very limited for now.

1

Rodolflying OP t1_jdv9vsg wrote

A basic example should be not asking frecuently asked questions again.

If you look into the json or csv file, you can filter past conversations. For example im always asking things about git and github, like step by step merge conflicts commands or many other things. Then you can also clasify your conversations and analize them.

I have now an article explaining the code of the first script

Api scraper medium article

And another one with the summary of the whole repo: repo summary

2

TEMPLERTV t1_jdr34xh wrote

This isn’t for the idiot unwashed masses, it’s for the people who understand why this is good. Crayons are fine, enjoy them

−2

mikeyj777 t1_jdrarde wrote

Is this a sore spot for you? your mom said she enjoys crayons over talking to you.

2

Natfan t1_jdrlxqh wrote

what compelled you to wake up in the morning, log into reddit.com and type this message?

2

TEMPLERTV t1_jdrtlxz wrote

The amount of people on here who complain about things they have no understanding of. The fact that we have AI and all this tech and people cant take 15 mins to just get a basic understanding of even the concepts of the tech.

0

mikeyj777 t1_jdtfn1i wrote

Yep, you got me. Mistaken that chatGPT only tracked the previous 4000 credits. I'm sure everyone that's learned about these new fangled computers for your fifteen minutes obviously knew that.

2