visarga
visarga t1_jdlonpq wrote
Reply to comment by kromem in [R] Hello Dolly: Democratizing the magic of ChatGPT with open models by austintackaberry
One way to speed this up is to make an extension for voluntary contributions of LLM interactions to open source. A user decides when a chat deserves to be donated to open source and pushes a button to share. I don't think OpenAI can object to users donating their data.
visarga t1_jdloh24 wrote
Reply to comment by light24bulbs in [R] Hello Dolly: Democratizing the magic of ChatGPT with open models by austintackaberry
Since RLHF finetuning is short, you can continue training your original model and RLHF again.
visarga t1_jdlo8hl wrote
Reply to comment by master3243 in [R] Hello Dolly: Democratizing the magic of ChatGPT with open models by austintackaberry
Closed source on the generation end, but even more open than open source on the usage end. LLMs lift the open source idea to the next level.
visarga t1_jdgd9f2 wrote
Reply to comment by ghostfaceschiller in [D] "Sparks of Artificial General Intelligence: Early experiments with GPT-4" contained unredacted comments by QQII
Maybe they left it intentionally to be found...
visarga t1_jd89ecr wrote
Makes sense, with AI it would be possible to tutor kids anywhere and administer remote medicine where doctors are not present. I would not want to raise kids in a place where they don't have access to education and medicine, or where there are no playmates.
visarga t1_jd8831w wrote
Reply to comment by basilgello in AI democratization => urban or rural exodus ? by IntroVertu
Just because it is a plot device in StarTrek, and in physics scientists demonstrated they could "teleport" a particle, it doesn't mean it will be possible to teleport a human. What they teleport is not the particle but its quantum state. If in general, you want to read the quantum state of something you destroy it.
visarga t1_jd782aa wrote
Reply to comment by No_Ninja3309_NoNoYes in The Age of AI has begun - Bill Gates by Buck-Nasty
All you need is a 7B model trained on 1T tokens with 8K context length (chatGPTs little brother that is free to run)
visarga t1_jd77xhq wrote
Reply to comment by even_less_resistance in The Age of AI has begun - Bill Gates by Buck-Nasty
They are the same thing.
Math is language, that's trivial. The other way around is proved by GPT's existence.
visarga t1_jd77vff wrote
Reply to comment by D_Ethan_Bones in The Age of AI has begun - Bill Gates by Buck-Nasty
The usual human level chat bot and painter we have always had since 1956?
visarga t1_jd77ok8 wrote
Reply to comment by Drunken_F00l in The Age of AI has begun - Bill Gates by Buck-Nasty
I, for one, would benefit from AI reminding me about something or keeping me up to date when some important information surfaces. I don't read all the Slack channels and mails.
visarga t1_jd773xf wrote
Reply to comment by turnip_burrito in The Age of AI has begun - Bill Gates by Buck-Nasty
There is a new trend started by Stability and picked up by OpenAI that will provide base models for fine-tuning for each country/language/social group. Various groups are reacting to one-size-fits-all AI models.
This is an excellent article showing how AI models could impact communities effort to preserve their language.
> OpenAI's Whisper is another case study in Colonisation
https://blog.papareo.nz/whisper-is-another-case-study-in-colonisation/
And a positive one:
> How Iceland is using GPT-4 to preserve its language.
https://openai.com/customer-stories/government-of-iceland
When you got just 300k speakers of a language, you don't want the TTS and language model to make the new generation learn it wrong because the model didn't have good enough training data and made many mistakes. Kids are going to use AI in their own language, hence the risk of low quality responses impacting their small community even more.
visarga t1_jd763xk wrote
Reply to comment by Last_Jury5098 in The Age of AI has begun - Bill Gates by Buck-Nasty
Apparently RLHF makes the model less calibrated. So the more morality you put into it, the less you can rely on its confidence.
visarga t1_jd75vay wrote
Reply to comment by HonestIbrahim in The Age of AI has begun - Bill Gates by Buck-Nasty
Don't worry, 10 years later AI will be better than us at everything so we all become its pupils. The hardest task for AGI will be to bring humans along. Think it was hard to bring AI to human level? You should see how hard it will be to bring humans to AI level.
visarga t1_jd75ooz wrote
Reply to comment by [deleted] in The Age of AI has begun - Bill Gates by Buck-Nasty
But your supposition is not falsifiable, right? Under what circumstances would you believe a billionaire philanthropist is sincere?
visarga t1_jd0j369 wrote
Reply to comment by TinyBurbz in Teachers wanted to ban calculators in 1988. Now, they want to ban ChatGPT. by redbullkongen
Today, of course it isn't. But it will be like the internet - with time passing all things will depend on it. Without internet commerce, industry and banking would crash. We used to be ok without electricity 120 years ago, but today we can't exist without it anymore.
visarga t1_jd0ipc9 wrote
Reply to comment by zifahm in Teachers wanted to ban calculators in 1988. Now, they want to ban ChatGPT. by redbullkongen
When I was a kid they taught a bit of set theory at kindergarten. Concepts like set, union, intersection, count, etc. They all can be done with pretty pictures and coloring books.
visarga t1_jd0hl3r wrote
Reply to comment by mrmelts in Teachers wanted to ban calculators in 1988. Now, they want to ban ChatGPT. by redbullkongen
Instead of posing a threat to education, I believe that GPT has the potential to benefit children by individualized tutoring in an engaging style. Customized instruction has proven to be highly effective, so AI instruction could be effective as well. Homework would be unnecessary since all activities would be addressed during the tutoring sessions. The AI could guide discussions to specific subjects and focus on them, simultaneously teaching and evaluating. The degree of personalization achievable and the meticulous attention to detail are unparalleled. An AI could assess your knowledge and fine-tune its teaching approach. I would be thrilled to have access to such a system even as an adult. It could seamlessly integrate spaced repetition into an interesting conversation that avoids monotony. But conversational AI tutoring has no age limit, it could be used even by kids. There are lots and lots of bored kids who don't have anyone older to play with.
visarga t1_jd0ga1q wrote
Reply to comment by blueSGL in Teachers wanted to ban calculators in 1988. Now, they want to ban ChatGPT. by redbullkongen
North Korea tries to survive without fertiliser and it's not pretty.
visarga t1_jd0es70 wrote
Reply to comment by jonesocnosis in Teachers wanted to ban calculators in 1988. Now, they want to ban ChatGPT. by redbullkongen
I think we have been domesticated by language long before. Language models are just the latest development in language evolution.
visarga t1_jd0egyt wrote
Reply to comment by MisterViperfish in Teachers wanted to ban calculators in 1988. Now, they want to ban ChatGPT. by redbullkongen
> I might get it to reword some things for me after I already typed it, and proof read it to figure out HOW it improved it.
Don't copy GPT style if you know what's best for you. Slip in a mistaek or two to prove it wasn't written by AI.
visarga t1_jd0akyj wrote
Reply to comment by magnets-are-magic in Teachers wanted to ban calculators in 1988. Now, they want to ban ChatGPT. by redbullkongen
This is an artefact of RLHF. The model comes out well calibrated after pre-training, but the final stage of training breaks that calibration.
https://i.imgur.com/zlXRnB6.png
Explained by one of the lead authors of GPT4, Ilya Sutskever - https://www.youtube.com/watch?v=SjhIlw3Iffs&t=1072s
Ilya invites us to "find out" if we can quickly surpass the hallucination phase, maybe this year we will see his work pan out.
visarga t1_jctfir1 wrote
Reply to comment by relevantmeemayhere in [R] ChatGLM-6B - an open source 6.2 billion parameter Eng/Chinese bilingual LLM trained on 1T tokens, supplemented by supervised fine-tuning, feedback bootstrap, and RLHF. Runs on consumer grade GPUs by MysteryInc152
Human Feedback is being boostsrapped by GPT3 predictions "stolen" against OpenAI's will (for just $500 API bills).
visarga t1_jcjqap8 wrote
Reply to comment by IndiRefEarthLeaveSol in Those who know... by Destiny_Knight
We have SOTA image generative models, when we get even a decent, good enough small-LLM we're off. We can get our hands dirty with unconstrained AI tools.
visarga t1_jcjptxg wrote
Reply to comment by anaIconda69 in Those who know... by Destiny_Knight
I think you can even use a GPT2 model tuned with data from GPT4 to play a bunch of characters in a game. If you don't need universal knowledge, a small LM can do the trick. They can even calibrate the language model so the game comes out balanced and diverse.
visarga t1_jdloqee wrote
Reply to comment by ZetaReticullan in [R] Hello Dolly: Democratizing the magic of ChatGPT with open models by austintackaberry
Most of our pre-2020 NLP skills are worthless now, what required bespoke models and datasets is just another emergent LLM ability. It's like a new starting line and we don't know what human skills will be valuable in the future.