visarga
visarga t1_ja52ibm wrote
Reply to comment by Sandbar101 in People lack imagination and it’s really bothering me by thecoffeejesus
Not even people working in the field have a good idea about 3 years ahead. Ten or twenty years ahead is just sci-fi.
visarga t1_ja50rdh wrote
Reply to comment by MrTacobeans in People lack imagination and it’s really bothering me by thecoffeejesus
Probably having to verify AI takes 50% of the time do do it manually, so the relative advantage is smaller.
But another advantage of teaming human+AI is that AI can be calibrated and ensure a baseline of quality. Humans might have higher variance, have a bad day, be tired, inattentive. So it is useful to increase consistency, not just volume.
visarga t1_ja5036a wrote
Reply to comment by Tall-Junket5151 in People lack imagination and it’s really bothering me by thecoffeejesus
> Humans are surprisingly adaptable, things that would have blown my mind even 5 years ago I take for granted now.
No way automation can keep up, we'll take everything for granted and still have to work to bring it to the next level.
visarga t1_ja4402m wrote
Reply to comment by civilrunner in Sam Altmans, Moores law on everything - housing by Pug124635
I think raw materials might not be needed anymore if we can recycle everything. At some point we will have to treat industrial materials like biology, they will have an ecosystem of their own. Of course if we want to expand we need new materials and space.
visarga t1_ja36ih0 wrote
Reply to comment by Kolinnor in Is multi-modal language model already AGI? by Ok-Variety-8135
We can have a model trained on a large video dataset, and then fine-tuned for various tasks like GPT3.
Using YouTube as training data we'd get video + audio which decompose in image, movement, body pose, intonation, text transcript, metadata all in parallel. This dataset could dwarf the text datasets we have now, and it will have lots of information that doesn't get captured in text, such as physical movements for achieving a specific task.
I think the OP was almost right. The multi-modal AI will be a good base for the next step, but it needs instruction tuning and RLHF. Just pre-training is not enough.
One immediate application I see - automating desktop activities. After watching many hours of screen casting from YT, the model will learn how to use apps and solve tasks at first sight like GPT-3.5, but not limited to just text.
visarga t1_ja3637d wrote
Reply to comment by turnip_burrito in Is multi-modal language model already AGI? by Ok-Variety-8135
A recent approach saves past experience data and loads it back for in-context-learning. The model itself can be task generic. So it learns by collecting new data.
visarga t1_ja2zhbj wrote
Reply to comment by genericrich in US Copyright Office: You Can't Copyright Images Generated Using AI by vadhavaniyafaijan
But it is still preferable to train on synthetic images than on the original works, don't you agree?
When the artist refuses to allow their images be used for training AI models, or it is impossible to get permission for other reasons such as not knowing the correct contact information, if the AI uses variations it won't learn to imitate the originals closely. Variations should be OK because they have no copyright, as the courts decided. Seems like a better compromise than either indiscriminate training or making AI impossible to train.
visarga t1_ja2yd3h wrote
Reply to AI technology level within 5 years by medicalheads
> Many AI researchers believe that solving the language translation problem is the closest thing to producing Artificial General Intelligence (AGI).
I call bullshit on this. Show me one researcher or paper claiming this. MT is not the closest to AGI, we have been doing ok in MT even before GPT-3. The most advanced AI we have now can solve problems and handles general chat. MT is a much simpler, basic task.
visarga t1_ja2vym8 wrote
Reply to comment by Frumpagumpus in An ICU coma patient costs $600 a day, how much will it cost to live in the digital world and keep the body alive here? by just-a-dreamer-
You don't need to do all that. Train a model on your data without destroying your body, just what can be logged from outside. It will be enough. chatGPT can enter a persona even with just a handful of hints. I think the AI of the future will be able to replicate any personality without fine-tuning.
visarga t1_ja2vlx9 wrote
Reply to comment by SpecialMembership in An ICU coma patient costs $600 a day, how much will it cost to live in the digital world and keep the body alive here? by just-a-dreamer-
You still need materials, it doesn't just create out of thin air.
visarga t1_ja2uz8e wrote
Reply to comment by Motion-to-Photons in Meta unveils a new large language model that can run on a single GPU by AylaDoesntLikeYou
Apparently 13B models feel comparable with chatGPT on a 3090 card with 24gb vram (source). So it would be fast!
visarga t1_ja2u514 wrote
Reply to comment by duffmanhb in Meta unveils a new large language model that can run on a single GPU by AylaDoesntLikeYou
But they documented how to make it by sharing paper, code, dataset and hyper-parameters. So when Stability wants to replicate, it will be 10x cheaper. And they showed a small model can be surprisingly good, that means it is tempting for many to replicate it.
The cost of running inference on GPT-3 was a huge moat that is going away. I expect this year we will be able to run a chatGPT level model on a single GPU, so we get cheap to run, private, open and commercial AI soon. We can use it for ourselves, we can make projects with it.
visarga t1_ja2tdeu wrote
Reply to comment by Ok-Ability-OP in Meta unveils a new large language model that can run on a single GPU by AylaDoesntLikeYou
> Could they get it to run on a phone one day? It would be awesome.
It would be Google's worst nightmare. Such a model could sit between the user and their ad-infested pages, extracting just the useful bits of information and ignoring the ads.
Using the internet without your local AI bot would be like walking outside without a mask during COVID waves. It's not just the ads and spam, but also the AIs used by various companies that don't have your best interest at heart. I expect all web browsers to have a LLM inside. Or maybe the operating systems.
It will be like "my lawyer will be talking to your lawyer" - but with AIs. You can't expose raw humans to external AI assault, humans need protection-AI just like we need an immune system to protect from viruses.
visarga t1_ja2r2fe wrote
Wouldn't it be better if people could donate their interactions with chatGPT, BingChat and other models? Make a scraping extension, it should collect chat logs and anonymise them. Then you got a diverse distribution of real life tasks.
I suspect this is the reason OpenAI and Bing offered their models for free to the public - to find the real distribution of tasks people want to solve with AI bots.
visarga t1_j9y7fro wrote
Reply to comment by Linear-- in [D] Isn't self-supervised learning(SSL) simply a kind of SL? by Linear--
Words in language are both observations and actions. So language modelling is also a kind of supervised policy learning?
So... Self Supervised Learning is Unsupervised & Supervised & Reinforcement Learning.
visarga t1_j9y79v0 wrote
Reply to comment by cthorrez in [D] Isn't self-supervised learning(SSL) simply a kind of SL? by Linear--
But the text coming from a human should be considered "manually" labelled, right?
visarga t1_j9y7624 wrote
Reply to [P] Introducing txtchat, next-generation conversational search and workflows by davidmezzetti
Does it do only one round of retrieval?
visarga t1_j9sib0x wrote
Reply to comment by TheLastVegan in And Yet It Understands by calbhollo
> The grounding problem is a red herring because thoughts are events rather than physical objects.
What? If they are events they are physical as well. The problem with grounding is that LLMs don't get much of it. They a grounded in problem solving and code generation. But humans are in the real world, we get more feedback than a LLM.
So LLMs with real world presence would be more grounded and behave more like us. LLMs now are like dreaming people, but it is not their fault. We need to give them legs, hands and eyes so they wake up to the real world.
visarga t1_j9qxt97 wrote
Reply to comment by 1973DodgeChallenger in [R] Provable Copyright Protection for Generative Models by vyasnikhil96
Well, you can't. Because it is really hard to extract any verbatim replications of training data from chatGPT. You need to put a considerable portion from the work as prompt, to put the model in the right place, and then sample your way ahead. Doesn't work for most stuff, like 99%.
visarga t1_j9qxgt2 wrote
Reply to comment by Disastrous_Elk_6375 in [R] Provable Copyright Protection for Generative Models by vyasnikhil96
If you go down to individual words or characters, everything is reused. If you go up, usually a random 10 word snippet is nowhere else in the internet. But boilerplate and basic things might be replicated in all shapes and forms.
visarga t1_j9qwzlf wrote
Reply to comment by currentscurrents in [R] Provable Copyright Protection for Generative Models by vyasnikhil96
> Honestly, kinda selfish. We'll all benefit from these powerful new tools and I don't appreciate you trying to hamper them.
They took their little pebble from the beach back home, that'll show them.
visarga t1_j9qwl8q wrote
Reply to comment by currentscurrents in [R] Provable Copyright Protection for Generative Models by vyasnikhil96
Diffusion models take about 1 byte of information from each training image - 5B images, 5Gb. So much less than a thumbnail.
visarga t1_j9qvckw wrote
Reply to comment by Wiskkey in [N] U.S. Copyright Office decides that Kris Kashtanova's AI-involved graphic novel will remain copyright registered, but the copyright protection will be limited to the text and the whole work as a compilation by Wiskkey
So, all you need to do is use a source image you made yourself, and then the whole image belongs to you. Nobody can extract just the AI contribution.
visarga t1_j9q9q7o wrote
Reply to comment by genericrich in US Copyright Office: You Can't Copyright Images Generated Using AI by vadhavaniyafaijan
That is glossing over the fact that nobody can actually demonstrate which of the source images were responsible for this derivation. Will you choose, or shall we pick one or ten at random, or just the closest by similarity score? We have no way of assigning merit.
And I suspect you think everything in a copyrighted work is protected by copyright. But it's not true. Only expression is protected, not the ideas. You can borrow ideas if you don't copy the exact expression. AI only learned basic concepts, it builds new images from first principles. By learning only ideas and not exact expression they can have free hand.
If you want to be 100% sure, then it is possible to train an AI with variations of the original works generated by another AI - this way only the ideas are transmitted and the new model has never seen copyrighted works, so it can never replicate them even by mistake.
visarga t1_ja52xcs wrote
Reply to comment by phillythompson in People lack imagination and it’s really bothering me by thecoffeejesus
In many ways it's been the same since 2010. We could talk, take photos, load web pages, use maps, set alarms and play games back then too, we even had Uber and AirBnb. Now the screens are a bit larger and the experience more polished.
I was expecting something more revolutionary - the phone is a pack of sensors, it has sight, hearing, touch, orientation, radio and many other sensors in the same package. But the amazing new applications didn't appear, except Pokemon Go?