It's unfortunate that AI can only be developed by large, well funded groups

Submitted by Scarlet_pot2 t3_1277dw4 in singularity

It takes millions of dollars to train an AI model on GPT-4 level or beyond. You need a large amount of GPUs. It takes a graduate degree and experience just to get your foot in the door for a ML / AI position. This is disappointing because it means what arguably will be the most important invention by humans will solely be in the hands of the corporations, government, or in other words the rich and powerful.

Morally speaking, A technology this important should be developed and fully available to all. This isn't the case though. Corpos will have control of this tool and they will get to decide how much of it is available to the people, if any. I'm just a software dev student. I've spent probably too much time thinking of how I could be in control of my own ASI / AGI. Get rich, get a graduate degree, start a company? Just isn't possible for normal people. This type of power is for people like Altman and Musk, people who come from very wealthy families and have Ivy league educations. This concentration of power is not good, but I don't see any realistic way around it. The only real hopes are the open-source community and the mercy of the corpos.

22

Comments

You must log in or register to comment.

TheKnifeOfLight t1_jed4kdv wrote

I do believe there is an ai that can run on consumer tech and can be trained on consumer tech that doesn’t need high end specs for the smaller models. I think it’s Alpaca/LLaMA and is comparable to Chatgpt from what I’m seeing online

8

smokingthatosamapack t1_jed6l4a wrote

the primary concern is training it to improve it. Sure fine tuning can be done but to make a substantial AI with significant changes and a unique model is its own feat that needs lots of funding.

4

Scarlet_pot2 OP t1_jed7tts wrote

Fine-tuning isn't the problem.. if you look at the alpaca paper, they fine tuned the LLaMA 7B model on gpt-3 and achieved gpt-3 results with only a few hundred dollars. The real costs are the base training of the model, which can be very expensive. Also having the amount of compute to run it after is an issue too.

Both problems could be helped if there was a free online system to donate compute and anyone was allowed to use it

1

smokingthatosamapack t1_jedmdli wrote

Yeah I see what you mean and it could happen but there's no such thing as a free lunch and even if there was a system it would probably pale in comparison to paid solutions for compute

1

Scarlet_pot2 OP t1_jed67k5 wrote

True alpaca is competent, but we need more models, better and larger models.. a distributed system where people donate compute could also be used to allow people to run larger models. maybe not 175 billion parameters, but maybe 50-100B as long as everyone donating compute isn't using it at the same time

that being said more smaller models like alpaca / LLaMA are needed too. if we made sufficient resources / training available to anyone, models like that could be created and made available more often

1

Akimbo333 t1_jefh3f9 wrote

Llama just needs much more training and fine-tuning and it'll be good

1

IronJackk t1_jed2dqn wrote

And I suppose you think you're the one to do it? I will be DEAD before I see ai in the hands of an Elf!

7

Scarlet_pot2 OP t1_jed2mpi wrote

As long as I'd have unrestricted access to the latest advanced models i wouldn't care. That's the real goal IMO. most advanced access to everyone

also nice LOTR reference lmao

3

NakedMuffin4403 t1_jed8ojv wrote

Who said you need to train models from scratch? The people doing this are actual SCIENTISTS. Startups will NOT be training models from scratch (similar to how they abstract compute to cloud providers).

What you need to do is use these already trained models and then train them to cater to your vertical. Not sure how doable this is now, but it is inevitably going to be the industry standard.

The two hardest parts will be finding the data to tailor-train your model and actually implementing it a meaningful way.

I am in kinda in a similar position. I actually made a post on my profile you can check out. I study CS but I currently lack the expertise to compete effectively - and I am working hard to fix that.

4

Scarlet_pot2 OP t1_jed9mxw wrote

I see your point about tailoring foundational models. The problem is that, do you think companies like OpenAI and Google are going to allow regular people to tailor train their models however they want? It's debatable. Even in the best case the corps will still put some restrictions on what and the models are tailor trained.

The best way to get around this is have open source foundational models. To do this you need available compute (people donating compute over the internet) and free training (free resources and groups to learn together). I'm sure tailoring corporate models will play a role, but if we want true decentralization we should approach it from all angles

1

NakedMuffin4403 t1_jeda9l7 wrote

>The best way to get around this is have open source foundational models. To do this you need available compute (people donating compute over the internet) and free training (free resources and groups to learn together). I'm sure tailoring corporate models will play a role, but if we want true decentralization we should approach it from all angles

It's not debatable as SaaS is going to be commodified and models are going to be the new hot thing.

This is a major paradigm shift from SaaS to MaaS (models as a service).

What's terrifying and exciting is that at some point in the future, there will barely be any proprietary software. Most software will be easily replicable with SIGNFICANTLY less engineers given the productivity boost of AI.

Imagine remaking a $100B Stripe with just $100m and 1/100 the human capital.

The MaaS providers are kind of like the people selling the shovels in a gold rush (and the silicon fabs are enabling the shovel sellers).

The software companies that will remain "proprietary" are going to be those that can implement AI more effectively than others, and those who have network effects like social media apps for instance.

3

tiselo3655necktaicom t1_jed5oln wrote

What? Change, revolution, progress etc has always been dictated by capital. The rest of us just have events and history happen to us. This is the way its always been. There's no reason or evidence to think it would be otherwise suddenly.

2

Scarlet_pot2 OP t1_jed747y wrote

Okay now that's just incorrect. Most of human innovations were made by small groups or even a single person, without much capital. Think of the wheel, agriculture, electricity, the light bulb, the first planes, Windows OS. The list goes on and on.

It's only recently that it takes super teams and large capital to make these innovations. I'm saying we should crowdsource funds, with free resources to learn from together, donating compute, etc. It's totally possible but modern people aren't very good at forming groups. Maybe its because people are too tired from work, or they have become much less social. For whatever reason, still, we could improve AI progress and decentralize AI if the people learned to talk and collaborate again

0

tiselo3655necktaicom t1_jed7uvh wrote

>Most of human innovations were made by small groups or even a single person, without much capital. Think of the wheel, agriculture, electricity, the light bulb, the first planes, Windows OS. The list goes on and on.

You have a childlike naivety about business and live in a fantasy world.

"Data shows US inventors aren’t just good at science—they come from rich families" (2017)

"Entrepreneurs come from families with money" (2015)

3

Scarlet_pot2 OP t1_jed8dir wrote

These articles are talking about in our modern society. Our technology is to the point where it takes a lot of effort to make modest improvements (in most areas). for most of time the innovations found didn't cost much, like how to make a bow, or how to smith metal. If you think all inventions were made by wealthy people, you are delusional. It wasn't the king that learned how to make chainmail armor, and it wasn't the noble that learned how to raise bigger crops.

P.S. Your insults don't help your point at all.

−1

tiselo3655necktaicom t1_jed9maa wrote

inventions throughout history were almost entirely made by rich people. because iterating and failing takes money. the fact that you cannot comprehend this at the outset means you are naive or a child. This is just a fact. It is self-evident. Your using high fantasy examples furthers the point that you live in...a fantasy land.

−1

Scarlet_pot2 OP t1_jedadfu wrote

talking to you is like a brick wall. I'm done. Keep idolizing rich people with your false narratives.

Yeah I'm sure the first person to learn how to raise crops was drowning in wealth. I'm sure the first person to make a bow was somehow wealthy, lmao. I'm sure the wealthy king walked into the blacksmiths place one day and just figured out how to build chainmail. The person who invented the wheel had so much wealth he didn't even need to get up if he didn't want to. all sarcasm. This belief you have is illogical.

In reality, most advancements were made by regular people, very poor people by modern standards, just trying to improve their lives, or discovering by accident, or other ways.

−2

tiselo3655necktaicom t1_jedavq8 wrote

Lol. stay in school kids. You could end up like this person.

0

Scarlet_pot2 OP t1_jedbizi wrote

you can't make a half decent argument so you result to insults and running away lmao. Very annoying type of person.

−1

tiselo3655necktaicom t1_jedcxcu wrote

I supplied multiple links to actual sources, all you can do is talk about blacksmiths and bows. This isn't D&D. Get out of the fucking basement. Come with citations.

>talking to you is like a brick wall. I'm done.

And yet, you're back with:

>you can't make a half decent argument so you result to insults and running away lmao.

hm. Where are your sources again?

5

Scarlet_pot2 OP t1_jedrsmn wrote

you are such a sad person. you're life is so sad you have to insult strangers on the internet to make yourself feel better. And you're so low IQ you can't even form a coherent argument. shut up and go back to work at your 9-5 restaurant job. reddit loser

Also: anyone can link a few irrelevant articles. you linked ones that have no relation to the topic at hand but you are too brain dead to be able to actually comprehend it.

Take your sausage fingers off the keyboard and go learn common sense.

And lose some weight while you're at it.

1

TemetN t1_jed01g3 wrote

I'd go so far as to call it dangerous honestly, and we can see that in the struggle for basic things. Many of which are much more normalized in less significant fields simply due to mass participation. On the plus side there is at least more public funding for research now, but I am glad to see larger calls for a massive public project on this.

1

Scarlet_pot2 OP t1_jed0f3l wrote

A public project would be great. I'm sure there are thousands of people willing to get involved. We probably have at least a few hundred on this sub. The main thing would be to get organized and spread the word, which is ironically difficult in this age of the internet

5

TemetN t1_jed0uhc wrote

LAION launched a petition for one, but the petition site is down for maintenance currently. Bad timing honestly.

Securing our Digital Future: A CERN for open source large scale AI research and its safety

3

Scarlet_pot2 OP t1_jed1kio wrote

That's definitely a positive move. The only issue is that people at LAION will probably decide who gets access and when. Still much better then corps or gov tho, but more projects would be good. Maybe a distributed training network where people could contribute compute over the internet? Along with a push to give anyone who wants it free training on ML / AI. Those two things would help decentralize AI

3

DarkCeldori t1_jeepkok wrote

This long training time with massive resources is by virtue of backpropagation and use of llm.

The human brain runs at about 100Hz at like 2% activity, and within a few years you can have a prodigy doing calculus, chemistry, chess and playing instruments with grasp of multiple languages. It is estimated the brain does the equivalent of around 100 trillion operations per second.

These models are being trained with the equivalent of millions of years of training on what is likely hw far more powerful than the brain.

It is likely brain like algorithms can allow far more modest hw to train in realtime and achieve agi performance.

1

Ago0330 t1_jed32kv wrote

Most of the rich and powerful are using their AI apps to protect their current interests. It’s way too early in the game to pick winners and losers because at this stage, the most likely outcome is they will be the “MySpace” of AI apps. They are tuning their AI Chatbots on trillions of parameters. This can’t be explained to the general public in a way that will cause a shift in societal thinking. The financial markets are setup for a massive change in wealth in the short term that will cause a paradigm shift and unlock new AI opportunities. The best is yet to come.

0