Viewing a single comment thread. View all comments

DaffyDogModa t1_j2c6m43 wrote

-Only- needs 584 GPUs for 3 months to train haha.


mishap1 t1_j2cd6gg wrote

$6M in A100 GPUs plus all the hardware necessary to run them. Seems totally manageable.


Admirable_Royal_5119 t1_j2cj5co wrote

80k per year if you use aws


shogditontoast t1_j2cps4g wrote

Wow I’m surprised it’s so cheap. Now I regret working to reduce our AWS bill as that 80k would’ve previously gone unnoticed spread over a year.


username4kd t1_j2coq39 wrote

How would a Cerebras CS2 do?


SoylentRox t1_j2f9pw8 wrote

I think the issue is the cerebras has only 40 gigabytes of SRAM.

Palm is 540 billion parameters - that's 2.160 terabytes in just weights.

To train it you need more memory than that, think I read it's a factor of 3*. So you need 6 terabytes of memory.

This would be either ~75 A100 80 GB GPUs, or I dunno how you do it with a cerebras. Presumably you need 150 of them.

Sure it might train the whole model in hours though, cerebras has the advantage of being much faster.

Speed matters, once AI wars get really serious this might be worth every penny.


nickmaran t1_j2e09em wrote

Let me get some pocket change from my Swiss account


quettil t1_j2dn46n wrote

A fraction of the resources wasted by cryptocurrency.


PeakFuckingValue t1_j2ct76l wrote

How big is the storage requirement though? I don't know if I have an accurate perspective on what's beyond terabytes. That's like describing light years to me. Good luck.

It seems like we already unlocked some incredible speed technology recently with quantum computers. That was many magnitudes beyond the standard deviation. Whatever the cutting edge is on quantum computing and AI research must be combining the two.

Yes it's all crazy to us as consumers, but don't worry. We're in a capitalistic world. Whoever brings it to consumers first gets all the money lmao. They will be so stupid rich as well. I wonder if the people who should work with AI will be the ones who get there first.


rslarson147 t1_j2cz1uh wrote

Wonder if I could use a few at work without anyone noticing


DaffyDogModa t1_j2czs09 wrote

Worth a try!


rslarson147 t1_j2czuh6 wrote

Developing a new hardware stress test


DaffyDogModa t1_j2czz2y wrote

Just need some kind of remote on/off switch that you can turn on when everyone has gone home for the day. But I bet next power bill somebody gonna notice lol


rslarson147 t1_j2d06gs wrote

It’s just one GPU compute cluster, how much power could it consume? 60W?


Coindiggs t1_j2ekopz wrote

One A100 needs about 200-250w each. This needs 584x250w = 146,000w so approximately 146kwH. Average price of power is like 0.3$/kwh right now so running this will cost ya 43.8$ per hour, 1051.2$ per day or 32,500ish USD per month.


DaffyDogModa t1_j2d09dx wrote

One GPU if it’s a bad boy can be hundreds of watts I think. Maybe a miner can chime in to confirm.


rslarson147 t1_j2d0cqe wrote

I actually work as a hardware engineer supporting GPU compute clusters and have access to quite a few servers but I’m sure someone in upper management wouldn’t approve of this use


XTJ7 t1_j2fhhjd wrote

This went right over the head of most people. Brilliant comment though.


tomistruth t1_j2eig9t wrote

A medium crypto farm has about 1000 highend gpus running for full year. Server costs will go down, but we will hit a CPU performance plateau soon. Still compared to tech just 20 years ago, we now have computers more powerful than desktop pcs in our smartwatches. Also the ai model probably won't run on our phones but be connected to a giant central server system via the internet.

But what happens once we give those personal ais acceess to our computers and data is terrifying.

It could be the end of free speech and democracy, because you could literally become transparent. The AI could predict your habits and needs and show you ads before you even realize you want that.

Scary thought.