Submitted by Dr_Singularity t3_ywdsks in singularity
visarga t1_iwkbncq wrote
Reply to comment by 94746382926 in Cerebras Builds Its Own (1 Exaflop) AI Supercomputer - Andromeda - in just 3 days by Dr_Singularity
One Cerebras chip is about 100 top GPUs in speed but in memory it only handles 20B weights, they mention GPT-NeoX 20B. They need to stack 10 of these to train GPT-3.
Viewing a single comment thread. View all comments