[D] Do we really need 100B+ parameters in a large language model? Submitted by Vegetable-Skill-9700 t3_121a8p4 on March 25, 2023 at 4:14 AM in MachineLearning 84 comments 101
Poseidon_22 t1_jdpyo9u wrote on March 26, 2023 at 7:29 AM Apparently, for linear improvement in accuracy, we would need exponentially more parameters. Gpt-4 with more than 1 trillion parameters would need to be trained on 6,700gpus for a whole year! Permalink 1
Viewing a single comment thread. View all comments