Poseidon_22 t1_jdpyo9u wrote on March 26, 2023 at 7:29 AM Reply to [D] Do we really need 100B+ parameters in a large language model? by Vegetable-Skill-9700 Apparently, for linear improvement in accuracy, we would need exponentially more parameters. Gpt-4 with more than 1 trillion parameters would need to be trained on 6,700gpus for a whole year! Permalink 1
Poseidon_22 t1_jdpyo9u wrote
Reply to [D] Do we really need 100B+ parameters in a large language model? by Vegetable-Skill-9700
Apparently, for linear improvement in accuracy, we would need exponentially more parameters. Gpt-4 with more than 1 trillion parameters would need to be trained on 6,700gpus for a whole year!