You shouldn't directly compare LLM training to human learning. LLM's are spawned with totally random weights, apart from the design choices of the architecture, the only learning signal they ever receive is from the training data. Human's are born with billions of years of information baked into them due to evolution. Comparing the two doesn't really make sense. I think this becomes way more obvious when you think about fine motor control instead of language modeling. I.e. a robot isn't going to learn how to walk as well as a human after the same amount of "training" time.
CaptainLocoMoco t1_jbmeg1l wrote
Reply to [D] Why are so many tokens needed to train large language models? by blacklemon67
You shouldn't directly compare LLM training to human learning. LLM's are spawned with totally random weights, apart from the design choices of the architecture, the only learning signal they ever receive is from the training data. Human's are born with billions of years of information baked into them due to evolution. Comparing the two doesn't really make sense. I think this becomes way more obvious when you think about fine motor control instead of language modeling. I.e. a robot isn't going to learn how to walk as well as a human after the same amount of "training" time.