I don't know if IQ is exactly a good metric here because LLMs merely replicate training data so it would be likely that the training data (which is very big) contains information regarding IQ tests. It would be an indirect comparison because you'd be comparing sheer training data amount with a person's ability to produce thoughts. It would be way more interesting to give GPT4 complex situations that require advanced problem solving skills. Say you got a message that you need to decode and it has multiple layers of encryption and you only have a few hints on how you might go about it, since there's no way to replicate responses based on previous training data I'd be curious to see how far it gets, or let's say a hacking CTF, which is something that not only takes pure coding skill, but also a creative thought process.
WindForce02 t1_je4zh7m wrote
Reply to comment by lostmsu in [D] Prediction time! Lets update those Bayesian priors! How long until human-level AGI? by LanchestersLaw
I don't know if IQ is exactly a good metric here because LLMs merely replicate training data so it would be likely that the training data (which is very big) contains information regarding IQ tests. It would be an indirect comparison because you'd be comparing sheer training data amount with a person's ability to produce thoughts. It would be way more interesting to give GPT4 complex situations that require advanced problem solving skills. Say you got a message that you need to decode and it has multiple layers of encryption and you only have a few hints on how you might go about it, since there's no way to replicate responses based on previous training data I'd be curious to see how far it gets, or let's say a hacking CTF, which is something that not only takes pure coding skill, but also a creative thought process.