It's not "complex patterns between neurons" we should care about, what will drive AI is more and better data. We have to beef up datasets of step by step problem solving in all fields. It's not enough to get the raw internet text, and we already used a big chunk of it, there is no 100x large version coming up.

But I agree with you here:

> whatever problems that remain over the horizon, there's a sort of exponential space that we are now in where those unknowns will quickly be reeled in

We can use language models to generate more data, as long as we can validate it to be correct. Fortunately problem validation is more reliable than open ended text generation.

For example, GPT-3 in its first incarnations didn't have chain-of-thought abilities, so no multi-step problem solving. Only after training on a massive dataset of code did this ability emerge. Code is problem solving.

The ability to execute novel prompts comes from fine-tuning on a dataset of 1000 supervised tasks. So they are Question-Answer pairs of many kinds. After seeing 1000 tasks, the model can combine and solve countless more tasks.

So it matters what kind of data is in the dataset. By discovering what data is missing and what are the ideal mixing proportions AI will advance further. This process can be largely automated, it mostly costs GPU and electricity. That is why it could solve the data problem. It is not dependent on us creating more data.