>If you have a follow up to Gato that's 10x or 100x larger, the ability to cross/interpolate its knowledge across learned skills, and has a context window larger than 8,000 tokens, then you're approaching something like a proto-AGI.

And exactly this is why I think we're missing some structural / architectural component / breakthrough - the current models have the feel of unrolled loops.