Honestly, I wouldn't be surprised if we're past this hurdle in a matter of weeks:

RWKV showed how you can get an order of magnitude increase in inference speed of LLMs without losing too much performance. How long until someone instruction-tunes their baselines like alpaca did to llama?

the pace of development on these things is frightening.