Another perspective: while new technological advances can certainly create their own demand, technology is also part of a complex web with social, cultural, and economic forces. It's not enough for a technology to just become possible; there also has to be a need, a demand for it. What problems do VR and AR actually solve?

The entertainment potential is tremendous, but entertainment is a luxury, and also has to be actually enjoyable first and foremost; both comfort and affordability are big factors there, and as others have pointed out, the hardware isn't yet good enough to consistently provide both to users just looking to play around with the tech for fun.

And other than that... It's mostly niche uses. You can come up with lots of specific cases where AR or VR could be useful, but not a lot of reasons for just everyone, like, average people, to need to use these technologies on a daily basis.

Smartphones, say, are useful as heck; convenient, practical, lots of reasons for the average person to want to always have one with then. Society as a whole might simply not have a use for AR or VR that's as broadly compelling. If the tech was mature to the point of it being affordable and easy to use and convenient, maybe some people would use this tech "just because." But then again, maybe not! Google Glass failed, in part, because most people seemed to not actually want to wear the tech (or to be around people who did...) And, it also works the other way around: with less demand for the tech, there's less of a drive for engineers and inventors to work on pushing it forward.

And agriculture turned happy bands of free, egalitarian hunter-gatherers into generations of miserable dirt-farming peasants toiling away for the glory of various dictatorial leaders.