Personally I think there are several things that need huge improvements for it to become mainstream. First one being higher resolution displays, the trouble with this is rendering very high resolutions requires better GPUs, as a high resolution increases the cost of rendering. Also these GPUs need to become affordable enough for it to be in a mainstream product at a competitive price.

Secondly, I think a better field of view, this is similar to the last point, as it requires more screen to be rendered therefore increasing the cost to render it. We can overcome these challenges with eye tracking and foviated rendering hopefully.

Next I think we need good intuitive controllers, something comfortable with good haptics to allow more interactions, theres are some cool ideas out there, search up VR gloves for examples.

We also just generally need more developers to start making full games for VR, this is tricky as the market is small so they would not see much return on their hard work. But we need someone to get the ball rolling to attract more people to the market. Once people see the unique experiences to be had in VR, they will be more interested to join in.

Lastly I think we need to increase comfort by lowering the weight and size of the headsets, a lot of people I know who own headsets say this is a big reason they dont use VR as much as they would like to.