But does Bing believe in the sentience of Blake Lemoine...?

Look into getting setup with a cloud solution, like Amazon Sage Maker or something similar. The potential gains to moving to a more powerful machine will be strongly dependent, however, on what is causing your code to take so long to execute. If it's just taking a long time because you're searching over a massive, high dimensional grid, moving to a better machine might offer limited improvements, and you might need to look into splitting the workload across a cluster of machines.

You can still compute the estimator with sparse inputs because the regularization term ensures the denominator is full rank. If the zeros are standing in for missing values, however, your estimates will be biased.

As for your second question, W* computed from only columns 2 and 4 will only yield the same values as W in the unrestricted model if the columns of X are orthogonal. Could you work with an orthogonal transform (e.g. PCA projection) of the X matrix?

Are you asking about the feasibility ridge regression with sparse inputs? Or about regularization to enforce a sparse solution?

Have you tried getting in touch with someone at Google? I wonder whether they have a procedure for folks with expertise in the local language to contribute materials (e.g. your translations, high quality translations from other sources) to improve the model for that language.