It might be. But then tensorflow/pytorch are out there, too, have documentation and have CPU-only modes.

I say this as someone who coded a number of numeric algorithms (BFGS, Nelder-Mead, etc) from scratch just to understand them, while knowing all along I wasn't competing with real OR libs. On a few occasions, my own implementations were handy for proof-of-concept work when adding 3rd party libs was a hassle.

If you are still learning, you can continue. But with the knowledge that your library likely won't be competitive with what's already out there.

There is life outside of NLP and CV.

GAs are not great for expensive-to-evaluate functions. And those have become kind of relevant lately.

See "deep learning".

Batch gradient descent (the usual method) does not require the entire dataset to fit into memory -- only one batch, as it were.