vsmolyakov
vsmolyakov t1_j22v7n7 wrote
Reply to comment by Obvious-Set-1981 in [D] Simple Questions Thread by AutoModerator
You may want to use spark sql for big data applications, scala for processing the data, MLlib for machine learning and parquet file format for saving the results.
vsmolyakov t1_j22uo5z wrote
Reply to comment by writerwritesalot in [D] Simple Questions Thread by AutoModerator
The reason for checking the validation loss against the training loss is to see how well the model is learning and whether it’s overfitting. You would need as many data points as necessary to make that assessment.
vsmolyakov t1_j22s8cc wrote
Reply to comment by kyolichtz in [D] Simple Questions Thread by AutoModerator
Consider Kaggle for a source of data science projects that look interesting to you. If you are looking for more ML engineering / cloud native type projects, I recommend “Practical MLOps” book by Noah Gift, you can also find his GitHub for ideas.
vsmolyakov t1_j225ap9 wrote
"Look within. Within is the fountain of good, and it will ever bubble up, if thou wilt ever dig." -Marcus Aurelius
vsmolyakov t1_j1tc0dg wrote
Reply to comment by sanman in [D] Simple Questions Thread by AutoModerator
Validation dataset is typically used to tune model hyper-parameters and for early stoping to tell whether the model is overfitting.
vsmolyakov t1_j1p17x0 wrote
Reply to [D] Simple Questions Thread by AutoModerator
Does anyone know of a multivariate time series Transformer models? Similar to https://huggingface.co/blog/time-series-transformers but for multiple time series?
vsmolyakov t1_j1p0eeg wrote
Reply to comment by throw1ff in [D] Simple Questions Thread by AutoModerator
I recommend building up a portfolio of CV and NLP projects on GitHub, it will help sharpen your deep learning skills. Also getting experience with model deployment and cloud certifications (AWS, Azure, GCP) is valuable for ML engineering positions.
vsmolyakov t1_j2qsqzi wrote
Reply to [D] life advice to relatively late bloomer ML theory researcher. by notyourregularnerd
Be ware of the length of the program, find out ahead of time average graduation time for advisors of interest; top schools tend to take longer to graduate!