Hi all, I'm using neural networks to solve a multi-output regression problem. Now I want to improve my results, but it is unclear how to proceed. There are many (hyper)parameters that I could adjust: batch size, optimizer, learning rate, number of layers, number of hidden units per layer, type of activation, etc. Since training the network takes a decent amount of time, how can I approach (hyper)parameter selection in an intelligent way? Additionally, how can we decide when a model should be tuned versus scrapped? Is there some intuition that a model 'will not work', regardless of parameter settings?

Comments

You must log in or register to comment.

JurgenSchmidthuber t1_j8zf6gn wrote on February 18, 2023 at 1:28 AM

Rub it in oil and herbs and place it in the oven on broil for 10-12 minutes

MugiwarraD t1_j9djpux wrote on February 21, 2023 at 3:15 AM

lol

Sim2955 t1_j8ytfmi wrote on February 17, 2023 at 10:43 PM

possibly randomised search https://scikit-learn.org/stable/modules/grid_search.html#tuning-the-hyper-parameters-of-an-estimator

arrowoftime t1_j8z0jp2 wrote on February 17, 2023 at 11:35 PM

Deploy it: https://youtu.be/oVHYOD7GaUg

Morteriag t1_j90stfo wrote on February 18, 2023 at 10:29 AM

Put it to test with real data. Putting a lot of effort into tuning a model on a fixed data set that will eventually be deployed is a waste of time. And dont freak out when it fails! Just add more quality data from when it is deployed.

a_man_of_music t1_j91clm6 wrote on February 18, 2023 at 2:12 PM

Grid search to reduce the search space on the hyperparameters, then random search on the reduced search space.

mreyesag t1_j91hr5l wrote on February 18, 2023 at 2:52 PM

Once trained (and happy w the testing results)…make it fail :) find perturbations and difficult inputs so you understand the limits of your models you can now focus on robustness