I trained a neural network. Now what?

Submitted by Ricenaros t3_114yiwj in deeplearning

Hi all, I'm using neural networks to solve a multi-output regression problem. Now I want to improve my results, but it is unclear how to proceed. There are many (hyper)parameters that I could adjust: batch size, optimizer, learning rate, number of layers, number of hidden units per layer, type of activation, etc. Since training the network takes a decent amount of time, how can I approach (hyper)parameter selection in an intelligent way? Additionally, how can we decide when a model should be tuned versus scrapped? Is there some intuition that a model 'will not work', regardless of parameter settings?

8

Comments

You must log in or register to comment.

JurgenSchmidthuber t1_j8zf6gn wrote

Rub it in oil and herbs and place it in the oven on broil for 10-12 minutes

32

Morteriag t1_j90stfo wrote

Put it to test with real data. Putting a lot of effort into tuning a model on a fixed data set that will eventually be deployed is a waste of time. And dont freak out when it fails! Just add more quality data from when it is deployed.

2

a_man_of_music t1_j91clm6 wrote

Grid search to reduce the search space on the hyperparameters, then random search on the reduced search space.

2

mreyesag t1_j91hr5l wrote

Once trained (and happy w the testing results)…make it fail :) find perturbations and difficult inputs so you understand the limits of your models you can now focus on robustness

2