Conquerer_Aegon t1_ix3ypa1 wrote on November 20, 2022 at 4:04 PM

Reply to comment by loly0ss in [D] Simple Questions Thread by AutoModerator

You have to have a non linear activation function at each layer otherwise your model won't learn any non linear relationship in your data. It will simply classify with a linear decision boundary.

loly0ss t1_ix40v2u wrote on November 20, 2022 at 4:18 PM

I have sigmoid in all hidden layers and output but it seems the model is only predicting one class. I tried balancing the datset, changing learning rate, shuffled data and iteration number and weight initilization yet still wrong :(

Conquerer_Aegon t1_ix4a0oq wrote on November 20, 2022 at 5:21 PM

What is the approximate proportion of the classes in the dataset?
Have you tried changing the no of hidden layers and activation function? What library are you using?

loly0ss t1_ix4c4ix wrote on November 20, 2022 at 5:36 PM

Yeah I've trid with no hidden layers and 2 hidden layers still the same. I've also tried Relu and softmax btu sigmoid was better. It's the mnist dataset, I'm trying to predict if the label is 1 or not 1. Since labels of 1 are 10% of the dataset. I reduced the dataset to around 40/60 , so 40% are labeled one and 60% are not ones, which I encoded them to 0.