Submitted by MLNoober t3_xuogm3 in MachineLearning
dumbmachines t1_iqx2g66 wrote
Reply to comment by MLNoober in [D] Why restrict to using a linear function to represent neurons? by MLNoober
>So if we ignore the implementation details to accomplish this for large networks, are there any inherent advantages to using higher-order neurons?
I don't know what that might be, but there is an inherent advantage in stacking layers of act(WX+b) where act is some non-linear function. Instead of guessing what higher level function you should use for each neuron, you can learn the higher order function by stacking many simpler non-linear functions. That way the solution is general and can work over many different datasets and modalities.
Viewing a single comment thread. View all comments