Viewing a single comment thread. View all comments

DataMonkey2 t1_iqmmmrz wrote

For machine learning, is anyone else seeing the tide change. At first it was a step(useless for optimising obv), then sigmoid/logistic in the 80s, then tanh going into the 00s, which was generally replaced by Relu. Whilst there's a lot more leaky relu now, is anyone seeing more significant uptake of newer activations like mish?

0