Viewing a single comment thread. View all comments

shanereid1 t1_j3dgdq0 wrote

Do you define transformers as Deep Learning? Cause if not then transformers.

1

junetwentyfirst2020 t1_j3dhtx6 wrote

Oh wow I never considered if I should see them as different. Transformers are a pretty big conceptual change, but they’re used in a deep way. 🤔 food for thought

2

currentscurrents t1_j3epeo7 wrote

Transformers are just deep learning with attention.

And attention is just another neural network telling the first one where to look.

4

junetwentyfirst2020 t1_j3ftm38 wrote

That makes sense. Convolutional neural networks were just deep learning with convolution.

1