Viewing a single comment thread. View all comments

sharp7 OP t1_iqu81jn wrote

Reply to comment by IntelArtiGen in [P] Small problems to test out transformers? by sharp7

Hmm interesting that it only took you 30 min for europarl and masked word prediction. Do you have any links to more information about that dataset and task? I'm not familiar with masked word prediction. But that's pretty fast. Although I only have an old GTX 1060 6GB. Not sure how much worse that is than your rtx2070.

1

IntelArtiGen t1_iqv7vu7 wrote

The task is described in the paper I linked (3.1, Task #1: Masked LM). Any implementation of BERT should use it, like this one.

2