Gody_Godee
Gody_Godee t1_j6ayw0r wrote
Reply to comment by bo_peng in [P] RWKV 14B Language Model & ChatRWKV : pure RNN (attention-free), scalable and parallelizable like Transformers by bo_peng
your idea looks like this one from 3 years ago: https://arxiv.org/abs/2006.16236
Gody_Godee t1_j5zl5ar wrote
Reply to [P] RWKV 14B Language Model & ChatRWKV : pure RNN (attention-free), scalable and parallelizable like Transformers by bo_peng
another underperforming linear transformer again? ¯\_(ツ)_/¯
Gody_Godee t1_j91z1cm wrote
Reply to [R] RWKV-4 14B release (and ChatRWKV) - a surprisingly strong RNN Language Model by bo_peng
could you test it again LRA please?