BungaBunga6767 t1_jcl6vf9 wrote on March 17, 2023 at 5:07 PM Reply to comment by harharveryfunny in [D] PyTorch 2.0 Native Flash Attention 32k Context Window by super_deap LongFormer does it but not with FlashAttention Permalink Parent 3
BungaBunga6767 t1_jcl6vf9 wrote
Reply to comment by harharveryfunny in [D] PyTorch 2.0 Native Flash Attention 32k Context Window by super_deap
LongFormer does it but not with FlashAttention