You're better qualified to know than nearly anyone who posts here, but is flash attention really all that's necessary to make that feasible?

I'm curious which memory efficient transformer variant they've figured out how to leverage at scale. They're obviously using one of them since they're offering models with 32k context but it's not clear which one.

100-200 is the likely range for number of B-21s built.

Nope, they live on. The B-21 replaces the B-1 and the B-2 but the B-52 continues. There are a lot of roles which don't need stealth but do need significant payloads, range, and the ability to bolt large, oddly shaped things under the wings. The B-52s theoretically take some pressure off the B-21s for things like chucking long range cruise missiles or deploying MALDs. You can technically do that out of cargo planes these days but the B-52s already exist, are a little better for the role, and don't really cost that much to operate.