I don't think that's true. It would imply that Bi-LSTMs reach good performance faster than Transformers, and Transformers catch up later during training.

I've never seen proof for that, nor do my personal experiences confirm this.