I'm probably not the target demographic here (work in mid-size? tech), but I have a couple vague thoughts:

• training speed == dev velocity, train more models -> either get things ready faster or make model better in same time
• training speed == training cost if you're using on-demand compute. Depending on the company, they might not use on-demand (or might not care about cost). What i usually have seen happen is this never-ending cycle of slow training -> optimize -> add something that ends up leading to performance regression (maybe a new feature slows dataloading -> optimize again -> ... forever. Because of this, i think fundamental training optimizations can be useful, but it's super easy to introduce regressions and just accept them bc it's not usually a priority
• For realtime systems powered by ML, latency == engagement. You can get substantial improvements in engagement from running inference of ranking models faster