In my experience (automating quality control in automotive industry), inference time is critical.
Every step in a production line must be perfectly timed. If the factory must produce, let say, 30 cars per hour to achieve profits, even a 1s delay in a single step may turn to inmediate losses.
On the other hand, that 1s gain may translate to huge profits.
​
Generally, clients specify the total amount of time available to perform all the calculations (Data acquisition, preprocessing, inference and postprocessing) so we have to find for the optimal solution in each case. (Spoiler: the best solution and the optimal one are rarely the same)
The lower the inference time is, the better the output of the other processes are, which translates in a better overall performance.
An improvement in training speed may allow you to train more models, optimizing the developing stage --> Hardware and human resources optimization.
​
To sum up:
Inference speed --> Normally bottleneck in industry implementations.
Training speed --> Better use of resources. Better service to clients.
​
Idk how different industries work, but I guess it is similar.
P.S. This is my opinion based on the experience I have in my niche. 😬
LordDGarcia t1_iu3hp6v wrote
Reply to [D] Do companies actually care about their model's training/inference speed? by GPUaccelerated
In my experience (automating quality control in automotive industry), inference time is critical.
Every step in a production line must be perfectly timed. If the factory must produce, let say, 30 cars per hour to achieve profits, even a 1s delay in a single step may turn to inmediate losses.
On the other hand, that 1s gain may translate to huge profits.
​
Generally, clients specify the total amount of time available to perform all the calculations (Data acquisition, preprocessing, inference and postprocessing) so we have to find for the optimal solution in each case. (Spoiler: the best solution and the optimal one are rarely the same)
The lower the inference time is, the better the output of the other processes are, which translates in a better overall performance.
An improvement in training speed may allow you to train more models, optimizing the developing stage --> Hardware and human resources optimization.
​
To sum up:
Inference speed --> Normally bottleneck in industry implementations.
Training speed --> Better use of resources. Better service to clients.
​
Idk how different industries work, but I guess it is similar.
P.S. This is my opinion based on the experience I have in my niche. 😬