Generally speaking, since response times and expected peak requests are usually defined in advance upgrading for speed will only happen if current hardware cant meet the requirements.
It's not really a "bring more value" thing, the customer will either have a clear target throughtput figure or we'll work one out together that satisfies what they want to do (usually trying to make it as low as possible because fast hardware = money) and then it's just a question of whether the current hardware is enough or not.
Ogawaa t1_iu21yuz wrote
Reply to [D] Do companies actually care about their model's training/inference speed? by GPUaccelerated
Generally speaking, since response times and expected peak requests are usually defined in advance upgrading for speed will only happen if current hardware cant meet the requirements.
It's not really a "bring more value" thing, the customer will either have a clear target throughtput figure or we'll work one out together that satisfies what they want to do (usually trying to make it as low as possible because fast hardware = money) and then it's just a question of whether the current hardware is enough or not.