op_prabhuomkar OP t1_j5k0h1j wrote on January 23, 2023 at 3:43 PM Reply to comment by Ok_Two6167 in [P] Benchmarking some PyTorch Inference Servers by op_prabhuomkar It’s actually easier to do for HTTP, will probably take that as a TODO. Thanks for the suggestion! Permalink Parent 1
op_prabhuomkar OP t1_j5i7oyj wrote on January 23, 2023 at 4:18 AM Reply to comment by kkchangisin in [P] Benchmarking some PyTorch Inference Servers by op_prabhuomkar Thank you for the feedback. I am looking forward to using the Triton's model analyzer possibly with different batch sizes and also FP16! Lets see how that goes :) Permalink Parent 2
[P] Benchmarking some PyTorch Inference Servers Submitted by op_prabhuomkar t3_10iqeuh on January 22, 2023 at 6:27 PM in MachineLearning 8 comments 27
op_prabhuomkar OP t1_j5k0h1j wrote
Reply to comment by Ok_Two6167 in [P] Benchmarking some PyTorch Inference Servers by op_prabhuomkar
It’s actually easier to do for HTTP, will probably take that as a TODO. Thanks for the suggestion!