op_prabhuomkar OP t1_j5i7oyj wrote on January 23, 2023 at 4:18 AM

Reply to comment by kkchangisin in [P] Benchmarking some PyTorch Inference Servers by op_prabhuomkar

Thank you for the feedback. I am looking forward to using the Triton's model analyzer possibly with different batch sizes and also FP16! Lets see how that goes :)

kkchangisin t1_j5if8hc wrote on January 23, 2023 at 5:29 AM

Depending on how much time I have there just might be a PR coming your way 😀…

Triton is really a somewhat hidden gem - the implementation and toolkit surrounding it is pretty impressive!