[P] What are the latest "out of the box solutions" for deploying the very large LLMs as API endpoints? Submitted by johnhopiler t3_11a8tru on February 23, 2023 at 9:09 PM in MachineLearning 11 comments 10
CKtalon t1_j9r2k9j wrote on February 23, 2023 at 11:19 PM Probably FasterTransformers with Triton Inference Server Permalink 3 whata_wonderful_day t1_ja3kh4d wrote on February 26, 2023 at 4:17 PM Yeah this is what the big bois use. It'll give you max performance, but isn't exactly user friendly Permalink Parent 1
whata_wonderful_day t1_ja3kh4d wrote on February 26, 2023 at 4:17 PM Yeah this is what the big bois use. It'll give you max performance, but isn't exactly user friendly Permalink Parent 1
Viewing a single comment thread. View all comments