johnhopiler OP t1_j9u0kpb wrote on February 24, 2023 at 3:38 PM Reply to [P] What are the latest "out of the box solutions" for deploying the very large LLMs as API endpoints? by johnhopiler There are also https://alpa.ai/tutorials/opt_serving.html and https://colossalai.org/docs/advanced_tutorials/opt_service/ Permalink 1
johnhopiler OP t1_j9u0kpb wrote
Reply to [P] What are the latest "out of the box solutions" for deploying the very large LLMs as API endpoints? by johnhopiler
There are also
https://alpa.ai/tutorials/opt_serving.html
and
https://colossalai.org/docs/advanced_tutorials/opt_service/