alibrarydweller t1_iuedvyb wrote on October 30, 2022 at 6:18 PM Reply to [D] Looking for suggestions on setting up autoscaling on GPU servers for AI inference (without kubernetes)? by fgp121 You might look at Nomad -- it manages containers like K8s, but it's significantly simpler. We run GPU jobs on it, although we don't currently autoscale. Permalink 3
alibrarydweller t1_iuedvyb wrote
Reply to [D] Looking for suggestions on setting up autoscaling on GPU servers for AI inference (without kubernetes)? by fgp121
You might look at Nomad -- it manages containers like K8s, but it's significantly simpler. We run GPU jobs on it, although we don't currently autoscale.