Have you thought about buying your own hardware?

For $700-900 You can get recertified dell R820 40 cores/80 threads 256gb ram server with RAID10 SAS drives (8 or 16)

Or if you need GPU $1,100 For an HP Z840 WORKSTATION 24cores/ 48 threads and 128gb ram and 8gb workstation GPU. (we got lucky found one that had 512gb ram and dual GPU)

You can pick these up multiple places or Amazon and Amazon offers no interest financing for 12 months equal payments on amazing credit card. They come with 90 day warranty but can buy an extended warranty

This will pay for itself if you are actively training models and also working with any big data as clouds charge for bandwidth in and out of their systems and offcourse storage

Colab

vast.ai is extremely cheap + allows you to specify custom docker images.

It would be nice if you can try out stochastic.ai and provide suggestions on how to improve it. I'd be happy to explain how to build ML cloud infrastructure yourself too.

Try modal.com.

Modal is an ML-focused serverless cloud, and much more general than replicate.com which just allows you to deploy ML model endpoints. But still extremely easy to use.

It's the platform that this openai/whisper podcast transcriber is built on: /r/MachineLearning/comments/ynz4m1/p_transcribe_any_podcast_episode_in_just_1_minute/.

Or here's an example of doing serverless batch inference: modal.com/docs/guide/ex/batch_inference_using_huggingface.

This example from Charles Frye runs Stable Diffusion Dream Studio on Modal: twitter.com/charles_irl/status/1594732453809340416

You can't deploy Colab, only link people to the notebook?

DigitalOcean

I don't think it's remotely feasible for most people to just buy a server for their projects, even excluding a GPU on top.

The same thing can be said for someone never standing up server in AWS, GCP, AZURE, VPS, DIGITAL OCEAN.

I don't get your point.

A 3090 VM around $0.50 an hour on a lot of providers (less if you look around). If you're a hobbyist experimenting 6 hours a day for a week that's $21.00.

Compare to $1100 for the machine you quoted.

Your cost did not account for storage and bandwidth

Using Runpod as an example: bandwidth is free and storage is $0.0013 per GB hour.

If you use a 100GB disk that's an extra $2.87.

so I cannot upload a notebook and use it from my service, right?

no, you won't be able to make requests to the notebook.

>It would be nice if you can try out
>
>stochastic.ai
>
> and provide suggestions on how to improve it. I'd be happy to explain how to build ML cloud infrastructure yourself too.

Thanks for your reply. I would be interested to learn more about building your own infrastructure

>Try modal.com.
>
>Modal is an ML-focused serverless cloud, and much more general than replicate.com which just allows you to deploy ML model endpoints. But still extremely easy to use.
>
>It's the platform that this openai/whisper podcast transcriber is built on: /r/MachineLearning/comments/ynz4m1/p_transcribe_any_podcast_episode_in_just_1_minute/.
>
>Or here's an example of doing serverless batch inference: modal.com/docs/guide/ex/batch_inference_using_huggingface.
>
>This example from Charles Frye runs Stable Diffusion Dream Studio on Modal: twitter.com/charles_irl/status/1594732453809340416

this looks real nice. I will give it a try

Is the 3090 VM on AWS? Sounds really interesting. I might give it a try

Interesting idea. Where do you look for these used computers? Are you adding GPUs to the machine or just using them as it is?

thanks for the clarification. so this one has a limited use for creating a website and use it as an API

I use RunPod.

Thanks, I signed up. Are you using the community cloud or the secure one?

Typically secure if it's available, community cloud if not. Have a look on "browse servers" for the community cloud instances, their specs can range quite a bit so make sure to get one that fits your use-case.

Please do! DM me your email and I'll approve your account.

You can start out by looking at deploying stable diffusion models with accelerated inference (https://github.com/stochasticai/x-stable-diffusion). You normally want to be containerizing the models when you deploy them, easier to maintain and scale. DM me if you have any questions.