[D] Is there an affordable way to host a diffusers Stable Diffusion model publicly on the Internet for "real-time"-inference? (CPU or Serverless GPU?)

Submitted by OkOkPlayer t3_zdfrnw in MachineLearning

I have a fine-tuned Stable Diffusion Model and would like to host it to make it publicly available. Both options (GPU, CPU) seem to be problematic.

  1. I can't find a "cheap" GPU hosting platform. AWS etc. are all > 200$ per month and they have no serverless option (only found banana.dev which seems to have relatively limited flexibility)
  2. CPU seems to be too slow for inference

I am currently running the model on my notebook CPU with 35s/it which is way too slow. Is it possible to host a Stable Diffusion on CPU with close to real-time responses (< 60s for ~100 inference steps) or is there a "cheap" GPU hosting platform I couldn't find yet?

8

Comments

You must log in or register to comment.

LetterRip t1_iz20yak wrote

You might consider uploading it to civitai rather than self hosting, then people can download it and/or make it available via a number of free and paid services.

3

3pinephrin3 t1_iz1is4i wrote

If you use ddim you can get decent results with as few as 8 steps

1

Inevitable_Host_1446 t1_iz50q5y wrote

Assuming speed is your problem rather than wanting to share it with others, a decent longer term solution would be to put together a cheap rig with a 3060 12gb. It has enough vram / tensor cores to do pretty well, relative to price, at least orders of magnitude faster than cpu.

1

vizim t1_j23t41o wrote

Did you get to a solution? I am currently looking right now as well

1

OkOkPlayer OP t1_j23tq7b wrote

No unfortunately not. replicate looked the most promising but for own custom models it is closed beta atm.

1

vizim t1_j23tuqe wrote

I saw replicate can also handle private models. Have you looked into that?

1

OkOkPlayer OP t1_j23u7ou wrote

No. I only saw "closed beta" but they have a documentation for it. But because my project is currently stopped due to other reasons I have not further looked into it.

2