Most of the existing stable diffusion demos rely on a server behind to run the image generation. It means you need to host your own GPU server to support these workloads. It is hard to have the demo run purely on web browser, because stable diffusion usually has heavy computation and memory consumption. The web stable diffusion directly puts stable diffusion model in your browser, and it runs directly through client GPU on users’ laptop.

This means there is no queueing time for the server’s response, more opportunities for client server co-optimizations, and friendly for personalization and privacy.

Github page: https://github.com/mlc-ai/web-stable-diffusion

Also comes with a online demo: https://mlc.ai/web-stable-diffusion/

Comments

You must log in or register to comment.

Educational_Ice151 t1_jcn7818 wrote on March 18, 2023 at 1:23 AM

Wow! 🤯 this is so useful

Shared to r/aipromptprogramming

kalakau t1_jcn6q2v wrote on March 18, 2023 at 1:20 AM

absolutely amazing, thanks for doing this!

WASDx t1_jcogs0j wrote on March 18, 2023 at 9:55 AM

How does it store the model? I assume it's not re-downloaded every time you visit the page, and I would not expect my browser to allow caching multiple gigabytes from a single domain.

crowwork OP t1_jcourze wrote on March 18, 2023 at 12:43 PM

models are stored in the browser cache via an explicit browser cache api. So the model is cached once.

WASDx t1_jcpeej0 wrote on March 18, 2023 at 3:18 PM

Explicit as in requiring user approval? My web development skills are really out of date, can you link to the API?

crowwork OP t1_jcpojd6 wrote on March 18, 2023 at 4:27 PM

Explicitly means we need to write js code to request the cache as oppose to browser's automatic caching. https://developer.mozilla.org/en-US/docs/Web/API/Cache

remghoost7 t1_jcnvpqh wrote on March 18, 2023 at 5:05 AM

Very interesting....

Reminds me of how some VR apps can run natively in browsers, using hardware acceleration I believe. I'm guessing this is something sort of similar to that....? Could be entirely wrong though.

Cool stuff though. Would be need to make an extension of this for A1111.... Not to diminish the work you've done, but it would probably get more exposure that way (since it's the most used Stable Diffusion front end out there).

LetMeGuessYourAlts t1_jcq33nb wrote on March 18, 2023 at 6:05 PM

The Unreal Engine let's you compile to browser with basically a few clicks so that might be how they're doing it and abstracts away a lot of the hard parts.

remghoost7 t1_jcq8emm wrote on March 18, 2023 at 6:41 PM

Ah, it was made with unreal.....? I didn't see that.

I always love adaptations of video game engines. One of the reasons I've been a huge fan of Unity for years. It's essentially just a wrapper for C# code with a pretty interface.

shadowknight094 t1_jco3sjk wrote on March 18, 2023 at 6:46 AM

When I click generate button in Chrome on my desktop nothing happens? Is this a service worker or a network call? I don't see anything in Chrome dev tools

asdf3011 t1_jcnz5d2 wrote on March 18, 2023 at 5:45 AM

Going to send this to people when someone shows curiosity to trying image generation. Thanks you!

race2tb t1_jcops9d wrote on March 18, 2023 at 11:51 AM

I think there is a need for a p2p torrent backend for models that needs to exist now.

marcus_hk t1_jcrdufd wrote on March 18, 2023 at 11:46 PM

For weights, yes, and for inference. If you can decompose and distribute a model across enough nodes, then you can get meaningful compute out of CPUs too — for instance for tokenization and smaller models.

marcus_hk t1_jcrgwqm wrote on March 19, 2023 at 12:09 AM

Just browsing on my phone and haven’t dug deep yet, but in the notebook it says that build.py targets M2 by default but can also target CUDA. What about CPU?

I’d love to see a super minimal example, like running a small nn.Linear layer, for pedagogical purposes and to abstract away the complexity of a larger model like Stable Diffusion.