mrpogiface t1_irwjphh wrote on October 11, 2022 at 4:00 PM

How much effort would it be to get this running in WASM / the browser?

ggerganov OP t1_irwlluz wrote on October 11, 2022 at 4:13 PM

I was thinking about this too.

Compiling the code is easy. The problem is you need to load 75 MB model data (this is the "tiny" model). I guess nobody would want to download 75 MB every time they load a page.

Even if we say you are OK with 75 MB assets, the next problem is WASM not supporting SIMD. So the performance would be much worse compared to native. How much? Not sure.

But nevertheless - it might be fun to try and run it in the browser.

mrpogiface t1_irz4hc9 wrote on October 12, 2022 at 2:50 AM

As a complete WASM novice, I'd appreciate you doing it as a learning exercise for me :) But yeah, everything you outlined makes sense.

ggerganov OP t1_itcgtpx wrote on October 22, 2022 at 4:13 PM

Hey, just in case you are still interested - today I finished the WASM port and the performance not really bad:

https://github.com/ggerganov/whisper.cpp/tree/master/examples/whisper.wasm

There is a link to a live demo page where you can play with it.

Cheers!

mrpogiface t1_itch4j5 wrote on October 22, 2022 at 4:15 PM

I am extremely interested! I'm excited to learn from it, thank you :)

ggerganov OP t1_is1tgok wrote on October 12, 2022 at 6:06 PM

Looks like WASM actually support SIMD:

https://emscripten.org/docs/porting/simd.html

Will definitely give this a try when I get some free time. I will post updates here, if you are interested in the progress:

https://github.com/ggerganov/whisper.cpp/issues/44

zzzthelastuser t1_is0oiz5 wrote on October 12, 2022 at 1:29 PM

I THINK it is possible to keep the file cached. So that if a user returns to the site the model doesn't need to be re-downloaded again.

Alternatively a user could download the model file manually and your website asks the user to drag and drop their model file to launch the service?