Viewing a single comment thread. View all comments

cheecheepong t1_is38h3m wrote

Reply to comment by Batuhan_Y in [Project] I've built an Auto Subtitled Video Generator using Streamlit and OpenAI Whisper, hosted on HuggingFace spaces. by Batuhan_Y

Great news! Different error this time:

https://www.youtube.com/watch?v=PlUvLBRwLbw

RuntimeError: The size of tensor a (316) must match the size of tensor b (3) at non-singleton dimension 3Traceback:File "/home/user/.local/lib/python3.8/site-packages/streamlit/scriptrunner/script_runner.py", line 554, in _run_scriptexec(code, module.__dict__)File "/home/user/app/app.py", line 258, in <module>main()File "/home/user/app/app.py", line 138, in mainresults = inference(link, loaded_model, task)File "/home/user/.local/lib/python3.8/site-packages/streamlit/legacy_caching/caching.py", line 573, in wrapped_funcreturn get_or_create_cached_value()File "/home/user/.local/lib/python3.8/site-packages/streamlit/legacy_caching/caching.py", line 557, in get_or_create_cached_valuereturn_value = func(*args, **kwargs)File "/home/user/app/app.py", line 81, in inferenceresults = loaded_model.transcribe(path, **options)File "/home/user/.local/lib/python3.8/site-packages/whisper/transcribe.py", line 181, in transcriberesult: DecodingResult = decode_with_fallback(segment)File "/home/user/.local/lib/python3.8/site-packages/whisper/transcribe.py", line 117, in decode_with_fallbackdecode_result = model.decode(segment, options)File "/home/user/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_contextreturn func(*args, **kwargs)File "/home/user/.local/lib/python3.8/site-packages/whisper/decoding.py", line 701, in decoderesult = DecodingTask(model, options).run(mel)File "/home/user/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_contextreturn func(*args, **kwargs)File "/home/user/.local/lib/python3.8/site-packages/whisper/decoding.py", line 633, in runtokens, sum_logprobs, no_speech_probs = self._main_loop(audio_features, tokens)File "/home/user/.local/lib/python3.8/site-packages/whisper/decoding.py", line 588, in _main_looplogits = self.inference.logits(tokens, audio_features)File "/home/user/.local/lib/python3.8/site-packages/whisper/decoding.py", line 145, in logitsreturn self.model.decoder(tokens, audio_features, kv_cache=self.kv_cache)File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_implreturn forward_call(*input, **kwargs)File "/home/user/.local/lib/python3.8/site-packages/whisper/model.py", line 189, in forwardx = block(x, xa, mask=self.mask, kv_cache=kv_cache)File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_implreturn forward_call(*input, **kwargs)File "/home/user/.local/lib/python3.8/site-packages/whisper/model.py", line 124, in forwardx = x + self.attn(self.attn_ln(x), mask=mask, kv_cache=kv_cache)File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_implreturn forward_call(*input, **kwargs)File "/home/user/.local/lib/python3.8/site-packages/whisper/model.py", line 85, in forwardwv = self.qkv_attention(q, k, v, mask)File "/home/user/.local/lib/python3.8/site-packages/whisper/model.py", line 97, in qkv_attentionqk = qk + mask[:n_ctx, :n_ctx]

2

Batuhan_Y OP t1_is39n5l wrote

Ran into same error now. Changing the model size worked somehow. I'm not sure what the problem is.

I tried it on my localhost many times. I think it might be related to huggingface spaces.

4

cheecheepong t1_is3cxb4 wrote

>Ran into same error now. Changing the model size worked somehow. I'm not sure what the problem is.
I tried it on my localhost many times. I think it might be related to huggingface spaces.

Interesting. What model size did you end up using? Did it work eventually on the huggingface spaces?

1

Batuhan_Y OP t1_is3d8cc wrote

Yes it worked on spaces. I've tried tiny, base and small models. Tried with 3 to 5 min. long videos.

2

cheecheepong t1_is3dq0j wrote

Definitely needs more development for robustness but otherwise a great start! I just got it working on a 1:30 video.

1

Batuhan_Y OP t1_is3dxsv wrote

You're right. Also needs more processing power. I'm still working on the app, trying to improve.

Thank you for your interest.

1

inglandation t1_is5ym5t wrote

Did you manage to solve this error? I'm getting it too. My video is unlisted and lasts 10:41 minutes.

I tried switching to the large model but it takes forever, it's still running.

1

Batuhan_Y OP t1_is60gfd wrote

HuggingFace gives 16GB RAM and 8 CPU cores for the spaces, when the app exceeds 16GB it crashes. In order to avoid that kind of situation I need deploy this model to one of the cloud providers with multiple GPU's. I can't afford that kind of heavy processing model at the moment. But I rebuild the hosted space so it works until it exceeds 16GB again :D

Small model works fine most of the time you can try using it.

I put it up on HuggingFace spaces for demonstration purposes.

1

inglandation t1_is61d3u wrote

Okay, thanks! I'd run it locally but it looks like it would be a bit much for my computer.

1

Batuhan_Y OP t1_is627j7 wrote

You can try on hf spaces, its up and running now. If you face an error keep clicking on Transcribe/Translate button (it worked for me on hf spaces :D, no errors on localhost).

1

inglandation t1_is68ong wrote

Okay, thanks! Very useful app btw. I'd be nice if I could somehow replace the autogenerated YouTube subtitles by those. They're much better.

1

Batuhan_Y OP t1_is6guu8 wrote

Do you mean it would be nice to edit the transcript and re-generate the video with that? If so, I actually tried to implement it 2 days ago, but couldn't make it work, I'll be working on that.

1