TruthAndDiscipline t1_j9avex4 wrote on February 20, 2023 at 4:09 PM

Reply to comment by DevarshTare in [D] What matters while running models? by DevarshTare

I'm using a 3060 (no ti) with 12GB VRAM and train locally as well. Performance is fine, too.

DevarshTare OP t1_j9b4u7s wrote on February 20, 2023 at 5:12 PM

Thats interesting. I was considering that purchase since it makes sense to run larger datasets or models on the rtx 3060. But the Tensor cores were significantly lower. The GPU would definitely run much larger models but at a lower speed I assume?

How has your experience been with larger models? Especially video or image based models ?

ggf31416 t1_j9clwen wrote on February 20, 2023 at 10:59 PM

I actually have a 3060 too, in theory a 3060ti should be up to 30% faster, but most of the times the 3060 is fast enough and faster than any T4.

For making a few images on stable diffusion maybe the difference will be 15 vs 20 seconds, for running whisper on several hours of audio it could be 45 minutes vs 1 hour. The difference will only matter if the model is optimized to fully use the GPU in the first place.

DevarshTare OP t1_j9ngofc wrote on February 23, 2023 at 5:45 AM

I've seen the same across multiple threads now, the VRAM does make a difference in being able to run a model or having to optimize it. This has been really helpful, thanks a lot guys!