Greetings!

Recently I was asked about a budget AI / ML workload, and decided to test it against some of my own lab GPUs.

I'll be adding more tests, and benchmarks over time, but below is a link to my website where I covered it. As well as the code I wrote to benchmark them.

Hopefully this helps someone out there. :-)

https://www.zb-c.tech/2022/12/26/pytorch-drag-race-tesla-k80-performance/

Comments

You must log in or register to comment.

Tom_Neverwinter t1_j1x5gyp wrote on December 28, 2022 at 1:30 AM

#1,104,973

Bought a few Tesla m40s now I need a motherboard with enough gpu slots. 1x slots have too much of a bottleneck whoops.

zveroboy152 OP t1_j1xi8km wrote on December 28, 2022 at 3:09 AM

#1,106,757

Replying to Tom_Neverwinter (#1,104,973)

Hi Tom,

I ran into that problem too. I ended up getting one of these for my Tesla's and AMD MI25's:

SUPERMICRO 1027GR-TRF

(not sponsored)

https://www.theserverstore.com/Supermicro-SuperServer-1027GR-TRFT-1U-GPU-Server

It's been great for my GPU workloads.

learn-deeply t1_j1xjm5p wrote on December 28, 2022 at 3:20 AM

#1,106,975

This benchmark is not representative of real models, making the comparison invalid. The model has ~5,000 parameters, while the smallest resnet (18) has 10 million parameters. You're essentially just comparing the overhead of PyTorch and CUDA, which isn't saying anything about the actual performance of the different GPUs.

zveroboy152 OP t1_j1xrwio wrote on December 28, 2022 at 4:29 AM

#1,108,155

Replying to learn-deeply (#1,106,975)

It sounds like I have a lot to learn for PyTorch then. :-)

I'll work on re-working the script to reflect better numbers. If you have any pointers, or ideas, I'd love to hear it.

learn-deeply t1_j1yd1z6 wrote on December 28, 2022 at 8:27 AM

#1,111,126

Replying to zveroboy152 (#1,108,155)

TorchBench (https://github.com/pytorch/benchmark) is used by PyTorch core developers to test performance across a wide variety of models. I've never used it personally though.

yaosio t1_j20mi1e wrote on December 28, 2022 at 8:14 PM

#1,125,369

Your conclusion is that the Tesla K80 is a great value, but your benchmark doesn't show that. It shows that the Tesla K80 12 GB is slower than the RTX 3070 TI (with unknown vram) on one synthetic benchmark. You don't provide performance per dollar. You also say it scales up well across multiple cards but don't show that.