for half, $12k, (and get two) I would go with something AMD with PCIe 5.0 and 4x GPU slots. And I would personally get used 3090, but if you want warantee new 4090. I haven't RMA anything for a long time but be warned could take over a year to get a replacement card. 4090 might not support the new up in coming 4bit LLM mode. Not sure.
Server rack, with 2.5gigE ethernet, 1-to-1 VRAM to RAM (with 20% more CPU ram), NVM 2tb or better, and 40-60TB storage.
deepstatefarm t1_j6oy6ps wrote
Reply to [D] DL university research PC suggestions? by seanrescs
for half, $12k, (and get two) I would go with something AMD with PCIe 5.0 and 4x GPU slots. And I would personally get used 3090, but if you want warantee new 4090. I haven't RMA anything for a long time but be warned could take over a year to get a replacement card. 4090 might not support the new up in coming 4bit LLM mode. Not sure.
Server rack, with 2.5gigE ethernet, 1-to-1 VRAM to RAM (with 20% more CPU ram), NVM 2tb or better, and 40-60TB storage.