I may do better going through a vendor, honestly. System76 doesn't do dual 4090s, but I think Exxact does.

I am also interested! I'm going in circles trying to decide, and I think a 2x4090 would be the best for me, too. Though I'm more likely to have it built at MicroCenter to save myself the stress.

How would a 2x A5000 system differ from a single A6000 in actual use? Are your cards treated as a single card by the software?

Here is a thread where we talked about this with GeForce cards. It's not treated as a single GPU and apparently you still need to parallelize. At least that's what I was told in that thread.

Thanks. So it really isn't the same as how the Quadro cards share vram. That's really confusing.

Huh. If it requires parallelization then why is the 3090 singled out as the one consumer GeForce card that is capable of memory pooling? It just seems weird. What exactly is memory pooling then, that the 3090 is capable of? I'm clearly confused.

edit: I did find this from Puget that says

> For example, a system with 2x GeForce RTX 3090 GPUs would have 48GB of total VRAM

So it's possible to pool memory with a pair of 3090s. But I'm not sure how it's done in practice.

So this means you cannot access 48GB of vRAM with a pair of 3090s and nvlink, with TF and PyTorch? I could have sworn I've seen that it's possible. Not a deal breaker for me, but a bummer to be sure. I will likely end up with an a6000 instead, then, which isn't as fast but has that sweet vram.

I think 2x 3090 will pool memory with nvlink, but not treat them as a single card. I think it depends on the software you're using. I'm pretty sure pytorch and tensorflow are able to take advantage of memory pooling. But the 3090 is the last GeForce card that will allow it. I hope somebody else comes into the thread with some examples of how to use it, because I can't seem to find any online.

I'm far from an expert but remember the 4090s are powerful but won't pool memory. I'm actually looking into a lighter setup than you with either an A6000 or, more likely, 2x 3090s with nvlink so I can get access to 48GB of vRAM. While the 4090 is much faster, you won't have access to as much vRAM. But if you can make do with 24GB and/or can parallelize your model, 2x 4090s would be awesome.

edit: Just re-read your post and I see I missed you mention parallelizing already. Still, if you can manage, 2x 4090 seems incredibly fast. I would do that if it was me, but I don't care much about computer vision.

What's a good way to get started with reinforcement learning, in particular for writing board game AIs? I have David Silver's videos on my to-watch list, but I'd prefer a good intro book I can work with, similar to how Hands On ML with scikit-Learn, etc. is a good intro to ML in general. I found that book really readable.

Yes, this is what made me think of how GANs are trained. I'll see what's out there so I don't have to reinvent the wheel, and tweak as needed!

Yes, I have seen the hide and seek results and I didn't even consider it! That's a great example.

Thanks!

Cool, thanks for the reply. With chess, I always assumed it was just examining the state as a pair (board,turn), regardless of who went first. I study the mathematics of combinatorial games and it's rare to ever consider who moves first, as it's almost always more interesting to determine the best move for any given game state.

Do you have any reading suggestions for understanding AlphaZero? I've read surface level/popular articles, but I'm a mathematician and would like to dig deeper into it. And, of course, learn how to apply it in my case.

There are a number of benchmarks out there already for that

I guess it's sharable via nvlink. Usually a pair of GeForce cards can't combine vram.

What about 2x3090 vs 1x4090? Cost vs. performance?