Viewing a single comment thread. View all comments

Thalesian t1_iun1w8t wrote

TBH the value of the M1 Macs is the RAM. Because CPU, GPU, and RAM are all on the same chip, you don’t have VRAM - just RAM. Not such a big deal on an M1 with 16 Gb, but perhaps something to consider when thinking about the kinds of models you can build on the M1 Max with 64 Gb or M1 Ultra with 128 Gb. The unified memory framework Apple uses is weird, and I haven’t seen aggressive testing of its limits. For example - does the M1 Ultra have 128 GB for GPUs, or the 64 Gb of each M1 Max component? Either way, if it is slower than a 3090, just use absurd batch sizes paired with a slightly faster learning rate.

That said, the big problem I’ve found is with OpenMP. It doesn’t adjust well to the dynamic CPU use Apple wants, and this leads to frequent crashes and/or low thread counts on anything more complex than an M1/M2 . With the RAM - CPU link XGBoost is an absolute beast until it crashes within a few minutes. There may be some leeway in setting threads lower, but hopefully development will catch up with the new chipsets, particularly since Intel is also pursuing the efficiency/performance core division.