Submitted by thedarklord176 t3_10vqxtp in MachineLearning
I know Python is the primary choice because it’s a simpler language for data scientists to use and a lot of ML libraries are made for Python. But, if you look at this, it is extremely inefficient with energy. And everyone knows big models like ChatGPT cost a ton to keep running. Maybe a more efficient but not too difficult language, like C#, is something we should consider giving more attention in ML?
Most inference/mlops solutions don’t really use Python despite being used to develop the model.
Stuff like Nvidia’s Triton inference server is used for speed up.
Pytorch is written in C++ and CUDA.
Python is really just an interface, with a minimal contribution to the execution time.
But isn’t everything in Python from C? By that logic I’d think that would make no difference because it’s still Python. Not saying you’re wrong, I don’t work in AI I’m just curious
[deleted]
[removed]
The critical factor to consider is whether the computation spends time in the python code or C/C++.
Many of the python language constructs are quite slow, and this is why libraries like numpy exist. The program spends relatively little time in the python code which is merely acting as an interpreted, rapid-to-modify "glue" between the compiled C/C++ library functions.
In the case of tensorflow and pytorch virtually all the computation is being done in C/C++ and python is basically acting as a highly flexible configuration language to do setup.
The answer is sort of but not really. A lot of the common packages in ML And data science are in python, but the computations happen in C/C++, Fortran, Scala and others to name a few.
All the computation is happening on the GPU. Python is just making a bunch of calls to the GPU drivers.
Researchers spend a lot of time making neural networks as fast as possible. If switching to another language would have given a substantial speed boost, they would have done it already.
Call me when logistic regression can generate a realistic detailed digital painting by greg rutkowski.
Perhaps when we can grow human-like brains and interface with them?
LOL spit my coffee out
The 'busy' core stuff will be written in a low level high efficiency language.
‘Hey, I know nothing about what I’m saying but…’
It’s not python, but the underlying amount of data, i.e. Tensor size and interdependence.
It matters way more what you doing than the language itself. I can easily make an infinite loop C program that uses more energy than a Haskell program.
Good points but python is NOT the problem.
Lol
No. All the energy intensive computations occur on dedicated hardware like GPU/TPU. These run a compiled instruction set that would not benefit from using a different language frontend. You have to tackle energy efficiency at the hardware level, and in this respect, the number of flops/watt has steadily gone up over the years. The ML tasks always grow to fill the extra computational efficiency though. At this point, progress in ML is the fruit of increased energy efficiency, not energy cost.
[deleted]
You can minimize the energy costs by hosting the primary interface on a COBOL mainframe, with eleventeen node-processors.
PredictorX1 t1_j7izzeg wrote
Just to be clear, deep learning is the energy consumer. "Shallow" machine learning (logistic regression, multilayer perceptron, tree induction, etc.) and related technologies cost pennies to fit.