Viewing a single comment thread. View all comments

Zealousideal_Low1287 t1_j0zqyr2 wrote

Even if you wrote in a language like Rust, Go, C, C++ you wouldn’t avoid ‘calling into a framework’ syndrome.

There are several reasons why things like the compute kernels available in PyTorch are fast / optimised, and it’s not as simple as they are compiled. Operations are written efficiently with hardware and memory access patterns in mind, and we also have to ensure we do things like correctly implement the API for backward mode autodiff.

If you wanted to swap over to writing all of this in bare C++ it would undoubtedly be much slower than using PyTorch. We use Python because it’s a nice convenient language for calling into the framework. The overhead from this usually isn’t particularly significant. If you have a use-case where it is significant then sure use C++.

22

vprokopev OP t1_j0zu1ke wrote

But I do want to use pytorch, I like it very much.

I just usually have a lot of specific modifications to data and find myself avoiding native python and loops/indexing, because it makes things way slower.

It would still be slower if I implemented it in C++ then in pytorch, but at least not like way slower and would not create a bottleneck

−1

Zealousideal_Low1287 t1_j0zuq39 wrote

I have no idea what you’re suggesting. Use C++ instead of vectorising properly and using PyTorch? Do you currently do much compute ‘outside’ PyTorch?

8

vprokopev OP t1_j0zy68v wrote

Mostly use vectorized pytorch operations.

Sometimes use just native loops and indexing.

Yes, unfortunately there are specific data preprocessing cases where I have to do stuff outside of pytorch, it's just more convenient.

And even when mostly using pytorch I still want the freedom to just use native functionality of a language without a huge hit to a performance.

But I know pytorch vectorized ops will still be faster and are suitable for majority of tasks

0

Zealousideal_Low1287 t1_j0zycf1 wrote

And you feel if you wrote raw C++ it would be as fast as the PyTorch ops, or you seek to replace the Python part, or something else?

6

vprokopev OP t1_j0zzfpg wrote

Just seek to replace a python part with something that is slower (obviously) but not like way slower then Pytorch vectorized ops.

And to have freedom to use more native structures and a bit less thinking about how to vectorize every algorithm possible

1