[D] Is Pytorch Lightning + Wandb a good combination for research?

Submitted by gokulPRO t3_11oh450 in MachineLearning

I am planning on moving from Pytorch to Lightning for more structured research. Is lightning + Wandb a good combination in the long run for research experimentations? Which tech stack do you use for research? And for code version control, what do you use? Also, for Kaggle, is it a good choice to use lightning and Wandb? Thank you

5

Comments

You must log in or register to comment.

shahaff32 t1_jbu0zmr wrote

In our research we ran into issues with Lightning. It is especially annoying when designing non-trivial layers or optimizers. Also, it is much harder to convert the code back to pure pytorch.

For example, in a recent peoject, Lightning caused each forward-backward to operate twice on each batch because we used a combination of two optimizers for a specific reason. And now we are working on rewriting everything without Lightning.

14

ghostfuckbuddy t1_jbwobij wrote

That just sounds like a bug. It might take a lot less effort to report it for patching than rewrite all your own code.

2

shahaff32 t1_jbxy6vj wrote

It might be a bug, and it might also be the case that we are the "idiot users" that don't get how to use the package correctly.

Either way, we decided to continue without Lightning (because of that reason and a few others), and therefore we need the code in pure PyTorch for future works that rely on it.

5

AerysSk t1_jbt5j4l wrote

My main problem is Lightning itself. I don’t find the flexibility of it like pytorch. Tried migrate an old code, gave up along the way, and still using pytorch now.

5

maxToTheJ t1_jbtjn8w wrote

The structure of pytorch-lightning makes sense for like 99% of workflows which is why other similar libraries use similar structures for what the generic DL process is like. If you are having trouble migrating you are either in a rare use case or really should think about how you structured your code

2

koolaidman123 t1_jbtkuif wrote

there are some fairly annoying things with pytorch lightning, and somethings are definitely harder to do in lightning due to how it's structured. but overall i find for practical purposes i've been liking lightning a lot more than pytorch + accelerate, especially now you can basically use colossal ai with lightning over deepspeed

4

ri212 t1_jbsohhm wrote

I can't say for sure whether it is the best combination for research in the long run, but if you do go down that route I have found this template very useful

3

paulgavrikov t1_jc4gnml wrote

I ditched lightning after a while because I spent more too much time fixing it than doing productive stuff. But I regularly use wandb to track experiments and it’s mostly great!

3