Viewing a single comment thread. View all comments

dopadelic t1_izh5hi6 wrote

It's been years since I've implemented ODE solvers in my engineering courses, but I recall that ODEs are inherently not parallelizable since the subsequent timesteps require the current time steps to be solved. I've only worked with Euler's and Newton's method though.

1

martenlienen OP t1_izi71ti wrote

You are correct except for parallel-in-time integration methods that we also mention in the paper. But the "parallel" in the title refers to solving multiple ODEs in parallel independently which is contrary to what is currently done in ML. At the moment, training on a batch of ODEs means that you treat the batch as one large ODE that is solved jointly. torchode solves them independently from each other but still in parallel by tracking a separate current state, step size, etc. for each sample.

3

fhchl t1_izjd2vy wrote

+1 for the paper on parallel in time! Are there any implementation of those algorithms for torchode or diffrax out there?

1

martenlienen OP t1_izk7bk3 wrote

I am not aware of any but would be very interested if you find one

1