PolygonAndPixel2 t1_ist25ao wrote
That sounds interesting. I didn’t get to read it completely yet but I have a couple of questions:
- You say "a mathematical program". What do you mean by that or rather what is a program that is not mathematical? Any computer program is just a concatenation of basic functions which can be derived. Throw in the chain rule and you can use AD for any program where the gradients are exact for the execution path.
- If I understand that right then p is a random variable (or rather the probability of a random variable to take a given value) that changes the outcome in a discontinuous way. Is it correct to say that the execution path of the program changes with different outcomes for p, i.e., if(random_event(p)) {return 1;} else {return 0;}? Or is this a different problem?
- I didn’t take a look into your code (and Julia isn’t my first language) but can you estimate how much work it is to incorporate this kind of AD in existing AD tools like CoDiPack?
ChrisRackauckas OP t1_ist3bu8 wrote
> What do you mean by that or rather what is a program that is not mathematical?
If it outputs strings or code it may not work with this method. It should output numbers in a way that has a well-defined (differentiable) expectation.
> Is it correct to say that the execution path of the program changes with different outcomes for p, i.e., if(random_event(p)) {return 1;} else {return 0;}? Or is this a different problem?
It can. One of the examples is differentiation of an inhomogeneous random walk, which is a stress test of doing this kind of branch handling.
> but can you estimate how much work it is to incorporate this kind of AD in existing AD tools like CoDiPack?
That's hard to say. I would say it wouldn't be "too hard", though it may be hard to add this without overhead for "normal" cases? It would make the code more complicated but the standard cases are just a special case here, so it should be fine.
PolygonAndPixel2 t1_ist8frv wrote
Thanks for your answer. That all makes sense and I'm going to take a closer look at the random walk.
Viewing a single comment thread. View all comments