schwagggg t1_isxz4cw wrote on October 19, 2022 at 2:58 PM

Reply to comment by ChrisRackauckas in [P] Stochastic Differentiable Programming: Unbiased Automatic Differentiation for Discrete Stochastic Programs (such as particle filters, agent-based models, and more!) by ChrisRackauckas

then this sounds like measure valued derivative a bit? you perturb then calculate derivative. then wouldn’t this be at least O(D) expensive for one layer, and O(LD) for L layers of D dim rvs?

ChrisRackauckas OP t1_isy96fg wrote on October 19, 2022 at 4:05 PM

O(LD) yes, so yeah you want reverse mode O(L+D) but without bias and at a low variance, and that's the next steps here.