Viewing a single comment thread. View all comments

dasayan05 t1_ir4kcfg wrote

Reply to comment by golljj in [R] DDIM Reconstruction Confusion by [deleted]

Sorry, but that's not really the correct interpretation. The "forward process" is not the encoder -- it's a stochastic process. The encoder is the "reverse of eq.14", which is integrating the ODE in eq.14 backwards in time -- that is not same as the "forward process".

1

golljj t1_ir930ya wrote

Do you mean that the forward process is what only adding pure gaussian noise with different $\beta$ to $x_0$ to obtain $x_T$, but the encoder here is adding the noise predicted by the estimator $\epsilon_{\theta}(x_t)$ to $x_t$?

1

dasayan05 t1_ir9d9s3 wrote

Your first part of the statement is correct -- that is called the "forward process" and it is only needed at training time.

Yes, the encoder in DDIM is basically adding a predicted-noise to travel back to x_T -- it's more like the "reverse of the reverse process", but we can't really call it the "forward process", can we? For example, the true "forward process" is almost entire random and you can skip to any x_t by re-parameterization. This isn't true for DDIM's "reverse of the reverse process" -- it must be sequential and deterministic.

1