Yeah I've read that paper you linked, but I have not really delved into trying to implement conditional SGM code myself (I've done work with conditional generative models in terms of GANs, VAEs, etc). I am also interested in lower dimensional data than images, so your code looked like a good starting point.

After some more reading, I'll give adding conditional capabilities to your code a shot.