I'm new to diffusion model but I've used few times GANs for sim2real domain adaptation. Now I'm wondering if there is already some project on this topic with diffusion models that might show better performance.

Comments

You must log in or register to comment.

AljoSt t1_isjbi3f wrote on October 16, 2022 at 12:45 PM

#115,922

Have not seen any projects, but tried it myself. Definitely looks promising, but didn't have much time to investigate fully

SeucheAchat9115 t1_isjcivg wrote on October 16, 2022 at 12:54 PM

#115,960

Would be interesting for a comparison. Since Diffusion Based Models seems like performing bad for faces, it might be interesting to see it perform image to image for synth to real like cycle-gan

Tomsen1410 t1_isjkhp1 wrote on October 16, 2022 at 2:02 PM

#116,374

Unlike GANs however, diffusion models are trained in a supervised fashion (they need an image which gets denoised). Not sure how one would use them for unsupervised translation like e.g. CycleGAN.

SeucheAchat9115 t1_iskb0z2 wrote on October 16, 2022 at 5:04 PM

#117,618

Replying to Tomsen1410 (#116,374)

Maybe with artificial noise? But I am not familiar with how diffusion is really trained, maybe thats how diffusion works.

MostlyRocketScience t1_isksjgk wrote on October 16, 2022 at 6:54 PM

#118,335

Replying to Tomsen1410 (#116,374)

Google has a nice paper on text-guided image2image translation by inferring the (random) init image and changing the prompt: https://github.com/google/prompt-to-prompt

new_name_who_dis_ t1_ispgfn7 wrote on October 17, 2022 at 6:51 PM

#126,305

I'm currently working on a similar problem and my current approach is to add some noise (but not a lot) on the image in domain A, and then denoise with network trained on generating images in domain B.

It's not perfect, but it works. I'd be interested to hear more discussion of this topic.

cygn t1_isx84ko wrote on October 19, 2022 at 11:16 AM

#141,080

Replying to new_name_who_dis_ (#126,305)

could you show some images please?

[deleted] t1_isx8wl0 wrote on October 19, 2022 at 11:24 AM

#141,122

[deleted]

new_name_who_dis_ t1_isy3qr2 wrote on October 19, 2022 at 3:29 PM

#143,340

Replying to cygn (#141,080)

It's proprietary data so I can't. If you have a public dataset (or I guess 2 for style/domain transfer), I could run my code on it and get back to you.

cygn t1_it25agy wrote on October 20, 2022 at 12:04 PM

#152,185

Replying to new_name_who_dis_ (#143,340)

you could use one of the datasets that are listed here: https://paperswithcode.com/task/domain-adaptation

Office-31 for example looks quite practical. It has product images from Amazon, DSLR and webcam images. The problems I'd like to solve are similar. Take good images of plants with diseases and adapt them to images taken from users with smartphones with lots of quality issues.