Viewing a single comment thread. View all comments

plocco-tocco t1_jao43p9 wrote

Reply to comment by mikonvergence in [P] A minimal framework for image diffusion (including high-resolution) by mikonvergence

Thanks for the input. I have seen some papers claiming SOTA in image segmentation using diffusion so I am also curious to see how they perform.

I have another question, if you don't mind. How difficult would it be to extend the code for image-to-image translation so that it works on 3D data (64x64x64 for example)?

2

mikonvergence OP t1_jao5zyg wrote

There could be a few simple solutions to extending this to 64x64x64 and each would have certain pros and cons. The two key decisions to make are in regards to the data format (perhaps there is a way to compress/reformat data so it's more digestible than direct 64x64x64) and in regards to the type of the underlying architecture (most importantly, do we use a 2D or 3D CNN, or a differnt type of topology altogether).

A trivial approach would be to use a 2D architecture with 64 channels instead of the usual 3, which could be very easily implemented with the existing framework. I predict that would be quite hard to train, however, though you might still try.

This is an area of active research (beyond DreamFusion and other popular papers I'm not very familiar with it), so exploring different solutions to this is still required, and if you discover something that works reasonably well then that will be really exciting!

3