SnooMarzipans3021 t1_jdzehws wrote on March 28, 2023 at 9:08 AM

Hello, does anyone have suggestions on how to do guided image upsacling?
Basically I have 6000x6000 image which im unable to load in network because of GPU memory. I had this idea of resizing the image to something like 1500x1500 and then upscaling it back to 6000x6000. But I have to do it without losing details and dont want to use super resolution models (im ofraid they will hallucinate and inpaint). If I already have the ground truth resolution, how can I use it to guide the upscaling?

GirlScoutCookieGrow t1_jdzv85v wrote on March 28, 2023 at 12:26 PM

I'm not sure I understand what you hope to accomplish. If you have the full size image, why do you want to downscale and upscale? This won't help you fit the full image on the GPU

SnooMarzipans3021 t1_je3x9ah wrote on March 29, 2023 at 6:07 AM

Im unable to load full res image into the model and train it even with batch size 1 and all sorts of optimizations. My idea is to add two small modules to my network. One at the front which downscales the image and one at the back which upscales the image.

The problem here is the upscaling, it will need to be some sort of super resolution model.

alyflex t1_je4uq2y wrote on March 29, 2023 at 12:59 PM

Another solution is to use a memory efficient neural network: https://arxiv.org/pdf/1905.10484.pdf With this type of neural network you can easily fit those size images into your neural network. However the problem with them is that they are very difficult to make (you manually have to code up the backpropagation). So depending on your math proficiency and ambitions this might just be too much.

SnooMarzipans3021 t1_je96o53 wrote on March 30, 2023 at 9:55 AM

Thank you for the suggestion. At first glance it does seem overwhelming, I will check it out. The problem im solving has to be rapidly tested, i will run out of time implementing this.