jellyfishwhisperer t1_iu4twl9 wrote on October 28, 2022 at 3:40 PM

Reply to comment by DigThatData in [D] DL Practitioners, Do You Use Layer Visualization Tools s.a GradCam in Your Process? by DisWastingMyTime

Great list. To add, in the CV space you should be very careful with many "xai" methods. Usually they're just fancy edge detectors. Been Kim is pretty good on this stuff.

https://arxiv.org/abs/1810.03292

DigThatData t1_iue7pne wrote on October 30, 2022 at 5:38 PM

very thought provoking stuff! I wonder if maybe an alternative interpretation of these observations might be something along the lines of deep image prior, i.e. maybe randomly initialized deep architectures are capable of performing edge detection just by virtue of how the gradient responds to the stacked operators?

jellyfishwhisperer t1_iuisl72 wrote on October 31, 2022 at 5:25 PM

That's about right. Convolution priors in particular lend themselves to edge detection. CV xai is weird in general though so I've stepped back a bit. Is a good explanation one that looks good or one that is faithful to the model or what? Everyone disagrees. So Ive moved to inputs with interpretable features (text, tables, science, etc).