The format itself is quite flawed as it is 1-bit encoding with very high sampling rate.

The high sampling rate is required to apply noise shaping to move the noise outside of the audible band (over 20 kHz) and increase the signal-to-noise ration within the audible band.

DSD files have a lot of noise above 20 kHz. In best case you won't hear it.
In worst case, that noise can intermodulate with the audible frequencies, causing the noise to be added to the audible band, shifted in frequency and lowered in volume to the point it could cause some sensation of brightness.

That's theoretical.

You could convert the DSD file into high bitrate PCM (24/44.1 would remove the ultrasonic noise, 24/192 would keep some of it) and do a volume-matched double-blind test.