As u/ZombieRickyB said, the short answer is that it distorts distances to the point that you can't rely on them in downstream clustering.

There are two papers that do a really good deep dive into it:

This one: https://www.biorxiv.org/content/10.1101/2021.08.25.457696v1 where they both show that the distances pretty much have to be distorted and that the minimizer of the objective is such that you can make the output look pretty much like anything while minimizing.

And this one: https://jmlr.org/papers/volume22/20-1061/20-1061.pdf that studies which aspects of the objective functions of these methods affect the structure at different scales.