Viewing a single comment thread. View all comments

jackilion t1_j63e6ah wrote

Reply to comment by Blutorangensaft in [D] Quantitative measure for smoothness of NLP autoencoder latent space by Blutorangensaft

There is no reason to assume your latent space will be smooth by itself. I remember a paper for image generation that had techniques for smoothing out the latent space that can be applied during training:

https://arxiv.org/abs/2106.09016

​

It's about GANs, not autoencoders, but maybe you can find some ideas in there.

1

Blutorangensaft OP t1_j654qyd wrote

Thank you for the reference, it looks very promising. I've heard of ways to smooth the latent space through Lipschitz regularisation, but then got disappointed again when I read "ah well it's just layer normalisation". So many things in ML come in a different appearance and actually mean the same thing once you implement them.

1