What about a VAE-GAN? It couples a VAE (encoder-decoder) network with a GAN by sharing weights between the generator and decoder. This way you can you use the encoder to obtain the latent variable of interest.