I can't go into every other website in the world and force them to send me GPG-encrypted emails. The emails are going to arrive unencrypted.

You could use a provider that stores your emails encrypted. But when the app requests the emails using IMAP, they're coming unencrypted, otherwise the email app wouldn't be able to read it and show it to you.

If the email app gets them unencrypted... it was all for nothing. So how does it solve the problem mentioned?