I feel like your overall message is fine, but this idea that people just "grow out of being an atheist/agnostic" doesn't sit right with me. Many people are atheist or agnostic and stay that way forever, they won't necessarily "turn back" to religion once they get older.

I just say "I like all genres but am less likely to read x genres".

It's easier for me to say what I don't read then try and remember every type of book I like to read on the spot.

To get better results you might want to ask this question on r/RomanceBooks.