I'm in my early 30s, I would consider myself somewhat nerdy but most of my friends have previously watched anime at least once in their lives. We grew up with toonami and shows like DBZ, Guran Lagaan, and Full Metal Alchemist.

I dont really obsessively follow new anime any more but my gf and I watched Akiba Maid War recently and she died laughing several times in that show. Girlfriend still somewhat follows the story of one piece and her sister which is in her early 20s watches some Yuri stuff.

I think anime watching is relatively common. Maybe as common as people who own xbox in terms of cultural prevalence.