It's really not a problem. The only problem is men in their 40s/50s who chronically only date women in and around their 20s. It would be a problem for you because those men aren't attracted to women over 35 or they know older women just catch on to their bullshit faster. Either way, you'll age out of the relationship, they'll cheat or whatever and you've wasted years with them. If your guy isn't one of these dudes then I don't see the issue