Mostly people won't judge your relationship as long as your relationship isn't obviously, to a 3rd party perspective, suffering from an unaddressed power imbalance.

If your partner is immature or left a relationship with someone in his age group to be with you, or never dates women his own age, he might get criticized for that too.

There are always outsiders who will judge a relationship. They aren't worth listening to, usually, as long as you keep your eyes open to the potential pitfalls yourself and steer clear of them.