Part of it is just a matter of deep lore, characterization, and proper world building.

People familiar with the broader context of Lord of the Rings know Gandalf is a Maia, and basically has all kind of god-like abilities. But he was sent to aid the Free Peoples, not become their lord or hero. He was explicitly forbidden from using his powers except in vague and undefined contexts.

He notably only really uses magic through the stories when faced with higher evils like the Balrog, Saruman, or the Nazgul. At other times, his efforts are physical or restricted to advising the course of events. And of those evils he uses his powers against, only the Balrog is one that he outright defeats himself.

Thing is most people know and criticize Gandalf by his clones in subsequent fantasy, which lack explanations for why the powerful wizard doesn't do powerful wizard stuff.