Yes. Like I don't know any of the people who compete in horse jumping and I suspect most of the world doesn't. But they're all famous. Someone like David Chalmbers is a complete and total rockstar in philosophy... but broadly the world doesn't know who he is.

The term celebrity simply refers to being known. Sometimes the word is modified "local celebrity" to refer to people who are well known only in your town or city.