I hadn’t thought about it before, but it does have to do with the ending. Nationalities ending in ‘-n’ get an ‘-s’ added to them, and I think all of the others don’t (at least the ones I could think of).

Examples German - the Germans French - the French English - the English Japanese - the Japanese Swiss- the Swiss

I wouldn’t be surprised to find out there are exceptions though (like everything else in English).