Binary was chosen for computers because it was easy to represent with transistors, 1 and 0 represent on and off states for a gate or a transistor.

Creating computers that use 3 positions (trinary) or multiple digits for bit like 0,1,2,3 are possible but the limitation is the hardware itself. You need a single stage electronic device (like a transistor) that can exist in multiple different states to represent those digits.

Such a device could be called a transtator, a replacement for a transistor that can exist in multiple different states. A Quantum computer could be based on such a device.

In theory such a computer would be able to perform significantly denser calculations and have higher speed. But our entire ecosystem of instruction sets and operating systems would also need to be completely redone. So it would have to be worth the significant effort.