Today, sure, but the original use of binary in computer systems came from "powered or unpowered," and binary data is stored or transmitted in a variety of ways and represents two opposing states: on or off, positive or negative, up or down, high frequency or low frequency, 0 degrees and 180 degrees phase states, black or white, etc.

Binary arithmetic is what it does with the information, while the binary itself is how the data itself is represented.