Lived in Brooklyn half my life, and the other half in Manhattan. I’m sure I’ve heard them described by color a couple of times, but 99% of the time it’s by trains running on it (especially since with express/local color alone doesn’t tell you if the train stops there or bypasses a given station).

People who are New Yorkers and live in Manhattan use the numbers and letters. I live well above Midtown. If someone had to go to South Ferry, I'd never say, "Hop on the red line.: I'd tell them to take the No. 1 Local to the first express stop, and then to take the 2 or 3 Express to Chambers Street and then to switch back to the No. 1. If you took the No. 1 all the way down the trip would take an hour. If you took only the 2 or the 3 you'd have an unnecessary walk to South Ferry, assuming you knew which stop to get off at before you landed in Brooklyn.