IIRC, station agents were required to stay in the booth at all times because they would get lured out and have their booths robbed of all the tokens and cash. In the old days, robbers would set fire to the booths to force the clerks out for the robbery.

With both OMNY and Metrocard, I think particularly more as we've moved cashless with OMNY, there's nothing really of value in the token booths anymore.