I would just give yourself as much extra time as possible. Worst case scenario, the drive is smooth and you spend a bit longer in Manhattan. Not like you'll be stuck kicking rocks in a parking lot.

Also, be sure to take tolls into account. Usually now they send a bill in the mail.