Restrooms are dirty places, and there’s a lot you have to touch when using them - the toilet seat, the flush lever/button, etc.

You’re better off washing your hands as much as possible, but definitely after using the restroom.