I never answer if I don't know the number. If it's important, they'll leave a message.

I also run RoboKiller on my phone to screen calls. If I know someone is calling (like for a Dr. appointment) I'll disable it for a few hours to ensure it comes in.