There's nothing to "hunker down" from. Dude said his piece and moved on. Probably forgot about your house as soon as he hit the sidewalk. Just keep your doors locked (as you should no matter where you live, unfortunately) and you'll be fine.