You may be required to have an escrow account, particularly if you're below a certain percentage of equity in your house. Lenders want to make sure that insurance is being kept up to date and taxes are being paid on properties that they have a significant amount of stake in.

But if you're allowed to drop it, then you may want to, assuming that you're organized enough and financially disciplined enough to weather a big tax payment every year instead of monthly installments. Your insurance company may give you a discount by paying your premiums annually rather than monthly, which you typically won't get when paying via escrow.