I would give him one chance to either remove himself, or you remove it without damaging if possible, and place it back onto his own property.

Either way, set up some sort of property marker so he doesn't make that mistake again