It doesn't matter if they're friendly or not. IF It's on your property, Remove it. Let the burden of proof fall on them. Have your papers in your hand when the police are called, they will walk away and your neighbor will have to go to court.