It's irritating but if you refuse to pay I can imagine the broker being annoyed and pointing out to the landlord that your lease is considerably below market (which it probably is at this point). My rent is going up $50 per month so If I was offered the chance for a one off payment of $50 instead I'd bite their hand off.