Yes and no. If you're trying to move data from many devices through a router, a faster connection gets the data through faster and allows the next set of packets to be sent. The latency increases when there router can't complete a request fast enough to respond to the other device.

To out it another way, 5 people are standing around. 3 people (Person A B and C) are handing items to 1 person (Person Y) who is handing them to the next (Person Z). The longer it takes for person Y to hand Person As things to person Z, the longer B and C have to wait. That is their latency.

A faster router cuts latency, and a bigger pipe also reduces latency.