You make an interesting point. I believe (possibly wrongly) that you are saying that one should only consider mass that is within the local Hubble volume to compute the force on test galaxies, in which case we are at the gravitational center of the frame (because it truly is a finite sphere centered on us). One should not compute the force from the test galaxy's frame, since it is non-inertial to us. I think I can buy that argument. To me, it hinges on the finiteness of the Hubble volume and the speed of gravity. If the universe is infinite and if Newtonian gravity acted instantly, I think one could still argue that the test galaxy would feel no gravitational force, even in our inertial frame.