>How do we account for red shift without knowing distance?

We don't. We know the distance, at least to within some margin of error that is always accounted for.

>AFAIK the shift itself is the only way we can measure distance at cosmological scales.

That isn't accurate. You probably want to do a little bit of reading into the cosmic distance ladder and how it is constructed. Redshift needs to be accounted for at all but the closest distances, and there are close to two dozen different ways of measuring differently-sized distances that cover overlapping distance ranges, and which are all in general agreement with each other within the overlapping ranges, as well as in agreement with the measured redshifts.

Hope that helps!