A star or star system as bright as Alpha Centauri passing the Sun at 0.2 light years outshines all other stars combined. It's almost certain some system passed closer than that at some point in the past, but we can't reconstruct it today.

If you count even shorter periods then a nearby supernova at some point in the past wins.

Other galaxies would not have made the sky brighter, at least not 4 billion years ago. Andromeda is the closest galaxy and it is barely visible to the naked eye as is. Cosmic expansion means other galaxies would have been closer to us in the past, but not as close as Andromeda. Maybe if you go back like 13 billion years to when galaxies were being first formed, the cosmic neighborhood would have been 8 times denser back then, and maybe multiple galaxies would have been visible in the sky, including some that Milky Way has since collided with.

What would have been visible 4 billion years ago is more brighter stars. The brightest stars are the heaviest but also the most short-lived. They are only visible for a few million years after they are formed, but make up most of the starlight reaching us. The number of heavy bright stars is proportional to number of new stars formed, but star formation in galaxies is always slowing down... There were more bright stars in the past, and there will never be as many bright stars as there are in the sky today, except for a short period of time in 5 billion years when Andromeda crashes into the Milky Way, causing a temporary burst in new star formation.

The early Earth has a closer, and hence larger and brighter Moon. The Moon drifts away at about 3.8cm/year. Given that the Moon itself is the second brightest object in the sky even a nearby passing star would likely be fainter than the Moon. So the brightest regular night skies would be a night with a full Moon when it was at perigee soon after formation.

Studies on the formation of the Moon also suggest the Earth may have had a ring system shortly after the collision with Theia, a ring system would also have made the sky brighter while it existed.

And of course shortly before the collision a massive Theia (Mars size) reflecting sunlight in the night sky would have been even brighter than the non yet existent Moon, but that wasn't a longer term or repeatable event.

And all this would have drowned out starlight.

In average, as the earlier we go back to the past, the brighter the night sky would have been since everything is accelerating away from each other in the universe.

Or any super novae event close to earth would have brightened the sky significantly since they were observed from cave men to early astronomers in China.

I wonder if the time period between the formation of the moon to the settlement of the moon into its current stable had bright night sky dominated by much closer moon.

[removed]

[removed]