That's where the radioactive dating comes in. It uses the half life of known elements to measure how long ago that ice formed, and those 2 measurements together can be used to figure out the environment at a certain time.

There are lots of clues. One of the simpler ones is counting seasons. I've only seen ice cores from the Arctic, which don't go as far back, but just looking at them you can tell summer ice from winter ice. Counting the layers give you years. There are also more accurate ways to measure the difference (you can look into what delta 18Oxygen means).

You will lose count if there was a period where summers got warm enough to melt more ice than was formed during the winter, but you can then use the other methods to try and correct for that.

Even so, being of by a 100 years isn't too bad when you are counting 100000.