Generally when people read clocks they just round the hour hand down. Also, some clocks (especially older ones) don’t gradually move the hour hand, making your point work for only some clocks.

I think the essence of the saying is that a clock represents a time based on how we read it and if the time isn’t changing on the clock then it is correct twice a day.