For cities that have been inhabited for a long period of time, a lot of it is literally just people dropping litter, or similar. For most of the history of human cities, getting rid of waste was not something anyone really made happen, they just relied on rain to wash stuff away, and for heavier solid waste, it just sat there. If you pulled an old building down to put a new one up in its place, you would clear out the bigger bits of rubble, but a lot would just sit there, or get pushed into the street.

For a while I lived in a city in the UK with a ~1000 year old church. You had to go down four or five steps from modern street level to get to the churchyard. The churchyard and church itself have been in continual use and kept maintained over that period. That land is an example of somewhere in a built up area that has indeed been kept free from junk and debris for 1000 years.