They didn't. They would recycle materials and make new buildings out of old ones or change the purpose of a building to a new one. They would also heavily use wood in building houses and things like that because it was available and was not that difficult to manage unlike stone.
