A decline in land productivity would easily have led to famine. The other explanation is that the people of Great Zimbabwe had to move in order to maximise their exploitation of the gold trade network. By 1500 the site of Great Zimbabwe was abandoned.
