It is colder, BECAUSE:
As we go up in the atmosphere, air pressure continually becomes less. That means the rising air is expanding. And a characteristic of any gas is that it is cooled by expansion. So the draft of warm air that starts rising from the earth's surface continually cools itself as it rises.
Hope this answers your question!