That statement is a lot like saying "A baby has two legs when it's a boy.". The statement is true as far as it goes, but it doesn't go all the way.
Yes, speeding up is one kind of acceleration, but it's not the ONLY kind. Slowing down and turning are also acceleration.
