Because the balloon is closed, the number of helium atoms inside doesn’t change as you cool it or warm it. But that doesn’t indicate that the balloon’s size is fixed. What is happening here is related to the relationship between three important properties of a gas: its temperature, its pressure, and its volume. As the balloon is cooled down, its volume decreases so as to keep the internal pressure in balance with the ambient air pressure. When the balloon is heated later on, the volume increases to maintain this same pressure balance.