In early America, only people with a certain amount of property in society could vote.
When America was still part of Britain, there were several different colonies that required a certain amount of land in order to vote. Along with this, the right to vote was usually restricted to white males over the age of 21. These requirements made it so that there was a very small percentage of the population that could vote in comparison to the overall population.