They just werent allowed into the buildings where people could cast their votes.
In some places they were required to pay a special Poll Tax, that most could not afford. In other places they had to show they could read at a level that the educational system did not always get them to. In some places, men wearing hoods and carrying guns and torches simply prevented them from going to town on election day.