As part of the Constitution, Article one, Section two, states that any person who was not free, would be counted as three-fifths of a free person for the determination of congressional representation. This so meant that the Southern States, received far more congressional representation because of the fact that non-free persons were accorded 60% of the value of a free person, even though they had no Constitutional rights; however, congressional representation was based upon the total population of each State, free and the non-free as proportioned. So then, at the conclusion of the Civil War, the three-fifths Clause to the Constitution became moot, replaced by the 15th Amendment, which thus permitted all those of legal age that were male, to thereby be franchised to vote. The significance of the 15th Amendment, in an era in which previously the south had 4 million peoples that had been enslaved, meant ultimately that there would be more congressional representation in the South, for each person within the South, would be equally counted as a full person, no matter the color of their skin or possible previous servitude or enslavement.
The salient problem though with the sudden status change of non-free persons that would now be counted as free people, would be, if subsequent democratic voting precluded those of color from voting, via intimidation, or the enforcement of prejudicial and spurious voting laws, which would mean that the South could then revert back to not only being controlled by the elites of plantation society, but that because the numbers of those so living within the borders of these Southern States, black and white, now counting for representative purposes as each being a full person, thus meant that the Southern collective representation would become greater in the House of Representatives. Indeed, while initially, the Southern States actually did have people of color being their representatives on a Federal, State, and local level, it was not too long, before those with real money and historical power asserted themselves to essentially take back control of their respective domain, thereby annulling not only the franchise of people of color, but also in the finding of additional innovative ways to exploit those that were formerly slaves.
Indeed, it must be said, sometimes the very best legislation, so meant to protect and serve those people that have historically being abused and harmed, actually ends up being utilized in a way and form that in some respects, makes things even worse for those that were already unjustly suffering a great deal to begin with. To a very large extent those of the North, lost the will power to do right by those that they freed, which pretty much signified that the majority of those in the North, thereby enacted a form of benign neglect, which ultimately meant the fate of those that were formerly enslaved was left, in essence, with those that had been their slaveholders.
The proximate problem then with democracy, is that when those that are legally entitled to be franchised, are essentially precluded from being able to vote, then the result, thereof, is going to favor those that are franchised who will thereby get enacted whatever pleases them, so done to the detriment of all those that are unrepresented, and powerless.