The
entire speech you extracted the quote from is about slavery. Your quote comes from the portion of the speech where Lincoln is arguing against the idea that the people in a territory should be able to decide for themselves whether to allow slavery. Here is the transition into that section of the speech:
But one great argument in the support of the repeal of the Missouri Compromise, is still to come. That argument is "the sacred right of self government." It seems our distinguished Senator has found great difficulty in getting his antagonists, even in the Senate to meet him fairly on this argument.
Some poet has said, "Fools rush in where angels fear to tread." At the hazard of being thought one of the fools of this quotation, I meet that argument---I rush in, I take that bull by the horns.
The paragraph you quoted from reads:
Whether slavery shall go into Nebraska, or other new territories, is not a matter of exclusive concern to the people who may go there. The whole nation is interested that the best use shall be made of these territories. We want them for the homes of free white people. This they cannot be, to any considerable extent, if slavery shall be planted within them. Slave States are places for poor white people to remove FROM; not to remove TO. New free States are the places for poor people to go to and better their condition. For this use, the nation needs these territories.
Brass Napoleon already pointed out that this nowhere says whites
only. Lincoln is pointing to the economic development of the territories, and its economic effects on the rest of the country, to establish the legitimacy of the rest of the country concerning itself with the question of slavery in Nebraska. He has other arguments to the same end:
There are constitutional relations between the slave and free States, which are degrading to the latter. We are under legal obligations to catch and return their runaway slaves to them---a sort of dirty, disagreeable job, which I believe, as a general rule the slave-holders will not perform for one another. Then again, in the control of the government---the management of the partnership affairs---they have greatly the advantage of us.... The slaves do not vote; they are only counted and so used, as to swell the influence of the white people's votes.... Now all this is manifestly unfair; yet I do not mention it to complain of it, in so far as it is already settled. It is in the constitution; and I do not, for that cause, or any other cause, propose to destroy, or alter, or disregard the constitution....
But when I am told I must leave it altogether to OTHER PEOPLE to say whether new partners are to be bred up and brought into the firm, on the same degrading terms against me. I respectfully demur. I insist, that whether I shall be a whole man, or only, the half of one, in comparison with others, is a question in which I am somewhat concerned; and one which no other man can have a sacred right of deciding for me.
And further:
I insist, that if there is ANY THING which it is the duty of the WHOLE PEOPLE to never entrust to any hands but their own, that thing is the preservation and perpetuity, of their own liberties, and institutions. And if they shall think, as I do, that the extension of slavery endangers them, [they should not] submit the question, and with it, the fate of their country, to a mere hand-full of men, bent only on temporary self-interest.
In short, the whole speech is about keeping
slavery out of the territories, not blacks. I hope I've done enough quoting from it to make this posting on topic in this thread.