There was a vote.
And that's the thing about secession in 1860, at least when it comes to South Carolina. The state was involved, since the governor called on the legislature to call a special election for a convention, and the people were involved, since they elected representatives to a convention, just as they had when they had considered ratifying the Constitution, and just as they had during the nullification crisis. The state and the people were both integral to that process. So it seems to me to be a clear exercise of 10th amendment reserved powers, with both the state and the people exercising the power.
The only way to get around that is to take the nationalist approach and insist that it had to be all the states or all the people. And there's all sorts of directions we could go there.