First off, the CSA never had control of all of its territory. The gaudiest example is, of course, West Virginia. Add to that East Tennessee, Western North Carolina & North Georgia. In Tennessee, the three great divisions of the state East, Middle & West had radically different reactions to secession. Shelbyville, the town where Bragg made his HQ after Stones River was referred to as "... that Union hole..." With the loss of Middle Tennessee after the fall of Fort Donelson, the most densely populated counties in the state were permanently outside CSA control. With the concurrent capture of Memphis, the single major population center in West Tennessee was lost, about 2/3rds of the state population was lost to CSA control. Resistance to CSA in East Tennessee began almost the second the ink dried on the state's secession declaration.
The most hapless, sad & tragic group of soldiers I am aware of were overage men in their 40's drafted by the CSA from Strawberry Plaines. After nearly freezing to death in the winter, they were sent off to Vicksburg just in time to suffer the siege. As a living history program, my wife & I have used the letters that a family where one of those men was sent to recover after the surrender. As he lay dying, he repeatedly expressed concern about his bee hives. Needless to say, he & his neighbors did not give a fig about fighting for the right to hold other human beings as property.
My point is that just because there was a certain number of men of military age in a given county in 1860, that does not translate into men of military age who can be brought into the ranks. As Jefferson Davis so rightly stated, the CSA did not have enough white males of military age to both fight the war & stay at home to control the servile population. Throughout the war, CSA military planning was caught on the forks of the dilemma created by maintaining the absolutely essential slave patrols & recruiting. The infamous 20 slave rule that was such a propaganda disaster sprang from that conundrum.
As the consolidation of regiments as the war went on demonstrates, whatever the raw number of white men in their middle 20's was, the CSA simply did not have any means of filling out its ranks. I am probably able to write this because my ancestor drew a straw to stay home when the State Army of Tennessee was formed before secession. It was recognized that not all the men willing to join up could go. A lottery was the best way to be fair & also maintain dignity. The fall of Donelson placed him firmly behind Union lines & safe from CSA enrollment officers for the duration. You might say that worked out just fine for me.