As for your problem understanding where all of Lee's men went after Second Manassas. Read Lee's letter to President Davis dated September 7, 1862, in which he told Davis that the "cowards of the army," the men you are counting as present for duty, had not crossed the river but instead had straggled away, to "leave their comrades" to face the rigors of the march and the battle to come, alone.
...so how exactly is it that all the reports of the time from people observing the marching Confederate army gave estimates consistent with a Confederate army around 70,000-80,000 strong?
How is it that the 42,000 men who apparently didn't cross the Potomac river got fed for two weeks? And where were they, given how many of them appear again even in the 22nd September return?
Of course, we don't have (to my knowledge) the other side of the correspondence. It's quite possible that what Davis was actually doing was exactly what Lincoln often did - taking the Aggregate Present and Absent and thinking of it as the PFD strength of the army.
What the young serious students should realize is that what their elders have left them as "history" is clap trap, that it's their choice whether to cling to the fantasy, or tear the story back down to its essential objective facts and reconstruct it, using the discipline of finding the facts and drawing objectively reasonable conclusions from them, something academic historians don't do, and which you are not doing here.
This is a very troubling statement. You're saying that academic historians "cling to fantasy"?
Lee's army trains, of which Longstreet's was one, were moving across the river on the night of the 14th which is why the 1,200 Union cavalry column, escaping from the Ferry, ran into it on the Hagerstown/Williamsport road and took 125 of his wagons and went north to Greencastle.
Fair, to some extent, but I'm not seeing how this leads to the idea of Lee crossing his trains partly to convince McClellan to attack - unless you think Lee had Jackson let those cavalry go.
As for your idea about Harper's Ferry being an indefensible position. It seems you have not actually been there. The Union line at Bolivar Heights was perfectly defensible. Its commander surrendered under circumstances that baffle understanding. Walker's division was merely to block the garrison escaping across the Shenandoah; it had no functional value to the attack of the Union position. Ditto for McLaws. As for Jackson, in one sentence you admit that the order specified that Jackson was to go to the Baltimore & Ohio RR at Martinsburg and sit there, waiting to capture any of the Union garrison that tried to escape to the west; but in the next you put Jackson where he ended up, not by virtue of the order's text, but by Le's "verbal instructions" which, of course, McClellan and, apparently, you, did not know.
Since the extreme range of a smoothbore artillery gun for bombardment on level ground (no height advantage) is about a mile, it should be clear that Walker's guns could bombard Bolivar Heights even without the advantage of height (and could cover AP Hill getting in behind Bolivar Heights, in fact). McLaws' guns could take the rear of the fallback defensive position under fire even without the extra range granted by height; with that ~100 yard elevation advantage they could shoot approx.:
Flat - Elevation 5 degrees, range 1,619 yards
First appoximation is that the gun's trajectory peaks at 800 yards
Muzzle velocity about 480 yards per second
So peak after ~2 seconds
Vertical motion s = 1/2 a t^2
Peak altitude about 20 yards
But here peak altitude is ~120 yards
120 = 5t^2
t ~ 5
3 seconds extra travel
About an extra 1,400 yards range
So with the 100 yard elevation advantage from the Maryland Heights a Napoleon smoothbore's extreme range is about 3,000 yards to a first approximation. That's not enough to play on the peaks of the Bolivar Heights but it is enough to play on the reverse slope.
Any rifles McLaws has only make this conclusion stronger, and of course the guns definitely render Harpers Ferry itself untenable; thus McLaws and Walker do have value to the attack on HF. Since the abandonment of the Heights was unexpected on the part of the Union I don't see a problem with the assumption that:
The Heights were considered the key point of defence
thus
McLaws was the one who was considered to be "taking the lead" in capturing Harpers Ferry.
As for Martinsburg, I don't see any part of Jackson's orders which instructs him to
stay at Martinsburg; he's told to take possession of the B&O and capture anyone at Martinsburg, but he can then move as required to capture any escapees from HF. Since it's an operational order and the operational moves are fundamentally correct to what actually happened, I don't see any major problem.
After Jackson did what the order told him to do, it instructed him to recross the river and join Lee at Boonesboro. Any one reading the order would not conclude that Jackson was to appear in front of Bolivar Heights. Thus, the reader, realizing the set up the order presents cannot possibly result in its capture, unless for some unfathomable reason the commander surrenders without a fight, realizes Lee is attempting to entice him to march to the Ferry in order to strike his right flank as he passes either into the Middletown Valley or into Pleasant Valley. This is why McClellan refers in his message to Halleck and his wife that he recognizes the order "is a trap" which he tells them he means to turn to his advantage.
But... hold on.
You're saying that the enticement is the threatened attack on Harpers Ferry in SO191, and that the clues that it's an enticement are in SO 191.
But that's the same source.
And you still haven't addressed why McLaws was in such danger. In fact, you've amplified it - if McLaws' movements were movements which Lee
could not anticipate would lead to the capture of Harpers Ferry, then McLaws has been placed in a highly vulnerable position and should have been captured; worse, SO191 makes that vulnerable position clear.
To my mind the much simpler explanation is:
Lee planned to move to the Boonsboro area, take Harpers Ferry, and then conduct further operations once that Union position was defeated. (This is him planning in detail the movements of his own army about 4-5 days ahead and in outline further ahead.)
He constructed operational orders so as to allow a large portion of his army to converge on HF and the remainder to secure his route north (to threaten Pennsylvania).
He did not anticipate McClellan moving as quickly as McClellan did in fact do, and was taken by surprise by the attacks at South Mountain.
His signals to McLaws and the others to abandon the siege reflect this "panic" state.
When HF surrendered, it let McLaws cross and rescue his portion of the Confederate army.
With McClellan having turned South Mountain, Lee fell back to reunite his army at Sharpsburg - holding just long enough to get his trains over the river, then withdrawing on the night of the 18th.
Absent McClellan's quick movement, Lee would anticipate either a battle on his own terms in the Cumberland Valley or a movement into Pennsylvania; however, he would want his force substantially closed up before offering battle.
SO191 being dropped is explained thus - it was a genuine accident, resulting from the unusual situation of DH Hill meaning that Jackson copied the order for DH Hill (i.e. two were sent to DH Hill - direct from Lee's CoS and from Jackson) and DH Hill only got one copy. Thus when the orders were lost nobody noticed, because Hill expected orders and he got orders.