1) What about ending the Peninsula Campaign after stopping the Confederates at Malvern Hill. Still had 100k men+ on the Peninsula, naval superiority and supply access and a generally intact army. Instead of pushing on after a bloody repulse of the enemy, he ended the campaign
2) I'm not familiar with the specifics of Mac's intelligence heading into or post Antietam. But I assume he didn't know his numerical superiority before or after the battle. (Was he still using Pinkerton at this time?) Had he had a correct estimate, would have supported further action. If I'm understanding your question correctly.
67th has already explored the general situation at Malvern, but I'll provide a map:
Malvern Hill ended pretty much at sundown, so any offensive action post-Malvern would have to be on the following day. Note that the Union navy would not bring supplies up past the batteries at City Point.
If McClellan hadn't stepped back after Malvern Hill, he would have been encircled on the hill without supplies. To attack post-Malvern would be to fling a hungry force low on supplies into an enemy force forming a "box" around Malvern Hill, with many enemy brigades undamaged by the battle on July 1 and the rest of them having had time to recover
from July 1; it had little chance of destroying Lee's army and a major risk of destroying McClellan's.
As for abandoning the campaign, McClellan had been promised reinforcements by Lincoln before Malvern; since Lee's army was if anything slightly larger than McClellan's, McClellan waiting for reinforcements before going back on the offensive is entirely defensible. Those promised reinforcements continued to be promised and never arrived for the whole month of July (during which Lee was himself reinforced).
As for Antietam, it's true that McClellan's numbers for Lee's force are quite large, but then again Lee's numbers for his own force have major omissions and it's not clear Lee knew how large his
own army was. The best modern estimates of Lee's force during the Maryland campaign are derived three ways:
1) Addition of the contemporary estimates of the individual moving forces in Maryland and Virginia.
2) Taking the post-Antietam returns and adding back the known Confederate casualties.
3) Estimating the size of the force after Second Bull Run and adding the reinforcements that came up from Richmond.
In PFD terms, these all add to about 75,000 men for Lee's campaign strength (i.e. PFD strength before straggling) . McClellan's own campaign strength in PFD was a bit larger, but not by much (it's something like an 8:7 to 4:3 advantage depending on which of McClellan's units you include in your count) and both sides suffered heavily from straggling.
Your claim is that if McClellan had a better estimate of how many men Lee had he could have ordered a resumed attack, but I don't think this is the case. Most of McClellan's army had been sent on the attack and repulsed, often with heavy casualties and in many cases with massive disorganization; what he's got left unspent is IIRC five brigades of 6th Corps forming his right and maybe 3-4 brigades of 5th Corps composing his centre. Lee has three brigades that basically haven't taken a casualty still in reserve, plus the rest of his army which has certainly been bloodied but which contains many units that are still able to fight.
If McClellan's attack with these troops fails, meanwhile, his army's all wrecked. The only thing preventing a Confederate counterattack destroying his right wing is that the five-brigade line of 6th corps is protecting it.
Meanwhile, of course, it's hard to see how McClellan could have formed the picture that there were few enough Confederates to attack successfully. Even if he'd somehow been able to count and identify every single Rebel brigade that his men engaged, there'd be at least seven unaccounted for (the three unaccounted for who were actually at the battle and the brigades left at Richmond); as far as we can tell the Rebel returns for September don't actually exist; if he correlated reports from eyewitnesses he'd come up with the result he only had a slight numerical advantage.