So something that occurred to me in the context of the question of this thread (and the replies to it when it was originally opened) is that basically it's two cliches clashing and people not really being able to evaluate them properly.
The Johnston cliche is "he wants to retreat in all cases even when it's not a good idea".
The McClellan cliche is "he's very slow when he advances even when he could press on faster".
The problem with these is that they can't actually both be true of the same campaign, though they can both be false.
Specifically, if Johnston's force is small enough that McClellan could easily roll over him, then slowly retreating up the Peninsula is a better way of buying time than being beaten in a battle. Conversely, if Johnston's force is large enough that he could fight it out with McClellan from a good position, then McClellan's caution is justified.
Of course, I have my own opinion... and it's that Johnston's army was close in size to that of McClellan, though not larger*. Johnston's attitude during the Peninsula is self consistent if we assume that he wanted to fight only from a position of advantage (since the armies are similar in size) and McClellan's attitude is consistent if we assume that he wanted to avoid fighting Johnston in a position to Johnston's advantage (ditto).
Thus Johnston holds at Yorktown until McClellan's siege guns are in position to blast through the forts in the position, then he falls back. Johnston then holds the third and final prepared defensive line on the Peninsula, Williamsburg, and McClellan outflanks it with the move to Eltham's Landing.
The next natural barrier is the Chickahominy, but it's too long for Johnston to hold and he fears being outflanked, so he pulls back to the Richmond fortifications as McClellan moves up to base at White House Landing. It's here McClellan's operations start being constrained by the order to fix his base there and operate against the rail lines north of the Chickahominy.
Johnston's aim during the march up the Peninsula was to pounce if McClellan's marches left a division exposed; in this light McClellan's caution seems well founded.
*In the Seven Days Lee's army had 112,000 PFD, split as:
Army of Northern Virginia
Longstreet: 14,291
A.P. Hill: 16,411
D.H. Hill: 12,318
Stuart: 2,109
Total: 45,129
Army of the Valley
Jackson: 9,604
Ewell: 6,353
Whiting: 5,537
Cavalry: 605
Total: 22,099
Confederate defensive forces south of the Chickahominy:
McLaws: 4,915
D.R. Jones: 4,503
Magruder: 5,671
Huger: 6,160
Holmes: 9,018
Reserve artillery: 1,680
Cavalry: 2,000
Richmond defenses: 9,136
Petersburg defenses: 1,909
Total: 44,992
(Harsh)
As of Seven Pines, the following Seven Days regiments are with Johnston's army.
Whiting's division: 9/9
Jackson's division: 1/18 (will carry over to Longstreet)
Ewell's division: 0/18
DH Hill's division: 16/23 (missing Ripley's brigade and a few scattered units)
DR Jones' division: 10/9 (1st KY extra, will carry over to McLaws)
McLaws' division: 9/10 (Kershaw not listed on Wikipedia but was present)
Magruder's division: 9/9 (some regiments missing on Wikipedia but were present, such as 17th MS and Cobbs brigade)
Longstreet's divison: 25/27 (missing 2 SC Rifles and 56th VA)
Huger's division: 15/27 (missing the two Dept. of NC brigades, plus 44th AL. I will treat it as 15/16 and assume the Dept. of NC is embraced in Holmes.)
AP Hill's division: 31/32 (missing 5th Alabama battalion)
Holmes: not present except for 26th VA. Will treat AP Hill's division as complete.
So the infantry PFD strength is roughly:
Whiting 5,537
16/23 of DH Hill 8,569
DR Jones 4,503
McLaws 4,915
Magruder 5,671
26/27 of Longstreet 13,762
15/16 of Huger 5,775
AP Hill 16,411
Richmond defences 9,136
Seven Pines casualties 6,134
That comes to an estimate of 80,400 PFD in the infantry formations. Given that the Seven Days infantry formation PFD is 104,000 (after Seven Pines casualties, so 110,000 adding them back) this implies that over June Lee got a total of 30,000 reinforcements, of which 25,000 were formed units (Jackson, Holmes, Ewell) and the remaining 5,000 was in loose regiments and Ripley's Brigade; this seems feasible.
Thus adding in the reserve artillery and cavalry from Yorktown Johnston's army comes out at about 85,000-87,000, or 75,000 for the field. Smaller than McClellan's, to be sure, but in a 3:4 ratio with McClellan rather than the oft-suggested 2:1.
ED: per the below Ripley and Walker had already arrived by Seven Pines. This puts Johnston's available force up to 92,000 by my estimate, which means that if he left Richmond with just 6,000 defenders Johnston could fight with 86,000 PFD against McClellan's 104,000.
This isn't quite equal numbers but it's pretty close, and the slightly different PFD-Effectives conversion factor the Confederates had makes it closer still. (Basically some of the non combat jobs done by PFD soldiers in McClellan's army were done by black non-soldiers, voluntarily or otherwise, in Johnston's.)
Assumptions made in this assessment include: no casualty returns-to-duty took place for Seven Pines casualties, no attrition took place in June for any division apart from Jackson and Ewell.