Even without Franklin the number of men defending Washington would amount to (PFD):
53,184 (historical Dept. Rappahannock plus historical Dept. Valley at the end of May 1862, i.e. Banks plus McDowell minus Franklin)
+ 12,845 (historical Dept. DC/Alexandria at that time)
So 66,000 at minimum. With Franklin it's 77,000, and in addition the Department of the Mountains has ~25,000 to call on (including Blenker).
I'm not sure how big Johnston was imagining this offensive to be (i.e. how big it could get with forces available to him before the end of April) but at the end of the Yorktown siege there were 58,500 effectives in the lines. As of the point when Johnston visited AP Hill's troops were included in the troops then in the lines, which means there were 38,100 effectives in the line.
That makes Johnston's offensive about 20,000 effectives, plus Jackson's Valley force (which wasn't very big). Call it a corps all told.
It's an interesting question however how much this could threaten Banks if Banks' forces were actually positioned for defence in the Valley. My understanding is that a big part of what made Banks so vulnerable in the historical Valley Campaign was that so much of his force was transferred to the Dept. of the Rappahannock (down at Fredericksburg, mostly) that he was down to only a couple of brigades.
This is on April 15 and Johnston is talking about anyone and everyone who has not already been sent down the Peninsula to Magruder, plus anyone he can possibly get from anyplace else. Lee is objecting to stripping the Carolinas and Georgia. Johnston wants all the new troops forming under the conscription law if he can get them. He will take them all up towards Washington and the Potomac -- plus every bit of supply and ammo he can pry loose, no doubt. Base any calculations on that desire of Johnston.
Meanwhile, please note that one of the participants in that conference has very different ideas about crossing the Potomac than what you think. Longstreet had already been communicating with Jackson about joining forces to sweep the Shenandoah. From Longstreet's memoir:
... About the 1st of April, Generals Johnston and G. W. Smith were called to Richmond for conference with the War Department, leaving me in command. On the 3d I wrote General Jackson, in the Shenandoah Valley, proposing to join him with sufficient reinforcements to strike the Federal force in front of him a sudden, severe blow, and thus compel a change in the movements of McClellan's army. I explained that the responsibility of the move could not be taken unless I was with the detachment to give it vigor and action to meet my views, or give time to get back behind the Rapidan in case the authorities discovered the move and ordered its recall.
I had been left in command on the Rapidan, but was not authorized to assume command of the Valley district. As the commander of the district did not care to have an officer there of higher rank, the subject was discontinued.
General Johnston, assigned to the Department of the Peninsula and Norfolk, made an inspection of his new lines, and on his return recommended that they should be abandoned. Meanwhile, his army had been ordered to Richmond. He was invited to meet the President to discuss military affairs, and asked General G. W. Smith and myself to go with him. The Secretary of War and General R. E. Lee were with the President when we met.
It was the first time that I had been called to such august presence, to deliberate on momentous matters, so I had nothing to say till called on. The views intended to be offered were prefaced by saying that I knew General McClellan; that he was a military engineer, and would move his army by careful measurement and preparation; that he would not be ready to advance before the 1st of May. The President interrupted, and spoke of McClellan's high attainments and capacity in a style indicating that he did not care to hear any one talk who did not have the same appreciation of our great adversary. McClellan had been a special favorite with Mr. Davis when he was Secretary of War in the Pierce administration, and he seemed to take such reflections upon his favorites as somewhat personal. From the hasty interruption I concluded that my opinion had only been asked through polite recognition of my presence, not that it was wanted, and said no more. My intention was to suggest that we leave Magruder to look after McClellan, and march, as proposed to Jackson a few days before, through the Valley of Virginia, cross the Potomac, threaten Washington, and call McClellan to his own capital.
As you can see, there is reason to suppose that Johnston, Smith and Longstreet
were not thinking the way you are. Johnston had recruited Smith to be the stalking horse to present the most aggressive part of the plan
before the meeting.