Hill had several of Lee's combat-experienced veteran units, including two of Pickett's brigades. Lee wanted them returned to him prior to the campaign, but Hill refused.
“General Lee supervised considerably more than just the Army (and Department) of Northern Virginia. He was responsible as well for operations throughout the Department of Virginia and North Carolina, focusing particularly on the possible enemy approaches to Richmond and on the defense of North Carolina's coastal region. Early in 1863, in response to a barrage of Federal threats to these areas, Lee had dispatched two brigades of his North Carolina troops, under Robert Ransom and John R. Cooke, to their native state. They were followed south shortly thereafter by Hood's and Pickett's divisions, under Longstreet. This cut in strength of 18,600 men left Lee to face Hooker's attacks at Chancellorsville with only 65,000 men of all arms. The battle's casualties came to 13,500. Thus simply to return the Army of Northern Virginia to its numbers at the beginning of the year would require 32,100 men." [Stephen Sears, Gettysburg, p. 47]
“In 1861 and 1862 D. H. Hill had served as a superb combat general in the Army of Northern Virginia, but he was so contrary and cross-grained an individual that when in January 1863 Governor Zebulon Vance of North Carolina called for native son Hill to lead the defenders of his state against the Yankee invaders, General Lee raised no objection. By spring, thanks to the infusion of troops from lee's army, the situation in North Carolina was stabilized. Then, during the Chancellorsville crisis, Hood's and Pickett's divisions were ordered back to the Rappahannock, and soon thereafter Lee sought the return of his other absent brigades. Harvey Hill began to grow very nervous. Hill first proposed a simple exchange--untested but fully recruited brigades from his command traded for veteran but depleted brigades from Lee's. Lee was disapproving. Such a scheme, he pointed out to Hill, would mean 'taking away tried troops under experienced officers & replacing them with fresh men & uninstructed commanders. I should therefore have more to feed but less to depend on.' In fact, there would be one such exchange--Junius Daniel's largely untried North Carolina brigade for Alfred Colquitt's Georgia brigade that had been bloodied at Chancellorsville, but Lee probably agreed to the trade as a convenient way to rid himself of the inept Colquitt. An essential point of difference remained: Lee was far less alarmed about the Federals' threat than was Hill, arguing that the enemy was actually reducing his forces along the Atlantic seaboard. 'It is of course our best policy to do the same & to endeavor to repel his advance into Virginia,' he observed. So far as General Lee was concerned, the central issue here was the return of four brigades he regarded as merely on loan to Hill. These contained some of the best fighting men in his army, commanded by four of his best generals--Robert Ransom and John R. Cooke, whose brigades were sent to North Carolina in January, and Montgomery Corse and Micah Jenkins, whose brigades had been attached to Pickett's division when it went south but had since been detached for Hill's purposes. In due course, enduring what he surely regarded as the cruelest of blows, General Lee would find himself crossing the Potomac without a single one of these brigades." [Stephen Sears, Gettysburg, pp. 48-49]
“It was finally determined that the two brigades Lee required to fill out Harry Heth's new division in the new Third Corps would indeed come from D. H. Hill's command. But t hey would not be any of those Lee had sought. One was a brigade of North Carolina troops with no appreciable experience, led by Johnston Pettigrew, who was returning to the field after being wounded and captured on the Peninsula. The second brigade comprised one North Carolina and three Mississippi regiments with at least some combat experience, led by a nephew of President Davis, Joseph R. Davis, who had no command experience at all. However badly this affair of reinforcements was handled, it was in the end a commentary on the severe manpower strains rending the Confederacy. 'I readily perceive the disadvantage of standing still,' President Davis wrote Lee on May 31, 'and sorely regret that I can not give you the means which would make it quite safe to attempt all that we desire.' " [Stephen Sears, Gettysburg, pp. 50-51]
“As he started his march north, Lee was optimistic that he would finally regain at least one of George Pickett's two brigades earlier expropriated by D. H. Hill. But, once again, it was not to be. A new spasm of Federal activity on the Virginia Peninsula so alarmed Richmond that Montgomery Corse's brigade was held back to guard the capital. When Pickett came up from Hanover Junction, it was with but three of his original five brigades, making his the weakest division in Longstreet's First Corps. At the same time, and for the same reason, Lee's hopes were dashed that another First Corps brigade, John R. Cooke's, would be returned to him." [Stephen Sears, Gettysburg, p. 95]