That he didn't think it immoral or not enough to do or say anything about it is where I struggle with him.
There was no policy requiring his army to invade the US to enslave free blacks, or enslave free blacks they came across in their own lands. There was no policy that black US POW's had to be used as human shields. Or a policy that surrendering blacks needed to be shot immediately. Lee promoted those who did those things or did them himself and not once ever reprimanded those actions.
And while yes there was later a policy that black POW's had to be either executed or enslaved, that doesn't make you honorable that you enforce it. Just because Hitler had a policy that Jews got the chamber doesn't make Heinrich Himmler an honorable man since he followed what Hitler wanted.
The policy wasn't to invade specifically to capture African-Americans, but it was to capture fugitives as they came across them in the course of their operations.
It was confederate policy for their army to capture African-Americans they deemed to be "fugitive slaves." Of course, how could they tell a free person from an escaped slave? Why, the escaped slave was black, of course!
In March of 1863, Lee sent out a circular to all his subordinate commanders in the Army of Northern Virginia implementing a policy promulgated in Richmond.
The citation for the circular is: "W. H. Taylor to General, 21 March 1863, Orders and Circulars Issued by the Army of the Potomac and the Army and Department of Northern Virginia, C.S.A., 1861-1865, NA Microfilm M921, reel I, frame 1391. Also see Orders and Circulars, Rodes and Battle's Brigade, Army of Northern Virginia, 1861-1865, NA, RG 109, Chap. 2, vol. 66, pp. 175-76. Lee's order to his subordinate commanders in that circular directed the army to comply with an early March directive by the Confederate Adjutant General (General Orders No. 25, Adjutant and Inspector General's Office, 6 March 1863, in OR, Ser. 2, vol. 5, pp. 844-45."
Here's the order:
https://babel.hathitrust.org/cgi/pt?id=coo.31924079609644;view=1up;seq=856
The order turned every confederate army into armies of slave catchers, and slave catchers were not known for being particular about which African American they accused of being a fugitive slave. Eyewitness testimony shows confederate soldiers rounding up every African American they could find for transport back to the confederacy in compliance with this order.
That they didn't catch more African-Americans shows how African-Americans fled as the ANV approached.
The African-Americans captured during the Gettysburg campaign were brought back to Virginia with the Army of Northern Virginia and there disposed of in accordance with above confederate policy.
The captured African-Americans would have been taken to one of the depots, likely Richmond, for disposition. Around the same time as this policy was developed, another policy, on slave impressment, was developed. You can see the bill here:
https://archive.org/details/billtobeentitled41conf
The bill eventually was passed in March of 1863 and likely any of the captured African-Americans who were not claimed by purported "owners" would be automatically impressed into confederate service to either work on food production or on fortifications.
We have this order, where Rev. T. V. Moore of Richmond interceded for one of the kidnapped African-Americans. Apparently friends in Pennsylvania got the documentary information to Rev. Moore showing Amos Bares was born free in Pennsylvania. Amos was lucky his friends knew Rev. Moore and could lay hands on documentary evidence.
CONFEDERATE STATES OF AMERICA, WAR DEPARTMENT,
Richmond, Va., December 14, 1863.
Brig. Gen. JOHN H. WINDER:
GENERAL: You will dispose of prisoners named below, embraced in Report, No. 146, of Maj. I. H. Carrington, indicated, viz: Joseph A. Marm, William Tennant, John E. Tennant; send to conscript camp. You will also deliver to Robert Ould, esq., commissioner for exchange, to be transferred to the United States by the first flag of-truce boat, Amos Bares, a free negro from Pennsylvania, whose release is applied for by the Rev. T. V. Moore, of this city, upon grounds which appear to the Department sufficient to justify an exceptional policy with regard to him.
By order of the Secretary of War:
J. A. CAMPBELL,
Assistant Secretary of War.
[OR Series II, Vol. 6, pp. 704-705]
In his book,
Mosby's Rangers, James J. Williamson, who was a member of Mosby's Rangers [Company A, 43rd Battalion, Virginia Cavalry], writes that on June 28, 1863, "It was Mosby's intention to join General Lee in Pennsylvania, but when we reached Mercersburg, where we expected to find a portion of the army, it had moved. Our number being so small, and as we were ignorant of the country as well as of the position of our army, Mosby determined to return to Virginia, which he did, but not until he had gathered up 218 head of cattle, 15 horses
and 12 negroes. Returning through Washington county, Maryland, we recrossed the Potomac without interruption." [pp. 79-80]