I'm glad you researched it on your own.
The ORs have the relevant messages:
HEADQUARTERS OF THE ARMY,
Washington, July 2, 1863.
Col. WILLIAM H. LUDLOW,
Agent for Exchange of Prisoners, Fort Monroe:
COLONEL: Your dispatch of the 1st instant is received.
It is stated that some of the officers and men of the colored regiments captured west of the Mississippi River have been hung by order of General Taylor, and that others (colored) have been sold into slavery under some pretended State authority. It is understood that General Grant has made a formal demand on General Taylor to know if these statements are true, and also that all such prisoners be treated in accordance with the stipulations of the cartel and the rules of civilized war. It is also stated that a portion of Colonel Streight's command captured have been refused the right of exchange under the cartel and are improperly retained by the enemy.
It is the duty of the United States to afford protection to all persons duly received into the military service, and if the enemy should violate the cartel and laws of war in the treatment of prisoners our Government will be reluctantly compelled to resort to retaliation. While we shall ask for nothing to which we are not entitled by well-established laws, we cannot permit a deliberate and systematic violation of the usages of civilized warfare to pass unpunished. However much we may wish to avoid any act by which the innocent may suffer for the crimes of the guilty, there are occasions where summary retaliation must be resorted to. I am fully aware that violations of law, both civil and military, will sometimes occur under any Government or organization, and complaints are not made where the proper authorities employ all legitimate means to rebuke and punish the offenders. It is hoped that the statements I have alluded to may be incorrect or mere exaggerations, as is not unusually the case on both sides, and that the matter may be properly and satisfactorily arranged.
In connection with this matter I inclose herewith a copy of a report of General Rosecrans upon General Bragg's letter in regard to his stripping Coburn's brigade of their blankets, clothing, &c. You will please again call Mr. Ould's attention to General Bragg's conduct as admitted by himself. Instead of depriving prisoners of war of their clothing we have issued to them large quantities of blankets to make them comfortable and have generally exchanged them in better condition than when captured. The enemy, on the contrary, has frequently treated our troops with great inhumanity and sent them back in a condition utterly disgraceful to the captors. It is hoped that this matter will be properly investigated and the abuse corrected.
Very respectfully, your obedient servant,
H. W. HALLECK,
General-in. Chief.
[OR Series II, Vol VI, p. 73]
HEADQUARTERS OF THE ARMY,
Washington, October 26
, 1863
.
Major-General BANKS,
New Orleans:
GENERAL: Your dispatches of October 16 and 17 are received.
In regard to our prisoners of war held by the enemy, I submit the following brief explanation of the difficulties in effecting any exchanges on account of the utter disregard of the cartel by the rebel authorities.
The enemy commenced the violation of this solemn agreement by refusing to deliver and exchange certain classes of officers and men, and as soon as they had in their possession a large number of their own, paroled by General Grant at Vicksburg and yourself at Port Hudson, they entirely ceased delivering ours as required by the cartel, but placed them in close confinement. They then proceeded to declare all of their own paroled prisoners "duly exchanged" without any equivalents delivered to us. In this way they have been able to return to duty in the field a much larger number of men than if they had made regular exchanges. This was a most shameless violation of the cartel and the general laws of war.
To now exchange the rebel prisoners in our hands for ours in the possession of the rebels would be to admit the legal exchange of the rebel prisoners already returned to duty.
Generals Hitchcock and Meredith have been doing their best to arrange this difficulty and to renew the system of exchanges established by the cartel, but it is almost useless to expect any justice or honesty from a rebel, who is described by Shakespeare "upon whom do swarm the multiplying villainies of nature."
Very respectfully, your obedient servant,
H. W. HALLECK,
General-in-Chief.
[OR Series II Vol VI p. 419]
WAR DEPARTMENT,
Washington, November 17, 1863.Major-General BUTLER, Fort Monroe: The whole subject of exchange of prisoners is under direction of Major-General Hitchcock, to whom, as commissioner of exchange, that branch of the service has been committed. He will be glad to have any idea or suggestion you may be pleased to furnish, but there should be no interference without his assent.
It is known that the rebels will exchange man for man and officer for officer, except blacks and officers in command of black troops. These they absolutely refuse to exchange.
This is the point on which the whole matter hinges. Exchanging man for man and officer for officer, with the exception the rebels make, is a substantial abandonment of the colored troops and their officers to their fate, and would be a shameful dishonor to the Government bound to protect them.
When they agree to exchange all alike there will be no difficulty. EDWIN M. STANTON, Secretary of War. [OR Ser II Vol 6 p. 528]
WAR DEPARTMENT, Washington City, December 5, 1863.
Mr. PRESIDENT:
A general summary of the military operations of the past year is furnished by the report of the General-in-Chief, herewith submitted. In the operations that have been alluded to, prisoners of war to the number of about 13,000 have fallen into the hands of the enemy and are now held by them. From the commencement of the rebellion until the War Department came into my charge there was no cartel or formal exchange of prisoners; but at an early period afterward a just and reasonable cartel was made between Major-General Dix and the rebel General Hill, which, until recently, was faithfully acted upon by both parties. Exchanges under that cartel are now stopped, mainly for the following reasons: First. At Vicksburg over 30,000 rebel prisoners fell into our hands, and over 5,000 more at Port Hudson. These prisoners were paroled and suffered to return to their homes until exchanged pursuant to the terms of the cartel.
But the rebel agent, in violation of the cartel, declared the Vicksburg prisoners exchanged; and, without being exchanged, the Port Hudson prisoners he, without just cause, and in open violation of the cartel, declared released from their parole. These prisoners were returned to their ranks, and a portion of them were found fighting at Chattanooga and again captured. For this breach of faith, unexampled in civilized warfare, the only apology or excuse was that an equal number of prisoners had been captured by the enemy. But, on calling for specifications in regard to these alleged prisoners, it was found that a considerable number represented as prisoners were not soldiers, but were non-combatants--citizens of towns and villages, farmers, travelers, and others in civil life, not captured in battle, but taken at their homes, on their farms, or on the highway, by John Morgan and other rebel raiders, who put them under a sham parole. To balance these men against rebel soldiers taken on the field would be relieving the enemy from the pressure of war and enable him to protract the contest to indefinite duration. Second. When the Government commenced organizing colored troops the rebel leader, Davis, by solemn and official proclamation, announced that the colored troops and their white officers, if captured, would not be recognized as prisoners of war, but would be given up for punishment by the State authorities. These proceedings of the rebel authorities were met by the earnest remonstrance and protest of this Government, without effect. The offers by our commissioner to exchange man for man and officer for officer, or to receive and provide for our own soldiers, under the solemn guarantee that they should not go into the field until duly exchanged, were rejected. In the meantime well-authenticated statements show that our troops held as prisoners of war were deprived of shelter, clothing, and food, and some have perished from exposure and famine. This savage barbarity could only have been practiced in the hope that this Government would be compelled, by sympathy for the suffering endured by our troops, to yield to the proposition of exchanging all the prisoners of war on both sides, paroling the excess not actually exchanged; the effect of which operation would be to enable the rebels to put into the field a new army 40,000 strong, forcing the paroled prisoners into the ranks without exchange, as was done with those paroled at Vicksburg and Port Hudson, and also to leave in the hands of the rebels the colored soldiers and officers, who are not regarded by them as prisoners of war, and therefore not entitled to the benefit of the proposed exchange. The facts and correspondence relating to this subject are detailed in the accompanying report of Major-General Hitchcock, commissioner of exchanges. As the matter now stands, we have over 40,000 prisoners of war, ready at any moment to be exchanged, man for man and officer for officer, to the number held by the rebels. These number about 13,000, who are now supplied with food and raiment by this Government and by our benevolent and charitable associations and individuals. Two prisoners, Captains Sawyer and Flinn, held by the rebels, are sentenced to death, by way of a pretended retaliation for two prisoners tried and shot as spies by command of Major-General Burnside. Two rebel officers have been designated and are held as hostages for them. The rebel prisoners of war in our possession have heretofore been treated with the utmost humanity and tenderness consistent with security. They have had good quarters, full rations, clothing when needed, and the same hospital treatment received by our own soldiers. Indulgence of friendly visits and supplies was formerly permitted, but they have been cut off since the barbarity practiced against our prisoners became known to the Government. If it should become necessary for the protection of our men, strict retaliation will be resorted to. But while the rebel authorities suffer this Government to feed and clothe our troops held as prisoners we shall be content to continue to their prisoners in our hands the humane treatment they have uniformly enjoyed.
Respectfully submitted.
EDWIN M. STANTON,Secretary of War.
[OR Ser II Vol 6 pp. 647-649]
CITY POINT, VA., August 24, 1863.
I propose that all paroles on both sides heretofore given shall be determined by the general orders issued by the War Department of the United States, to wit, No. 49, No. 100, and No. 207 of this year, according to their respective dates, and in conformity with paragraph 131 of General Orders, No. 100, so long as said paragraph was in force. If this proposition is not acceptable I propose that the practice heretofore adopted respecting paroles and exchanges be continued. In other words, I propose that the whole question of paroles be determined by the general orders of the United States, according to their dates, or that it be decided by former practice. Re. OULD,
Agent of Exchange.
In reply to my demand for the release of Colonel Streight and his command I was informed that they were in Richmond held as other prisoners of war, and will be exchanged when exchanges of officers are resumed. In relation to Doctor Rucker, Mr. Ould referred me to his letter of August 16, which I have the honor to forward herewith.
**
To my demand "that all officers commanding negro troops, and negro troops themselves, should be treated as other prisoners of war, and be exchanged as such," Mr. Ould declined acceding, remarking that they (the rebels) would "die in the last ditch" before giving up the right to send slaves back to slavery as property recaptured, but that they were willing to make exceptions in the case of free blacks. He could not exactly tell me how his authorities intended to distinguish between the two (free and slave), but presumed that evidence as to the fact of freedom would be taken into consideration. As their laws put slave and free upon the same footing no comment is necessary.**
An informal proposition was made to the following effect: "To exchange officer for officer of the same grade, except such as are in command of negro troops;" which was declined.
Mr. Ould expresses a willingness to release all chaplains, provided that one Septimus Cameron, who, he stated, had been in prison for a year, should be released, or indicted for any offense he may have committed. On my inquiring about and urging the release of the members of the Sanitary Commission, I was informed that they would be set free on making a statement in writing that they had at any time been of assistance to rebel soldiers. General Neal Dow has been handed over to the Governor of Alabama. Lieutenant-Colonel Powell is in Libby Prison, Richmond. I have notified the rebel authorities in relation to the two above-named officers, as directed in yours of the 18th ultimo [instant].
The rebel authorities wish to continue exchanging non-commissioned officers and privates as usual, returning as many as we send. I have given you, I believe, the substance of all that took place, according to your suggestion. I avoided much discussion. No agreement as to exchanges was arrived at.
I am, general, very respectfully, your obedient servant,
S. A. MEREDITH,
Brigadier-General and Commissioner for Exchange. [OR Ser II Vol 6 pp. 225-226]
From the above, we can see the exchanges were halted before Grant took command of Union armies.
Grant continued to carry out the policy that had been established before he was promoted.
HEADQUARTERS ARMIES OF THE UNITED STATES,
In the Field, Culpeper Court-House, April 17, 1864.
Maj. Gen. B. F. BUTLER, Comdg. Dept. of Virginia and N. Carolina,
Fortress Monroe, Va.:
GENERAL:
Your report of negotiations with Mr. Ould, C. S. agent, touching the exchange of prisoners, has been referred to me by the Secretary of War with directions to furnish you such instructions on the subject as I may deem proper. After a careful examination of your report the only points on which I deem instructions necessary are:
First. Touching the validity of the paroles of the prisoners captured at Vicksburg and Port Hudson. Second. The status of colored prisoners. As to the first, no arrangement for the exchange of prisoners will be acceded to that does not fully recognize the validity of these paroles and provide for the release to us of a sufficient number of prisoners now held by the Confederate authorities to cancel any balance that may be in our favor by virtue of these paroles. Until there is released to us a sufficient number of officers and men as were captured and paroled at Vicksburg and Port Hudson not another Confederate prisoner of war will be paroled or exchanged. As to the second, no distinction whatever will be made in the exchange between white and colored prisoners; the only question being, were they at the time of their capture in the military service of the United States. If they were the same terms as to treatment while prisoners and conditions of release and exchange must be exacted and had in the case of colored soldiers as of white soldiers. Non-acquiescence by the Confederate authorities in both or either of these propositions will be regarded as a refusal on their part to agree to the further exchange of prisoners, and will be so treated by us.
I am, general, very respectfully, your obedient servant,
U.S. GRANT, Lieutenant-General.
[OR Ser II, Vol 7, pp. 62-63]
In October of 1864, R. E. Lee proposed an exchange of prisoners to Grant:
HEADQUARTERS ARMY OF NORTHERN VIRGINIA,
October 1, 1864.
Lieut. Gen. U.S. GRANT, Commanding Armies of the United States:
GENERAL: With a view of alleviating the sufferings of our soldiers, I have the honor to propose an exchange of the prisoners of war belonging to the armies operating in Virginia, man for man, or upon the basis established by the cartel.
With much respect, your obedient servant,
R. E. LEE,
General.
[OR Series II, Vol VII, pp. 906-907]
Grant was receptive to this proposal, but insisted that Black troops be exchanged as well:
HEADQUARTERS ARMIES OF THE UNITED STATES,
October 2, 1864.
General R. E. LEE, Commanding Army of Northern Virginia:
GENERAL: Your letter of yesterday proposing to exchange prisoners of war belonging to the armies operating in Virginia is received. I could not of a right accept your proposition further than to exchange those prisoners captured within the last three days and who have not yet been delivered to the Commissary-General of Prisoners. Among those lost by the armies operating against Richmond were a number of colored troops. Before further negotiations are had upon the subject I would ask if you propose delivering these men the same as white soldiers.
Very respectfully, your obedient servant,
U.S. GRANT,
Lieutenant-General.
[OR Series II Vol VII p. 909]
Lee then refused to include the Black troops he considered to be escaped slaves. How he would know which were escaped slaves and which weren't is unknown:
HEADQUARTERS ARMY OF NORTHERN VIRGINIA,
October 3, 1864.
Lieut. Gen. U.S. GRANT,
Commanding Armies of the United States:
GENERAL: In my proposition of the 1st instant to exchange the prisoners of war belonging to the armies operating in Virginia I intended to include all captured soldiers of the United States of whatever nation and color under my control. Deserters from our service and negroes belonging to our citizens are not considered subjects of exchange and were not included in my proposition. If there are any such among those stated by you to have been captured around Richmond they cannot be returned.
Very respectfully, your obedient servant,
R. E. LEE,
General.
[OR Series II Vol VII, p. 914]
Grant then properly declined to participate in the exchange since the rebels refused to treat all US soldiers equally:
HEADQUARTERS ARMIES OF THE UNITED STATES,
October 3, 1864.
General R. E. LEE, Commanding Army of Northern Virginia:
GENERAL: Your letter of this date is received. In answer I have to state that the Government is bound to secure to all persons received into her armies the rights due to soldiers. This being denied by you in the persons of such men as have escaped from Southern masters induces me to decline making the exchanges you ask. The whole matter, however, will be referred to the proper authority for their decision, and whatever it may be will be adhered to.
Very respectfully, your obedient servant,
U.S. GRANT,
Lieutenant-General.
[Ibid.]