This is taken from the second link that
@USS ALASKA provided giving further insight into motivations for halting of prisoner exchange:
"From the three-score years that preceded Bennett’s comments, there emerged a literary effusion of articles, expositions, and anecdotes about prisons. Veterans from the North and South were uniformly convinced “that their jailors had subjected them to treatment heinously designed to reduce their ranks by starvation and disease.” Moreover, ex-prisoners called into question, some in published form, the leaders and policies that had created and protracted their time in the valley of the shadow. Thus despite what Bennett held to be true in the twentieth century, few veterans believed to be true in the immediate postwar period of the nineteenth century.
The first published accounts of prison life in the South appeared in the late summer of 1862 following the exchange agreement between opposing governments. The initial accounts were written to incite a public outcry against the inhumane treatment of soldiers by southern “barbarians.” Yet because the very nature of an uninterrupted exchange reduced the overall time that individuals were held in captivity, the original published accounts failed to produce much response, humanitarian or otherwise, in the North.
When autumn approached a year later in 1863, however, Union policy regarding the cartel changed, and the exchange was halted. The Lincoln administration attributed their suspension of the cartel to several factors: the South’s refusal to validate its parole and exchange of the Port Hudson and Vicksburg prisoners; the desire of Secretary of War Edwin Stanton and General Ulysses S. Grant to curb desertion and bounty jumping by federal soldiers; and most importantly, Grant’s belief that the most expeditious way to relieve all inhumane suffering and treatment was to end the war. The northern policy change, then, was in keeping with Grant’s desires to end the war, one he called “a military necessity.”
Secretary Stanton utilized the stoppage of the exchange, moreover, to inaugurate a renewed prison literature campaign on the national level to help deflect growing criticism from Washington onto Richmond. Stanton wanted to convince northerners, and the rest of the world, that the southern confederacy had erected and maintained a “deliberate system of savage and barbarous treatment and starvation” of northern prisoners. He prodded the U.S. House Committee on the Conduct and Expenditures of the War as well as aid societies like the United States Sanitary Commission and Christian Commissions to make public their interviews with former prisoners, audits of camps, and abstracts of their official reports and findings.
By the early summer of 1864, the U.S. House committee finished its investigation of ex-prisoners from an Annapolis hospital and made its official report. The report, containing thirty pages and eight photographs, summarized the testimony of prisoners held in Richmond’s Libby Prison and on nearby Belle Isle on the James River in Virginia. It was quickly distributed throughout the north among the various presses. The House committee said that the “evidence proves, beyond all matter of doubt . . . that the inhumane practices . . . are the result of the determination on the part of the rebel authorities to reduce our soldiers in their power, by privation of food and clothing, and by exposure.” Secretary of War Stanton added fuel to the report when he said, “The enormity of the crime committed by the rebels towards our prisoners for the last several months is not known or realized by our people . . . [as there] appears to have been a deliberate system of savage and barbarous treatment and starvation.”
Thus heading into the fall elections of 1864, Stanton’s incendiary rhetoric, combined with the report of the House Committee on the Conduct and Expenditures of the War, served as the spark that helped inflame the northern populous behind the Union war effort and President Lincoln’s reelection bid. Northerners seemed to accept that the war must continue, and thus bolstered the decidedly Republican policy objectives."
(Sources are included in link)