In the case of CW sailor Frederic Augustus James, he would wait until he got paid, then would go to the nearest Adams Express office and give them the money, which I gather they would then either telegraph or forward to his wife back in Boston. There is a note on Wikipedia that says Adams Express served as paymaster for both the North and the South.
In the case of the soldiers captured at Plymouth, NC, they had just gotten paid and had cash in hand when they were captured, and one of the terms of surrender was that they could keep all of their belongings, and so they took the money with them to Andersonville.