Yes, the photo is of Belle Plain. Here is a similar Matthew Brady photo. Note the same packet-type boats to right of wharf as in the OP.
And
Yes, your photo is most probably of Confederate prisoners being transported north.
In May 1864, Union planners searching hurriedly for a supply base in the Fredericksburg area chose the stretch of shoreline known as Belle Plain. Situated on the south side of Potomac Creek near its junction with the Potomac River, Belle Plain offered many logistic advantages: it was closer to Fredericksburg than were the nearby Aquia Creek landings, it did not share Aquia's dependence upon a connecting railroad, and it had hosted Federal supply operations earlier in the war. Thus Union forces reoccupied Belle Plain by May 10, 1864 and on May 12 constructed the first of two pontoon wharves there: one on the west side of Whipsawason's Point, one-half mile east of (downstream from) modern Belle Plain Landing ("upper wharf"), and another ("lower wharf") at Pratt's Point, a mile downstream from Whipsawason's.
View attachment 320549
Matthew Brady's photo and your photo show the Union supply base (area known as "Lower Wharf") along the Potomac River at Belle Plain, VA. on about May 16, 1864 in support of General U. S. Grant's Overland Campaign. The photo depicts the US Military Railroad Construction Corps (USMRCC) built lower wharf/dock made of pontoon construction. It shows army supply wagons on the wharf being loaded onto ships to furnish goods to the Union army scattered throughout the area. In addition to a number of supply tents seen in the foreground, ships are anchored waiting provisions. By May 23, 1864 the Belle Plain depot was being evacuated as Grant's army moved further south, necessitating the relocation of the Union supply base to Port Royal, Va.