It was political of course. And because of this battle the Joint Committee on the Conduct of the War was started. This comittee turned out to be basically a radical republican "witch hunt". Here is the wikipedia explanation.
Baker's death and the action at Ball's Bluff would have serious consequences for Stone, and also affect the way the American Civil War would be prosecuted. In his official report about the battle on October 24, McClellan did not hold Stone personally responsible for the defeat, saying "The disaster was caused by errors committed by the immediate Commander—not General Stone."
Arrest and imprisonment
Stone bore the brunt of much public criticism; the
U.S. Congress Joint Committee on the Conduct of the War was established in the wake of Baker's congressional eulogies and anger over the defeat. This seven-man group called Stone as one of their first witnesses about the Ball's Bluff affair, and all testimony given by him and 38 others was kept secret. Before the end of October 1861, Stone's official report about Ball's Bluff had been leaked to the
New York Tribune newspaper, and in it Stone praised Baker's bravery but made clear his shortcomings as a field commander.
Baker's congressional allies, among them Governor Andrew and Senator Sumner, openly denounced this report and began to point accusing fingers at Stone, not at Baker. Stone's loyalty to the Union and position on slavery were more in question than his military abilities and decisions. The committee's questions accused him of improper and frequent communications with the Confederates, of not re-enforcing Baker, of using his men to protect slaveholder property in Maryland, and of returning runaway slaves to their owners—despite the last two of these following Maryland as well as Federal law. Another problem for Stone defending himself was an order from McClellan forbidding him to give testimony "regarding his [McClellan's] plans, his orders for the movement of troops, or his orders concerning the position of troops." This made it impossible for Stone to explain his movements to the committee, but kept McClellan out of the investigation as well.
Under a cloud for suspected disloyalty and treason, Stone was arrested just after midnight on February 8, 1862, on orders of Maj. Gen. McClellan, who was acting under orders from
Secretary of War Edwin M. Stanton, dated January 28. Awaiting Stone near his home in Washington were 18 soldiers led by Brig. Gen.
George Sykes. When Stone approached, Sykes stated "I have now the most disagreeable duty to perform that I ever had—it is to arrest you." When Stone angrily asked why, Sykes said "I don't know. It's by order of Maj. Gen. George B. McClellan, general-in-chief of the army... I may as well tell you that you are to be sent to
Fort Lafayette." This shocked Stone, who stated "That's where they send
secessionists! I have been as true a soldier to the Government as any in service."