Didn't both sides have regulations as to what could and couldn't be worn?
Yes. and these regulations allowed for the wear of the regulation uniform, as would be worn on formal occasions and on parade, as well an "undress" for particular occasions. There was also allowance for a "fatigue" dress. However, if one reads carefully, there is not much specific to dress in the field or on the march. US troops generally just wore their fatigue dress on the march, etc.
Union officers generally often dressed informally in the field, taking their cues from the higher ups. Like army commanding general McLellan, who wore a sack coat cut and trimmed something like a regulation uniform coat... but without the black velvet collar/cuffs, etc., and with side pockets...
As commanding General in '64-'65, Grant wore his uniform coat unbuttoned, and in the last just a plain citizen's sack with shoulder straps...
And from 1864 officers in the field could dispense with regulation rank insignia...
General Orders 286 November 22, 1864
Officers serving in the field are permitted to dispense with shoulder straps and the prescribed insignia of rank on their horse equipments. The marks of rank prescribed to be worn on the shoulder-straps will be worn on the shoulder in place of the strap. Officers are also permitted to wear overcoats of the same color and shape as those of the enlisted men of their command. No ornaments will be required on the overcoat, hats or forage caps; nor will sashes or epaulets be required.
BY ORDER OF THE SECRETARY OF WAR
The references to the sashes etc. is interesting, as according to Winfield Scott Hancock, writing to the Gettysburg cyclorama painters, the sashes weren't worn in the field generally by Gettysburg, over a year before the army "officially" allowed for them to be dispensed with for the purpose. (see Union Uniforms at Gettysburg, by Thomas Publications).
Here's some Union officers in the west prepared to march...
On the Southern side, for example, in mid-1862, the Confederate War department ordered that all officers in the field should wear less conspicuous "fatigue dress," including uniform coats without all of the rank insignia, or "jackets" (without specifying the type, whether long, short, standing or falling collar, etc.)...
The officers frequently wore slouch hats like the men in lieu of any kind of regulation cap...
General Lee at Gettysburg is described as wearing his uniform coat on the early morning of the second day, but otherwise dressed informally in a long gray jacket... identically to a country farmer by several accounts.