I wish people who say "Regulation" would check what the 1861 US Army Regulations actually says before posting information.
When you do, you'll find the following:
FORAGE CAPS (note: this is the official Army term, not "bummer's caps".)
1493. For fatigue purposes, forage caps, of pattern in the Quartermaster-General's office: dark blue cloth, with a welt of the same around the crown, and
yellow metal letters in front to designate companies.
1494. Commissioned officers may wear forage caps of the same pattern, with the distinctive ornament of the corps or regiment
in front.
(this can be found in many places on line, I'm quoting this one)
http://www.unionvolunteers.com/uv/FILES/USMANUALS/Union Volunteers - REVISED UNITED STATES ARMY REGULATIONS OF 1861.pdf
Notice the words "in front". This means the company letter is to be worn on the BAND of the cap, NOT on the circular crown --
according to regulation. And that's all there is - leave the branch of service insignia, regimental number and for Heaven's sake the eagle, cord and feathers for your regulation dress HAT.
Now as we all know - Americans of all generations in all wars have thought the most fun can be had with unimportant regulations by disregarding them. (Of course one's officers and NCOs may have a different view.) But as the photos previously posted show, this regulation was often honored principally "in the breach". Still, please don't spread misinformation by calling a practice "regulation" when it clearly was not.
(P.S. I first discovered this as a 12 year old kid interested in painting military miniatures. Why grown men presuming to be reenactors can't seem to find it still eludes me...)