A star pinned or sewn to the hat is, of course, best associated with troops from the Lone Star State; however, it was also very popular with Mississippians and was worn by some Confederate troops from other states as well.
The lone star was widely viewed as a symbol of secession and independence at the time. Different variations of lone star flags were flown in the Texas Revolution and adopted as the official flag of the Republic of Texas, the final design becoming the state flag. It also appears on many early secessionist banners and Southern state flags of the Civil War.
One of the most famous, the Bonnie Blue Flag, was presented to the Mississippi Secession Convention amid cheers after the Ordinance of Secession was passed on January 9, 1861. It was soon after applied to the state flag. That might explain why the lone star was also worn by so many Mississippi troops. It is believed that the Bonnie Blue and possibly other flags were inspired by the banner of the Republic of West Florida (as explained Here), one of the first to bear a single star after the province declared itself independent from Spain in 1810.
Secessionists may have also seen the lone star as symbolic of the removal of their state's star from the American flag, thence their state from the Union.
Whatever inspired them, many Southerners heading off to war pinned or sewed stars to their hat or uniform. If not only to make a political statement, if any, it certainly gave some of them a jaunty, audacious look that expressed the attitude of the typical young volunteer. Obviously, for Texans it was also a symbol of state pride and many continued to wear the lone star throughout the war.
This thread is for compiling any photos of Confederates troops wearing a star on their hat or uniform (other than rank insignia). I'll post a few from Texas first, then Mississippi and other states.