I thought many of them were mounted on mules. Since the regiments were not intended to fight while mounted, mules would have been sufficient.
After the Union took Nashville, Confederate General Morgan flanked the Union lines and moved north towards Louisville to destroy supply lines and Wilder was tasked with chasing them down. Then, on page 23:
"Then in a vain attempt to overtake the Confederates Colonel Wilder mounted a detachment of the 17th Indiana on some of the wagon train mules. This decision proved to be disastrous. The mules bucked, jumped on each other's backs and tried to run under each other's bellies in order to dispose of their riders."
This event was the catalyst for Wilder to try and mount his infantry. He accomplished this by stealing/commandeering horses from the citizens in the area. It was also the catalyst from him to issue his men hatchets as they had frozen to the ground because they couldn't bivouac well.
I think that by the time the Battle of Chattanooga occurred the regiments were separated and assigned to cavalry brigades.
Between Chickamauga and Chattanooga they were assigned to Gen. Crook's Cavalry but notably did not fight as cavalry. They maintained their mounted infantry tactics throughout and regular infantrymen revered them for it. Most notably with Crook they pursued Wheeler's cavalry while Wheeler tried to destroy Union supply lines in the west, it was probably their hardest fought campaign in the war. From page 112:
"In only nine days they had traveled three hundred miles over rugged terrain on two days' rations, fought three skirmishes and one pitched battle. But they had accomplished their mission; they had kept open the supply lines to the besieged troops in Chattanooga."
At the end of this campaign they had captured so many horses that they rode out with 1000 more horses than men.
Anecdotally, I drive the terrain they covered by car and sometimes
that struggles with the steepness. I can't imagine doing it on horseback, in the pouring rain, dragging cannons behind me, with much less in the way of road infrastructure.
You're right they were broken up but they were reunited for the Atlanta campaign alongside Col. Minty and Col. Eli Long as part of General Garrard's 2nd Cavalry Division sans Eli Lilly's artillery battery which had reached the end of enlistment in 1863.