On the other hand, a cavalry trooper on temporary duty as provost might still carry his pistol out of habit, especially if he was with headquarters of a battle division.
Habit had nothing to do with it. The CS was chronically short of pistols. Even at the end of the war there were CS Cav units sans pistols carrying only sabres instead even in the very well armed ANV. By late war the condition was worsening on an almost daily basis. A Cav trooper sans horse but still in service would find himself in the Infantry w/ Infantry gear or without gear altogether. His tack, weapons & gear going to equip troops with horses. A man working as a clerk in the Provost office would not likely have a pistol, sword or anything as his job was considered most certainly as a rear area job out of the way of combat.
Now if he was detached on Provost Duty would he keep his arms and horse? That is a question I don't know the answer to but I defer to those who were there who often viewed those without a horse in sight as Infantry. The horse culture of some Texas units is best exemplified by a Tennessee infantryman remarking on watching members of a Texas Cav unit mounting their horses to cross the street. While at the time I first read that I thought the soldier was talking through his hat until I read several other observations of the same thing.
Typically an enlisted clerk was a position that dealt with a LOT of paperwork and documentation. He had to have a readable hand and be observant enough to take notes and such as well as read and triage correspondence. These men were often taken from the ranks of the army who had been clerks, teachers or others of a similar profession in civilian life. It was a much sought after job as it came with an additional pay stipend and kept the soldier well to the rear under normal circumstances. A unit certainly acts differently on active campaign vs on garrison duty. But a Provost on active campaign also has dramatically different roles than in garrison.
A soldier being assigned to a different duty than the field would likely find his gear and horse passed to another Cav trooper to maximize unit strength. In theory when he returned to the unit he would receive gear back of the same type and quality he passed on before he departed... if such was still available.
Keep in mind that the officer of the CS generally obeyed regs to the best of their ability though this decreased the farther one got from Virginia to the point that units west of the Mississippi in what we today call the Trans Mississippi were a whole different critter than the ANV. The CS Army had a definite pecking order with the ANV getting forst pick of everything from horses to arms and equipage. Next came the AoT and it rolled down hill from there, quite noticeably after 1863 and the loss of the Mississippi.
As a note the CS Army was no different than any army in history, those who got the cushy jobs were largely despised by the rank and file. Today the US military refers to them semi affectionately as "Pogues" or less affectionately as REMF's. Clerks and those who were able to dodge their time on the line had a very real and often underappreciated task. They were at the heart of the bureaucracy that was the army. They were the men that filed the needed requisitions, pay vouchers and pay call documentation as well as taking notes and filing the final disposition paperwork for ordnance returns, deserters, POWs, invalided troops, hospital correspondence etc. A literal mountain of paperwork that was agonizingly dull but necessary and I'll wager they rarely let anyone forget how vital they were.
FWIW I can't even fathom a clerk trying to keep track of the back pay owed Wheelers men as most were never again paid after the spring of 64.