nor did he take any of his wagons
That implies he had wagons to take and did not. I don't think he had wagons. Tony had pointed out that on 05/06 Pemberton did order Gregg to bring his wagons, but the day before Gardner telegraphed that he could not move until the wagons returned. As I said, while it isn't conclusive that the wagons Gardner was awaiting the return of were those wagons, it does seem more likely than not that the wagons used to transport Gregg went back to PH. My issue here is if Pemberton wanted Gregg to move with wagons he should have stated that in his original movement order to Gardner, but he did not.
A dispatch written 5 days later, well after the movement had commenced, carrying that order is well beyond what a commander ought to think is a reasonable request. I think a personality flaw existed in Pemberton in that when he issued an order he expected it to be executed, no matter how much clarity, completeness, and
timeliness the order lacked. I don't think Pemberton ever understood that Gregg was without wagons. He never asked and no one ever said that Gregg had no wagons. Pemberton just assumed it because he ordered it.
Walker bringing up his wagons and rations.
Again, implying Walker had his own wagons. I don't believe Walker came with his own wagons. If you look back to how the order to Beauregard for reinforcements was carried out , you will see that both brigades were infantry supports for seacoast defense and had not even been brigaded together until the 05.01 order. That movement is a great story all by itself! I can't say for sure, but it seems probable that there were no brigade wagons.
How do you think his men were supplied during this time?
Why Alan, the spirits of the ladies of the UDC are frowning down upon you now. You know the dear ladies of Raymond spread out a delicious feast for the men upon their return from victory in the field, you know how well that worked out.

Otherwise, I could only surmise they foraged best they could while on the march and again once they made camp in Raymond. I know Davidson refers to going foraging and being given some onions (I think).
I think the follow-up question would be, "Why were they unable to get rations and wagons in Jackson?"
So here's where I get a little wonky and a lot subjective in my reply, what's that saying, "the devil is in the details"?
1. While at Port Hudson any wagons Gregg could have accessed belonged on the books of the 3rd District, so wagons would not have been carried on his Master Essential Equipment (MEE) list. IIRC, Gregg was moved to PH via the Mississippi and since PH was a post with it's own staff departments maybe Gregg's wagons (if he had any) stayed behind. I think they had been on duty on the Yazoo when ordered to PH, but no matter.
2. As soon as Gregg boarded the trains in Osyka, Gardner's command authority ended and administrative (ADCON) and operational (OPCON) control passed from Gardner to Pemberton. So Gardner's QM detailed for the movement took control of his wagons and headed back to PH.
3. Since Gregg's brigade wasn't a part of a division Gregg would have been a direct report unit (DRU) to the DMEL commander. So the ADCON logistical support of Gregg's brigade becomes the responsibility of the DMEL staff, QM, Ordnance, and Commissary.
4. Unfortunately for Gregg, all the staff Chiefs were busily executing Pemberton's 05.07 order to evacuate Jackson. So if I were to hazard a guess, I'd say there were only so many wagons at Jackson belonging to DMEL and the QM (Mimms) had a written order to move. Perhaps Gregg went to Mimms, showed him his orders, confident he could get the wagons, but Mimms showed Gregg his orders and refused Gregg any wagons. This scenario could have played out just as well with Theo Johnston, the Commissary Chief.
Considering Pemberton's obsession over logistics, if anyone were going to deviate from an order, it wasn't going to be Majors Lafayette Mimms or Theo Johnston whose heads rolled!