I wanted to provide direct quotes from Julia's memoirs and other sources regarding some of what has been referred to in this thread:
For the sake of context here is a basic timeline:
Aug. 1848 - Grant returns from the Mexican War to St. Louis & marries Julia
1848-1852- Grant stationed at various posts, Julia with him or at White Haven or at the Grant home in Bethel OH.
1852-1854- Grant stationed on west coast, Julia stays behind at White Haven with 2 children
Spring 1854- Grant resigns and returns to White Haven
Spring 1855 - The Grant's move into Wish-ton-wish on the White Haven estate
Fall 1856 - The Grant's move into Hardscrabble built on the 60 acres given to her by her father on the White Haven estate
January 1857 - Julia's mother dies and the Grant's move back into White Haven
Spring 1858 - Col. Dent leaves White Haven for St. Louis city home
Early 1859 - The Grant's move into St. Louis city and Grant manumits slave William Jones
Spring 1860 - The Grant's move to Galena IL
1861-1863 - Grant at warfront, Julia when not with him stayed with his parents in Covington KY or with her father at White Haven
Early 1864 - Julia in Washington DC
Summer 1864 - Julia returns for the summer to stay with her father at Wish-ton-wish on White Haven estate
Fall 1864 - Julia moves into a home in Burlington NJ
Julia states that Grant was operating the entire White Haven estate after Julia's mothers death ( January 1857)....
"My husband conducted his own farm and father's as well. Father was getting quite old and missed dear mamma deeply."
This is interesting because it shows that Col. Dent gave Grant some level of authority over the operation of the entire farm and therefore the slaves on it. Grant would state in a letter around this time that
"I have now three negro men, two hired by the year and one of Mr. Dent's..." Just because he was managing the farm for his father-in-law obviously does not automatically mean he or Julia assumed ownership of his slaves.
As Grant prepared to leave White Haven he wrote a letter to his father in Covington, KY regarding the possibility of moving there and working for him. In the letter he included a proposal to bring along one of Julia's slaves...
"Mr. Dent thinks I had better take the boy he has given Julia along with me, and let him learn the farrier's business. He is a very smart, active boy, capable of making anything; but this matter I will leave entirely to you. I can leave him here and get about three dollars per month for him now, and more as he gets older." -USG to Jesse Grant (October 1, 1858)
This is interesting for a lot of reasons: 1. Why would Jesse want a slave around when he doesn't want his son living among slaveholders. 2. Why does he bring up Mr. Dent, is it because he still has say in what happens to the slave, or just because he received advice from him. 3. Why reference Mr. Dent at all, knowing Jesse's disposition towards him, it would not have made the situation any more desirable. Purely speculative but perhaps it was to distance himself from his father seeing him as owner? One important note is that as we see in this instance and in a later passage from Julia, they did hire out slaves for profit. Whether they did this as actual owners or simply as a permissive arrangement with Mr. Dent is unknown.
When they moved from White Haven to city of St. Louis (1859) Julia states...
"...we had sold our farm, horses, farming implements, and crops. The four slaves we kept...We had brought into the city with us the four servants my father had given me, but these were young, ranging from eighteen to twelve years of age. They were born at the old farm and were excellent, though so young."
When they left St. Louis for Galena (spring 1860) she states...
"We...hired out our four servants to persons whom we knew and who promised to be kind to them. Papa was not willing they should go with me to Galena, saying the place might not suit us after all, and if I took them they would, of course, be free..."
As stated before, their disposition of the servants illustrates a level of control over their fate, at least to the extent of being able to rent them out. Interesting use of language in the following sentence that her father "was not willing" they should go. Is this just her receiving the advice of her father or does he still exert some level of authority over the situation. It seems as though it's moot point though as the final statement holds true that the enslaved could not be kept enslaved in Illinois.
This also brings up Julia's perspective on slavery. She, like many of her time in that area, had a familial connection with her slaves calling them "dear old black uncles" and "our colored people." She would have felt that the slaves she grew up with were "hers" in a sense regardless of legal ownership. She did state that her father was "the kindest of masters to his slaves" but more often used possessive references like describing "old black Kitty" as being "my nurse" and her black childhood playmates as "my dusky train" and "these little maids of mine." This sense of possession may have led her to state things in a certain way later in her memoirs. Descriptions of White Haven closely resemble the slave society described in the book
Slavery in Missouri, 1804-1865.
"There is not one word of truth in the statement made by a late chronicler of General Grant that he ever offered to sell one of these dear family servants of mine, nor in the statement that one of these girls was left with, or given to, Mr. Long for debt....Eliza, Dan, Julia and John belonged to me up to the time of President Lincolns' Emancipation Proclamation." (Note: The slave names, other than John, match with an account by her sister Emma Dent Casey who said Julia's slaves were the cook Colored Julia, housemaid Eliza and houseman young Dan)
She is again alluding to the fact that Her and Grant had the authority or permission to dispose of the servants as they pleased. She is mistaken of course when it comes to the Emancipation Proclamation as that did not apply to slavery in Missouri. It's possible she was just stating this as the signal at which she believes her servants ceased to feel as though they were bound and took their leave (as we know they did by her later statement that they were gone by early 1864.)
"When I visited General Grant during the war, I nearly always had Julia with me as a nurse. She came near being captured at Holly Springs."
This is an somewhat of an overstatement as the time Julia spent with Grant in 1864 (Black) Julia was no longer with her. She states that sometime in the winter of 1863-64
"At Louisville, my nurse [Black Julia]...left me, as I suppose she feared losing her freedom if she returned to Missouri. I regretted this as she was a favorite with me."
When she returned home to St. Louis for the summer of 1864 she states...
"Our colored people had all left, their places were readily filled by German and French men and women, who were most excellent substitutes."
Despite calling them "excellent substitutes" she goes on to say that without their old servants it was more difficult to entertain the number of visitors they were receiving. It's important to note that Julia lived without slaves in Detroit MI & Sackett's Harbor NY in the 1850's and Galena IL & Burlington NJ in the 1860's. She had servants of different sorts in some of the locations and at some of the Headquarters homes she shared with General Grant during the war. All this to say she had already had experience with paid servant labor by 1864.
One reality that has to be taken into consideration when evaluating Julia's recollections (recorded many years after the fact) is that they contain known errors and rather "optimistic" characterizations of persons and events at times. She tends to romanticize situations and sometimes downplays difficult or contentious moments. So knowing that she was somewhat loose with her interpretations at times in her work it casts some level of doubt on the material. In this situation it's nice to have come corroborating evidence to help reach a conclusion.
Ulysses, Julia and "Colonel" Dent at Long Branch NJ ca. early 1870's