I think Varina did feel that Davis's heart had never truly been 100% hers, and I say that because I could swear I read that she advised her daughter on that particular issue, but I will later look through my book and see if I can find the quote.
(ETA: I'm scanning, and though I haven't found it yet, I think the advice was for a friend's daughter, not her own.)
There's no judgment towards Davis from me though. I think a second marriage after the death of the first spouse is just going to be complicated, period. From Davis's point of view, I can understand wanting to visit Sarah's grave, looking to pay respect and perhaps silently asking for approval. I get that kind of loyalty.
From Varina's, I can completely understand if that was unnerving for her. And if she convinced herself out of fear that Davis was never 100% hers, I question whether or not he could have done anything to put her mind at ease. (In other words, he could have truly been totally devoted, but she would never have believed it.)
Makes me wonder...how many "true love"s does each person have? Is it really only one? My mother is sure there is more than one person out there with whom you can make that kind of connection ~ there are just too many personality combinations and possibilities to limit a person's life to one true love. With fear, doubt, and guilt cast aside, it is possible that a man in Davis's position could fully and happily commit to Varina. (Whether or not Varina believed it is another story.

)