Yes.
"
President Grant invited him to the White House in 1869, and he went."
From:
http://en.wikipedia.org/wiki/Robert_E._Lee
Here is an account of that short conversation they had
May 1st, 1869 in the White House - as described in Douglas Southall Freeman's biography of Robert E. Lee:
"The following conversation occurred between the President and General Lee, which lasted about five minutes:
"
General Lee: Mr. President, I called today, in accordance with your kind invitation, with my friends here, Mr. and Mrs. Taggart
[sic], of Baltimore, to thank you for the honor you have done me.
"
President: I did wish, General, to have a somewhat lengthy conversation with you in regard to matters relating to your section of the country, if such will be agreeable to you.
"
General Lee: Mr. President, I would much prefer that you should not take my opinions and views as representing those of the people of Virginia and the South, and I do not think I could give any useful information on that subject. If you will excuse me, Mr. President, I will repeat my thanks for your invitation, and bid you goodday."
All this may conceivably have been said, in substance, but it certainly is not all that was said. Lee would not have been so abrupt.
[...]
The interview was short, and all that Grant could remember afterwards was that they spoke of building railroads, and he said playfully to Lee, 'You and I, General, have had more to do with destroying railroads than building them.' But Lee refused to smile, or to recognize the raillery. He went on gravely with the conversation, and no other reference was made to the past. Lee soon arose, and the soldiers parted. . . ."
From:
http://penelope.uchicago.edu/Thayer/E/Gazetteer/People/Robert_E_Lee/FREREL/4/Appendices/6*.html
That was the last time the two men had seen each other.
From:
http://deadpresidents.tumblr.com/post/20787674562/grant-and-lee
"
The two men who had been such a huge part of each other's lives would never see each other again. Perhaps the most amazing thing is that Grant and Lee could come together at all after chasing each other throughout the country and killing thousands of Americans while trying to destroy each other. It's a tribute to the two men that they were living in a country still needing time to heal, and they stepped forward — leading the way just like they did while waging war — to model for Americans how to wage peace".