I can't dig up specific sources for this statement at this moment, but I believe quite a few Southern women were angry about the surrender. There were large swaths of the Confederacy who never saw a Union soldier but the locals still suffered the loss of loved ones.
Many, if not most men on both sides, who were doing the actual fighting, developed a respect for their opponents and knew when the game was over. They had done their best. For some women and other non-combatants however, who had no reason or desire to call it quits until deaths had been avenged and their choices validated, the surrender was a bitter pill. But I've read of no one who blamed Robert E. Lee. He was beyond reproach.