It's easier to deal with oppressive structures when enough centuries have passed that the emotions can die down.I said this on another web site and got a lot of "upvotes": If the British had torn down all their structures that had any bad history associated with them, we would then have to imagine what castles looked like, or the Tower of London, or royal palaces, or majestic old mansions. If European nations did the same thing, European towns would be as charming and quaint as a bowl of plain oatmeal. And goodbye to the incredible old fortresses and castles of Europe. Yeah! Let's be politically correct and tear it all down!
When you still have the residual emotions and memories of an oppressive system, and you still have some fellow Americans pining for the confederacy, the plantations will continue to have a stigma as labor camps. And they should have that stigma.
As long as we still have people advocating for the confederacy, we will have people advocating for obliterating its memory.