Confederate Memorial Hall is great (and the National World War Two Museum across the street if you are interested in that era as well). The Herman Grima House is interesting as well.
The US Mint in the city is another place worth visiting.
Camp Parapet's powder magazine is still around, but there isn't much else.
The Chalmette Battlefield in St Bernard is pretty nice if you are interested in the Battle of New Orleans.
If you are willing to travel a little bit then here are a few places to visit:
-Camp Moore- Largest confederate training camp in the state near Kentwood, Louisiana
-Fort Jackson in Buras (I am not sure if it is still open as it received a whole lot of damage during Katrina and the hurricanes that passed through that area since).
-Fort Pike in the Rigolets near Slidell (it is state owned and is no longer open to the public, but it still a nice fort. It was also a stopover on the Trail of Tears)
-Port Hudson just north of Baton Rouge
-Baton Rouge- Magnolia Cemetery, the Old Federal Arsenal, and the Old State House (Mark Twain called it one of the ugliest buildings he's ever seen). All have connection to the Battle of Baton Rouge and the subsequent federal occupation of the city.
-Although Ship Island is in Mississippi, its a short drive from the city. This is where Butler's men staged before the invasion of Louisiana.