In addition to everything else that has come up so far in this thread, average train speeds also had a lot to do with railroad economics, which is a complex subject. All railroads (other than those that had been seized by the government during the war) were private businesses, so every decision was ultimately a financial one. Railroad history is littered with failed companies that were not good at figuring out how much it would cost to make trains go faster, or at estimating how much the customers would be willing to pay extra for faster delivery, or in finding the right balance point between the two sides of this equation.
Dispatching was a key factor. Everybody knew that passengers and mail were highly time-sensitive, and everybody knew what to do about it. Those trains were kept shorter, and given the highest dispatching priority, to keep their average speeds close to the track speed limit. Other loads were quite different. With bulk commodities like wood, coal, textiles, stone, bulk liquids, and some dry agricultural materials, speed was not important at all. Shippers didn't care if a delivery took two weeks as long as the cost was low. The way to make money moving bulk commodities was (and still is) to make the trains as long as possible, loaded to the point where the locomotive could barely make it to the top of the steepest hill on the route (known as the ruling grade). The trick was keeping those trains out from in front of passenger and mail trains. Making matters more complicated was the fact that manufactured items were more time-sensitive than bulk commodities, but less so than passengers, so the dispatcher had a third priority level to complicate his plans and calculations. Time estimates for meets used to be done on a chalkboard or on paper. Then the detailed train orders would be sent out to station operators via telegraph. Station operators gave written orders to train crews and also controlled station sidings.
Engineering also played a key role. We already talked about the importance of building a straight, flat track to whatever extent it was feasible, using bridges, high fills, and tunnels to fight nature in hilly country. Again, spending the right amount of money was essential. Railroads often went out of business due to spending too much to building a line that was too perfect, crushed by debt and interest payments when customers wouldn't pay high enough rates for the extra speed. Spending too little could also be fatal, especially if there was a competitor in the area that ran faster and more safely. Many lines got sold at heavy discounts by bankruptcy courts, due to these management failures. The other way the engineering department could help with speed at that time was by adding more sidings, giving the dispatchers more options for meets that resulting in less time being spent by freights sitting around waiting for a higher priority train to show up from either direction.
Double track lines were still off in the future.
Don't assume that southern railroads were always inferior to those in the northeast, since the southern economy was so strong before the war. Also, some lines in the South were built with money and expertise from the North. A great future was seen in expanding cotton and tobacco plantations further back from the riverfronts, free from total dependence on farm wagons.
After the war and on into modern times, northeastern companies and investors have recognized and acted upon railroad business opportunities in the South.