You seem to be confusing "over the years" with the CW. Read the 1858-1864 Raleigh newspapers and you will see the exact problems I have mentioned.
The newspapers didn't necessarily always know what was going on inside John M. Morehead's mind, and he was the prime driving force behind most of NC's railroad progress during that era; first as governor of the state, then as president of the North Carolina Railroad. He may have promised to various people at various times that he was always going to favor NC ports, but then he pushed for the Danville connection, over the determined opposition of the Quaker voters who dominated Greensboro at that time, and over the objections of NC port operators. The Quakers preferred a calm, quiet, simple lifestyle, and did not equate urban growth with anything good. As a smart, forward thinking businessman, Morehead knew that trains would continue to grow bigger, longer, heavier, faster, and more important to the economy, and he could look at a map and see tremendous potential for growth associated with a direct north-south route from Washington to Atlanta and on to New Orleans. To him, more connections in more directions simply meant more growth in rail traffic and revenue, and eventually the port situation would sort itself out. The end result would be more jobs in the state's interior, and higher tax revenue for the state.
Morehead lived out his retirement years in downtown Greensboro, in a home on property adjoining the railroad. The house survives, as a museum.
Meanwhile, the Quaker no-growth mindset continued to play a major role in Greensboro's lack of development for more than a hundred years, as various large corporations were turned away. They even passed an ordinance against building an airport anywhere inside the city limits, back when every other city was in a rush to build their own.
Various economic interests in Norfolk, mostly tied to shipping companies, have tried several times over the decades to form stronger connections to NC, with very little success. The old original Norfolk Southern Railway that ran from Norfolk through Raleigh to Charlotte (closest connection to Greensboro was near Sanford) survived for many years, but was never high traffic. The NF&D ran along the state line, barely on the Virginia side; never high traffic, now abandoned.
Railroads other than the NCRR had their own agendas. The CW-era Wilmington and Weldon eventually became the Atlantic Coastline; always fiercely loyal to the port of Wilmington even though it did eventually acquire a low traffic connection to Norfolk, plus more heavily used lines to Charleston and Savannah. Predecessor lines that eventually became the Seaboard promoted trade through the port of Charleston because it was convenient to Hamlet Yard, the center of their empire. They reached as far west in NC as Monroe, near Charlotte, on their Atlanta line. The Virginia and Tennessee, later Norfolk and Western, built more branch lines over the Virginia state line into NC than any other company, but none of them evolved into high traffic routes.
Today, only two rail crossings of the NC-VA state line have enough traffic to justify a second track, and most of that traffic is just passing through NC on the way somewhere else. They are the old ACL south of Petersburg, and the NCRR / R&D / Southern Railway / Norfolk Southern line from Greensboro to Danville. State efforts to keep the port of Morehead City alive have been mostly given up, with poor results. What little port action we have left is about 90% in Wilmington. About five years ago, for the first time in decades, NS re-established a daily train between Greensboro and Norfolk, using the old N&W east of Lynchburg (can't turn right at Altavista). It's double stacked containers.