James McPherson in Battle Cry of Freedom notes several failed instances of attempted invasions of Cuba from New Orleans, prior to the war ("filibusters" - that's where the word originated in English). Then, of course, there was Tennesseean William Walker: http://en.wikipedia.org/wiki/William_Walker_(filibuster)
"William Walker convinced many Southerners of the desirability of creating a slave-holding empire in tropical Latin America. In 1861, when U.S. Senator John J. Crittenden proposed that the 36°30' parallel north be declared as a line of demarcation between free and slave territories, some Republicans denounced such an arrangement, saying that it 'would amount to a perpetual covenant of war against every people, tribe, and State owning a foot of land between here and Tierra del Fuego.'"
"William Walker convinced many Southerners of the desirability of creating a slave-holding empire in tropical Latin America. In 1861, when U.S. Senator John J. Crittenden proposed that the 36°30' parallel north be declared as a line of demarcation between free and slave territories, some Republicans denounced such an arrangement, saying that it 'would amount to a perpetual covenant of war against every people, tribe, and State owning a foot of land between here and Tierra del Fuego.'"
Let's face it...the easiest places to expand happened to be those owned (but not necessarily settled) by Spain/Mexico....who's the closest? People from southern states (although I noticed a lot of "patriots" in the State Cemetery on Saturday who were born in New York and New England states!)