North of Jacksonville is Fort Clinch, a well preserved Third System seacoast fort of unusual (for North America) polygonal design with caponiers for flanking fires and <very unusual> carnot scarps.
There's an NPS park in Jacksonville near the site of the piratical French colony that was destroyed by the Spaniard Pedro Menendez in 1565. There's a phony replica of the French fort Menendez stormed and put its garrison to the sword.
Menendez founded St. Augustine, about 30 miles to the south. Much history there including the crown jewel of NPS forts, the Castillo de San Marcos (where I was an NPS volunteer). A magnificently maintained example of gunpowder era bastioned fortification complete with glacis, covered way, fine ditch and a ravelin.
Lots of other history too—the site of Fort Mose where black Spanish militia and their families lived (slaves who escaped the English colonies to Florida were accepted by the Spanish and turned into Spaniards). At Fort Mose a British force was destroyed by the Spanish during the siege of 1740.