Richard Dale Owen and wife Anne Eliza Neef of NewHarmony Indiana.
Edit - I will give credit for this answer. Although Watertown, WI seems to be more emphatic about claiming to be the home of the first kindergarten, the following website says that New Harmony can make the same claim. Richard Dale Owen did serve as a Civil War officer.
Margarethe Meyer-Schurz, wife of Major General Carl Schurz (also U.S. Senator and Secretary of the Interior), founded the first kindergarten in the U.S. in Watertown, Wisconsin, in 1854.
Officer: Carl Schurz (2 March 1829 – 14 May 1906)
Wife: Margarethe Meyer Schurz (27 August 1833 - 15 March 1876)
State: Mrs. Schurz's Kindergarten was started in Watertown, Wisconsin