If the story is correct and he was captured at Fort Henry that is Jesse Taylor of the 8th co. Tennessee Artillery (McCown's). There is very little in his combined service record - I would have expected a prisoner of war record from Camp Chase. But he appears in a list of officers surrendered at Ft Henry.
According to a genealogical web page, he later became a major, but I don't see evidence of this in his service record:
(Major) Jesse Taylor (1831 - 1903), son of Jesse and Lydia. A Confederate soldier, Capt. Jesse Taylor, commanded the heavy artillery at Fort Henry during the bombardment February 1862, and was surrendered at that place. He married Lucy Browning of Kentucky. Lucy, a teacher, had nursed soldiers during the war. After the war she resumed teaching near Mifflin, Tennessee. She married Major Taylor in 1864. In the 1870's Jesse and Lucy Taylor founded the Springdale Institute, a school for young ladies. Lucy's pupils came from Virginia, Kentucky and Tennessee, and she was said to have instructed some of the first ladies of the old commonwealth of Kentucky.
His brother, John May Taylor, was also a captain. You can see that he somewhat resembles the man in the photo.