Lincoln was inagurated March 4, 1861. In 1861, one star was added, representing Kansas, bringing the total number of stars to 34. Kansas, was admitted to the Union on January 29, 1861. This flag flew from 4 July, 1861, to 3 July, 1863. Even after the Southern states seceded from the Union Lincoln would not allow any star to be removed from the American flag. The next star was added to rtepresent West Virginia in 1863, bringing that current total to 35 and was used July 4, 1863 – July 3, 1865.
So, my answer would be at Lincoln's inaguration on March 4, 1861 there were 33 stars on the flag, the 34th star represented Kansas and flew on the flag starting July 4, 1861.