After the capture of Morris Island, General Gillmore was unable to capture Sumter. Both he and Dahlgren tried amphibious assaults but as the Rebs read their signals, were well prepared. Gillmore wanted Dahlgren to sail right into Charleston Bay to attack Charleston and force its surrender. Dahlgren refused to sail past Sumter as it had not been silenced. Sumter by now ceased to be an artillery post and had all its artillery removed and placed elsewhere. Its importance was to guard the torpeodos that guarded the entrance to Charleston Bay. Stymied, Gillmore resorted to long distance artillery.
By April, 1864, Bureaugard pulled out most of his troops to protect Petersburg. While this momentarily gave Gillmore a huge advantage in numbers, Gillmore and 15,000 of his men were pulled out by Grant in May, 1864 to become part of Beast Butler's Army of the James. This left John Foster in command and he continued the siege. Foster continued shelling Sumter and Charleston. This was about the time when the Confederates placed Union officers in Charleston as hostages who were exposed to Foster's shellfire. In retaliation, the Union placed 600 Confederates on Morris Island where they would be exposed to counterfire by the Confederates. These 600 became known as the "Immortal 600." Anyway, Gillmore fell out with Butler and was relieved by Grant (initially by Butler but Grant had Butler withdraw the order and Grant issued one himself). Gillmore was allowed to return to the Department of the South where he finally, with the cooperation of Sherman's approaching army, was able to capture Charleston (which was abandoned by the Corn-feds).
BTW, this will be covered in Chapter 10 of my book about the blackpowder sharpshooter.