I did a quick review of the 1860 Census for the 6th Ward which seems consistent with the 1855 State Census. Excluding the City Prison, there were approximately 25,000 individuals recorded in the 6th, of which roughly 289 were African-American, or slightly more than 1%. I only have the district map from 1840, but assuming it remained roughly the same, the lowest percentage of African-Americans was the 3rd District which contains Five Points. Of the approximately 6600 residents, only 13 were African-American (0.2%). Interestingly, the higher density, poorer population appears to have been in the 2nd District to the west bounded by Centre, Pearl, Broadway and Canal. The recorded population was approximately 9,500 with 199 African-Americans (2.1%).
Again, this assumes the districts remained relatively the same from 1840. I would not be surprised if they changed as the demographics of the 2nd seem much more in line with the contemporary portrayals.