Sort of like old telephone numbers, just so simple. Back then such as on Shelter Island, the post office clerk probably knew everyone on the island. The people would probably pick up their mail at the local P. O., which the clerk also could have served a dual role as grocer, etc.
When you consider giant towns now were just little places back then, everyone had a chance to at least know about everyone else. Most of those look like little places anyway, so everyone would indeed know everyone else.