Where It All Goes Down
Since it's quite foggy in these parts, and we know the fog hovers over "harbor and city" we can assume we're in a place that has these kinds of weather patterns. We don't get any specifics as to the name of this particular city, but we don't really need any. Fog tends to look and feel the same no matter where you go.
But perspective matters in this poem, so although it's foggy, we also know that the speaker is watching the fog in a place that allows him to see its approach. So maybe we're in a city with skyscrapers, Chicago perhaps (the collection that this poem appears in is called Chicago Poems), where we know there's fog, harbors, and spots where people can watch weather patterns roll in from.
And that backdrop of the city is really what helps to establish the tension in this poem. The fog comes mysteriously creeping in, around a whole city of unsuspecting people. Is it dangerous? What will it do to them? Ah, nothing (as it turns out)—but the city setting is what allows for the hint of threat in the first place.