Where It All Goes Down
This poem doesn't give us very many specifics when it comes to its setting. We do know that it's nighttime, and that we're near the docks—the perfect place to dump a body, do some illegal business, or mourn your lost love.
The wharves, the river meeting the sea, and the sea itself are all big parts of the geographical setting, and are really important for the metaphor the poem builds up around the woman as the dangerous sea ("in you everything sank!") and the speaker as a shipwrecked captain, who is "Deserted like the wharves at dawn" (2).
The mental environment, however, is a whole other can of worms. The speaker's mind is full of sadness and despair, and he can't shake thoughts of his lost love, which consume him the same way she consumes everything:
You swallowed everything, like distance.
Like the sea, like time. In you everything sank! (9-10)
And so, whether we're in the metaphorical seas of the woman's descriptions, or the speaker's own tortured imagination, the setting is shaped by the abandonment and despair, continually reminding us of the speaker's heartbreak. As they say, it may be a nice place to visit, but…