Where It All Goes Down
This poem is set in the Big Bad Apple, a.k.a. New York City. The speaker tells us exactly where he is in New York: the poem starts with him telling us that he's walking along 44th Street (which is in midtown Manhattan). He heads over to Sixth Avenue, a famous avenue in New York, before heading downtown.
The poem immerses us in this urban setting. There are "pedestrians" walking around (31), we see the "pavement sparkling with sunlight" (8), and we see the "pigeons fluttering off the curb" (9). The city's important as a location because jazz is very deeply associated with New York. Sure, jazz first got going in New Orleans, way down there in the south, but it eventually moved north. In fact, New York became a hotbed of jazz music during the Harlem Renaissance, which is considered to be the music's greatest period.
So the speaker's emphasis on the New York setting calls our attention to the long relationship that jazz has with the city. He's in a New York state of mind, or in other words: jazzed.