Symbolism, Imagery, Allegory
Paul spends so much time peering in windows that we're pretty sure he's going to end up with a restraining order some day. After his night ushering at Carnegie Hall, he follows the lead soprano back to her hotel, where he sees the "the windows of its twelve stories glowing like those of a lighted cardboard house under a Christmas tree" (1)—in other words, like a total fantasyland that might as well be a dollhouse for all Paul is ever going to enter it.
In case we don't get it, Cather spells it out later in the same paragraph: "the rain was driving in sheets between him and the orange glow of the windows above him. There it was, what he wanted…but mocking spirits stood guard at the doors" (1). Paul is on the outside looking in. Windows symbolize the frustration of Paul's longing for a better life—one that smells a lot less like cooking.
This becomes really clear once Paul makes it to New York. There's still a storm outside, but now he's on the other side of the window. In his hotel room, he looks out at "the snow…whirling so fiercely outside his windows that he could scarcely see across the street" (2). Lucky for Paul, he's inside where "the air was deliciously soft and fragrant" (2).
Later that night, he can hardly even drag himself away from the window to go to bed, "watching the ranging storm from his turret window" (2). See? Storms are great, as long as you get to be the one inside.
Tear Down the Wall
Here's the thing about windows. They're like walls, because they separate people. But if rich people live behind thick walls, it's no big deal. You can fantasize about what their lives are like, but you can't actually see in and know for sure that they're living it up with champagne and pretty dresses while you're out soaking your feet in a chilly puddle.
Windows, on the other hand, let you know exactly how good it could be. They give you a glimpse of a different life, a little peek behind the curtain. And that's Paul's problem, or at least part of it. Without windows, he'd never know what he was missing.
P.S.—There's also some window-related symbolism around the whole "flowers blooming in glass cases" business. Check out "What's Up With the Title?" for more about that.