The very end of The Zoo Story has Peter screaming "OH MY GOD!" and then Jerry says, "Oh…my…God," and then he dies.
So what's with everyone shouting out to God right at the end there? Part of it is ironic; this is not a religious kind of play. The Zoo Story is a story about animals wandering about confusedly and bumping into each other, often painfully. A fight over a bench is not a great deed in the divine sense. So the play ends with "oh my God" to emphasize that this not a play with a holy, divine plan behind it.
The "oh my god" might also be important not because of what it says, but just because it's repeated. Peter says it—in fact, he says it a bunch of times as the play lumbers toward its end. Then Jerry says it. You could see this as Jerry mocking Peter—making fun of his meaningless references to a religion he probably doesn't really believe (imagine Jerry saying, "Oh…my…god" with a valley girl accent), or you could see it as Jerry trying to be Peter; imitating him because Jerry sort of, secretly, wishes he were Peter. Or it could be both—in fact, the stage directions say it is spoken with "a combination of scornful mimicry and supplication" (285). Jerry thinks Peter is ridiculous and wants to be Peter at the same time. Which is somewhat confusing—no wonder he calls on God to help him sort it out.