The speaker of the poem sneaks into a church after making sure it's empty. He lets the door thud shut behind him and glances around at all the fancy decorations, showing his ignorance of (or indifference to) how sacred all this stuff is supposed to be. After a short pause, he walks up to the altar and reads a few lines from the notes that are sitting on a lectern. After this, he walks back out of the church and slides an Irish sixpence into the collection box, which is basically like donating an old shirt button.
The speaker thinks that the place wasn't worth stopping to check out. But he also admits that he did stop, and that this isn't the first time he's done so. He can't help but wonder what he's looking for when he keeps coming back to this place, and also asks himself about what will happen to churches when there are no more believers left in the world. He wonders if they'll make museums out of the churches, or if they'll just leave the buildings' doors open so that sheep can hang out inside them.
Nearing the end of the poem, the speaker asks what will happen to the world when religion is gone altogether. Then he wonders what the very last religious person will be like. Will they be an obsessive compulsive, who just can't stop wanting to smell incense? Or will they be more like the speaker, someone who's bored and ignorant about the church, and just passing by without knowing what they're looking for?
Finally, the speaker just comes out and admits that he's pleased by the church because it's a serious place for serious questions. Humanity, he concludes, will always have a hunger to ask those big questions like "Why are we here?" and "Where do we go when we die?" And for this reason, the kind of urge that created religion in the first place will never go away, even if organized churches do. Sorry, atheists. If you were looking for a poem that just trashes religion and calls spiritual people stupid, you'll have to look someplace else.