The speaker in this poem is a guy who doesn't know all that much about churches and religion or any of that kind of stuff. You can tell this from the way he clumsily enters the church doesn't really seem to respect its sacredness of the church. He sees its holy objects inside as "some brass and stuff" (5).
However, that's not all there is to this guy. He also hears a "tense, musty, unignorable silence" in the church, which tells us that the speaker does feel some sense of tension or awe inside the building. He knows he's supposed to do something to show respect, but since he's not wearing a hat, he chooses to "take off/ [his] cycle clips in awkward reverence" (9). In other words, the guy doesn't know much about church, but he's not just coming into the church to make fun of it. He just isn't all that smooth when it comes to dealing with all this holy stuff.
In all, the speaker is someone who can't really get behind the whole religion thing, but he definitely has a hunger for some sort of spirituality. And maybe that's the best way to describe him: he's spiritual, but not religious. The speaker feels like church and faith might be declining in the modern world, and he's curious about what will happen to the church when the last of the believers are gone. While he doesn't come to any great conclusions, he figures that something of the church's influence will remain. He just doesn't know what.
And you know what? He seems okay with that, really. As a whole, the speaker in this poem is characterized by his tendency to ask questions. In fact, the poem is almost all questions between line 23 and 52. He doesn't seem to come up with many definitive answers, but that doesn't stop him from asking in the first place.