"Church Going" seems like a very simple and straightforward title, just as the poem itself seems to be simple and straightforward. On the most literal level, it refers to the way that regular "church goers" attend mass every week. For the speaker of this poem, though, church going has a completely different set of meanings, because he's not connected to the official teachings of Christianity. Church going for him refers to the way that he continues to return to the church even though he can't find anything in it that's believable. This double meaning of "church going" helps to highlight the tension this poem explores between traditional religious meaning and the speaker's personal relationship to the church.
On another level, "Church Going" could refer to the fact that the speaker of this poem spends much of his time wondering about what will happen to churches once people's belief in religion has vanished from the Earth. In other words, the title also hints at the possibility that the church might "go" away someday and never come back. And guess what? That's exactly what a huge section of this poem is about. Pretty clever stuff, right?