Symbolism, Imagery, Allegory
As Mr. Vladimir explains, it won't do for Verloc to arrange an attack on a head of state, church, or parliament building, because all of these attacks could be explained as acts of angry violence undertaken by a crazy individual against an institution.
In order to really make people sit up and listen, Vladimir says, Verloc needs to initiate an attack on an institution of science, which is the thing that people in Conrad's time had total confidence in:
"Any imbecile that has got an income believes in that. He does not know why, but he believes it matters somehow. It is the sacrosanct fetish." (2.111)
The Greenwich observatory is also on the Prime Meridian, which sets the clock time for the entire world and, because of this, structures the way everyone lives out their days. The fact that this building is never so much as touched by the blast that kills Stevie adds another note of absurdity and futility to this already bleak, hope-depleting book.