The golden dome symbolizes Colin Sullivan's dreams of becoming a powerful man. Frank Costello has always taught him to have a lot of ambition, and Colin has turned this ambition toward the non-criminal world. Nevertheless, when a friend catches him looking at the golden dome of the State House, he immediately sees what Colin is thinking and says:

BARRIGAN: What are you looking at? Forget it. Your father was a janitor, his son's only a cop.

In other words, Colin has done well to get where he is. But his friend thinks he needs to know when to quit.

Well, it's pretty clear from Colin's choice of apartment that he doesn't know when to quit, because he specifically picks a place with a great view of the State House. As his realtor says:

REALTOR: Well, there's a great view of the State House. Beacon Hill, you can see it. I mean, you move in, you're upper-class by about Tuesday.

In his heart, Colin's true dream is to never stop rising up in the world. He's smart enough to do it; his only problem is that he's tied to the criminal world through his relationship with Frank Costello, and this relationship ends up getting him killed.

It's especially poignant, then, that in the movie's final shot, we see a rat running in front of the golden dome. This shot suggests that Sullivan could have been so much more if he hadn't gotten bogged down in the rat-like world of crime and violence.