After the Tattaglia family assassinates Luca Brasi, the Corleones' feared hit man, they send a message to the Corleones: a fish wrapped in Brasi's bulletproof vest. The message is clear: He "sleeps with the fishes," meaning that his corpse is weighed down at the bottom of a river (maybe the East River, a historically great place to dump dead bodies).

This needed to be mentioned, since it's effective symbolism. But there's no deeper layer of meaning to it. Puzo's book apparently coined the term, and it has popped up many other places since, like on The Simpsons, where a gangster named Fat Tony mimics classic mob archetypes.