In relation to sports like football and hockey, where guys smash into each other with the approximate mass and speed of elephant seals, baseball is pretty safe. Even so, you have to watch out for injuries that can affect your performance.

The worst involve damage to your hands or feet. It's hard to throw with accuracy when you have to stifle a scream whenever you pick the ball up, and pivoting on your ankle to make that throw gets problematic if your bones start grinding against each other. (Pitchers in particular have to watch out for hand and arm injuries. They want you throwing that ball until your arm falls off, and if it's already falling off, it spoils their fun.)

Coaches tend to worry about more mundane injuries to knees, elbows, hamstrings, and anything else that keeps you from moving the way you normally would (source).

Staying away from those little boo-boos means practicing carefully. Don't over-exert yourself during practice or try to hobble around on a pulled hamstring. Take your time warming up and learn to spot the warning signs if you're pushing yourself too hard. So if the pain goes from "irritating" to "stabbing daggers of agony," you probably should take a break.