Stress
Considering you're working with potentially dangerous tools and elements that could melt your face off in a nanosecond, we'd say there's going to be a certain level of stress involved.
That said, taking all the necessary precautions to ensure your safety and the safety of your co-workers ensures that your potential anxiety about that part of the job will be kept to a minimum.
Your family and friends may have concerns that you'll start glowing in the dark, but that's not something you spend much time thinking about. You're going to have to ease their trepidation with your incredible knowledge and understanding of nuclear science. You're about as likely to develop superpowers as you are a glowing skin tone.
There's another, less immediate stress that you may have to deal with, though. With scientific developments come opinions about said scientific developments. Whether or not those opinions are based on facts is irrelevant; with nuclear science,, there's always some level of public backlash, so you'll have to learn to deal with the occasional protest chant.
You're not just a scientist when you go into this field—you're also a scientific mouthpiece. You know what goes on in the lab, plant, radiology ward, or physics classroom, and it's up to you to communicate that to the folks around you. The thing that may stress you most is having to dumb down your speech patterns for the rest of us.
You may be a genius, but the rest of us are just trying to figure out how you're not an evil genius.