From a lyrical standpoint, the hallmark of "Criminal" is Fiona Apple's use of extended metaphor. In this case, likening herself to a criminal on trial.
Though her sins are never enumerated, she confesses to "breaking" a boy "just because she can," and alludes to the pain that she has caused her lover. While it would make sense for an adversary—her paramour, perhaps—to occupy the position of prosecutor, it's the narrator herself—or more accurately, her conscience—who serves that role. The narrator even refers to herself as "the devil" during the song's bridge.
The judge, whose identity remains a mystery, stands as a beacon of wisdom and moral righteousness. The narrator appeals to them repeatedly throughout the song:
I've done wrong and I want to suffer for my sins
I've come to you 'cause I need guidance to be true
Save me from these evil deeds before I get them done
Let me know the way, before there's hell to pay
Although the narrator is eager to receive her punishment, the judge never hands down a decision, leaving the narrator in a state of moral limbo.
In numerous interviews, Fiona Apple has stated that she uses songwriting as a way to work out her personal issues. Since she was a young girl, the singer has kept extensive journals filled with lyrics and poems, many of which form the basis for future songs.
Apple discovered the cathartic power of songwriting at the age of ten, after being deemed mentally unstable and sent for psychiatric evaluation. The singer was overheard at school saying, "I am going to kill myself, and I'm going to bring my sister with me" (source).
Songwriting became an outlet for repressed emotions, a way for the young Apple to deal with the conflict and pain in her life. Ultimately, though, it became an avenue for understanding and self-knowledge.
As she told MTV in 1996, "I'm such an incredibly, stupidly sensitive person that everything that happens to me, I experience it really intensely. I feel everything very deeply. And when you feel things deeply and you think about things a lot and you think about how you feel, you learn a lot about yourself." (Source)
In her lyrics, Apple has consistently explored themes of strength, defiance, ownership, and moral righteousness. She explained to Spin in 1997, "I've spent my whole life not being listened to, or taken seriously, or respected for my opinions. That makes me need to say what I feel and have complete control over the way that it's done. And to make hundreds of people sit down and listen. That's kind of sick when you think about it, that I have to get up there and grind my hips in front of a bunch of people." (Source)
For Apple, songwriting is an empowering endeavor, one that allows the singer to express opinions and emotions on her own terms. But as a woman, she has to "grind her hips" to get the message across. It's a game of tug of war.
So, "Criminal" can be viewed as Apple's unique and deeply personal meditation on sin, guilt, and the essential conflict between desire and obligation.