Fitzgerald’s use of the word "curious" might seem anachronistic (i.e., old-fashioned) by now; you’re used to "curious" describing a person who is intrigued about learning something. But "curious" can also mean "odd" or "peculiar," and that is the way in which Fitzgerald uses it here. Benjamin Button’s case really is curious; he’s born as an old man and ages backwards over the course of his life.
The atmosphere set by the title fits quite well with the set-up of the first paragraph of "Benjamin Button," in which we meet the narrator. As we discuss in "Narrator Point of View," the nameless storyteller introduces the tale as though it really happened, as though he bore witness to the curious case and is now passing it on to you, the reader. Additionally, the understated nature of the title – merely a "curious case," not a "medical miracle" or "bizarre phenomenon," fits with the whimsical nature of the story. Part of the humor of "Benjamin Button" is that no one reacts to such a curious case as we might expect. They simply order Benjamin to stop being so ridiculous and age properly, already. More on that in Benjamin's "Character Analysis."