Modernism
The title of this novella helps guide us toward the genre, so let's start there. In Death in Venice, who, or what, dies in Venice? That would have to be Aschenbach, obviously—dude is dead as they come at the end—but what does Aschenbach stand for? In other words: What dies with him in Venice?
We have more (like, way more) to say about Aschenbach in the "Characters" section, but here's the lowdown: Aschenbach starts out as the portrait of an ideal writer, a man who's reached his prime, and is able to exercise perfect, disciplined control over his creative faculties. Then he loses this control as he is overtaken by desires and impulses that lead to his destruction—but also produce some of his finest writing.
From this perspective, what dies in Death in Venice is not just Aschenbach (though he's totally dead at the end, too), but an idealized conception of the artist as the master of his talents. Ironically, that "death" also becomes the birth of a new and tragic creativity. And you know what genre is all about questioning the ideals of the past and finding paths toward the future? That would be modernism. So we can officially say hello to our genre.
But modernism also likes to mess with form. So what about form here? This is not a novella that develops highly innovative forms of writing—its theme and subject matter might be shocking, but it's not trying to be a "new" kind of literature. So Death in Venice is not necessarily modernist in form, though it is modernist in its particular use of irony, giving us the strange sense that "old" ideals are being both upheld and cleverly mocked.