The evils of genre labeling fiction

After being annoyed by countless discussions online on what genre one book or another belongs to. I decided to address the issue. I thought I'd post it here.

I am a fan of both what we call literary fiction, and science fiction. And as someone who reads many literary works in both categories, it disturbs me how much emphasis we place on categorizing fiction. What does it matter except to limit someone who doesnt like one category or the other from reading sometimes excellent works?

Many times the genre is no more descriptive of a work, than the cover itself. While I can see why someone might debate whether Kazuo Ishiguro's Never Let Me Go is literary fiction or science fiction; there needs to a recognition that sometimes a book can be both literary and still explore themes found in science fiction. Look at George Orwell's 1984 or Margaret Atwood's Handmaid's Tale or M. John Harrison's Things That Never Happen. As someone who finds that today, genres limits what one reads or watches or listens to, I suggest that maybe we should begin to explore works for merit rather than trying to classify them so that they fit comfortably within the boundaries that we have defined.

Note: I did not address the argument that genrefication is beneficial for categorization, placement in a bookstore, or that well we need some sort of genrefication to study certain movements, or illuminate how we think about certain books aspects of the argument, because clearly some classification is needed, and has its purposes. It would be stupid to argue against all genrefication. However it seems to me, when someone in a discussion group is asking, hey is this ficiton or science fiction, all they are looking for is to be spoon fed a point of view, and a preconcieved notion of what the book is going to be about, which I do have a problem with. Please feel free to comment.