Middle Earth: Post-Apocalyptic Hell Hole
We always seem to think that The Hobbit and the Lord of the Rings took place in some forgotten distant past. But I don't think Tolkien was a historian, he was a clairvoyant futurist. You see, Middle Earth isn't in the distant past; it's in the far future, after the world has recovered from the nuclear holocaust that we all know is waiting around the corner.
It's obviously in the far, far future. The wreckage of our civilization has long since wasted away. There's no memory of a suitcase bomb, or an exchange of ICBMs, just a genetic memory that at one time in the past something went horribly wrong.
And while the radiation caused huge mutations, enough time has gone by that new species have arisen from our failed attempt at world domination. The giant spiders, the flying lizards, the Great Eagles, even those walking talking trees, although they aren't really trees, but a camouflage adaptation gone wild -- can't see the Ents for all the trees!
We can assume that most of the intelligent creatures are descendants of humans. Hobbits are an obvious adaptation to food scarcity and nuclear winter after the bombs drop. It doesn't take as much food to power a three foot, furry creature. The hairy feet are perfect for the long wintry conditions that follow the Bomb. Eventually the world warms up, but they keep their timid ways, learned from scurrying through the wreckage of a world gone wrong.
Dwarves also come from the need to be smaller, but they went the route of being feistier, combative little creature. Again, they're hairy from the long cold years, and have endurance reserved for creatures that had to go days without food or water.
I assume that Orcs are the city dwellers that stayed in the radioactive wreckage like those bomb worshipers in Planet of The Apes. In fact, they're still radioactive. Bilbo's sword, Sting, phosphoresces when Orcs are near -- the radiation they put off is enough to make whatever it is in the metal glow.
There are still Men, although I'm sure the Man Genome Project would show some serious differences from the Human Genome Project. They retained the form of humans today, but they are resistant to the radiation that still lingers in Middle Earth. I'd wager their ancestors were survivalists who holed up in the mountains, which explains a bit about their culture.
Wizards look a lot like Men, but I think they are a separate branch of humans that evolved in their own way. These would be the scientists and the politicians and, yes, the military men (someone had to remember how to make fireworks) who were able to keep technology going after The Fall. Gandalf's staff is probably an amazing, bit of tech, able to store massive amounts of energy and discharge plasma bursts, or simple light, or even modulated pulses to open old-tech security systems.
Every now and then some really fine tech shows up -- Sauron might be an ancient, self-aware supercomputer or cybernetic creature. This would help explain some of the new tech that finds its way into the world, like a cloaking device with an indeterminate power source that can be worn as a ring.
But, I know you're thinking, what of Elves? Elves are, of course, true humans. Well, as true as they can be after millennia in space. These are the colonists of other worlds, come back at relativistic speeds to find the Earth a strangely twisted place. They are far more advanced than we are today, with genetic engineering to extend their lifespans to nearly immortal.
And like most immortals, they're bored. Why else would the hang out with the mutant barbarians of Earth. But when they finally bore of the sword and sorcery thing, they head 'across the sea' in the style of the Queen song '39. They aren't really going 'across the sea' they're going to another planet. The only reason the go to the sea itself it that they probably have a spaceport floating offshore where the creatures of Middle Earth can't mess with.
But what of all the lore and history Tolkien tells in all the other books? What of the songs of the ancient history? You might as well ask, why doesn't the Bible and the Geologic Record tell the same story? A world thrown into chaos from nuclear war isn't likely to keep a lot of records, but it is likely to be the backdrop for a hell of a story.
And then the backdrop for a heck of a rationalization.